EVALUATION OF SECONDARY PREVENTION IN THE COMMUNITY, FAMILY AND YOUTH RESILIENCE PROGRAM IN ST. LUCIA, ST. KITTS AND NEVIS AND GUYANA FINAL BASELINE REPORT August 2019 This publication was prepared independently by Stephanie Gimenez Stahlberg, Alberto Díaz-Cayeros, Corinna Bordewieck, Daniel Sabet, Jordan Robinson, Catherine Caligan, and Julia Kresky of Social Impact. It was produced at the request of the United States Agency for International Development as part of the Democracy, Human Rights, and Governance – Learning, Evaluation, and Research activity. EVALUATION OF SECONDARY PREVENTION IN THE COMMUNITY, FAMILY AND YOUTH RESILIENCE PROGRAM IN ST. LUCIA, ST. KITTS AND NEVIS AND GUYANA FINAL BASELINE REPORT August 2019 AID-OAA-M-13-00011 DISCLAIMER The author’s views expressed in this publication do not necessarily reflect the views of the United States Agency for International Development or the United States Government. ACKNOWLEDGMENTS This evaluation has required extensive coordination between many different groups and individuals. First and foremost, we would like to thank our colleagues at the United States Agency for International Development (USAID) for continually pushing and advocating a rigorous evaluation design despite the many challenges this entailed. We are grateful to Morgan Holmes, Andrew Greer, Ted Lawrence, Ryssa Brathwaite, Mansfield Blackwood, Chloe Noble, and Sharon Ramsaran. The evaluation required the implementer Creative Associates to make numerous compromises and invest a great deal of time and energy into ensuring both strong programming and a rigorous evaluation. We want to thank Debra Wahlberg, Jason Wilks, Carlo Arze, Donald Cole, Dwynette Eversley, Waqas Mahmood, Sara Danish, Enrique Roig, Guillermo Cespedes, Chuck Katz of Arizona State University, and University of Southern California colleagues Karen Hennigan and Kathy Kolnick. The National Opinion Research Center (NORC) at the University of Chicago led much of the data collection, and we appreciate the efforts of Ron Wendt, Kareem Kysia, Alexander Rigaux, Letitia Onyango, and Jeff Telgarsky. We would also like to thank data collection partners in the three countries, including The Solomon Group, Specialists in Sustained Youth Development and Research (SSYDR), and the St. Lucia Central Statistical Office as well as Social Impact (SI) field coordinators Carol English, Sacha Harris, and Marecia Pemberton and former SI staff who supported the Democracy, Human Rights, and Governance – Learning, Evaluation and Research (DRG￾LER) mechanism, Alison Miranda and Meredith Feenstra. Finally, we are grateful to the many youth, parents, and caregivers who participated in the Youth Services Eligibility Tool (YSET) assessment and caregiver survey. CONTENTS LITERATURE REVIEW – REDUCING YOUTH RISK LEVELS FOR ENGAGING IN CRIME AND VIOLENCE 3 THE STATE OF YOUTH VIOLENCE IN THE CARIBBEAN 3 THE CAUSES OF YOUTH VIOLENCE IN THE REGION 3 WHAT WORKS IN PREVENTING YOUTH VIOLENCE 4 EVALUATION PURPOSE, QUESTION, AND METHODOLOGY 10 EVALUATION PURPOSE 10 EVALUATION QUESTIONS 10 EVALUATION DESIGN 10 THE YSET AND SELECTION OF YOUTH 10 CAREGIVER SURVEY 13 COMMUNITY SURVEY 14 HUMAN SUBJECTS PROTECTIONS 14 ASSUMPTIONS AND RISKS 14 COMMUNITY CONTEXT: RELEVANT FINDINGS FROM THE CFYR COMMUNITY SURVEY 17 CRIME VICTIMIZATION 18 PERCEPTION OF CRIME PREVALENCE 21 OVERALL SOCIAL CAPITAL INDEX 23 YOUTH SOCIAL CAPITAL 24 ATTITUDES TOWARD THE LAW 26 VIEW OF THE POLICE 27 RESOURCES FOR YOUTH 28 SELF-EFFICACY 29 INDEX SUMMARY 30 FINDINGS BASED ON THE YSET ASSESSMENT 36 QUESTION 1: DO SECONDARY PREVENTION EFFORTS IMPACT OVERALL YOUTH RISK, CRIME AND VIOLENCE RELATED BEHAVIOR, AND YOUTH RISK AND RESILIENCE FACTORS? WHY OR WHY NOT? 36 UNDERSTANDING RISK ACROSS THE FULL YOUTH POPULATION 36 SCALE B: WEAK PARENTAL MONITORING 40 SCALE C: CRITICAL LIFE EVENTS 42 SCALE DE: IMPULSIVE RISK TAKING 43 SCALE F: GUILT NEUTRALIZATION 44 SCALE G: NEGATIVE PEER INFLUENCE 46 SCALE H: PEER DELINQUENCY 47 SCALE T: FAMILY ANTI-SOCIAL INFLUENCE 48 SCALE IJ: SELF-REPORTED DELINQUENCY 50 SCALE FRD: RISKY GROUP BEHAVIORS 52 COMPARISON BETWEEN HONDURAS AND CARIBBEAN YOUTH IN TARGETED COMMUNITIES 54 BASELINE FINDINGS FROM THE CAREGIVER SURVEY 56 Q2: DO SECONDARY PREVENTION EFFORTS IMPROVE THE FAMILY ENVIRONMENT? WHY OR WHY NOT? 56 INTRODUCTION 56 CAREGIVER SURVEY RESPONDENTS AND FAMILY STRUCTURE 56 FAMILY CHARACTERISTICS 59 FAMILY COHESION 64 FAMILY DECISION-MAKING 66 PARENTING ATTITUDES: LOCUS OF CONTROL 68 CAREGIVER PRESENCE 70 NEGATIVE FAMILY INFLUENCES 76 KNOWLEDGE OF YOUTH BEHAVIOR 77 CONCLUSIONS 80 ANNEX I: ELIGIBILITY DETERMINATION AND RANDOMIZATION 82 DETERMINATION OF ELIGIBILITY 82 MATCHING AND RANDOMIZATION 82 ANNEX II: DATA COLLECTION INSTRUMENTS 84 YSET TOOL 85 CAREGIVER SURVEY 110 COMMUNITY SURVEY 123 ANNEX III: REGRESSION RESULTS 156 THRESHOLDS 156 ANNEX IV: SUPPORTING TABLES AND FIGURES 159 MEASURES OF INCOME 159 RELIGION 160 ANNEX V: INDEX CONSTRUCTION 165 ANNEX VI: GENDER DISAGGREGATIONS 167 YSET 167 CAREGIVER SURVEY 176 REFERENCES 187 TABLES Table 1: Sample sizes in various surveys used in this report............................................................... 13 Table 2: Assumptions and risk mitigation strategies.............................................................................. 15 Table 3: Index Summary............................................................................................................................... 31 Table 4: Percent of assessed youth at risk in the overall YSET and on each scale (n=5,109) ..... 37 Table 5: Odds ratios with Guyana, females, in school, ages 10 to 14, and ethnicity-Black/Afro￾Guyanese as the base (n=5,101) ................................................................................................................ 38 Table 6: Differences in predicted probabilities for each group of respondents scoring at risk/eligible (n=5,101) ................................................................................................................................... 39 Table 7: Percent of assessed youth at risk for weak parental monitoring (n=5,109).................... 41 Table 8: Percent of assessed youth at risk for critical life events (n=5,109).................................... 42 Table 9: Percent of assessed youth at risk for impulsive risk taking (n=5,109)............................... 43 Table 10: Percent of assessed youth at risk for guilt neutralization (n=5,109) ............................... 45 Table 11: Percent of assessed youth at risk for negative peer influence (n=5,109)....................... 46 Table 12: Percent of assessed youth at risk for peer delinquency (n=5,109).................................. 47 Table 13: Percent at risk for family antisocial influence (n=5,109)..................................................... 49 Table 14: Percent of assessed youth at risk for self-reported delinquency (n=5,109).................. 50 Table 15: Percent at risk for risky group behaviors (n=5,109)........................................................... 53 Table 16: T-tests of means with entire samples (eligible and ineligible)............................................ 55 Table 17: T-tests of means with eligible youth only .............................................................................. 55 Table 18: Caregivers: primary (by gender) and additional household members (n=941)............. 57 Table 19: Share of household members who are gainfully employed (n=942)................................ 61 Table 20: Average number of gainfully employed household members (n=947)............................ 62 Table 21: Number of youth between the ages of 16 and 18, by employment and school status (n=537) ............................................................................................................................................................. 62 Table 22: Youth behaviors as reported by self, caregivers, and differences between the two ... 78 Table 23: Share of matches and mismatches in reporting youth behavior, by type of primary caregiver (n=1,135 for youth and 1,002 for caregivers)....................................................................... 79 Table 24: Average scores for 34 highest risk youth and final thresholds across nine risk modules in Guyana......................................................................................................................................................... 82 Table 25: Results of stage 1 and 2 randomizations in Guyana ............................................................ 83 Table 26: YSET, caregiver survey, and implementation dates............................................................. 84 Table 27: Odds ratios with St. Lucia, females, in school, ages 10 to 14, and ethnicity-Black/Afro￾Guyanese as the base.................................................................................................................................. 156 Table 28: Odds ratios with St. Kitts and Nevis, females, in school, ages 10 to 14, and ethnicity￾Black/Afro-Guyanese as the base............................................................................................................. 156 Table 29: Comparing shares of risk on each YSET scale by country, by setting the risk thresholds to be the same......................................................................................................................... 157 Table 30: Logistic regressions comparing shares of risk on each YSET scale by country, by setting the risk thresholds to be the same ............................................................................................ 158 Table 31: PCA with polychoric correlations......................................................................................... 166 FIGURES Figure 1: Number of youth in communities, by eligibility status, and interviewed for each survey ........................................................................................................................................................................... 12 Figure 2: Crime victimization reported by respondents in the last 12 months by country and community....................................................................................................................................................... 18 Figure 3: Crime victimization among other members of the household in the last 12 months by community and country ............................................................................................................................... 19 Figure 4: Crime victimization: Count of reported crimes of different type by community.......... 20 Figure 5: Crime victimization: Count of reported crimes of different types by country.............. 21 Figure 6: Average Perception of Crime Prevalence Index scores by community and country ... 22 Figure 7: Rankings: Perceptions of crime prevalence versus reported crime victimization by community....................................................................................................................................................... 23 Figure 8: Average Overall Social Capital Index scores by community and country....................... 24 Figure 9: Average Youth Social Capital Index scores by community and country ......................... 25 Figure 10: Rankings: Average Overall Social Capital Index and Youth Social Capital Index scores by community ................................................................................................................................................. 26 Figure 11: Average Attitudes Towards the Law Index scores by community and country ......... 27 Figure 12: Average View of the Police Index scores by community and country........................... 28 Figure 13: Average Resources for Youth Index scores by community and country...................... 29 Figure 14: Average Self-Efficacy Index scores by community and country....................................... 30 Figure 15: St. Lucia community risk factor visualizations with captioned occurrences represented by a red dot and non-occurrences by a black dot.......................................................... 33 Figure 16: St. Kitts and Nevis community risk factor visualizations with captioned occurrences represented by a red dot and non-occurrences by a black dot.......................................................... 34 Figure 17: Georgetown, Guyana community risk factor visualizations with captioned occurrences represented by a red dot and non-occurrences by a black dot.................................. 35 Figure 18: Average standardized YSET scores from 0 to 1 on each scale for eligible and ineligible youth (n=5,109) ............................................................................................................................................. 40 Figure 19: Responses to select questions on parental monitoring among eligible youth (n=1,138) ........................................................................................................................................................................... 41 Figure 20: Responses to select questions on parental monitoring among eligible youth (n=1,138) ........................................................................................................................................................................... 42 Figure 21: Responses to questions on critical life events in the last year among eligible youth (n=1,139) ......................................................................................................................................................... 43 Figure 22: Responses to questions on impulsive risk taking among eligible youth (n=1,134)...... 44 Figure 23: Responses on select questions on guilt neutralization among eligible youth (n=1,137) ........................................................................................................................................................................... 45 Figure 24: Responses on select questions on guilt neutralization among eligible youth (n=1,135) ........................................................................................................................................................................... 46 Figure 25: Responses on negative peer influence among eligible youth (n=1,133)......................... 47 Figure 26: Reported involvement of peers in delinquent behavior by eligible youth (n=1,135).. 48 Figure 27: Responses on family antisocial influence questions among eligible youth (n=1,132).. 49 Figure 28: Response on family antisocial influence question among eligible youth (n=1,133) ..... 50 Figure 29: Self-reported delinquency among eligible youth in the last six months (n=1,136)...... 51 Figure 30: Self-reported delinquency by eligible youth with a group of friends (n=44;1,017;160;520;203;12;847;460;810;726;513;124) ....................................................................... 52 Figure 31: Eligible youth reporting on behaviors their friends have engaged in during the last six months (n=1,008).......................................................................................................................................... 53 Figure 32: Family structures in St. Lucia, St. Kitts and Nevis and Guyana in caregiver survey (n=941) ............................................................................................................................................................. 58 Figure 33: Number of assets in households by country as reported in the caregiver survey (n=947) ............................................................................................................................................................. 59 Figure 34: Age distribution of mothers of eligible youth (regardless of caretaking role or family structure .......................................................................................................................................................... 60 Figure 35: Age distribution of fathers of eligible youth (regardless of caretaking role or family structure .......................................................................................................................................................... 60 Figure 36: Levels of educational achievement by country and relationship to program youth (n=941) ............................................................................................................................................................. 61 Figure 37: Share of youth ages 16-18 that are neither employed nor in school (n=537)............. 63 Figure 38: Average frequency of religious service attendance, by country (n=944)...................... 64 Figure 39: Responses to questions on vertical family relations (n=1,000) ....................................... 65 Figure 40: Distribution of scores in the Family Cohesion Index, by country (n=943) .................. 66 Figure 41: Responses to questions on family decision-making among households with more than one caregiver, n=872 .................................................................................................................................... 67 Figure 42: Caregiver responses to questions on family decision-making, by country (n=872).... 67 Figure 43: Caregiver responses to questions on parenting locus of control (n=947)................... 68 Figure 44: Caregiver responses to questions on parenting locus of control (cont) (n=947)....... 69 Figure 45: Caregiver responses to questions on parenting locus of control, by country (n=947) ........................................................................................................................................................................... 70 Figure 46: Caregiver responses to questions on parenting locus of control (cont.), by country (n=947) ............................................................................................................................................................. 70 Figure 47: Number of days in a typical week caregivers report select behavior for each eligible youth................................................................................................................................................................. 71 Figure 48: Number of days in a typical week caregivers report select behavior for each eligible youth (n=996)................................................................................................................................................. 72 Figure 49: Caregiver responses to questions on caregiver presence for each eligible youth (n=998) ............................................................................................................................................................. 73 Figure 50: Caregiver responses to questions on caregiver presence for each eligible youth (n=1,001) ......................................................................................................................................................... 74 Figure 51: Caregiver responses to questions on communication with each eligible youth (n=1,001) ......................................................................................................................................................... 74 Figure 52: Caregiver responses to questions on caregiver presence, by caregiver’s gender (n=1,001) ......................................................................................................................................................... 75 Figure 53: Responses to select questions on caregiver presence (n=1,001) ................................... 76 Figure 54: Share of matches and mismatches between youth and caregiver answers, for each behavior (n=1,135 for youth and 1,002 for caregivers)........................................................................ 79 Figure 55: Household socioeconomic ranking according to assets reported in caregiver survey (n=947) ........................................................................................................................................................... 159 Figure 56: Share of caregiver self-reported religious affiliation, in St. Lucia (n=413)................... 160 Figure 57: Share of caregiver self-reported religious affiliation, in St. Kitts and Nevis (n=155) 161 Figure 58: Share of caregiver self-reported religious affiliation, in Guyana (n=377) .................... 161 Figure 59: Caregiver reported crimes that household members have been charged with in the last ten years (n=76)................................................................................................................................... 162 Figure 60: Caregiver reported household crime victimization in the last 12 months (n=42).... 163 Figure 61: Shares of matches and mismatches of youth behavior, by type of primary caregiver ......................................................................................................................................................................... 164 Figure 62: Responses to select questions on parental monitoring, by gender.............................. 167 Figure 63: Responses to select questions on parental monitoring, by gender.............................. 167 Figure 64: Responses to questions on critical life events, by gender............................................... 168 Figure 65: Responses to questions on impulsive risk taking, by gender.......................................... 169 Figure 66: Responses to questions on impulsive risk taking, by gender.......................................... 169 Figure 67: Responses on select questions on guilt neutralization, by gender................................ 170 Figure 68: Responses on select questions on guilt neutralization, by gender................................ 170 Figure 69: Responses on negative peer influence, by gender ............................................................ 171 Figure 70: Responses on negative peer influence, by gender............................................................ 171 Figure 71: Responses to questions on peer delinquency, by gender............................................... 172 Figure 72: Responses on family antisocial influence questions, by gender...................................... 173 Figure 73: Responses on family antisocial influence question, by gender....................................... 173 Figure 74: Responses on self-reported delinquency, by gender........................................................ 174 Figure 75: Responses on risky group behavior, by gender................................................................. 175 Figure 76: Responses to questions on family leadership, by caregiver’s gender, n=104 Male, 767 Female ............................................................................................................................................................ 176 Figure 77: Responses to questions on parenting, by caregiver’s gender......................................... 176 Figure 78: Responses to questions on parenting, by caregiver’s gender......................................... 177 Figure 79: Number of days in a typical week caregivers report select behavior, by caregiver’s gender ............................................................................................................................................................. 177 Figure 80: Number of days in a typical week caregivers report select behavior, by caregiver’s gender ............................................................................................................................................................. 178 Figure 81: Responses to select questions on caregiver presence, by caregiver’s gender........... 178 Figure 82: Responses to select questions on caregiver presence, by caregiver’s gender........... 179 Figure 83: Responses to select questions on caregiver presence, by caregiver’s gender........... 179 Figure 84: Responses to select questions on caregiver presence, by caregiver’s gender........... 180 Figure 85: Responses to select questions on caregiver presence, by caregiver’s gender........... 180 Figure 86: Responses to select questions on household drug problems, by caregiver’s gender ......................................................................................................................................................................... 181 Figure 87: Responses to question on crime in the last 12 months, by caregiver’s gender......... 181 Figure 88: Responses to question on crime in the last 12 months, by caregiver’s gender......... 182 Figure 89: Responses to question on crime in the last 12 months, by caregiver’s gender......... 183 Figure 90: Responses to question on crime in the last 12 months, by caregiver’s gender......... 184 Figure 91: Responses to question on gang membership, by caregiver’s gender ........................... 185 Figure 92: Responses to select question on community, by caregiver’s gender........................... 185 Figure 93: Responses to select question on community, by caregiver’s gender........................... 186 Figure 94: Responses to questions on police perception, by caregiver’s gender........................ 186 ACRONYMS Acronym Definition CBT Cognitive Behavioral Therapy CFYR Community, Family and Youth Resilience DRG-LER Democracy, Human Rights, and Governance – Learning, Evaluation, and Research ESC Eastern and Southern Caribbean FACES Family Adaptability and Cohesion Scale GUY Guyana GRYD Gang Reduction and Youth Development IDB Inter-American Development Bank LAC Latin American and Caribbean LST LifeSkills Training MDES Minimum Detectable Effect Size MICS Multiple Indicator Cluster Survey NEET Not in Education, Employment, or Training NORC National Opinion Research Center PCA Principal Component Analysis RCT Randomized Control Trial SI Social Impact SKN St. Kitts and Nevis SLU St. Lucia SSYDR Specialists in Sustained Youth Development and Research UNICEF United Nations Children’s Fund USAID United States Agency for International Development USC University of Southern California YES Youth Empowerment Services YSET Youth Services Eligibility Tool i EXECUTIVE SUMMARY In response to crime and violence in the Eastern and Southern Caribbean (ESC), in 2016 the United States Agency for International Development (USAID) launched the Youth Empowerment Services (YES) project, which includes the Community, Family and Youth Resilience (CFYR) initiative. Implemented by Creative Associates, CFYR employs a public-health informed, place-based strategy in 15 communities in Guyana, St. Lucia, and St. Kitts and Nevis. This impact evaluation focuses on CFYR’s Family Matters secondary prevention efforts, a key component of the CFYR intervention. The Prevention and Intervention Family Systems Model (PIFSM) used by the Family Matters program is comprised of two interrelated components: 1) the Youth Services Eligibility Tool (YSET) diagnostic process which identifies nine risk factors predictive of crime and violence behaviors and 2) the seven phases of family systems intervention which reduces the nine risk factors. A strict, randomized selection process was established in which the YSET was applied to a large percentage of the population of youth between the ages of 10 and 17 in the 15 communities. CFYR then used the YSET to determine which youth were at higher risk and in need of secondary prevention efforts. Eligible youth were randomized into a treatment group, which would be eligible for one year of the PIFSM right away, and a control group, which would be eligible for the program in the following year1. There was no ethical dilemma presented by the delayed treatment, given that the size of the treatment group corresponds with the capacity constraints of the implementing party. This randomization strategy allows us to compare similar youth to estimate the effect of the intervention on treatment youth and their families vis a vis control youth and their families after one year. The tools used for the evaluation include a community survey conducted by CFYR, the YSET for youth, which is applied at baseline, after six months, and after one year, a caregiver survey of the youth’s primary caregiver, which is conducted at baseline and at the end of the one-year program, and qualitative fieldwork. The evaluation of the intervention itself will not be available until a year from now, but this report presents findings for the baseline community survey, YSET, and caregivers survey. Key findings include the following: 1. While not as well evaluated as some other approaches, a family-based approach to countering youth violence is supported by the existing literature on the causes of youth violence in the Caribbean and Latin America. 2. The 15 communities included in the study are suffering from varying degrees of crime victimization with variation in the composition of the crimes suffered by each community. Many residents have feelings of insecurity, and both victimization and perceptions of crime prevalence are particularly strong in Sophia in Georgetown, Guyana. 3. We find that overall social capital and a sense of self-efficacy is high in all 15 communities, suggesting that there are important opportunities to leverage connections and community 1 It should be noted that in Creative Associate’s previous implementation of the PIFSM, there was no separation between applying the YSET and commencing with treatment. In the Family Matters program, there was a time delay between the YSET and when the family first received a visit from their assigned family counselor. Also, the application of the YSET for Family Matters was done by enumerators instead of the family counselors that would begin working with the family. Although hiring enumerators for data collection was essential for a robust impact evaluation, this change in the PIFSM probably made it harder for family counselors to engage the family in their initial encounter. ii resources in supporting youth resilience programs. Residents are generally not satisfied with services for youth, although respondents are somewhat more satisfied in St. Kitts and Nevis (particularly in Basseterre) than the other two countries. 4. Attitudes towards the rule of law are relatively positive and consistent across communities and countries. On average, respondents also have a generally trusting view of the police, although there is variation across communities. 5. Although there are differences between countries, there is typically greater variation across communities than between countries. This is a positive finding for the evaluation design, as it suggests comparability between the three countries on key indicators. 6. The YSET tool was adapted by the University of Southern California (USC) team for the three countries, allowing for the identification of higher risk youth eligible to be included in the program. One-fifth of youth surveyed scored above the eligibility cutoff. 7. Single risk factors are highly correlated with the YSET composite scoring of eligibility, suggesting a relative robustness of the tool. These single risk factors identify a similar share of youth engaged in risky behavior or exposed to risk factors in all communities. 8. Males are at significantly higher risk than females in the overall YSET and in each scale, except for family antisocial influence, where the difference between males and females is not statistically significant. Older youth (ages 15-17) are at higher risk than younger youth (ages 10-14) in all YSET scales. 9. Other personal characteristics correlate with CFYR’s risk determination. Out of school youth are at higher risk on most scales, and East Indian/Indo-Guyanese youth are at significantly lower levels of risk than other ethnic groups in Guyana. Country differences are not significant for determining eligibility, but the level of risk by country does vary across the different YSET scales. Youth in the islands are more likely to score at risk in the weak parental monitoring and negative peer influence scales, even when the scale thresholds are set to be the same for the three countries (see full table in Annex III). 10. A surprisingly large amount of eligible youth have had some critical life event happen in the last year. Some of the events included in the critical life events scale can be stressful or even traumatic, such as hearing about deaths or injuries caused by violence in their neighborhoods (which happened to 71 percent of eligible youth), having a fight or problem with a friend (67 percent), or having someone close to them die, get seriously injured, very sick or leave the country (64 percent). 11. Youth seemed to be relatively forthcoming in reporting behaviors. They were asked if they had engaged in a set of twelve negative behaviors, such as fighting or stealing, in the last six months. The caregivers of eligible youth were also asked if they thought the youth had engaged in those same behaviors in the last six months. The youth answered yes much more often than their caregivers did in the caregiver survey. Some of the percentage point differences in the reporting are as high as 36 points, and no single type of caregiver (e.g. parent, grandparent, etc.) seemed to be misreporting more often. Either caregivers are not very aware of how often youth are engaging in these negative behaviors, and which behaviors they engage in, or they are not willing to report these behaviors, presumably due to embarrassment or fear that the youth would get into trouble. 12. The caregiver survey allowed us to create a more comprehensive picture of the family composition and family life of the youth that are engaged in the program (eligible families in iii the treatment and control groups). The majority of primary caregivers are mothers (72 percent), while 11 percent are fathers and 17 percent are grandmothers, aunts, sisters and other family members. The most typical eligible household is one with a single mother and no father present in the household (52 percent). On average, around two additional household members, besides the primary caregiver, participate in care for the youth. This family composition is similar across countries, although Guyanese households tend to have more children. 13. On average, eligible households have 1.63 members who are gainfully employed. Employment levels are much higher in St. Kitts and Nevis (SKN) compared to the other two countries. 14. About 24 percent of youth ages 16-18 in eligible households are neither in school nor employed. There is significant variation across countries, with only four percent of treatment youth in St. Kitts and Nevis being out of school and unemployed, compared to 31 percent in Guyana. 15. Nearly half of respondents (47 percent) believe that community conditions are making it more difficult for them to keep their children out of trouble, but the vast majority (91 percent) think that parents can have a major influence on their kids. Caregivers already appear to have a relatively strong internal locus of control when it comes to raising their children. 16. 57 percent of caregivers report always knowing where their child is when not at home or in school, and 49 percent report always knowing who they are with. About 10 percent never or rarely know where their child is when outside the home or school. Only half report being able to always discuss problems with their child. This suggests that there is room for improvement in the quality and frequency of communication between youth and their caregivers. 17. A higher percentage of male caregivers report being listened to than female caregivers. Males are also more likely to report that they are able to talk to their child and solve their problems and less likely to report saying mean things or pushing or hitting their child when angry. It is not clear if these differences reflect actual behavior differences or gender differences in reporting. 18. In seven percent of eligible households, caregivers report that drug use has created a problem in the household. In these households, marijuana was mentioned 67 percent of the time and alcohol 36 percent. 1 INTRODUCTION Social Impact (SI) is contracted with the United States Agency for International Development (USAID) to evaluate the Community, Family and Youth Resilience (CFYR)’s Family Matters program. CFYR includes 15 communities with relatively high levels of crime and violence, distributed evenly across the countries of Guyana (GUY), St. Kitts and Nevis (SKN), and St. Lucia (SLU). The Family Matters secondary prevention program is targeted towards high-risk youth, and it is one of four components of CFYR’s programing. Crime and violence are growing problems in the Caribbean region. With increased globalization, the region has experienced a proliferation of trans-national crime and drug-trafficking that is unprecedented in scope and scale (CARICOM, p. 16). A CARICOM Commission on Youth Development found that crime and violence are primary concerns for youth in the Caribbean, as young people speak of “fear, perceptions of lack of safety and concern for their general well-being as a result of the increased crime and violence; of self-imposed curfews, diminished participation in community activities, restriction of night￾time activities, and changes in social practice as a consequence” (CARICOM, pp. xvi-xvii). Constant exposure to crime and violence can deeply affect a person’s well-being, through increases in stress levels, emotional scarring, grief, and creating a sense of loss (CARICOM, p. xvi). Although the level of social and economic development varies in the three countries included in this project, all three have marginalized communities where youth face significant hardships at the personal, family and community levels. Each of the countries exhibits profound dislocations in their family structures, characterized by single-mother-headed households, missing male role models, and a relatively large transient population due to migration.2 Local gangs that control drug sales and instigate violence afflict some of these communities. The communities are all urban and some are low-income, although they show some variation in educational, health and public services provided. The main goal of the CFYR intervention is to reduce youth involvement in crime and violence. CFYR’s Family Matters program is a secondary prevention intervention consisting of counseling sessions for youth aged 10-17 and their families. Family counselors who have been trained on the PIFSM engage with the families to identify the problems behind the youth’s potential anti-social behavior and work on ways to improve the family environment, parental involvement, and ultimately the youth’s behavior. The population of youth in each community was identified through schools and community outreach, with enumerators knocking on doors. All youth who had the proper consent were then interviewed using the Youth Services Eligibility Tool (YSET), which was first designed by researchers at the University of Southern California (USC) to identify youth who were at the highest risk for violence associated with gangs in Los Angeles and which has been adapted to the Caribbean context.3 The intervention targeted youth considered to be at a higher risk level. 2 Census and other national data for three countries was analyzed as background for this evaluation. 3 The YSET tool has been validated as a gang risk assessment tool. See Hennigan, Karen M., et al. "Identifying high-risk youth for secondary gang prevention." Journal of Crime and Justice 37.1 (2014): 104-128; Hennigan, Karen M., et al. "Targeting youth at risk for gang involvement: Validation of a gang risk assessment to support individualized secondary prevention." Children and youth services review 56 (2015): 86-96. 2 The opportunities afforded by these countries to understand engagement with criminal activity and youth violence are unique. While we know much about how these processes unfold in cities and neighborhoods in the United States (and increasingly more knowledge is being generated in Central America), the opportunity to study prevention strategies in these small, tightly knit communities may shed light on how to create successful programs that can impact local social dynamics in many other developing countries, including the rest of the Caribbean. In the three CFYR countries, at-risk youth will be offered counseling services in very supportive social environments before serious escalations of violence or gang activity have taken place. In that sense, this study can shed light on youth services that can be effective when early warning signs first appear, before organized crime and youth recruitment to gangs become so daunting that it may be too late to intervene. Two of the countries chosen for the intervention are very small, rendering it unfeasible (financially and in terms of the statistical power of a quantitative approach) to randomize treatment across communities. However, the evaluation team designed a multi-site, randomized controlled trial (RCT), whereby youth were randomized into control or treatment groups within 15 pre-selected, high risk communities.4 As such, the impact evaluation allows programs to be encompassing and comprehensive in the sense that they cover 15 high risk communities, rather than a random selection of treated and untreated communities within hotspot territorial areas. This baseline report begins with a brief literature review on the state of youth violence and violence prevention policies and programs. Next, we cover the evaluation purpose and evaluation questions, background on the CFYR project, evaluation design, methods and limitations, and, finally, the data analysis findings. The results include background information on the communities studied from a community survey, findings on youth behavior and beliefs from YSET analysis, and information regarding the home environment and parenting style of caregivers from the caregiver survey. It is important to note that this is only a baseline report and we are not yet able to report on the impact of Family Matters program. 4 In Guyana, the communities selected for the intervention were not necessarily the ones at highest risk, as CFYR excluded from its selection process several high risk communities with programming implemented by the Inter-American Development Bank (IDB). In St. Lucia, due to small population sizes and limited disaggregated data to inform community selection, some of the “communities” include areas from different parishes/counties. 3 LITERATURE REVIEW: REDUCING YOUTH RISK LEVELS FOR ENGAGING IN CRIME AND VIOLENCE THE STATE OF YOUTH VIOLENCE IN THE CARIBBEAN Citizen security has become a greater concern for citizens of the Caribbean, where the level of violent crime is particularly high and on the rise. Violent crime, including homicides, assaults and threats, appears to be more common in the Caribbean than in all other regions covered by the International Crime Victimization Survey database. Nearly one in three Caribbean residents reported witnessing a violent act in which another person got injured or died. Youth (18-24) and young adults (25-30) are disproportionally likely to be victims or perpetrators of violent crime, in consideration of their share of the population overall (Sutton and Ruprah, 2017). A 2013 Caribbean youth assessment prepared by SI noted among respondents a perception of heightened crime and violence involving youth in the region. Thefts and house break-ins, which are considered to be the predominant youth crimes, were on the rise and becoming more sophisticated. Key informants in St. Vincent and the Grenadines, Dominica and St. Lucia reported that stolen articles are handed off to collaborators on boats and then transported and sold in other islands. This level of organization within local and inter-island networks also suggests that a gang problem is on the rise (Social Impact, 2013). This perception of high levels of youth involvement in crime and violence in the Caribbean appears to be based on fact. A 2016 report titled “Prevalence and Patterns of Troublesome Youth Groups in the Caribbean” found that youth themselves report high levels of involvement in these activities. Two-thirds of surveyed students reported having committed criminal and/or violent offenses in the previous year, and this number was even higher for students who acknowledged being gang members (Katz and Nuño, 2016). This report suggests that gang involvement starts at a very young age, as early as nine years old. THE CAUSES OF YOUTH VIOLENCE IN THE REGION Set against a backdrop of drug trafficking, organized crime, weak judicial and law enforcement systems, and a lack of opportunities and support for youth who live in deprived communities, Laura Chioda (2017), Social Impact (2013), and Katz and Nuño (2017) identify several factors that drive youth participation in crime and violence in the Latin American and Caribbean (LAC) region. Laura Chioda (2017) finds that LAC youth are at a higher risk of committing and falling victim to violence, which bears important consequences for their life trajectories and for society as a whole. In her study using a panel of LAC countries, Chioda (2017) also finds that although the general level of employment was not related to crime levels, youth unemployment was.5 5 Micro-level data for Mexico and Brazil also revealed that it was not just the employment status that matters for the relationship between labor markets and criminal offending, but also the quality of employment. 4 For the Caribbean region, in particular, youth development workers interviewed for the 2013 Social Impact youth assessment reported that besides economic pressures, “deteriorating social support systems, lack of positive male role models, and the growing number of unattached males in an age group that tends to be less risk-averse combine to create opportunities for disorder and crime” (Social Impact, 2013, p. vii). One reason cited for increased gang membership in Guyana was the high number of parents who live elsewhere due to better economic opportunities, thus leaving their children with a single parent/caretaker or even with an older sibling. As household or family ties fail, youth seek to form other ties, including by engaging in promiscuous behaviors and joining gangs. Also, in Guyana, almost half of interview respondents characterized crime as incidents related to getting “fast cash” to fulfill basic needs and pursued primarily by young women in the form of transactional sex. In terms of organized crime, a third of interview respondents said that youth are often used by adult perpetrators as front-line workers in drug trafficking and prostitution rings (Social Impact, 2013). By interviewing youth in the study countries, Katz and Nuño (2017) were able to identify the risk factors and reasons for joining what they term troublesome youth groups (TYGs). Among the risk factors found to be significant, they list low self-control (all countries), low commitment to school (St. Lucia), negative peer commitment (St. Lucia and Guyana), and lack of parental supervision (Guyana). The most commonly cited reasons for joining TYGs were to make friends and because someone in the family was a group member. In Guyana, another reason was to participate in group activities. One stakeholder explained it this way: “Generally speaking . . . persons do not ‘join’ a gang, per se. It is simply a continuation of the community socialization process. Gangs are found in communities where many of their members reside; therefore, they are known to most persons in the communities and [have] socialized with them at some point. Being part of the group simply requires one to ‘lime’ with the gang in their customary location” (Katz and Nuño, pp. 12-3). WHAT WORKS IN PREVENTING YOUTH VIOLENCE According to a 2015 Inter-American Development Bank (IDB) study entitled Closing Knowledge Gaps, youth violence prevention strategies can be classified by their social ecology: “treatment-specific (i.e. cognitive behavioral therapy (CBT), counseling and social skills training), family-based (i.e. behavioral parent training and home visitation), and school- or community-based (i.e. mentoring and after-school programs, as well as social capital building outside of the family)” (Jaitman and Guerrero Compeán, 2015, pp. 16-7). The report identifies the most successful youth violence prevention programming to date, most of which were implemented in the United States, by reviewing various studies and evaluations. Cognitive-behavioral interventions, such as the Big Brother/Big Sister program, which is a long-running mentoring match between youth at risk and adults, were the most successful at reducing anti-social behavior over time, more than double the measured effect of counseling therapy (Lösen and Beelmann, 2003). CBT and family therapeutic interventions are usually also identified as successful at preventing violence. However, one study that reviewed eight randomized controlled trials in the United States found that multisystemic therapy, a type of intensive family and community-based treatment, had no statistical effect on arrests and convictions (Littell, Popa, and Forsythe, 2005), suggesting that some of the optimism regarding these programs should be tempered. A more recent study that did a randomized controlled trial of a Functional Family Therapy (FFT) program in Philadelphia found that the effectiveness of such a program on recidivism varied by levels of risk of gang membership. Among youth that were at a high risk 5 for joining a gang, the study found significantly lower recidivism rates, compared to the control group that was receiving the “treatment as usual.” For youth that were at a lower risk for gang membership, the study did not identify consistent differences between the treatment and control groups (Thornberry et al, 2018). In a very thorough review of the literature on violence in Latin America and policies targeting children and youth, Chioda (2017) found that adolescence and young adulthood are critical ages for policy intervention “because of loosening of parental control, the increased pull of peers, the transition from youth to adult roles, and the development of three different regions of the brain, which are responsible for the regulation of automatic or instinctive reactions, risk-taking behavior, self-control, and reflective reasoning” (p. 34). The most effective behavioral interventions identified by Chioda’s review are CBT, life skills training and family-based interventions.6 School-based interventions can be successful, particularly when they are designed to alter behavior, thinking patterns, attitudes and beliefs. Those that discourage students from dropping out during their final years of secondary education yield benefits even decades later, in the form of reducing participation in violent and property crime (Chioda, 2017). One such school-based intervention is the LifeSkills Training (LST) program which was implemented in schools in Jamaica and which received the first evaluation for a program of this kind in a developing country context. LST is a 3-year intervention for middle and junior high school students, targeting violence as well as tobacco, alcohol and marijuana use. Youth receive daily supervision at school from 8am until 2pm, receiving instruction, socialization and “training in social skills, self-management, and social resistance skills to address a number of cognitive, psychological, attitudinal, and social factors related to risky behaviors (including violence)” (Guerra 20137 as cited in Chioda, 2017, pp. 239-40). LST was extensively evaluated and has shown positive results in the short and long-term, acquiring the “effective” program label by the Blueprints Initiative. Although participants in the treatment group did not differ from their counterparts in the control groups in terms of “triggers” of aggression, as measured by a propensity scale, they showed better self-control by being less likely to act on those triggers (Chioda, 2017). Policies and programs that target the early years of life have also shown to be effective in preventing violence later in life.8 Finally, family-based interventions constitute a type of violence prevention programming that has also resulted in some successes. Developmental research findings suggest that the family environment includes key risk factors for involvement in crime and violence, namely household structure (single-parent households, siblings as caregivers, etc.), parent’s age at first birth, and the stability of the home 6 Cognitive behavioral therapy (CBT) is a short-duration intervention that helps participants recognize and mitigate unhelpful thoughts and behaviors, or in a way, gets them to be “thinking about thinking” (Chioda, 2017). Participants learn to recognize situations where automatic, intuitive decision-making kicks in and results in negative outcomes, as a means of prevention. The CBT-based becoming a Man program for minority male youth in Chicago received considerable attention from the Obama administration and the press after an impact evaluation showed very positive results. Participation in the program raised an index of school engagement and performance by 0.14 and 0.19 standard deviations in the program and follow-up years, with a resulting 10-23 percent increase in the graduation rate relative to the control group. Program participants also experienced a 44 percent reduction in violent crime arrests and a 36 percent decline in arrests related to vandalism and weapons crimes during the program (Chioda, 2017). CBT has been successfully implemented in other cities in the United States. 7 The original Guerra 2013 study could not be located. 8 Among the US programs, the most well-known are probably the perinatal home visitation programs, such as the Nurse-Family Partnership, and the HighScope Perry Preschool Program for children living in poverty in Chicago. Program that target the early years help build human capital by investing in the health, and cognitive and psychosocial skills needed for adequate development, which determines a person’s life outcomes for many years to come (Chioda, 2017). 6 environment (Chioda, 2017). Family history and criminal background, particularly of the parents, are among the strongest predictors of involvement with criminality, even stronger than income or employment status (Chioda, 2017). Gang membership has also been shown to be a significant risk factor for offending and involvement in delinquency (see discussion in Thornberry et al, pp. 955-6). Improving parent-adolescent communication can lead to better mental health, better social functioning and more positive behavioral outcomes (UNICEF, 2017). The most common types of interventions that seek to help families achieve better functioning include functional family therapy, multisystemic therapy, and brief strategic family therapy. Through therapy, these interventions seek to replace dysfunctional family behaviors with healthier ones to improve antisocial behavior in youth. Participating families will have one or more youth who are struggling with serious antisocial behavior or are even already in trouble with the law and will work with a trained therapist over the course of three months or longer. Each of these three types of family-based therapies have been evaluated multiple times and were found to reduce incidences of anti-social behavior, arrests and recidivism (Chioda, 2017). However, Chioda’s review does not include the Littell, Popa and Forsythe (2005) study mentioned earlier, which found no statistically significant effect of multisystemic therapy on arrests and convictions. Some programs include more than one strategy and component. The Gang Reduction and Youth Development (GRYD) program in Los Angeles includes six interrelated approaches, which were: 1) primary prevention (for instance, a Gun Buy-Back program and community forums where GYRD staff present information on gang risk factors), 2) secondary prevention (targeting youth ages 10-15 who are assessed as high risk), 3) intervention – family case management (youth ages 14-25 who are engaged in gang activity), 4) intervention – incident response (immediate response to gang-related violent incidents, to control rumors and mitigate tensions that might lead to further issues such as retaliatory violence), 5) community engagement (engage the community and police in a community policing capacity, and incorporating community engagement into all of GYRD’s work), 6) suppression (the GYRD office continually communicates with law enforcement). Therefore, this program includes family-based programming but also goes beyond that to include five other pillars seen as essential to violence prevention (Cahill et al, 2015). An evaluation of this program found that youth clients and parents improved family functioning and problem￾solving over the course of the study. The process evaluation clearly showed very important shifts in behavior and perception. However, when assessing changes in outcomes that could be attributed to the intervention, the study had mixed findings. While there were large reductions in the incidence of crime and violence and a shrinking of crime hot spots, these general trends were not limited to the treatment communities. The evaluation found no statistically significant difference suggesting that treatment communities did better than the control group (Cahill et al, 2015). Hence, although the evaluation generally showed encouraging findings in terms of reported improvements by the participant youth and the community, it is not altogether clear that crime levels changed as a consequence of the program. The Proponte Más program in Honduras, which was also designed by Creative Associates, was based on the family systems secondary prevention model implemented in Los Angeles. By strengthening family relationships and youth support and supervision, the program sought to build resistance to gang recruitment. Individual and group activities were carried out through seven phases to promote change in youth high-risk behaviors inside and outside the family environment. The group activities were designed 7 to observe and intervene in high-risk behaviors by the youth in an environment outside of the family. Although the Proponte Más program is similar to Family Matters in terms of the family counseling provided to participants, there are important differences. For instance, Proponte Más worked with referrals instead of blanketing the communities and visiting schools to implement YSETs, as was done for Family Matters. In addition, with Proponte Más family counselors applied the YSETs instead of enumerators. A total of 300 youth were referred to Proponte Más, and 108 were found to be at high risk and eligible for treatment using the risk assessment tool. Out of these 108, 81 went into the treatment group receiving the intervention, and 27 in the control group. An evaluation showed a reduction in the percentage of youth scoring at-risk for different risk factors, before and after the intervention (at 6 months and 1 year). The risk factors that had the greatest reduction were: inadequate supervision, critical life events, the tendency to take more impulsive risks, and neutralization of guilt (Creative Associates International, 2015). Despite these positive findings, the evaluation was not an experimental study with a control group and estimation of the counterfactual. While many interventions have been studied in isolation, a multi-year, multi-method and multi-country evaluation measured the effect of USAID’s efforts in combatting crime and violence in Central America. The list of activities and programs evaluated included planning by municipal-level committees, crime observatories and data collection, better environmental design (such as street lighting and cleaned up public spaces), programs for at-risk youth, and community policing. Both quantitative and qualitative data pointed to a significant improvement in treatment communities with the program. By comparing treatment and control communities, the most significant effects identified were a difference in the reported cases of murder (51 percent), extortion (51 percent), people avoiding walking through dangerous areas (35 percent), sales of illegal drugs (25 percent), robberies (19 percent), community organization to prevent crime (18 percent), and report of young people in gangs (14 percent). These incidents were measured as reported to interviewers, not to the police (Berk-Seligson et al., 2014). Another comprehensive study, a meta-review that examined thirty different strategies, also provided important insights and lessons on what works in reducing violence (Abt and Winship, 2016). The authors identified elements of effectiveness, which seemed to be common among successful interventions: focusing on the people, places and behaviors that are most at risk for violence, being proactive in preventing violence, working together with partners and creating a legitimate loop between formal (police) and informal social control (communities), good capacity in terms of resources and good implementation, and a clear theory of change (Abt and Winship, 2016). Of the 30 different strategies examined in their review, two stood out as particularly successful: focused deterrence and CBT.9 9 Focused deterrence is also known as the “Pulling Levers” strategy, and consists of: 1) selection of a crime problem, usually youth or gun homicide; 2) identify and analyze they key group of offenders and their behaviors; 3) assemble a multi-sector task force that includes the police, service providers and community representatives; 4) target these groups for special enforcement operations, aiming to alter their behavior by any legal means necessary; 5) supplement the law enforcement work with legitimate offers of assistance to these groups, and engage the “moral voice of the community:; 6) communicating clearly, often, and face￾to-face with these groups to let them that they are under scrutiny, and their actions will have either positive or negative consequences, individually and for the whole group. Therefore, this strategy reaches out to offending groups directly, tracks and communicates with them clearly and often, and by “pulling every lever” with credible threats of enforcement and assistance, it seeks to alter their behavior. This intervention also has a clear focus, for instance on reducing the number of shootings, rather than seeking to stop all crime or convince members to leave their gangs. In Chicago, participants were 23% less likely to be 8 USAID CFYR PROJECT BACKGROUND AND THEORY OF CHANGE In response to crime and violence in the Eastern and Southern Caribbean (ESC), in 2016 USAID launched the Youth Empowerment Services (YES) project, which includes the CFYR initiative. Implemented by Creative Associates, CFYR employs a public health informed, place-based strategy in fifteen communities in Guyana, St. Lucia, and St. Kitts and Nevis. Communities were selected by CFYR through a multistep process that involved ranking communities using available quantitative crime and demographic indicators followed by a qualitative analysis entailing focus groups, interviews, and asset mapping in potential communities, and concluding with validation of results with stakeholders.10 Consistent with the public health approach, CFYR programming includes primary prevention, focused on the general youth population, secondary prevention, focused on youth at a higher risk level, and tertiary prevention, aimed at those who have already been involved in crime and violence. This impact evaluation focuses on CFYR’s secondary prevention efforts, rather than CFYR as a whole. The secondary prevention family model that CFYR is based on was first developed by the Los Angeles Mayor’s Office of GRYD between 2009 and 2011. As mentioned above, it was one component of that city’s comprehensive strategy to reduce gang activity. In 2012, the city of Los Angeles and USAID signed an agreement to share best practices in violence prevention used in Los Angeles to inform programming in Central America, Mexico and the Caribbean. The Northern Triangle of Central America began in 2013 with a pilot in five high risk communities in Tegucigalpa, Honduras. Later, USAID funded Proponte Más, also in Honduras, to implement six-month and one-year cycles of treatment with families in the secondary and tertiary levels of risk (CFYR Family Counseling Training Manual, 2017). The secondary prevention approach is grounded on family system theory and practice, including Structural, Multigenerational, Communications and Strategic approaches. The theory of change postulates that all the risk factors and associated behaviors are reinforced by relations within the family. If family member behaviors and practices associated with risk factors can be identified and modified, family dynamics and relations should improve and risk factors and associated negative behaviors should be reduced (CFYR Family Counseling Training Manual, 2017). Other factors that have been identified as being important for improving anti-social behavior include increasing youth’s resilience factors, such as a more stable home environment, the presence of positive male role models, higher self-esteem, peaceful conflict resolutions skills, and decreasing overall risk factors, such as low literacy levels, unemployment, domestic violence and child abuse. The risk assessment tool that is used by these programs is the above-mentioned YSET, and it measures the accumulation of risk factors at individual, peer and family level domains (Hennigan et al., 2013). Youth are eligible for programming if they have four or more of the nine risk factors, all of which are weighted equally. The nine risk factors included in the Caribbean YSET tool are: Weak parental monitoring, Critical life events, Impulsive risk taking, Guilt neutralization, Negative peer influence, Peer delinquency, Family involved in shootings and 32% less likely to become a gunshot victim in the year after treatment. Focused deterrence has been successfully implemented in other cities in the United States as well (Abt and Winship, 2016). 10 CFYR (March 2017). Draft CFYR Community Selection: St. Lucia. 9 antisocial influence, Self-reported delinquency, and Risky group behaviors. After the initial YSET assessment, the intervention is carried out for a year in a collaborative relationship between the family, their counselor, and a strategy team that provides on-going consultation. After six months, the youth is reassessed using the YSET Retest (YSET-R), and after another six months of treatment the youth receives the final YSET-1 year assessment. These two six-month cycles are each composed of six phases, which last one month each. During each phase, the counselor meets with the family at least twice and meets at least once individually with the eligible youth. After each family meeting, family members are given “homework” that is designed to identify emotional assets, build cohesion at the multi-generational vertical family level, and improve problem-solving capacity at the horizontal family level. The seven phases of intervention are: Referral and collaboration, Building agreements, Redefining, Celebrating changes, Integrating, Next level agreements, and Re-evaluation (CFYR Family Counseling Training Manual, 2017). 10 EVALUATION PURPOSE, QUESTION, AND METHODOLOGY EVALUATION PURPOSE The purpose of this evaluation is to measure the impact of the CFYR secondary prevention intervention on participating youth on overall risk and across nine risk factors. In addition, the evaluation team designed a caregiver survey to gather more background information on the families, which allows us to see if the intervention impacts caregiver behaviors and attitudes, providing variables that can be used as control in regressions measuring the impact of the program on youth. EVALUATION QUESTIONS This evaluation seeks to answer two core questions: 1) Do secondary prevention efforts impact the participating youth’s overall risk, crime and violence-related behavior, and their individual risk and resilience factors? Why or why not? and 2) Do secondary prevention efforts improve the family environment? Why or why not? The first question will be answered primarily with YSET data analysis, and the second primarily with information collected from the caregiver survey. EVALUATION DESIGN The evaluation team considered many options when designing an impact evaluation of the CFYR program, and, in collaboration with colleagues at CFYR and USAID, ultimately decided to compare youth eligible for secondary prevention efforts inside 15 treatment communities selected by CFYR. Given the small size of St. Lucia and St. Kitts and Nevis and other donor interventions in Guyana, it was not possible to identify an adequate number of eligible high-risk communities to randomize treatment at the community level or select comparison communities. However, it was possible to identify youth eligible for treatment and randomize those youth into treatment and comparison groups. The evaluation is, therefore, a multi-site, RCT without comparison communities but with comparison youth within treatment communities. Youth randomized into the control group are not denied treatment but will be able to obtain treatment in the second year of CFYR programming. CFYR and its partners lacked the capacity to treat all eligible youth at the same time, and this delayed treatment approach minimizes the ethical considerations to withholding treatment for some youth. THE YSET AND SELECTION OF YOUTH In the ESC, the YSET was used as an assessment tool to determine eligibility for treatment and by the evaluation team as an evaluation tool to measure changes on key outcomes over time.11 Prior to the intervention, CFYR and the National Opinion Research Center at (NORC) at the University of Chicago, working in collaboration with the evaluation team and data collection partners, conducted an extensive 11 In other contexts, Creative Associates also uses the YSET as a means by which to engage with families, begin a relationship with the family, and enter into the “family unit.” This trust building was undermined by emphasizing the evaluative aspect of the YSET and having enumerators rather than counselors administer the YSET. 11 outreach campaign in all 15 communities to reach the population of youth aged 10 to 17. This included outreach through schools both inside and outside of targeted communities and door-to-door canvassing to target youth out of school or that were previously missed for any reason.12 After all youth YSET assessments had been conducted, CFYR’s partners at USC established cut-points for eligibility on a country-by-country basis and determined which youth were eligible for treatment (see Annex I for greater detail). SI stratified households with eligible youth across communities and randomized households and their eligible youth into treatment and control groups, ensuring balance across gender, age, and risk assessment scores.13 Figure 1 shows the flow of steps in the identification youth that have been collected for both the intervention itself and the evaluation baseline (highlighted in color). CFYR undertook a detailed effort to identify the population of youth age 10-17 in the schools and identified 5,845 youth (3,162 in St. Lucia, 1,362 in Guyana, and 1,321 in St. Kitts and Nevis). Other out of school youth were identified through household canvassing and made up 6.3 percent of the final sample. While we do not know the population of youth out of school, if we assume that they make up at least 6.3 percent of the total population of youth age 10-17, then this population was at least 6,236. Assessments required informed consent by the youth and by a parent or guardian, and in total, 5,109 youth were assessed, representing as much as 82 percent of the potential population. Some additional youth may have been assessed through referrals from police, schools, or other sources. These were excluded from the evaluation as randomizing these youth into the control group might have undermined CFYR’s relationships with its referral networks. Of the 5,109 youth, 2,086 are in Guyana, 2,076 in St. Lucia and 921 in St. Kitts and Nevis (see Table 1 below.) Divided by gender, the sample is 51.5 percent female and 48.5 percent male. Youth range in age from 10 to 17, with 69 percent between the ages of 10 and 14, and 31 percent between the ages of 15 and 17. After the YSET, 1,133 (22 percent) were considered eligible for treatment and randomized into control and treatment groups. This includes 437 in Guyana, 468 in St. Lucia, and 208 in St. Kitts and Nevis (see Table 1). 12 Baseline YSET assessments occurred through various iterations. Originally, CFYR was responsible for applying the YSET in all three countries in collaboration with data collection partners, including Specialists in Sustained Youth Development and Research (SSYDR) (Guyana), the Central Statistical Office (St. Lucia), and other partners in St. Kitts and Nevis. For a variety of reasons, DRG-LER co-contractor NORC was later brought on to take over data collection oversight from CFYR in Guyana and St. Kitts and Nevis. SYDDR continued to conduct the assessments in Guyana and NORC contracted the Solomon Group (TSG) to complete the YSETs in St. Kitts and Nevis. In St. Lucia, CFYR and its partner the Central Statistical Office continued to lead the YSET process. All partners participated in the same trainings and -- with some minor, agreed to deviations across countries -- followed the same protocols. Originally, the evaluation was expected to last two years and as such CFYR followed a careful sampling approach to ensure replicability in the second year. This entailed a mixed strategy of randomly sampling youth from schools that served CFYR communities, including schools outside of community boundaries, and randomly selecting households in hot spot communities. The school sampling had the advantage of accessing numerous youth, and the household sampling had the advantage of identifying youth that were not enrolled in school. After it became clear that the number of youths were fewer than expected, the evaluation was shortened to one year, and it no longer became necessary to follow a strict random sampling approach that could be replicated the following year. As a result, the approach shifted to identifying and YSET assessing the population of youth between ages 10 and 17 in the selected communities. Data collection teams targeted the population of youth in the identified schools, door to door sampling was increased, and even referral networks were tapped into to help identify youth. We do not know the population of youth in the CFYR communities, and as such we cannot know for sure what percent of the population was reached, but we estimate that it was not greater than 82 percent. Without information on the full population, we do not know if there was sampling bias in the youth and families that were reached or agreed to participate. It would be reasonable to suggest that certain high-risk families would be more likely to not be identified or refuse to provide consent. Nonetheless, we feel confident that the vast majority of youth were identified and YSET assessed. 13 As the intervention occurs at the household level, it was necessary to randomize youth in the same household to the same treatment arm. The randomization process is described in greater detail in Annex I. 12 We should note that an error was discovered in the final report revision involving the erroneous inclusion of 26 invalid observations (not included in the figures above), which were to have been dropped for various reasons including duplicates, missing documentation, youth out of the targeted age range or targeted community. This affects only some tables and figures that contain observations that were erroneously included in the YSET analysis. The error involves 15 observations in St. Lucia, 11 in St. Kitts and Nevis, and none in Guyana. While we strive to present the most accurate data, given that this error (a) only affects two percent of the eligible youth, (b) does not impact the conclusions below, and (c) the final baseline figures will shift slightly in the endline report, we have opted not to revise all the YSET tables and figures. Figure 1: Number of youth in communities, by eligibility status, and interviewed for each survey Using the YSET, the evaluation will measure changes in overall risk and changes across all nine risk factors (Weak parental monitoring, Critical life events, Impulsive risk taking, Guilt neutralization, Negative peer influence, Peer delinquency, Family antisocial influence, Self-reported delinquency, and Risky group behaviors). The youth in both treatment and control (delayed treatment) groups are to be assessed again after six months of treatment and again after 12 months of treatment. 13 CAREGIVER SURVEY Another key source of data for the evaluation is a survey of caregivers for eligible treatment and control youth. This survey is used to (1) collect caregiver-level outcome data to answer the second evaluation question, (2) collect additional family-level data that will inform the youth analysis and help contextualize the family environment in which youth live, and (3) validate youth responses to the YSET.14 In addition, at endline this survey will (4) measure additional caregiver output indicators focused on participation in the intervention. The survey includes question batteries on the family environment, including family structure, family characteristics, and potentially negative family influences (e.g., family involvement in crime and violence), community engagement, and family attitudes and behaviors that might change as a result of the intervention, including family cohesion, family decision-making attitudes (locus of control in raising children), caregiver presence, and knowledge of youth behavior. The caregiver survey targeted those with primary caregiver responsibility for the youth. While this was most often the mother, it was at times fathers, siblings, aunts, grandparents, and guardians. As seen in Table 1, out of the 1,133 eligible youth, we interviewed caregivers for 1,002, or 88 percent (12 percent non-response). This included interviews with 946 caregivers; in households with more than one eligible youth, caregivers responded to questions about each eligible youth within that household. The remaining caregivers either refused to be interviewed or could not be located. The caregiver survey includes 404 interviews from Guyana, 434 from St. Lucia and 164 from St. Kitts and Nevis. Caregiver non-response rate was highest in St. Kitts and Nevis (21 percent). Table 1: Sample sizes in various surveys used in this report Country Community survey YSET youth Eligible youth Eligible youth with caregiver survey Percent without caregiver survey Guyana 1,602 2,086 437 404 8% St. Lucia 1,500 2,076 468 435 7% SKN 1,412 921 208 164 21% Total 4,514 5,109 1,133 1,002 12% Data collection for the caregiver survey occurred in Guyana between April and May 2018 and in St. Lucia and St. Kitts and Nevis between June and August 2018. Interviews typically took place in the household and caregivers provided informed consent. Challenges in conducting the survey included caregivers that could not be reached, creole language barriers, respondents with serious disabilities, noisy environments, and the presence of drugs and alcohol. 14 The caregiver survey was designed to validate youth responses to questions about negative behaviors (e.g., theft, fighting) that youth might be dishonest about in their YSET responses. As will be discussed in the findings section of the report, this goal was not achieved, as caregivers generally underestimated youth participation in these behaviors. 14 COMMUNITY SURVEY A final source of data that informs this report is a survey of community members conducted by CFYR. The CFYR community survey was collected between April 2017 and February 2018 with around 300 surveys conducted in each community, including 1,602 respondents from Guyana, 1,500 from St. Lucia and 1,412 from St. Kitts and Nevis (see Table 1). Community members were stratified into youth and adults and selected randomly, in order to ensure a representative sample of the community. This survey is not part of the evaluation design, per se; however, it provides important information on how the countries and the communities compare to one another. HUMAN SUBJECTS PROTECTIONS All survey participants provided informed consent to participate in the YSET and caregiver survey and youth also required the consent of a parent or guardian. Data was captured electronically on encrypted tablets and securely stored. The evaluation team conducted the analysis with de-identified datasets to limit opportunities for a breach of confidentiality, while still linking caregivers to eligible youth. ASSUMPTIONS AND RISKS A few assumptions must hold to minimize the risk of arriving at erroneous conclusions about the impact of secondary prevention efforts. These are summarized along with mitigation strategies in Table 2. 15 Table 2: Assumptions and risk mitigation strategies Assumption and risk Risk mitigation strategy Attrition will be low and control youth will participate in the follow￾up YSET interviews at the same rate as those in treatment. There is a risk that control youth will have less incentive to participate in endline data collection. It might also be harder to obtain parental consent for follow￾up interviews with control youth. This could be problematic if those that decline to participate are different than those that participate and if differences between the control and treatment group are introduced. The initial YSET outreach includes detailed contact information including name, cellphone, address, parents/guardians’ names, and their cellphone numbers to allow the evaluation team to follow-up. Second, by conducting a 6-month and 12-month follow-up, we hope that regular contact will increase contact rates and thereby reduce attrition. Third, the evaluation team will offer control youth a small in-kind incentive (e.g. cellphone minutes) to participate in endline. Fourth, we will conduct an attrition analysis to detect any bias potentially introduced by variable attrition. CFYR, NORC and data collection partners will systematically apply the YSET according to established protocols at baseline, midline and endline. The evaluation is not adequately powered to control for differences across different data collection units contracted by CFYR and NORC across the three countries. NORC, CFYR, and their data collection partners have used the same protocols and attended one another’s trainings to minimize variation across data collection teams. CFYR and its partners will respect the YSET scoring and the randomization. Failure to do so (e.g., control youth receive the intervention) will reduce the comparability between the treatment and control groups. CFYR has committed to this. The evaluation team will be able to monitor this at midterm through CFYR’s monitoring data. Social desirability bias and the testing effect of using the YSET at baseline, midline and endline can be minimized. We assume that measurement bias in key outcome variables will be minimal and that any bias will be the same across treatment and comparison youth. This is a risk to the evaluation, as youth receiving treatment might feel a stronger social desirability bias than those not receiving treatment. Social desirability bias was discussed as part of enumerator trainings. The evaluation team had hoped to use data from police/schools and from the caregiver survey to validate youth responses; however, police and school data are inconsistently available, and the baseline shows that caregivers tend to underestimate undesirable youth behavior. The program is implemented consistently across countries and locations. Another risk for the evaluation is implementation problems in one or more of the 15 sites. Implementation quality could be compromised by problems with a site, with a partner, or with individual counselors. If there CFYR trains counselors consistently across the three countries and uses a common counselor manual. CFYR has also implemented a monitoring system 16 are problems with a site or with a partner that result in the elimination of one or two sites from the study, then the minimum detectable effect size (MDES) will jump considerably (meaning less power). While minor adaptations are expected to be made to accommodate different realities across countries, we assume no substantial differences in the nature or intensity of the CFYR intervention between countries or sites. The greater the variation by site, the lower the statistical power. across the fifteen sites. The evaluation team will review monitoring data and collect qualitative data at the midpoint in the intervention. The evaluation will be adequately powered. The evaluation design assumed a greater number of youth in the fifteen communities and treatment youth than occurred in practice. The design assumed 800 treatment youth; however, for a variety of reasons only 450 are expected to receive treatment. This will reduce the study’s power and undermine the evaluation’s ability to speak confidently about heterogeneous effects across gender and age group. This number will be further eroded by attrition – both from the program and from the survey – which could reach 20 percent. As discussed above, SI, CFYR, and NORC have taken actions to reduce attrition. There is little that can be done beyond this. Spillover is minimal and can be understood through additional data analysis. Spillovers represent a potential challenge for the evaluation. Because treatment and control youth live in the same communities and might have overlapping social networks, it is possible that a treatment youth will (a) be positively affected by the program, (b) interact with control youth, and in so doing (c) have a positive influence on those control youth. This is expected to be a greater problem in smaller communities. It will be necessary to explore the potential of spillovers through alternative means. As noted above, first, we will ask control youth at end-line if they have noticed positive or negative behavior changes in their friends across a few different types of behaviors. Then we will attempt to link any positive changes to the program. In the analysis, we will then compare those who are more likely to experience spillover effects to those who are less likely to experience spillover effects. Second, we will explore the issue of spillover qualitatively in focus groups with youth receiving treatment and those not receiving treatment. 17 COMMUNITY CONTEXT: RELEVANT FINDINGS FROM THE CFYR COMMUNITY SURVEY CFYR’s community survey provides important information about the local environment, information which could not be obtained from available national-level data. This survey is particularly helpful to the evaluation because it allows us to understand differences between communities and countries on key indicators. If communities were found to differ mostly between countries, rather than within them, it would be difficult to justify an evaluation design that examines all three countries together. In order to provide the context of the communities in which the CFYR project is working, only some select data from the community survey is presented here. Relevant information about crime, policing, the rule of law, and community support is given, using summary statistics, indices and some maps. Our main conclusion is that although there are differences between countries, there is typically greater variation across communities within countries. Except for higher perceptions of crime in Guyana, greater resources for youth in St. Kitts and Nevis, and some demographic differences (e.g., larger family sizes in Guyana), practically all the metrics estimated from the baseline survey show that the three countries are quite comparable on the variables of interest. In the discussion that follows, we first begin with reported crime victimization. We then create and examine a series of indices measuring different aspects of the community context. The indices provide a single metric based on several questions in the community survey that measure characteristics of the social fabric, attitudes towards crime and law enforcement and agency within those communities. The following indices are calculated: • Perception of crime prevalence • Social capital, subdivided into two groups • Attitudes towards the law • View of the police • Resources for youth • Self-efficacy These indices were constructed using polychoric principal component analysis (PCA), a method for extracting a single indicator from a group of correlated variables. This single indicator, presented in the form of a unitless score, simplifies analysis and interpretation of the larger group of variables, while retaining a reasonable representation of the information contained in the original variables. Details on this methodology, including the principal component factor loadings (the correlation of each underlying variable to the resulting index), are given in Annex V. 18 CRIME VICTIMIZATION One of the most critical aspects of risk that must be assessed in order to understand the CFYR communities is to know how unsafe residents feel in these communities. The instrument included several questions on crime victimization. In particular, respondents were asked whether they had been a victim of any type of crime in the past 12 months or whether any other person living in the same household had been a victim in the same period. When a crime was reported, respondents were asked to specify the type of crime according to the categories. In most countries where many crimes go unreported, such victimization surveys tend to offer better measures of crime than official statistics. Nonetheless, victimization surveys suffer from recall error, and certain crimes with a social stigma, such as rape, sexual assault, and domestic violence, are frequently underreported. Figure 2 shows that the average rate of self-reported crime victimization is similar in each of the three countries. Responses are given for each community and for each country in aggregate; the percent of respondents who did not answer the question is captured on the right side of the figure in purple. Victimization ranges from six percent in St. Kitts and Nevis to nine percent in Guyana, with greater variation across communities, ranging from three percent in Sandy Point, St. Kitts and Nevis to 14 percent in Sophia, Guyana. Figure 2: Crime victimization reported by respondents in the last 12 months by country and community 19 We find a similar pattern in reported victimization of respondents’ family members in the last 12 months. As shown in Figure 3, the average rate of reported household crime victimization ranges from four percent in St. Kitts and Nevis to seven percent in Guyana, and from three percent (Corriverton) to ten percent (Sophia). In sum, victimization figures suggest relative similarities between the target countries with greater variation across communities. Figure 3: Crime victimization among other members of the household in the last 12 months by community and country It should be noted that reliability of self-reported victimization is difficult to ascertain in many contexts. For this reason, the proportion of “missing” responses are also included in the figures. This percentage averaged around one percent for self-reported crime victimization and around four percent for reports of household crime victimization; however, the proportion of respondents who declined to respond at all was substantially highest in the St George community in Nevis (16 percent) than any other community. Figure 4 shows what specific crimes were reported by victims, including 1) unarmed robbery, no assault or physical threats; 2) Unarmed robbery with assault or physical threats; 3) Armed robbery; 4) Assault but not robbery; 5) Rape or sexual assault; 6) Domestic violence; 7) Kidnapping; 8) Vandalism; 9) Burglary of your home while you were not at home; 10) Burglary of your home while you were at home; 11) 20 Extortion/blackmail; 12) Bullying; and 13) Other crimes. Only two percent of respondents reported being the victim of two crimes, and fewer than one percent reported being the victim of three or more crimes, so most observations correspond to one individual. Zero respondents stated that they had been a victim of kidnapping; therefore, this crime is not presented on the chart. Given that only a small percentage of respondents report victimization, Figure 4 provides a count of the number of incidences of each crime rather than a percentage of crimes. Given that there is some variation in the number of respondents in each community, the (broken) x-axis shows the total respondents in each community. For example, while we observe a large incidence of reported crime in Sophia, it is important to recognize that there were more respondents in this community. The most common crimes are unarmed robbery and burglary. While the graph illustrates the heterogeneity in type of crimes across communities, it should be clear that the incidence overall is quite low. Figure 4: Crime victimization: Count of reported crimes of different type by community Figure 5 provides a similar count (rather than percentage) of reported crime incidence by country. Although there is some variance across countries, the general composition of crime is quite similar. As noted above, a large part of the reported crime is comprised of robbery and burglary. Armed robbery is reported more frequently in Guyana; St. Lucia stands out for a greater share of bullying; and in St. Kitts and Nevis there is a larger share of domestic violence. There appears to be a greater number of reports of rape and domestic violence cases in St. Lucia than the other two countries, although it is not clear if this is the result of a higher incidence rate or a greater willingness to report these stigmatized crimes. 21 Figure 5: Crime victimization: Count of reported crimes of different types by country PERCEPTION OF CRIME PREVALENCE We created a Perception of Crime Prevalence Index from the questions listed below, with responses solicited on a Likert scale of Never to Always (with one exception, for which the scale was reversed). A higher score indicates a perception of higher crime prevalence and a lower score indicates a perception of lower crime prevalence. How often do the following situations occur within your community? • Vandalism or property damage • People using or selling drugs • People getting robbed • People getting sexually assaulted or raped • During the last year, do you think that crime in your community has [increased/decreased/stayed the same]? (scale reversed) • How safe do you feel walking in your community during the daytime? • How safe do you feel walking in your community after dark? Figure 6 shows each community’s and each country’s average score on this index. Individual scores ranged from 2.60 to 14.76. Error bars indicate a five percent error margin for the value of each average. In other words, assuming no bias, we are 95 percent confident that the true population average falls within the range shown by the error bands. It is important to note that the index values do not have a substantive 22 interpretation. The scores are standardized according to a normal distribution, such that the mean is the underlying central tendency of all the respondent attitudes, according to an underlying latent variable (assumed to be a normal distribution too) captured in their responses to the individual items on the categorical scales. Nonetheless, the index is still helpful to compare differences in crime perceptions across communities and countries. As can be seen in the figure, crime perceptions are somewhat higher in Guyana than the other two countries, with the worst crime perceptions in Sophia, Guyana, followed by Basseterre, St. Kitts and Nevis, followed by Castries, St. Lucia. As with crime victimization, there is more homogeneity across countries than individual communities. Figure 6: Average Perception of Crime Prevalence Index scores by community and country Interestingly, the communities with the highest perception of crime prevalence do not always correspond to those with the highest reported victimization. For example, as shown in Figure 7, Anse-La-Ray in St. Lucia has a low average victimization rate but a high average perception of crime score. In St. Kitts Nevis, the communities of St. John (Nevis) and Basseterre (St. Kitts) found themselves on opposite ends of the spectrum from one ranking to another. In Guyana, community rankings for both metrics were more closely aligned for all five communities. 23 Figure 7: Rankings: Perceptions of crime prevalence versus reported crime victimization by community OVERALL SOCIAL CAPITAL INDEX There are several questions in the survey that serve as a proxy for “social capital,” a concept with varying definitions, but frequently understood as social networks and their associated norms of trust and reciprocity.15 We divide social capital into an Overall Social Capital Index and a Youth Social Capital Index. While there are numerous approaches to measure social capital in surveys, questions usually hinge on issues of interaction, trust, and reciprocity. The first index is derived from the following questions: • How long have you lived in this community? • Would you say that people in this community are very trustworthy, somewhat trustworthy, not very trustworthy, or untrustworthy? (scale reversed) • What is the likelihood that people from your community would try to stop a fight if one broke out in front of their house, with someone being beaten? (Very Unlikely to Very Likely) A high score corresponded to high social capital, and a low score to low social capital. The resulting index showed a high level of uniformity across both countries and individual communities. Individual scores ranged from 1.95 to 9.23, but country and community averages ranged only from 5.69 to 6.62, indicating both homogeneity and a relatively high feeling of community trust and cohesion. 15 See for example Robert D. Putnam (2000) Bowling Alone: The Collapse and Revival of American Community. New York: Simon and Schuster. 24 Figure 8: Average Overall Social Capital Index scores by community and country YOUTH SOCIAL CAPITAL The Youth Social Capital index was constructed from a different set of questions and a different subset of respondents. Only responses from interviewees aged 10 to 17 years were used to create this metric, and the index compiled responses to the following set of input questions given on a Likert scale from Strongly Disagree to Strongly Agree. As with the Overall Social Capital Index, a higher score corresponds to stronger social capital: • I feel like a member of this neighborhood • I belong in this neighborhood • I have a say about what goes on in my neighborhood • People in this neighborhood are good at influencing each another • I feel connected to this neighborhood • I have a good bond with others in this neighborhood As seen in Figure 9, scores on this index were somewhat less uniform than above. It is important to note that sample sizes were smaller and therefore the margin of error for each estimate was greater, particularly in Guyana. Individual scores range from 2.39 to 16.75, and country and community scores 25 ranged from 9.77 to 13.20. We observe both similarity across communities and countries and relatively strong feelings of trust, belonging, and respect in their neighborhoods. Figure 9: Average Youth Social Capital Index scores by community and country Figure 10 compares how communities ranked in terms of the Overall Social Capital Index versus the youth index. Although each index was constructed from different data, the second index was youth-specific and can be thought of as a proxy for how well young people feel supported and nurtured in their communities. Making this assumption, a comparison of the two can draw out disparities in terms of youth perception of their communities versus the overall population, and whether they feel exceptionally supported or exceptionally isolated. Community rankings for both indices were relatively similar, with the notable disparities of Corriverton, Guyana and Soufriere, St. Lucia; both communities had high rankings on the first index, and low rankings on the second, indicating that there may be some rift in how young people in these communities feel integrated and supported. It is possible that some of the inconsistencies between the two sets of averages are explained by the large margins of error in the youth averages. 26 Figure 10: Rankings: Average Overall Social Capital Index and Youth Social Capital Index scores by community ATTITUDES TOWARD THE LAW The Attitudes Toward the Law Index was constructed from responses to the following set of questions. Responses were solicited to several statements on a Likert scale of Strongly Disagree to Strongly Agree. This scale was reversed (with one exception) so that a higher score indicates positive attitudes toward the law and a lower score indicates negative attitudes toward the law: • The law does not protect my interests (scale reversed) • If the police can break the law, then so can I (scale reversed) • I try to obey the law, even if it goes against what I think is right • I do not trust the law to protect me (scale reversed) • Most people would commit crime if they know they would not get caught (scale reversed) • In order to catch criminals, police should be able to occasionally bend the rules (scale reversed) The Attitudes Toward the Law index showed uniformity across both countries and communities. Individual scores ranged from 2.10 to 15.07, but country and community scores clustered very closely around a value of 8.91 with very little variation. In fact, when considering the error bands, the communities are barely distinguishable. This means that most of the variance in this perception comes from individual respondents, regardless of the community they live in. 27 Figure 11: Average Attitudes Towards the Law Index scores by community and country VIEW OF THE POLICE We derive the View of the Police Index from responses to the following set of input questions in the form of a Likert scale of Very Dissatisfied to Very Satisfied. A high score indicates a positive view of the police, and a low score indicates a negative view of the police: • Responsiveness of the police to incidents of crime and violence • Police patrols of the community • Approachableness of the police if you require assistance from them • People in my neighborhood are willing to call the police to help their neighbors who are in trouble (scale reversed) As seen in Figure 12, the View of the Police index shows general uniformity across countries. Individual scores ranged from 1.94 to 13.03, and community scores fall under a much narrower range from just 6.41 to 8.35. 28 Figure 12: Average View of the Police Index scores by community and country RESOURCES FOR YOUTH The Resources for Youth Index was constructed from several questions asking respondents to rate their satisfaction with youth services provided in their communities on a scale from Strongly Dissatisfied to Strongly Satisfied. A high score indicates strong satisfaction with services, and a low score indicates strong dissatisfaction with services. • Programs for youth not enrolled in school • School programs meeting the needs of students in the community • Care and support for residents injured in crime and violence by hospitals/clinics • Programs to help youth recently released from prison • Victim support services for residents hurt by crime and violence The Resources for Youth Index also showed general uniformity across both countries and communities, with a somewhat higher greater satisfaction in St. Kitts and Nevis. Individual scores ranged from 2.22 to 15.42 and community averages range between 5.00 and 7.71. This indicates a neutral or low satisfaction for resources for youth across all communities and countries. 29 Figure 13: Average Resources for Youth Index scores by community and country SELF-EFFICACY Last, we constructed a Self-Efficacy Index from the responses to a series of questions about one’s own sense of agency (or lack thereof). Interviewees were asked to respond on a scale of Strongly Disagree to Strongly Agree. A high score on the index corresponds with high self-efficacy, and a low score corresponds with low self- efficacy. • I have little control of things that happen in my life (scale reversed) • I do not have any solutions for some of the problems I am facing (scale reversed) • I cannot do much to change things in my life (scale reversed) • When I face life difficulties, I feel helpless (scale reversed) • I feel my life is determined by others and fate (scale reversed) • I believe things happening in my life are mostly determined by me • I can finish almost everything that I am determined to do This scale results in similar average scores across both communities and countries, and relatively high scores overall. The minimum individual score was 2.61, and the maximum individual score was 18.29, with 30 a country/community average of 13.64. Average scores suggest that respondents are generally able to determine the course of their lives and make changes if they desired. Figure 14: Average Self-Efficacy Index scores by community and country INDEX SUMMARY Although there are differences between countries, there is typically greater variation across communities than between countries. This is a positive finding for the evaluation design, as it suggests comparability between the three countries on key indicators. In addition, while there are differences across communities in some of the indicators, the similarities are more common (see Table 3).16 This is a promising feature of the study, in that it suggests most variation comes from the individual youth and not effects between communities. 16 We define “similar” as the majority (two or more for countries and four or more for communities) of error bars overlapping, and we define "divergent" as the majority of error bars failing to overlap. 31 Table 3: Index Summary INDEX COUNTRIES COMMUNITIES Crime Victimization (percentages) Similar Divergent Perception of Crime Prevalence Similar Divergent Overall Social Capital Similar Similar Youth Social Capital Similar Divergent Attitudes Towards the Law Similar Similar View of the Police Similar Similar Resources for Youth Similar Similar Self-Efficacy Similar Similar VISUALIZATION OF COMMUNITIES The community survey provides important contextual information about the intervention sites. In addition, it is possible to use the geocoding of the survey respondents to understand that, notwithstanding some territorial dispersion, these are relatively homogenous territorial units with characteristics that make them comparable with each other. In particular, it is possible to map some characteristics related to stability and permanence of the residents, as measured by the years they have lived in the community; the trust towards neighbors; how prevalent crime may be as measured by reported victimization; whether respondents feel unsafe when walking in their community after dark; and some basic socio-demographic patterns that may be associated with risky youth behavior or less stable family environments, specifically the presence of single mothers who are unemployed, or households characterized by employed married couples. As seen in Figure 15, in St. Lucia, the communities can be clearly visualized throughout the island, with a higher territorial concentration of households in Castries, where people live in very close contiguity, while in Vieux Fort there is a greater distance between households. The communities are very stable, with very few respondents reporting having lived in the community for three years or less. The neighbors in the communities are perceived as trustworthy, there is very little victimization and, in general, residents feel safe walking after dark. The family composition shows a high prevalence of single mothers that are unemployed, and a relatively low presence of married employed households. There is no particular territorial distinctiveness in these patterns between communities, which confirms the previous findings suggesting that the communities are somewhat similar in their demographics and feelings of insecurity and social trust. Figure 16 presents similar images for St. Kitts and Nevis. The figure shows that communities in St. Kitts and Nevis are also very stable in terms of the years of residence, but they exhibit less trust across communities, particularly in Basseterre and Sandy Point. There is very little victimization, but in contrast with St. Lucia, a lot of the residents feel unsafe when walking in their neighborhoods after dark. In St. Kitts and Nevis there are more stable family structures; only a few scattered households report having single unemployed parents, while there is greater prevalence of employed married households. In Guyana, the visualization in Figure 17 only shows Georgetown, although the patterns are similar in Corriverton. There is a marked difference in terms of residential stability, in the sense that there are many 32 households that have only been living in the community for three years or less. Sophia is somewhat different in that most respondents report living there for a longer time. There is a strong feeling of insecurity in terms of the possibility of walking in the community after dark; however, the communities are relatively similar to those in St Kitts and Nevis and St. Lucia, in terms of the actual victimization, with no particular concentration in specific places, perhaps with the exception of La Penitence. Finally, the composition of the households in terms of single unemployed versus married employed is similar to what was found in the other countries, with no particular distinctiveness for any specific community. The territorial visualizations suggest that, in general, while there are differences between communities, these do not imply distinctive dynamics of insecurity or risk for youth in particular locations, nor is there a suggestion that specific streets or blocks within the communities are particularly problematic in terms of their public safety environments. There is no particular concentration of victimization in specific territories within the communities, nor is there any particular demographic pattern that may be found within a few blocks inside a community. 33 Figure 15: St. Lucia community risk factor visualizations with captioned occurrences represented by a red dot and non-occurrences by a black dot 3 years or less living in the Community People in this community are untrustworthy Victimization last 12 months Feel unsafe walking in community after dark Single unemployed households Married employed households 34 Figure 16: St. Kitts and Nevis community risk factor visualizations with captioned occurrences represented by a red dot and non-occurrences by a black dot 3 years or less living in the Community People in this community are untrustworthy Victimization last 12 months Feel unsafe walking in community after dark Single unemployed households Married employed households 35 Figure 17: Georgetown, Guyana community risk factor visualizations with captioned occurrences represented by a red dot and non-occurrences by a black dot 3 years or less living in the Community People in this community are untrustworthy Victimization last 12 months Feel unsafe walking in community after dark Single unemployed households Married employed households 36 FINDINGS BASED ON THE YSET ASSESSMENT QUESTION 1: DO SECONDARY PREVENTION EFFORTS IMPACT OVERALL YOUTH RISK, CRIME AND VIOLENCE RELATED BEHAVIOR, AND YOUTH RISK AND RESILIENCE FACTORS? WHY OR WHY NOT? Our analysis of the YSET data included disaggregation based on country, sex and age. According to the United Nations Children’s Fund (UNICEF), results of research done on adolescents should be disaggregated for reporting, by sex, age and locality. We have followed their age divisions and separate youth into those 10-14 years of age and 15-17. The importance of age disaggregation stems from the fact that some interventions may be appropriate for younger adolescents but less so for older adolescents and young adults, or vice-versa (UNICEF, 2017). In the analysis that follows, YSET scales for the total population of youth are reported with sex and age disaggregation. Individual questions are typically reported in aggregate, and only when there are significant country differences on a certain set of questions are those questions disaggregated by country. UNDERSTANDING RISK ACROSS THE FULL YOUTH POPULATION As noted in the methodology section, the method applied by CFYR and their colleagues at USC categorizes a YSET respondent to be at risk or not at risk across nine risk scales, including weak parental monitoring, critical life events, impulsive risk taking, guilt neutralization, negative peer influence, peer delinquency, family antisocial influence, self-reported delinquency, and risky group behaviors. Youth that are determined to be at risk for four or more of the nine scales are considered to be at a higher level of risk and thus eligible for treatment. Table 4 reports the percentage of youth that were found to be at risk by country, sex, and age. The findings show that males and older youth (ages 15 to 17) are at a higher risk on every single scale, and there is country-level variation across scales. The scales with the highest numbers of youth scoring at risk are guilt neutralization and impulsive risk taking. In the following sections we examine each of these risk factors in turn. 37 Table 4: Percent of assessed youth at risk in the overall YSET and on each scale (n=5,109) Total Country GUY Country SLU Country SKN Gender Female Gender Male Age 10 to 14 Age 15 to 17 Eligible: at-risk in four or more scales 22% 21% 23% 24% 17% 27% 18% 32% Weak parental monitoring 19% 10% 29% 20% 11% 28% 16% 26% Critical life events 24% 25% 20% 29% 22% 26% 20% 33% Impulsive risk taking 35% 36% 34% 34% 33% 36% 32% 40% Guilt neutralization 40% 45% 41% 24% 36% 43% 35% 50% Negative peer influence 18% 11% 19% 32% 16% 20% 16% 23% Peer delinquency 16% 18% 16% 13% 14% 19% 13% 24% Family antisocial influence 19% 17% 19% 20% 18% 19% 16% 25% Self-reported delinquency 23% 20% 24% 26% 18% 28% 19% 31% Risky group behaviors 12% 13% 11% 13% 8% 16% 11% 15% It is important to note that a higher percentage of at-risk youth in a given country is not necessarily an indication of higher risk levels in that country, as the thresholds vary by country. For instance, in the weak parental monitoring scale, CFYR and USC determined that youth would be classified as at risk if they scored an average of at least 2.5 in Guyana, 2 in St. Lucia and 2.3 in St. Kitts and Nevis. Therefore, a higher share of youth scoring at risk in St. Lucia in weak parental monitoring does not necessarily mean that a higher share of youth in that country selected “risky” answers. Table 5 presents the results of a logistic regression testing the impact of country, sex, age group, school status, and ethnicity on risk levels. This analysis allows us to see if there are specific factors that make it more likely for certain groups to score at risk, meaning that they are more vulnerable. The table presents odds ratios, which are the difference in likelihood of being eligible or scoring at risk on each of the scales, by group. For instance, the odds ratio of 4.106*** on the independent variable St. Lucia and the dependent variable weak parental monitoring can be interpreted as the odds that a youth in St. Lucia will be considered at risk for weak parental monitoring are 4.1 times greater than the odds that a youth in Guyana will be considered at risk. In this case, Guyana is not included in the equation and is considered the base country by which St. Lucia and St. Kitts and Nevis are compared. The three stars indicate that this difference is statistically significant at the 99 percent confidence level. The results in this table that are most striking are: 1) youth in the islands are much more likely to score at risk in the weak parental monitoring and negative peer influence scales compared to Guyana. Because of the different risk thresholds set for each country, higher average scores in the islands could stem from the lower risk thresholds rather than from real country differences. We tested for this and found that when setting the same threshold for the three countries, the differences in average scores are still statistically significant but the magnitude is smaller. 2) In other scales, such as impulsive risk taking, guilt 38 neutralization and peer delinquency, Guyanese youth are more likely to be at risk. When the thresholds were set to be the same for the impulsive risk taking and guilt neutralization scales, the differences remained similar between Guyana and St. Lucia, but were reversed for Guyana and St. Kitts and Nevis (with the same thresholds, St. Kitts and Nevis youth are now at higher risk than Guyanese youth in three out of four cases). For the peer delinquency scale, the threshold is the same for all three countries so different average scores reflect real differences in risk levels. 3) Males are more likely than females to be at risk on all scales, except for family antisocial influence, where the difference is not statistically significant. 4) Older youth are more likely to score at risk on all scales. 5) Youth who are out of school are more likely to be at risk on most scales. 6) While Amerindians are not statistically significantly more at risk than Black/Afro-Guyanese youth, respondents who are East Indian/Indo-Guyanese are much less likely to be at risk on all scales. For instance, East Indian/Indo-Guyanese youth are half as likely to have had a critical life event occur in the last year compared to Black/Afro-Guyanese youth. Table 5: Odds ratios with Guyana, females, in school, ages 10 to 14, and ethnicity-Black/Afro￾Guyanese as the base (n=5,101) SLU SKN Male Ages 15 to 17 Out of school Ethnicity - Amerindian Ethnicity - East Indian/Indo Guyanese Eligibility 1.006 1.048 1.804*** 2.037*** 1.828*** 0.817 0.503*** Weak parental monitoring 4.106*** 2.551*** 3.370*** 1.773*** 2.104*** 1.589* 0.652** Critical life events 0.620*** 1.034 1.331*** 1.824*** 1.655*** 0.678* 0.500*** Impulsive risk taking 0.834** 0.832** 1.126** 1.334*** 1.521*** 0.985 0.655*** Guilt neutralization 0.717*** 0.327*** 1.300*** 1.781*** 1.677*** 0.774 0.597*** Negative peer influence 1.771*** 3.565*** 1.296*** 1.618*** 1.279 1.099 0.720** Peer delinquency 0.777*** 0.611*** 1.491*** 2.066*** 1.220 0.798 0.577*** Family antisocial influence 0.981 1.037 1.092 1.665*** 1.519*** 0.899 0.523*** Self-reported delinquency 1.192** 1.356*** 1.701*** 1.768*** 1.976*** 0.850 0.561*** Risky group behaviors 0.685*** 0.858 2.160*** 1.249** 1.548*** 0.730 0.518*** Note: significance is * p<.1, ** p<.05, ***p<.01. When St. Lucia is the base country (reference group) instead of Guyana, we can see that St. Lucian youth are more likely to score at risk on a few scales (weak parental monitoring, guilt neutralization, and peer delinquency). St. Kitts and Nevis youth are more likely to be at risk on a few others (critical life events, negative peer influence, and group risky behaviors). Given the importance of age and sex, Table 6 explores differences in risk in age and sex for each of the three countries using differences in predicted probabilities rather than odds ratios. In this table, we look at the differences in risk/eligibility considering the differences in location (country), gender, age group, 39 school status and ethnicity that exist in the samples. The first column suggests that all else being equal, males are ten percentage points more at overall risk than females. This difference is consistent in Guyana, St. Lucia, and St. Kitts and Nevis. In terms of individual risk factors, the impact of sex is generally consistent across countries once all other factors are considered, with the strong exception of parental monitoring. For this risk factor, males are 17 percentage points more at risk than females across all countries; however, there is only a ten-percentage point difference in Guyana and a 23-percentage point difference in St. Lucia. Predicted differences in risk across age groups are also fairly consistent across the three countries. The predictive impact of age matters somewhat less for negative peer influence in Guyana (ten percentage points) than it does in St. Lucia (11 percentage points). As with the community survey, these generally consistent findings are encouraging for the evaluation, as they suggest that -- at least in terms of sex and age -- risk dynamics are similar across the three countries. Table 6: Differences in predicted probabilities for each group of respondents scoring at risk/eligible (n=5,101) Male - female Male - female in GUY Male - female in SKN Male - female in SLU Ages 15 to 17 - ages 10 to 14 overall Ages 15 to 17 - ages 10 to 14 in GUY Ages 15 to 17 - ages 10 to 14 in SKN Ages 15 to 17 - ages 10 to 14 in SLU Eligibility 0.10 0.10 0.10 0.10 0.13 0.13 0.13 0.13 Weak parental monitoring 0.17 0.10 0.18 0.23 0.08 0.05 0.09 0.11 Critical life events 0.05 0.06 0.06 0.04 0.11 0.12 0.12 0.10 Impulsive risk taking 0.03 0.03 0.03 0.03 0.07 0.07 0.07 0.07 Guilt neutralization 0.06 0.06 0.05 0.06 0.14 0.14 0.11 0.14 Negative peer influence 0.04 0.03 0.06 0.04 0.07 0.05 0.11 0.08 Peer delinquency 0.05 0.06 0.04 0.05 0.10 0.12 0.09 0.10 Family antisocial influence 0.01 0.01 0.01 0.01 0.08 0.08 0.08 0.08 Self-reported delinquency 0.09 0.08 0.10 0.09 0.10 0.10 0.11 0.10 Risky group behaviors 0.08 0.09 0.08 0.07 0.02 0.03 0.03 0.02 Figure 18 compares standardized scores of eligible and ineligible youth across all nine risk factors.17 Eligible youth have higher average scores on every scale, and in the case of three scales, in particular, they have 17 Because the scales had different numbers of questions and maximum scores, we standardize the scales to allow for comparison in one chart. For example, in the scale Critical Life Events, eligible youth had a mean of four events, compared to a mean of two events for ineligible youth. Because the number of questions in that scale is eight, with a maximum total of eight events, these means translate in the chart to a 0.5 (or 50% of 8) for eligible youth and 0.25 (or 25% of 8) for ineligible youth. 40 much higher average scores. In the self-reported delinquency scale, eligible youth report that they have done certain delinquent behaviors in the last six months with much higher frequency than ineligible youth. In the scale critical life events, eligible youth are much more likely to have had important and serious (and in some cases, traumatic) events happened to them in the last year. And in the scale risky group behaviors, eligible youth report with much higher frequency that their peers engage in negative behaviors. Figure 18: Average standardized YSET scores from 0 to 1 on each scale for eligible and ineligible youth (n=5,109) SCALE B: WEAK PARENTAL MONITORING In the following sections we examine each scale in detail. Each scale section begins with a table that shows the percentage of youth that scored “at risk” for that scale, including all eligible and ineligible youth (this is the same information presented in Table 4). In addition to looking at the scales as a whole, we also report results for many of the individual questions. As noted in the chart titles, when examining specific questions, we use only data on eligible youth, because that is the population of interest for the evaluation. When there are interesting or striking results by disaggregating country or gender, these are included. The remaining charts containing gender disaggregation can be found in Annex VI. The first scale includes six questions on the level of parental monitoring and awareness of youth’s whereabouts and who they are with (see Annex II for the full questionnaire). For each of the six questions (e.g., how often they return home on time), the youth respondents were asked to select from one of five options: always=1, often=2, half the time=3, rarely=4 or never=5. Responses were then averaged across the six questions. Youth that score an average of at least 2.5 in Guyana, 2 in St. Lucia and 2.3 in St. Kitts and Nevis were classified as at risk in this scale. Table 7 repeats the percentage of youth that were found 41 to be at risk for weak parental monitoring across countries, gender and age group, with 19 percent of respondents overall scoring at risk and considerable variation across the different disaggregated results. Females and younger youth report higher levels of parental monitoring of their whereabouts and company. Table 7: Percent of assessed youth at risk for weak parental monitoring (n=5,109) Total GUY SLU SKN Female Male Ages 10 to 14 Ages 15 to 17 Weak parental monitoring 19% 10% 29% 20% 11% 28% 16% 26% When we limit the sample to just eligible youth, we find that a minority of these respondents report that they rarely or never tell their caregivers where they are going (18 percent), nor return home at the time the caregivers asked (20 percent), and that caregivers do not know where they are when outside of school (13 percent) (see Figure 19). Conversely, over 60 percent of eligible youth report that they always or often tell their parents or guardians where they are going and that their parents always or often know where they are. Figure 19: Responses to select questions on parental monitoring among eligible youth (n=1,138) Similarly, more than half of eligible youth report that their caregivers often or always know who they are with (56 percent) and they tell their caregivers where they are going even if not asked (60 percent). The vast majority of these respondents said that their caregivers care about where they go and what they do. 42 Figure 20: Responses to select questions on parental monitoring among eligible youth (n=1,138) SCALE C: CRITICAL LIFE EVENTS Scale C includes eight questions about difficult things that happen in young people’s lives and asks whether any of these happened to them in the last year. A count of at least four events in Guyana and St. Lucia, and three in St. Kitts and Nevis classified youth as at risk in this scale. About a quarter of respondents scored at risk, with a slightly higher percentage (29 percent) of youth in St. Kitts and Nevis. Table 8: Percent of assessed youth at risk for critical life events (n=5,109) Total GUY SLU SKN Female Male Ages 10 to 14 Ages 15 to 17 Critical life events 24% 25% 20% 29% 22% 26% 20% 33% Among eligible youth, the least often reported critical life events regard school, namely failing or dropping out of a class or failing to go on to the next grade (25 percent) or getting suspended/expelled/thrown out/transferred to another school (25 percent). The most common critical event in the lives of eligible youth respondents in the last year was hearing about deaths or injuries caused by violence in their neighborhoods, which occurred with 71 percent of respondents. 67 percent of respondents also said that they had gotten into a fight or had a problem with a friend. These important and serious events can have significant and negative impacts on youth’s wellbeing and behaviors. Two of the events included, namely going on a date for the first time or starting to hang out with a new group of friends, are not as clearly negative, and may in fact be positive events. The questionnaire does not dig deeper into these so we cannot confirm whether or not all of these had negative impacts. 43 Figure 21: Responses to questions on critical life events in the last year among eligible youth (n=1,139) SCALE DE: IMPULSIVE RISK TAKING The impulsive risk-taking scale asks about respondents’ level of agreement with four statements about doing dangerous actions or things that might get them in trouble. Responses include strongly agree=5, agree=4, neither agree nor disagree=3, disagree=2 or strongly disagree=1. Youth that had an average score of at least 3 in Guyana and St. Lucia and 3.3 in St. Kitts and Nevis are considered to be at risk. A total of 35 percent of respondents scored at risk in this scale; one of the most common risk factors. There was not as much variation across countries and gender as in other scales. Table 9: Percent of assessed youth at risk for impulsive risk taking (n=5,109) Total GUY SLU SKN Female Male Ages 10 to 14 Ages 15 to 17 Impulsive risk taking 35% 36% 34% 34% 33% 36% 32% 40% When we limit the sample to just eligible youth, we find that about 60 percent of these respondents agreed or strongly agreed with the statement that they like to do something dangerous for the fun of it 44 and to do things without thinking about whether they will get in trouble. Slightly under half (48 percent) agreed or strongly agreed that they find it exciting to do things that might get them into trouble, and 71 percent agreed or disagreed that they like to have fun when they can, even if they will get in trouble for it later. This reflects a higher level of agreement than other items in the YSET. Figure 22: Responses to questions on impulsive risk taking among eligible youth (n=1,134) SCALE F: GUILT NEUTRALIZATION This scale lists six statements that attempt to measure rationalization (or guilt neutralization) of lying, stealing, and beating others. Youth are asked if they strongly agree=5, agree=4, neither agree nor disagree=3, disagree=2 or strongly disagree=1 with these behaviors. Youth are considered at risk if they averaged 2.5 or above in Guyana and St. Lucia and 3 in St. Kitts and Nevis. This scale had the highest number of respondents scoring at risk, at 40 percent. Respondents in St. Kitts and Nevis were much less likely to be scored at risk; however, as demonstrated in Annex III where we set the risk thresholds to be the same for all three countries, this difference is due to the higher threshold – when the threshold is set to be the same, St. Kitts and Nevis youth’s responses are at a higher risk level in three out of the four cases tested. 45 Table 10: Percent of assessed youth at risk for guilt neutralization (n=5,109) Total GUY SLU SKN Female Male Ages 10 to 14 Ages 15 to 17 Guilt neutralization 40% 45% 41% 24% 36% 43% 35% 50% Turning to the eligible youth, we see that half of these respondents agree or strongly agree with the statement that it is okay to lie to keep themselves or their friends from getting in trouble (Figure 23). Similarly, a majority of eligible youth are also able to rationalize causing harm through fighting, as 70 percent believe it is okay to beat people up if they were hit first or were standing up for themselves. By contrast, these respondents are less willing to rationalize stealing; only eight percent agree or strongly agree that is okay to steal something from somebody who is rich (Figure 23). Figure 23: Responses on select questions on guilt neutralization among eligible youth (n=1,137) 46 Figure 24: Responses on select questions on guilt neutralization among eligible youth (n=1,135) SCALE G: NEGATIVE PEER INFLUENCE Scale G has four questions asking respondents how likely they are to go along with their friends in certain situations. Responses include yes, definitely=5, probably=4, not sure=3, probably not=2, no, definitely not=1. The threshold for risk in this scale is an average score of at least 2.3 in Guyana and St. Lucia and 2 in St. Kitts and Nevis. A total of 18 percent of respondents scored at risk in this scale, with a higher share of youth in St Kitts and Nevis at 32 percent. Table 11: Percent of assessed youth at risk for negative peer influence (n=5,109) Total GUY SLU SKN Female Male Ages 10 to 14 Ages 15 to 17 Negative peer influence 18% 11% 19% 32% 16% 20% 16% 23% Figure 25 provides responses to the four questions in this scale for eligible youth. About 60 percent of these higher-risk youth report that they would definitely or probably not go along with their friends if they were doing something wrong or bad. Responses are similar when eligible youth are asked about doing something that would get them in trouble at home or in school. A much larger percentage (85 percent) of eligible youth say that they would definitely not or probably not go along with their friends if they were getting in trouble with the police. 47 Figure 25: Responses on negative peer influence among eligible youth (n=1,133) SCALE H: PEER DELINQUENCY The peer delinquency scale lists a series of six actions and asks respondents to count how many of their friends did each action during the last six months. The module also asks how many friends their caregivers would consider to be “bad company.” Respondents selected from the options all=5, most=4, half=3, a few=2 or none=1. An average score of 1.9 or higher in any of the three countries classifies the respondent as at risk, and a total of 16 percent of respondents scored at risk in this scale. The largest difference in the disaggregation was based on age; older youth are much more likely to have more friends engaging in these negative behaviors or considered to be bad company. Table 12: Percent of assessed youth at risk for peer delinquency (n=5,109) Total GUY SLU SKN Female Male Ages 10 to 14 Ages 15 to 17 Peer delinquency 16% 18% 16% 13% 14% 19% 13% 24% Figure 26 lists eligible youth responses to the six actions in order from least common to most common. The most serious actions were also the least common, including breaking into a building to steal something, participating in crime and violence, using or selling illegal drugs, and attacking someone with a weapon. For example, 78 percent of youth reported that none of their friends had broken into a building to steal something whereas only 26 percent of eligible youth reported that none of their friends had skipped 48 school. Four-fifths of eligible respondents said that at least some of their friends would be considered “bad company” by their caregivers. Figure 26: Reported involvement of peers in delinquent behavior by eligible youth (n=1,135) SCALE T: FAMILY ANTI-SOCIAL INFLUENCE The family anti-social influence scale contains three questions about the respondents’ and their family members’ involvement with crime and violence. Respondents were asked how many of their family members think the respondent will be involved in crime in violence someday, how many family members have been involved in crime and violence, and if they think they will be involved with a group engaging in crime and violence A respondent who answered zero for the first two questions and no on the third scored a three, going up by increments of one if the answer was higher than zero or no. The threshold for at risk is a six for all three countries. A total of 19 percent of respondents scored at risk for this scale, and there was not a significant variation across countries and sex. 49 Table 13: Percent at risk for family antisocial influence (n=5,109) Total GUY SLU SKN Female Male Ages 10 to 14 Ages 15 to 17 Family antisocial influence 19% 17% 19% 20% 18% 19% 16% 25% Just over half (53 percent) of eligible youth have no family members who believe that they (the respondents) will one day become involved in crime and violence (see Figure 27). Most respondents (66 percent) report having at least one family member who has been involved in crime and violence, and 18 percent have four or more. Three-quarters of respondents answered no when asked if they thought that they would one day become involved in crime and violence, or if they are involved now. Only seven percent answered probably or yes (see Figure 28). Figure 27: Responses on family antisocial influence questions among eligible youth (n=1,132) 50 Figure 28: Response on family antisocial influence question among eligible youth (n=1,133) SCALE IJ: SELF-REPORTED DELINQUENCY The self-reported delinquency section is divided into two parts: first, the YSET asks youth if they did or did not engage in a series of behaviors in the last six months; second, if they answered yes to any of those behaviors, they are asked if they engaged in that behavior with the group of friends they hang out with. Every time the respondent says yes, he/she gets one added to the total score for this section. CFYR considers a youth to be at risk if they score a sum of six or higher in any of the countries. A total of 23 percent of interviewed youth scored at risk, with some variation across countries, sex and age groups. Table 14: Percent of assessed youth at risk for self-reported delinquency (n=5,109) Total GUY SLU SKN Female Male Ages 10 to 14 Ages 15 to 17 Self￾reported delinquency 23% 20% 24% 26% 18% 28% 19% 31% Figure 29 ranks delinquent behaviors from least common to most common among eligible youth. Hitting others and getting into fights were the most commonly reported behaviors, followed by skipping classes without an excuse or permission. Breaking into a building, selling drugs, and stealing were the least commonly reported behaviors by eligible youth. 51 Figure 29: Self-reported delinquency among eligible youth in the last six months (n=1,136) As noted above, if youth reported engaging in a behavior, they were then asked if they engaged in the behavior with the group of friends they hang out with. As shown in Figure 30, out of the behaviors that youth reported doing in the last six months, the most commonly done with a group of people they hang out with were: selling and inhaling drugs, skipping school, and getting into serious group fights. 52 Figure 30: Self-reported delinquency by eligible youth with a group of friends (n=44;1,017;160;520;203;12;847;460;810;726;513;124) Males are more likely than females to report engaging in the various behaviors. There are, however, two exceptions. Females are more likely than males to report hitting someone with the purpose of hurting him/her and skipping classes without an excuse or permission (see Figure 74 in Annex VI). SCALE FRD: RISKY GROUP BEHAVIORS The final scale concerns the behaviors of the group of friends the respondent hangs out with. The respondent is asked to answer yes or no to a series of questions regarding risky, illegal and belligerent group behavior. Each yes answer equals one, and a count of three or higher classifies the respondent as at risk in this scale. A total of 12 percent of youth scored as at risk in this scale. Risk does not vary much by country but the share of males at risk is twice as high as females. This means that the group of friends that females hang out with is less likely to engage in risky behaviors. 53 Table 15: Percent at risk for risky group behaviors (n=5,109) Total GUY SLU SKN Female Male Ages 10 to 14 Ages 15 to 17 Risky group behaviors 12% 13% 11% 13% 8% 16% 11% 15% The most commonly reported risky behaviors among eligible respondents’ group of friends were getting into serious fights, using weapons in fights, claiming or defending a territory as their own, and selling things that do not belong to them (see Figure 31). A total of 29 percent of eligible respondents said that their group of friends is known by others in the neighborhood as being violent (compared with 18 percent for the total youth sample). Figure 31: Eligible youth reporting on behaviors their friends have engaged in during the last six months (n=1,008) 54 COMPARISON BETWEEN HONDURAS AND CARIBBEAN YOUTH IN TARGETED COMMUNITIES Given that a very similar YSET tool was used in both Honduras and the Caribbean,18 the evaluation team thought it would be informative to compare youth answers and test for differences by module. While Honduras experiences higher levels of violence than any Caribbean country, the samples of youth included in the two programs might or might not reflect these differences, and the samples can be compared using their YSET answers. For this exercise, we included only questions that were identical or nearly identical across the two YSETs, and we used the same method for both countries/regions when calculating average scores. The two samples are, of course, different in many regards, and we were able to test for gender and age differences. We found that the Honduran sample is more male (68 percent compared with 49 percent in the Caribbean) and the Caribbean sample is older (average age is 13.14 years compared with 12.14 for Honduras, where the minimum age for YSET interviews was eight years old). These gender and age imbalances could account for some of the differences in average scores between Honduras and the Caribbean youth in targeted communities. The following table contains the mean scores for Honduran and Caribbean youth, by module, and the p￾value for the two-sample t-tests. A low p-value (typically, smaller than 0.05) tells us that the means for the two samples are statistically different. When comparing the entire two samples, with both eligible and ineligible youth, we see that Honduran youth have higher average scores in all modules but self-reported delinquency (where their means are not statistically different), and family antisocial influence (where Caribbean youth have higher average scores). The family antisocial influence module here includes two questions only, and their wording is slightly different in the two YSETs: in Honduras, youth were asked how many members of their family thought that they would one day become a gang member, and how many of their family members are in a gang; in the Caribbean, the words gang and gang member were replaced by being involved or engaged with crime and violence. The more general terms used in the Caribbean YSET might explain at least some of the difference in average scores in the family antisocial influence module. 18 By Caribbean we mean the three countries included in this study, and not the overall Caribbean region. 55 Table 16: T-tests of means with entire samples (eligible and ineligible) Honduras (n=2,181) Caribbean (n=5,109) p-value Weak parental monitoring 1.80 1.58 0.000 Critical life events 2.83 1.88 0.000 Impulsive risk taking 2.74 2.59 0.000 Guilt neutralization 2.58 2.36 0.000 Negative peer influence 2.01 1.50 0.000 Self-reported delinquency 1.05 1.08 0.429 Family antisocial influence 0.79 1.08 0.000 Risky group behaviors 0.82 (n=2,062) 0.76 (n=3,950) 0.043 When comparing only eligible youth, we found that Honduran youth had higher average scores in all modules except for self-reported delinquency, family antisocial influence and risky group behaviors. The risky group behaviors about risky behavior among the respondent’s group of friends, and the higher average age in the Caribbean sample might help explain the higher average scores in that module. Table 17: T-tests of means with eligible youth only Honduras (n=665) Caribbean (n=1,128) p-value Weak parental monitoring 2.35 2.11 0.000 Critical life events 3.63 3.17 0.000 Impulsive risk taking 3.46 3.36 0.007 Guilt neutralization 3.10 2.93 0.000 Negative peer influence 2.22 1.95 0.000 Self-reported delinquency 2.25 2.82 0.000 Family antisocial influence 1.65 2.56 0.000 Risky group behaviors 1.54 (n=646) 1.69 (n=985) 0.041 56 BASELINE FINDINGS FROM THE CAREGIVER SURVEY QUESTION 2: DO SECONDARY PREVENTION EFFORTS IMPROVE THE FAMILY ENVIRONMENT? WHY OR WHY NOT? INTRODUCTION Family structures and the environment at home can impact the behavior of youth, particularly when parents are either ill-equipped or absent. A survey of caregivers, parents or those with primary caretaker responsibility, was designed to understand at the baseline some of the family dynamics and the characteristics of the parents or others who may be in charge of youth at such a critical time of their lives. It also aims to measure parenting, family attitudes, and behaviors targeted by the intervention and test if the intervention leads to positive changes in these attitudes and behaviors over time. This section first introduces the caregivers participating in the survey and looks at the overall family structure. We then look at diverse family characteristics, including income and age, and examine potential negative family influences, such as drug and alcohol use and past engagement in criminal activity. These will serve as potentially important variables in exploring variation in risk and programmatic impact at endline. We then transition into items that might improve as a result of the intervention, including family cohesion, family decision-making, parenting attitudes (e.g., locus of control in raising children), caregiver presence, and knowledge of youth behaviors. We conclude by looking at families’ community engagement. Surveys were conducted with the primary caregiver of eligible youth in both the treatment and control groups. Out of the 1,139 eligible youth, we interviewed caregivers for 1,002 recipients, or 88 percent of eligible youth. This includes interviews with 947 caregivers; in households with more than one eligible youth, caregivers responded to relevant questions separately for each eligible youth. The remaining caregivers either refused to be interviewed or could not be located. The caregiver survey includes 404 interviews from Guyana, 434 from St. Lucia and 164 from St. Kitts and Nevis. As noted in the methodology section, the caregiver non-response rate was highest in St. Kitts and Nevis (21 percent). CAREGIVER SURVEY RESPONDENTS AND FAMILY STRUCTURE Variations in family structure might help explain risk levels and the success of a family-based intervention. The majority of the primary caregivers in Guyana, St. Kitts Nevis and St. Lucia (72 percent) are mothers; however, there is a small minority of fathers (11 percent) who responded to the survey as being the primary caregiver. These men are, according to the survey, primarily single parents, unlikely to be married or in common law unions. Mothers who take primary responsibility for the youth usually do so without the help of a father (an important finding further discussed below). Another 17 percent of the primary caregivers are grandmothers, aunts, sisters, and other guardians. This group of non-parental caregivers is overwhelmingly female. 57 While the survey focuses on a primary caregiver, there are usually additional household members who contribute to the care of the eligible youth. We asked respondents to list out members of the household and to note those with caregiver responsibility for the eligible youth. In addition to the primary caregiver, there are on average around two additional household members who participate in care. These additional caregivers mostly include fathers (and some mothers), grandparents, aunts and uncles, and a large number of older siblings. Table 18 provides the breakdown of who contributes, according to the survey, to the care of the youth eligible to participate in CFYR. The first two sets of columns show the reported primary caregiver, broken down by gender. The additional providers of care are reported in the third set of columns. This includes primarily spouses and grandparents but also a large number of siblings, aunts, uncles and other members of the extended family. When families have either fathers or grandparents living in the household, they are almost always involved in the care of the youth. Aunts and uncles may live in a household and not be involved in the care of the youth, most likely because they are taking care of a child of their own. Table 18: Caregivers: primary (by gender) and additional household members (n=941) Male Primary Caregiver freq. % of primary caregivers Female Primary Caregiver freq. % of primary caregiver Add’l Caregiver Freq. % of add’l Total caregivers Freq. % of total Father/Mother 102 11% 677 72% 419 23% 1198 43% Brother/Sister 3 0% 24 3% 713 38% 740 26% Grandparent 2 0% 65 7% 207 11% 274 10% Uncle/Aunt 4 0% 27 3% 236 13% 267 10% Self 0 0% 1 0% 28 2% 29 1% Others 9 1% 27 3% 257 14% 293 10% Total 120 13% 821 87% 1860 100% 2801 100% Overall, the most typical household in the caregiver survey is one with a single mother with no father (52 percent). Only a quarter of the households are nuclear families where both a father and a mother live together in the household and participate in the care (26 percent). There is a significant number of households where neither father nor mother appear as primary caregivers (12 percent); and a number of homes where the father is the single caregiver, with no mother present in the household (nine percent). It is important to note that the family structure in households with eligible youth is rather similar across the three countries. Families do, however, differ across countries in terms of size. The family size of eligible households in Guyana is larger, which had an average 2.46 household members, than in St. Kitts and Nevis (2.06) and St. Lucia (1.91). In order to visualize these profiles more clearly, Figure 32 provides a breakdown of the family structure in each of the three countries. The axes in the figure are a ratio. A 1:1 ratio would be one where there is a particular family member for each youth. For example, in the case of St. Lucia, approximately 80 percent of youth have a female parent while only roughly 40 percent of youth have a father living in the same household. Over 60 percent have both a male and female sibling, and roughly 20 percent have a female 58 grandparent in the household. This standardization is done in order to make the data comparable across countries and so that gender gaps and other family structure differences are not the consequences of differences in the way the graphs are scaled. The figures show a disproportionate number of mothers, a relative absence of fathers, similar numbers of male and female siblings (although many more young children in Guyana) and a somewhat greater presence of grandmothers in St. Kitts and Nevis. Figure 32: Family structures in St. Lucia, St. Kitts and Nevis and Guyana in caregiver survey (n=941) In St. Lucia and Guyana, we are able to draw on Multiple Indicator Cluster Survey (MICS)19 to compare eligible households with the rest of the nation. These health surveys are designed to understand various aspects of caregiving, with detailed information regarding mothers and children, as well as data on household characteristics. There are some limits to comparability, as MICS4 surveys were carried out in Guyana in 2015 and St. Lucia in 2012 and there are some differences in question wording. Nonetheless, the MICS provide a useful point of comparison. In the case of St. Lucia, we find that nationally the share of single mothers identified in the MICS is virtually identical to the one found in the caregiver survey: 52.3 percent of the households had a mother with no father present. In addition, for St. Lucia, in 4.3 percent of the households neither a father or a mother was living in the household. And only 1.6 percent of the households had a father with no mother living in the household. The MICS survey includes some information on caregiving responsibilities, but only for primary care. The largest difference with respect to our caregiver survey is that the MICS survey in St. Lucia found 41.7 percent of the households had nuclear families compared with 28 percent in the caregivers survey. Keeping in mind that there are some limitations to comparability, single mother patterns found among eligible households are not atypical of a country like St. Lucia; however, the share of nuclear families in our sample of eligible youth seems to be much lower than for the country as a whole. In the case of the MICS for Guyana, when the whole country is considered, the share of single mother households is 32.7 percent, with most homes being nuclear families. However, when only the urban area 19 MICS is a UNICEF initiative to collect data on the situation of women and children, with governments typically carrying out the data collection with technical and financial support from UNICEF and partners. Data are collected in nationally or sub￾nationally representative samples of households. 59 of Georgetown is analyzed, the share of single mother household increases to 54.1 percent, again virtually the same as what was found in the caregiver survey. FAMILY CHARACTERISTICS In addition to the family structure, there are other family characteristics that might be important to explaining variation in risk or variation in programmatic impact. For example, income levels, caregivers’ age, and their education levels might influence risk and resilience to violence in the larger community. INCOME LEVELS AND ASSET INDEX Income levels and socioeconomic status are notoriously difficult to measure even in relatively sophisticated survey instruments. There are many reasons why respondents may find it difficult to estimate their own household income, which may be driven by seasonality, complex income streams coming from different family members, as well as imperfect recall and other sources of imprecision. In addition, there are social desirability pressures that may prevent respondents from truthfully revealing their own income or socioeconomic levels. Annex IV provides information on three distinct methods for estimating household socio-economic status. In Figure 33, we provide a simple count of assets. The variables included are whether the household has a flush toilet, television, computer, internet connection, refrigerator, washing machine, stove, microwave, telephone, cellular phone, water pump, hair dryer, heater, satellite or cable television, vehicle and a boat. Many of these household goods are widespread, while others are rare within the caregiver sample. Only three households had all 16 items, and only two of them reported not having any of them at all. The median household has eight of these items. There are some differences between countries, which reflect the relative development of each of them, but as can be seen in the histograms reported in Figure 33, the most important feature is that there is a great deal of overlap in the asset indexes across countries. The average for St Kitts Nevis is highest at 9.8 assets, Guyana has the lowest average at 7.3, while St. Lucia falls in between with an average of eight. Figure 33: Number of assets in households by country as reported in the caregiver survey (n=947) 60 AGE Mothers who are the primary caregivers of the eligible youth are no younger or less educated than the fathers of the youth (living in the same household) (see Figure 34 and Figure 35). This is the case for all the countries, even though there are some differences, particularly in education, across the countries. It is not possible to compare the age profiles to the non-eligible youth, but comparisons across gender and country reveal some salient features. Fathers who participate as caregivers are slightly older than the mothers. The distributions are relatively dispersed in St. Lucia and much tighter around the mode in St. Kitts Nevis. In the case of the mothers, the distribution looks very similar across countries, with a slightly lower age in St. Lucia. Figure 34: Age distribution of mothers of eligible youth (regardless of caretaking role or family structure Figure 35: Age distribution of fathers of eligible youth (regardless of caretaking role or family structure EDUCATION As with age, it is not possible to use the caregiver survey to compare the education characteristics with non￾eligible households. However, some comparisons across family relationships and countries are very illustrative. Education is measured by a rank ordering, where 5 = tertiary (college/university), 4 = secondary, 3 = primary, 2 = less than primary, and 1= no formal education. Vocational training was placed in the same ranking as secondary (high school) education, with a score of 4. Average education levels across countries, groups, and genders are presented in Figure 36. While sample sizes are small across multiple disaggregation, the figure suggests the following: females tend to have a slightly higher level of education than males, siblings tend to have a higher level than parents, and those in St. Kitts and Nevis have higher levels than those in the other two countries. Perhaps surprisingly, the profile of education in St. Lucia is not so different from the one in Guyana, even though Guyana is clearly a less developed country.20 20 In UNDP’s 2018 Human Development Index (HDI), Guyana ranks 125th out of 189 countries, St. Lucia ranks 90th and St. Kitts and Nevis 72th. The HDI measures progress along three dimensions: a long and healthy life, being knowledgeable, and having a decent standard of living. 61 Figure 36: Levels of educational achievement by country and relationship to program youth (n=941) EMPLOYMENT Caregiver survey respondents were asked about gainful employment for each household member who is aged 16 and above. Although we do not know the number of hours worked or salary they receive, we know whether each member has a source of income through work. Of all the primary caregivers that answered the survey, 64 percent are gainfully employed. Among single mothers, 65 percent are working, and among single fathers, 84 percent are working. Grandparents who live in the household are working in about 31 percent of cases. In our sample, employment levels are much higher in St. Kitts and Nevis compared to the other two countries, which is primarily driven by higher employment rates for mothers. Table 19: Share of household members who are gainfully employed (n=942) All countries SLU SKN GUY Primary caregiver (n=942) 64% 59% 87% 60% All mothers (n=751) 64% 60% 92% 58% Single mothers (n=489) 66% 59% 90% 62% All fathers (n=332) 86% 83% 88% 88% Single fathers (n=83) 84% 85% 82% 85% Grandmothers (n=159) 31% 23% 41% 36% Grandfathers (n=53) 32% 32% 37% 29% Aunts (n=119) 68% 67% 91% 60% Uncles (n=112) 71% 55% 92% 81% An important factor for household income level and economic stability is how many household members are gainfully employed. On average, households have 1.63 persons aged 16 and above employed, and this number is lower for households that have a single parent or no parent at all (i.e. no mother or father are 62 reported to live in the household). With the exception of single-father households, St. Kitts and Nevis again shows higher levels of employment. That finding is especially interesting when we consider that Guyanese households are larger on average. Table 20: Average number of gainfully employed household members (n=947) All countries SLU SKN GUY All households (n=947) 1.63 1.41 1.96 1.73 Single-mother households (n=489) 1.53 1.24 1.84 1.72 Single-father households (n=83) 1.41 1.36 1.27 1.52 Households without any parents (n=118) 1.49 1.09 2 1.59 Youth employment levels are measured for those aged 16-18, including both eligible youth and other youth in the household. Among this group, 24 percent are neither in school nor working. The case of not being in school nor working is sometimes referred to as NEET (“not in education, employment or training”) in the literature and is typically considered a strong risk factor. Table 21: Number of youth between the ages of 16 and 18, by employment and school status (n=537) In school Employed Yes Employed No Total Yes 25 (5%) 327 (61%) 352 (66%) No 57 (11%) 128 (24%) 185 (34%) Total 82 (15%) 455 (85%) 537 (100%) Figure 37 breaks down the number of NEET youth by country and compares all youth in the caregiver survey sample. The share of youth that are NEET is much higher in Guyana compared to the other two countries, with 35 percent in Guyana and only six percent in St. Kitts and Nevis. St. Lucia falls somewhere in between at 17 percent. If breaking down the numbers of NEET youth by community in Guyana, this share becomes 32 percent of all sample youth in Sophia, 35 percent in East Ruimveldt, 36 percent in East La Penitence, 40 percent in Lodge and 43 percent in Corriverton. Therefore, compared to the Georgetown communities, the town of Corriverton appears to have a larger share of youth in our sample that is not currently studying or working. 63 Figure 37: Share of youth ages 16-18 that are neither employed nor in school (n=537) RELIGION AND RELIGIOUS ATTENDANCE There is a large degree of reported religious diversity in our sample of caregivers and considerable differences across countries. The largest denomination among respondents in St. Lucia is Roman Catholic (60 percent); in St. Kitts and Nevis it is Methodist (19 percent); and in Guyana it is Pentecostal (38 percent). Religious diversity is the highest in St. Kitts and Nevis where similar percentages identify as Methodist, Baptist, Pentecostal, Anglican, and other. Pie charts of self-reported religious affiliation can be found in Annex IV. Out of those reporting a religion, about half attend religious services more than once a month (see Figure 38). There are some differences in the frequency of religious service attendance by country, with respondents in St. Lucia attending somewhat less frequently than respondents in Guyana and St. Kitts and Nevis. 64 Figure 38: Average frequency of religious service attendance, by country (n=944) FAMILY COHESION To measure family cohesion, the caregiver survey uses a series of questions, mostly drawn from the Family Adaptability and Cohesion Scale (FACES), an instrument designed to measure family interactions. We inquire about the frequency with which a youth calls or visits older family members such as grandparents and uncles and whether the family shares traditions or family stories. According to caregivers, about one￾fifth of eligible youth do not ever call or visit older family members, and one-quarter do not ever or rarely get traditions shared with them. As noted above, if there is more than one eligible youth in the household, caregivers are asked separate questions about each youth. As such, sample sizes are somewhat higher than the number of caregivers interviewed. 65 Figure 39: Responses to questions on vertical family relations (n=1,000) To facilitate analysis, the data from several additional family cohesion questions are reduced, through a factor analysis, into an index in a single dimension. The items included in the index include evaluations by the caregiver of whether family members are involved in each other’s lives, whether they feel very close to each other, if are supportive of each other in difficult times, and if they can discuss problems calmly with each other. The index also includes questions about the quality of communication between family members, the family’s ability to resolve conflicts, and the amount of time spent together as a family. The factor analysis shows a very strong correlation among all these variables.21 The graphs show that this index of family cohesion is quite similar among all three countries, with most households bunched toward the lower end of the family cohesion spectrum. This skew is slightly stronger in St. Kitts and Nevis and the distribution somewhat more normal in Guyana. 21 The survey also asked a question related to anger in the family. The question is the following: “When angry, family members seldom say negative things about each other.” The factor analysis for the index showed that this question was not correlated with the rest of the variables measuring family cohesion. The lack of correlation with all the other family cohesion measures suggests that this variable triggers a very different aspect of family dynamics, probably related to anger. 66 Figure 40: Distribution of scores in the Family Cohesion Index, by country (n=943) FAMILY DECISION-MAKING To provide a measure of how decisions are made in families, caregivers in households with more than one caregiver were asked how often they disagree and argue with the youth’s other caregivers in front of the youth. More arguing, fighting, and bad mouthing signifies a less unified parenting front and can give youth mixed messages about how they are expected to behave. As shown in Figure 41, more than half (61 percent) of caregivers admitted to very often/often/sometimes disagreeing with other caregivers on how to discipline the youth in front of them. Talking badly about the other caregivers or arguing with them in front of the youth are reported to be far less common. 67 Figure 41: Responses to questions on family decision-making among households with more than one caregiver, n=872 Although there is variation across countries, as shown in Figure 42, no single country stands out as having a higher or lower share of caregivers reporting to disagree or argue with other caregivers in front of the youth. Figure 42: Caregiver responses to questions on family decision-making, by country (n=872) 68 PARENTING ATTITUDES: LOCUS OF CONTROL Caregivers were also asked six questions about their parenting beliefs, particularly about whether parents and the community have an influence on kids’ likelihood of getting into trouble, as a measure of parental locus of control. As show in Figure 43, while the vast majority (91 percent) of respondents believe that parents can have a major influence in their kids, a similar majority (85 percent) also believe that children will get into trouble no matter what their parents do. Therefore, most believe that their children’s behavior can be influenced but not ultimately controlled, and children will still get into trouble. Nearly half of respondents (47 percent) believe that community conditions are making it more difficult for them to keep their children out of trouble, while the rest does not necessarily think so, even though they all live in communities deemed to be vulnerable. Figure 44 shows that the vast majority (95 percent) of caregivers believe it is their responsibility to keep their children out of trouble. They also report that in the last 12 months they have improved as a parent (91 percent) and that their influence on their children has increased (87 percent). These findings suggest that caregivers already have a relatively strong internal locus of control when it comes to raising their children. Figure 43: Caregiver responses to questions on parenting locus of control (n=947) 69 Figure 44: Caregiver responses to questions on parenting locus of control (cont) (n=947) Respondents in St. Kitts and Nevis are more likely to agree that parents can have a major influence on their kids, but also that children will get into trouble no matter what they do (see Figure 45). Guyanese caregivers agree in higher percentages (58 percent) with the statement that the community conditions prevent them from keeping children out of trouble. St. Kitts and Nevis caregivers were more likely to strongly agree with all three questions in Figure 46, although the sum of agree and strongly agree is similar for all three countries on all questions. 70 Figure 45: Caregiver responses to questions on parenting locus of control, by country (n=947) Figure 46: Caregiver responses to questions on parenting locus of control (cont.), by country (n=947) CAREGIVER PRESENCE The survey asked caregivers how often they spend time with their child doing a few different specified activities over the course of the month prior to the survey. Caregivers were also asked about how often 71 they know where their child is, who they are with, and when they are out of school. The survey asked about parent-child communication, including whether they say mean things or hit or push when angry and whether they have other adults to count on to help raise their child. The last question asks them whether they are proud of their child. Figure 47 and Figure 48 show how many days a week caregivers report sitting down to eat, helping with school work, doing something fun as a family, and doing something religious as a family. The activity that caregivers and youth most often do together is sharing meals. 42 percent of caregivers report that they have meals together seven days a week and only 12 percent report not having any meals together. Other activities are less common, including doing homework together, doing something fun as a family, or doing something religious as a family. Doing something fun as a family is the least likely activity to take place every day of the week, with only 10 percent of respondents saying they did that seven days a week last month, and close to a quarter (23 percent) of caregivers reporting not having done anything fun as a family at all. Figure 47: Number of days in a typical week caregivers report select behavior for each eligible youth Note: While many caregivers with youth out of school did not answer the question on helping youth with school work, several did. Most of these stated that they helped youth zero days a week, but some did report helping with school work, which could have been understood as other educational programming. Excluding out of school youth, the percentage stating zero would drop from 34 percent to 32 percent. 72 Figure 48: Number of days in a typical week caregivers report select behavior for each eligible youth (n=996) The share of caregivers who report always knowing where their child is when not at home or school is 57 percent, and the share who report always knowing who they are with is 49 percent (see Figure 49). About ten percent never or rarely know where their child is when outside the home or school. These numbers are similar to the YSET youth responses, where 18 percent reported that they rarely or never tell their caregivers where they are going, and 13 percent that their caregivers do not know where they are when outside of school. More than half of youth (63 percent) reported that their parents or guardians often or always know where they are when they are outside of school, which is close to the 57 percent of caregivers reporting the same thing. 73 Figure 49: Caregiver responses to questions on caregiver presence for each eligible youth (n=998) About a third of respondents said that when the youth breaks family rules, he/she only faces consequences half the time/rarely/never (see Figure 50). Only 55 percent of caregivers say that their child always listens to them when they have something important to say, and about ten percent admit they only listen to their child when he/she has something important to say half the time or less often (see Figure 51). Only half report being able to always discuss problems with their child. When angry, a quarter of caregivers (26 percent) always or often say mean things to their child and 11 percent may push or hit them. These responses suggest that there is room for improvement in terms of parental monitoring and caregiver-child effective communication. 74 Figure 50: Caregiver responses to questions on caregiver presence for each eligible youth (n=1,001) Figure 51: Caregiver responses to questions on communication with each eligible youth (n=1,001) 75 There are gender differences in reporting on communication with eligible youth. A higher percentage of male caregivers report being listened to when compared to female caregivers (see Figure 52). Males are also more likely to report that they are able to talk to their child and solve their problems. Male caregivers are also less likely to report saying mean things or pushing or hitting their child when angry. It is not clear if these differences reflect actual behavior differences or gender differences in reporting. Figure 52: Caregiver responses to questions on caregiver presence, by caregiver’s gender (n=1,001) As shown in Figure 53, about 17 percent of respondents report not having another adult to count on to help raise their child. Despite the challenges, parents are overwhelmingly proud of their children. Only three percent said that they are never or rarely proud. 76 Figure 53: Responses to select questions on caregiver presence (n=1,001) NEGATIVE FAMILY INFLUENCES Caregivers were asked if members of the family had drug or alcohol problems or if they had been in conflict with the law (this was done without indicating specific family members.) In seven percent of households, caregivers report that drug use has created a problem in the household. In those households, marijuana was mentioned in 67 percent of these cases and alcohol in 36 percent. It is worth mentioning that enumerators noted instances where they observed signs of drug use. One enumerator noted that he/she “saw [a family member] smoking marijuana on the street” while doing the interview. Another noted that the “interview conducted outdoors and there was a smell of marijuana on respondent's person.” A third included a comment regarding “the uncles” who “were smoking marijuana,” and a final one reports that “the mother bought marijuana in front of me.” When disaggregating the data by caregiver’s gender, we found that three percent of male-led household reported that drug-use creates problems in the household, compared with seven percent of female-led households. Out of those who reported a problem, no male caregivers mentioned alcohol, and they all mentioned marijuana. In seven percent of households, respondents reported that a member had been charged with a crime in the last ten years. The most common types of crime were robbery, assault without robbery, and domestic violence (see Figure 59 in Annex IV for a breakdown of reported crimes). In two percent of households, respondents reported that a member had been a member of a gang, defined as a group of people who engage in violent and/or criminal acts together. This question does not include relatives, such as a parent, uncle, cousin or older sibling who currently live somewhere else. It is difficult to know if there is 77 underreporting due to social desirability bias, but these responses indicate a relatively low level of household members in conflict with the law. The survey also asked caregivers if members of the household had been a victim of a crime in the last 12 months. Only four percent of respondents reported victimization. This is lower than the average rate of household victimization found through representative samples of the fifteen communities reported above: nine percent in Guyana, eight percent in St. Lucia, and six percent in St. Kitts and Nevis. For those who said yes, the most common types of crime were assault without a robbery and home burglary (see Figure 60 in Annex IV). KNOWLEDGE OF YOUTH BEHAVIOR Caregivers were asked about the youth’s behavior in the last six months across the same set of behaviors that youth were asked in the YSET. The main reason for employing this strategy was to compare caregiver and youth responses as a means to estimate youth bias when self-reporting negative behavior. Interestingly, youth reported engaging in these behaviors in larger numbers than their caregivers, in all categories. While this battery does therefore not provide a useful validation tool, it does provide a measure of caregiver knowledge about youth behavior. Table 22 provides a comparison between youth and caregiver responses. The largest differences can be seen in the cases of hitting someone with the purpose of hurting him/her and being involved in fights, where 48 percent and 36 percent of youth, respectively, said yes, but only 12 percent and less than one percent of caregivers, respectively, said yes. Breaking into a building was the least commonly reported behavior, and for this no differences were identified in reporting between youth and their caregivers. These results indicate two possible phenomena: either caregivers were not very aware of how often youth were engaging in these negative behaviors and which behaviors they engaged in, or they were not willing to say yes even when they thought the answer was yes, presumably due to embarrassment or fear that the youth would get into trouble. 78 Table 22: Youth behaviors as reported by self, caregivers, and differences between the two Item Share of eligible youth that has done in the last 6 months (n=1,135) Share of caregivers reporting that youth has done in the last 6 months (n=1,002) Percentage point difference between eligible youth and caregivers Hit someone with the purpose of hurting him/her 48% 12% -36 Were involved in fights 36% 0% -36 Involved in serious group fights 39% 11% -28 Skipped classes at school without an excuse of permission 38% 18% -20 Purposely damaged or destroyed property that belonged to others 30% 6% -24 Carried a hidden weapon for protection 31% 6% -25 Inhaled drugs or substances that make you "high" 30% 9% -21 Lied about age to get into a place or to buy something 26% 4% -22 Attacked someone with a weapon 14% 3% -11 Stole or tried to steal something valuable 10% 3% -7 Sold marijuana or other illegal drugs 9% 2% -7 Broke into a building to steal something 1% 1% 0 Figure 54 shows where caregivers and eligible youth agree and disagree. The chart is ordered so that least commonly reported behaviors by youth appear first, starting with breaking into a building, and most commonly reported behaviors appear last, ending with hitting someone. Among the least commonly reported behaviors, there are larger shares of agreement between caregivers and youth. Conversely, among the most commonly reported behaviors, there are larger shares of disagreement, particularly with the youth reporting having done that behavior in the last six months, and the caregiver saying no. Overall, looking at the mismatches, it is far less common for the youth to say no and for the caregiver to say yes, than the other way around. 79 Figure 54: Share of matches and mismatches between youth and caregiver answers, for each behavior (n=1,135 for youth and 1,002 for caregivers) As shown in Table 23, different types of caregivers (parents, grandparents, and others) are equally likely to say a youth did not commit an act when the youth reported that s/he did. Other caregivers (e.g., aunt, sibling) were somewhat more likely to say that a youth did something when youth reported that s/he did not. Table 23: Share of matches and mismatches in reporting youth behavior, by type of primary caregiver (n=1,135 for youth and 1,002 for caregivers) Parents Grandparents Other caregivers Match 74% 75% 73% Youth yes, caregiver no 23% 22% 22% Youth no, caregiver yes 3% 3% 6% 80 CONCLUSIONS The analysis of the community survey using the indices created by the evaluation team indicates the following general findings. (1) There are relatively low rates of reported crime, but there are heterogeneous perceptions of crime prevalence, and perceptions of crime prevalence do not always match self-reported crime victimization statistics. (2) There is relatively strong community trust and cohesion, as measured by two different indicators of social capital. (3) Community members have somewhat neutral attitudes toward the law and toward police. (4) There are similar levels of resources for youth across the three countries, and higher variation across communities in each country. (5) There are generally high feelings of self-efficacy and self-determination. Although the levels of victimization and crime in the communities in the program are not as high as in other countries of Latin America or the Caribbean, there are substantial numbers of youth at risk within these communities. The YSET identified that more than one fifth of the youth to whom the instrument was applied were above a threshold of risky attitudes and behaviors to warrant the family intervention. As far as what is learned from analyzing the YSET data, implementation of the same YSET tool in the three countries provides a useful snapshot of the degree of risk that youth face in each location, and if there are differences by gender and age. Males and older youth (ages 15-17) are at a higher risk in all scales. Males stand out in relation to females in one scale in particular: weak parental monitoring, implying that caregivers monitor the whereabouts and company of female youth more closely, and in St. Lucia especially. Youth in the islands are more likely to score at risk in the weak parental monitoring and negative peer influence scales, even when the scale thresholds are set to be the same for the three countries (see full table in Annex III). When the thresholds are set to be the same, St. Kitts and Nevis has a higher share of youth scoring at risk in very single scale except for one, scale H – peer delinquency. Predicted differences across gender and age groups suggest that risk dynamics are similar across the three countries (see Table 6). Impulsive risk taking and guilt neutralization are the most common risk factors. These scales measure levels of agreement with statements about enjoying doing things that are dangerous or that might get them in trouble, and rationalizing behaviors such as lying, stealing and beating others. A surprisingly high number of eligible youth have had some critical life event happen in the last year. Some of the events included in the critical life events scale can be stressful or even traumatic, such as hearing about deaths or injuries caused by violence in their neighborhoods (which happened to 71 percent of eligible youth), having a fight or problem with a friend (67 percent), or having someone close to them die, get seriously injured, very sick or leave the country (64 percent). The frequency for negative behaviors as measured by the IJ scale was also noticeably high, with the mean for all the Caribbean youth a little higher than (but not statistically different from) the mean for Honduras. Eligible Caribbean youth had a higher mean score in the IJ scale than Honduran eligible youth. These data also show that there is significant underreporting of youth negative behaviors by caregivers. We believe that either caregivers are not very aware of how often youth are engaging in these negative behaviors, and which behaviors they engaged in, or they are not willing to report these behaviors, presumably due to embarrassment or fear that the youth would get into trouble. Through the caregiver survey, we are able to obtain a more comprehensive picture of the family composition and family life of the youth that are engaged in the program. The majority of primary 81 caregivers are mothers (72 percent) and the most typical household in the program is one with a single mother and no father present in the household (52 percent). According to caregivers, almost a quarter of household youth between the ages of 16 and 18 (24 percent) are neither in school nor working. Employment levels in St. Kitts and Nevis are much higher than in the other two countries, for youth and for household members in general. In our sample, Guyanese youth in particular are more likely to be in the risky situation of being out of school and out of work. Parents generally have a relatively strong internal locus of control, believing both that they have a responsibility to keep youth out of trouble and that they are able to influence youth. In addition, very few caregivers report problems of drug and alcohol problems in the family or family participation in gangs or criminal activities. Questions about caregivers’ knowledge of youth whereabouts and company when not at home or in school show that improvements can be made in the level of parental monitoring. Many caregivers are either unaware or unwilling to report that their youth engages in normatively undesirable behavior even when their children do so. Many caregivers are also struggling with communicating with youth about important topics they want or need to address, with only half reporting they are always able to discuss problems with their child. These seem to be problem areas that the family counseling intervention may be apt to address. One of the most challenging aspects of this evaluation is related to the small size of the communities (and even countries) involved, which limits the statistical power of the inferences that may be obtained regarding the effects of the intervention. After implementation began, several processes have generated a smaller sample size than what was originally envisaged. One is related to the willingness of families and youth to become beneficiaries, which can partly be attributed to the time that passed between YSET consent to the enumerator and the family counselor’s first visit. A second one is the consequence of mobility and migration of youth and families that had been participating in the program. Even though survey teams have gone to great lengths to find and survey all the youth and their families, some of them are not located. A third one is that there is likely to be some attrition in numbers due to youth and families that cease to participate for the full duration of the intervention. The feasibility of measuring minimally detectable effects will depend on ensuring that efforts continue to collect data as comprehensibly and with as full of a coverage as possible, at least for those aspects that are under the control of the implementer and the evaluator. 82 ANNEX I: ELIGIBILITY DETERMINATION AND RANDOMIZATION This annex summarizes the approach used to determine eligibility and randomize youth into treatment and control. DETERMINATION OF ELIGIBILITY Creative Associate’s project sub-contractor USC determined youth eligibility for the intervention. The USC team added up the scores for each YSET module, identified the highest risk youth across all modules, and then calculated the average module scores across this highest risk group. These averages were then used in conjunction with the distribution of the remaining youth to inform a purposively selected cut￾point for each module on a country-by-country basis. Using Guyana as an example, Table 24 provides the average risk scores for the 34 highest risk youths identified and the final thresholds. The thresholds are somewhat different across each country, and as such, raw scores cannot be compared across countries. Table 24: Average scores for 34 highest risk youth and final thresholds across nine risk modules in Guyana B C DE F G H T IJ EG Average scores for highest risk youth 2.21 3.97 3.45 2.92 2.24 2.11 6.24 5.91 4.47 Final cut-points 2.5 4.0 3.0 2.5 2.3 1.9 6.0 3 3 This approach is similar to that used by USC in Los Angeles and Central America, although in these locations the scores of gang members were used to identify the thresholds. In the Caribbean, where gang membership is far less common, this approach could not be used. MATCHING AND RANDOMIZATION Given that the intervention targets families, one of the challenges with randomization was ensuring that two eligible youth in the same household were randomized to the same treatment condition. After matching youth into households, we followed a two-stage randomization process. In the first stage all eligible households were stratified by community and randomized into treatment and control. We then conducted balance tests with the individual youth level data on several variables including sex, age, grade, working or not, in school or not, and a summary scores on each of the nine criteria. The randomization procedure was written to re-randomize until all variables balanced. In the case of Guyana, this resulted in 229 treatment and 228 control youth. This exceeds CFYR’s capacity to treat, which was 150 youth. Therefore, the second step divides the treatment and control youth into primary study youth and replacement study youth. This step was done to maximize distribution of cases across communities and subsequently maximize power. In the case of Guyana, 150 youth were assigned to the primary study group and the remaining 79 were assigned to the replacement group. Using Guyana as an example, Table 25 83 shows the number of youth by community randomized into treatment and control at both stage one and stage two. Table 25: Results of stage 1 and 2 randomizations in Guyana Stage I Treatment Stage I Control Stage I Total Stage 2 Treatment Stage 2 Control Stage 2 Total Corriverton 43 43 86 30 30 60 East La Penitence 17 14 31 17 14 31 East Ruimveldt 41 41 82 30 33 63 Lodge 30 32 62 30 30 60 Sophia 98 98 196 43 43 86 Total 229 228 457 150 150 300 84 ANNEX II: DATA COLLECTION INSTRUMENTS Table 26: YSET, caregiver survey, and implementation dates YSET application dates Caregiver survey dates Implementation start date Guyana Sept 2017-March 2018 April 2018 - June 2018 Jun-18 SLU Sept 2017-April 2018 June 2018 - August 2018 Aug-18 SKN October 2017-June 2018 July 2018 - August 2018 Aug-18 85 YSET TOOL GUYANA EXAMPLE 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 CAREGIVER SURVEY Impact evaluation of secondary prevention: Caregiver Survey March 14, 2018 Contents A. Preload Field Control .......................................................................................................110 B. Introduction/Gatekeeper speech ......................................................................................111 C. Informed consent .............................................................................................................111 D. Roster and respondent information ..................................................................................111 E. Family cohesion (from the FACES instrument) ................................................................113 F. Parenting .........................................................................................................................114 G. Relationship with the youth .................................................. Error! Bookmark not defined. H. Youth behavior and relationships .....................................................................................116 I. Other family members......................................................................................................118 J. Community ...................................................................................................................119 K. Household characteristics ............................................................................................120 L. Contact information ......................................................................................................121 M. End matter................................................................................................................122 A. Preload Field Control Enumerator name Enumerators ID number Unique identifier of youth interviewed Unique identifier of caregiver Country Community EA Name EA Code Number of siblings in the study Unique identifier of youth interviewed Unique identifier of youth interviewed Unique identifier of youth interviewed Unique identifier of youth interviewed Timestamp 111 B. Introduction/Gatekeeper speech Hello, my name is XXX and I work for The Solomon Group a local research firm conducting a study of youth in (St. Lucia/Guyana/St. Kitts and Nevis). In the last few months, a youth from this household participated in a survey for a project funded by the United States Agency for International Development (USAID) called the Community, Family and Youth Resilience Project. We would like to conduct some follow-up from that survey. I am looking to speak with the primary caregiver of XXXX. By primary caregiver I mean the adult in the household that most looks after XXXX. This is often the mother of the household but it can also be the father, grandparents, or other relatives or guardians. Is he/she available? [If Yes skip to informed consent] [Add a note to the enumerator with the name of the person who originally consented for the child to participate in the YSET] [If No] Do you know a good time for me to return so I can speak with XXX’s primary caregiver? [Record name and time for follow-up] A. Informed consent Hello, my name is XXX and I work for The Solomon Group, a local research firm conducting a study of youth in (St. Lucia/Guyana/St. Kitts and Nevis). XXXX recently participated in an interview and now we would like to conduct a follow-up survey with you as the primary caregiver to XXXX. This survey is funded by the United States Agency for International Development as part of an evaluation of a project being conducted in your community called the Community, Family and Youth Resilience Project. The project is implemented by a US organization in collaboration with the government of (St. Lucia/Guyana/St. Kitts and Nevis). The goal of this study is to better understand how to provide support and services to your community and communities like yours throughout the Caribbean. Please note that while we work in close coordination with the Community, Family and Youth Resilience Project, we are not part of the project. The survey should take no more than 45 minutes to complete. Your participation in this survey is voluntary. Deciding not to participate will not affect your child’s or ward’s ability to get services. If you are unable or do not want to answer a question, you do not have to. All of your responses will be kept strictly confidential, which means that we will never reveal your name. If you have any questions or concerns about this study now or later you may contact XXXXX at XXXXXXXXXX so that we may assist you. CONSENT. Are you willing to proceed? 1 Yes 2 No PROG: SKIP TO CLOSURE 112 D. Roster and respondent information HOUSEHOLD ROSTER: I want you to think about your household. By ‘household’ I mean a group of people living in the same dwelling space and who eat meals together at least three times a week. We would like to learn a little about who lives in this household. First, list yourself. Then list every other member of your household. We do not need to know their names, so you can just give me the first letter of their name or nickname. For each person I will ask a few questions, the answers to which will all be kept confidential. Former household members who have been gone for more than a year should not be listed. [ID] 1. [Name ] 2. [Relation to HH] 3. [Sex] 4. [Raising ] 5. [Age] 6. Education 7. In School 8. Emplo yment ID Please provide the first names of all of the people who are part of this household starting with yourself? • Probe: is there anyone else? What is their relationship to XXX? 1. Father/Mother 2. Brother/Sister 3. Brother-in￾Law/Sister-In￾Law 4. Grandparent 5. Uncle/aunt 6. Spouse/Partner 7. Son/Daughter 8. Step-Child 9. Father-in￾Law/Mother-in￾Law 10. Other (specify): __________ What is […]’s gender? 1. Male 2. Female 3. Other Does […] help raise XXX?22 1. Yes 2. No Approximat ely how old was […] on their last birthday? (Enter number of Years) _______ 98 - Don’t know What is the highest level of education completed by […]? 1 No formal education 2 Less than primary 3 Primary 4 Secondary 5 Tertiary (college/u niversity) 6 Vocational Answer only for those 6 to 18 years old: Did […] attend school this year? 1. Yes 2. No Answer only for those 16 years and older: Is this person currently employed in a paying job? 1. Yes 2. No RES P1 Interviewee : _________ ___ RES P2 RES P3 RES P4 RES P5 RES P6 RES P7 RES P8 RES P9 22 Guidance: Helping to raise the child means being responsible for them. Parents typically “raise” their children, but grandparents, older siblings or other extended family members might also help raise a youth. Raising is different than temporarily babysitting or looking after a youth. 113 9. Does anybody else, who does not live in this house, play a role in raising XXX? 1-Yes, 2-No 10. [If yes] Who? 1-Another parent, 2-Another relative, 3-A family friend, 4-Teacher or counselor, 5- Priest, pastor or other religious leader, 6- Other (specify) Thank you. Now I would like to ask you a few questions about yourself. 11. [If country = SKN or St. Lucia] What is your present civil status? 1-Single/never married, 2-Married, 3- Common law, 4-Widowed, 5-Divorced, 6- Separated, 7-Visiting at least once a week 12. [If country = Guyana] What is your present marital status? 1-Single/never married, 2-Married, 3- Common law, 4-Widowed, 5-Divorced, 6- Separated, 7-Visiting at least once a week 13. What is your religion? 1-None, 2-Anglican, 3-Baptist, 4-Hindu, 5- Methodist, 6-Muslim, 7-Pentecostal, 8- Rastafarian, 9-Roman Catholic, 10-Seventh Day Adventist, 11-Church of God, 12- Evangelical, 13-Other 14. How often do you attend religious services, on average? 1-Everyday, 2-Every week, 3-More than once a month, 4-Once a month, 5-Several times a year, 6-Once a year, 7-Less than once a year, 8-Never E. Family cohesion (some questions are from the FACES instrument) Now I want to ask you a few questions about your family. 1. How often does XXX visit any family members who are older than you, such as grandparents or great aunts and uncles? Visits include in-person meetings only, not over the phone. 1- Very often, 2-Often, 3-Sometimes, 4- Rarely, 5-Never, -99 RA N/A 2. How often does XXX call any family members who are older than you, such as grandparents or great aunts and uncles? These may include phone or video calls. 1- Very often, 2-Often, 3-Sometimes, 4- Rarely, 5-Never, -99 RA N/A 3. How often do family members share family traditions or family stories with XXX? 1- Very often, 2-Often, 3-Sometimes, 4- Rarely, 5-Never, -99 RA N/A Would you agree or disagree with the following statements about this family or household: 114 4. Family members are involved in each other's lives [Potential prompt: “Involved can include a number of activities, including regular communication, attending events together, and monitoring progress in school.”] 1-Strongly agree; 2-Agree, 3-Neither agree nor disagree, 4-Disagree, 5-Strongly disagree 5. Family members feel very close to each other 1-Strongly agree; 2-Agree, 3-Neither agree nor disagree, 4-Disagree, 5-Strongly disagree 6. Family members are supportive of each other during difficult times. 1-Strongly agree; 2-Agree, 3-Neither agree nor disagree, 4-Disagree, 5-Strongly disagree 7. Family members can calmly discuss problems with each other. 1-Strongly agree; 2-Agree, 3-Neither agree nor disagree, 4-Disagree, 5-Strongly disagree 8. When angry, family members seldom say negative things about each other. 1-Strongly agree; 2-Agree, 3-Neither agree nor disagree, 4-Disagree, 5-Strongly disagree How satisfied are you with… 9. The quality of communication between family members? 1- Extremely satisfied; 2- Very satisfied; 3- Generally satisfied; 4- Somewhat dissatisfied; 5-Very dissatisfied 10. Your family’s ability to resolve conflicts? - Extremely satisfied; 2- Very satisfied; 3- Generally satisfied; 4- Somewhat dissatisfied; 5-Very dissatisfied 11. The amount of time you spend together as a family? - Extremely satisfied; 2- Very satisfied; 3- Generally satisfied; 4- Somewhat dissatisfied; 5-Very dissatisfied F. Family leadership [Section to be asked in households where another person is listed as having a role in raising XXX in the Roster section [Raising] or the follow-up section. If there is nobody else, this section should be skipped.] For the following questions I want you to think about ALL THE PEOPLE that play a role in raising XXX, who may live in this house or not. 10. How often do you disagree on how to raise or discipline XXX in front of XXX? 1-Very often, 2-Often, 3- Sometimes, 4- Rarely, 5- Never 11. How often do you talk badly about these other people who help raise XXX in front of XXX? 1-Very often, 2-Often, 3- Sometimes, 4- Rarely, 5- Never 12. How often do you argue or fight in front of XXX? 1-Very often, 2-Often, 3- Sometimes, 4- Rarely, 5- Never 115 G. Parenting Now I would like to ask you some questions about parenting. For each question, I would like for you to tell me if you strongly agree, agree, neither agree nor disagree, disagree, or strongly disagree. 1. Children will get into trouble no matter what their parents do. 1-Strongly agree; 2-Agree, 3-Neither agree nor disagree, 4-Disagree, 5-Strongly disagree 2. The conditions in this community prevent me from keeping my children out of trouble. 1-Strongly agree; 2-Agree, 3-Neither agree nor disagree, 4-Disagree, 5-Strongly disagree 3. Parents can have a major influence on their kids. 1-Strongly agree; 2-Agree, 3-Neither agree nor disagree, 4-Disagree, 5-Strongly disagree 4. It is primarily my responsibility to keep my children out of trouble. 1-Strongly agree; 2-Agree, 3-Neither agree nor disagree, 4-Disagree, 5-Strongly disagree 5. In the last 12 months, I have improved as a parent. 1-Strongly agree; 2-Agree, 3-Neither agree nor disagree, 4-Disagree, 5-Strongly disagree 6. In the last 12 months, my influence on my children has increased. 1-Strongly agree; 2-Agree, 3-Neither agree nor disagree, 4-Disagree, 5-Strongly disagree H. Caregiver Presence [Module repeats if multiple study youth in the household. Names of youth should auto-populate into the question] In answering the following questions, I would like you to think about your experience over the last month. 1. In a typical seven-day week, how many days do you sit down to eat together with XXX? [synonyms for typical: normal, usual] Enter number from 0 to 7] 2. In a typical seven-day week, how many days do you help XXX with school work? Enter number from 0 to 7. -97 if youth is not enrolled in school] 3. In a typical seven-day week, how many days do you do something fun as a family with XXX, such as play a game, play sports, go to the beach, have a BBQ, and so forth? Enter number from 0 to 7] 4. In a typical seven-day week, how many days do you do something religious as a family with XXX such as go to church or temple, pray or read the scriptures together? Enter number from 0 to 7] 116 Now I would like to ask you about your relationship with XXX. For each question, I would like for you to tell me if it happens always, often, half the time, rarely, or never. 7. When XXXX goes out, he/she tells me where he/she is going either in-person or through a text, WhatsApp, or phone-call 1-Always, 2-Often, 3-Half the time, 4-Rarely, 5-Never 8. When XXXX is not at home or school, I know WHERE he/she is. 1-Always, 2-Often, 3-Half the time, 4-Rarely, 5-Never 9. When XXXX is not at home or school, I know WHO he/she is with. 1-Always, 2-Often, 3-Half the time, 4-Rarely, 5-Never 10. If XXXX does not let me know where he/she is going, he/she lets another adult in the household know. 1-Always, 2-Often, 3-Half the time, 4-Rarely, 5-Never 11. There are consequences if family rules are broken. 1-Always, 2-Often, 3-Half the time, 4-Rarely, 5-Never 12. When I have something important to say, XXXX listens to me. 1-Always, 2-Often, 3-Half the time, 4-Rarely, 5-Never 13. When XXXX has something important to say, I listen to him/her. 1-Always, 2-Often, 3-Half the time, 4-Rarely, 5-Never 14. XXXX and I are able to talk and solve our problems. 1-Always, 2-Often, 3-Half the time, 4-Rarely, 5-Never 15. When I’m angry, I may say mean things to XXXX. 1-Always, 2-Often, 3-Half the time, 4-Rarely, 5-Never 16. When I’m angry, I may push or hit XXXX. 1-Always, 2-Often, 3-Half the time, 4-Rarely, 5-Never 17. I have other adults I can count on to help me raise my child. 1-Always, 2-Often, 3-Half the time, 4-Rarely, 5-Never 18. I am proud of XXXX. 1-Always, 2-Often, 3-Half the time, 4-Rarely, 5-Never I. Youth behavior and relationships (from the YSET) Now, we would like to talk about XXXX’s behavior. We are going to ask you about a series of actions and if he/she has done any of these in the last 6 months. Please remember that your responses are confidential. Answering yes to any of the questions will not get anyone in any trouble. We know some of these questions are sensitive, but we are asking them to all parents of kids aged 10 and above. We hope you feel confident in answering these questions honestly. Please let us know at any time if you have any concerns. [If respondents demonstrate concern during this set of questions, remind them of the material in this introduction and that their responses are confidential. You might also mention: “when the researchers analyze the survey responses they only look at all the survey respondents together. For example, they are 117 interested in if 10% or 20% of youth committed these acts – not if a particular household answers yes or no.”] , In the last 6 months, has XXXX… [If the respondent answers “Yes” to any of the questions] How sure are you of this? 1. Inhaled (sniffed, smoked or drank) drugs or substances that make you “high”? 1-Yes, 2-No, -98 DK, -99 RA 1-Very sure, 2- Somewhat sure, 3- Not at all sure 2. Skipped classes at school without an excuse or permission? (skipped school = ditched or not turned up for school) 1-Yes, 2-No, -98 DK, -99 RA 1-Very sure, 2- Somewhat sure, 3- Not at all sure 3. Lied about his/her age to get into some place or to buy something? 1-Yes, 2-No, -98 DK, -99 RA 1-Very sure, 2- Somewhat sure, 3- Not at all sure 4. Purposely damaged or destroyed property that did not belong to him/her? 1-Yes, 2-No, -98 DK, -99 RA 1-Very sure, 2- Somewhat sure, 3- Not at all sure 5. Carried a hidden weapon for protection? 1-Yes, 2-No, -98 DK, -99 RA 1-Very sure, 2- Somewhat sure, 3- Not at all sure 6. Stolen or tried to steal something valuable? 1-Yes, 2-No, -98 DK, -99 RA 1-Very sure, 2- Somewhat sure, 3- Not at all sure 7. Broken into a building to steal something? 1-Yes, 2-No, -98 DK, -99 RA 1-Very sure, 2- Somewhat sure, 3- Not at all sure 8. Hit someone with the purpose of hurting him/her? (clarify: intending to hurt someone, not just playing) 1-Yes, 2-No, -98 DK, -99 RA 1-Very sure, 2- Somewhat sure, 3- Not at all sure 9. Attacked someone with a weapon? (attacked = jumped with gun, knife or other weapon) 1-Yes, 2-No, -98 DK, -99 RA 1-Very sure, 2- Somewhat sure, 3- Not at all sure 10. Used a weapon or force to get money or things from people? 1-Yes, 2-No, -98 DK, -99 RA 1-Very sure, 2- Somewhat sure, 3- Not at all sure 11. Been involved in group fights? 1-Yes, 2-No, -98 DK, -99 RA 1-Very sure, 2- Somewhat sure, 3- Not at all sure 12. Been involved in gang fights? 1-Yes, 2-No, -98 DK, -99 RA 1-Very sure, 2- Somewhat sure, 3- Not at all sure 13. Sold marijuana and illegal drugs? 1-Yes, 2-No, -98 DK, -99 RA 1-Very sure, 2- Somewhat sure, 3- Not at all sure 14. Thinking about whether this has ever happened, at any point in time… 1-Yes, 2-No 118 Have you ever discovered drugs in XXX’s room or belongings? 15. [if yes] In the last 6 months? 1-Yes, 2-No [For the enumerator to answer] 16. Was there any indication that the respondent did not answer the questions honestly? (Below we will ask you to explain any YES responses) 1-Yes, 2-No J. Other family members Now, I would like to ask you a few questions about the household members in general. Some of these questions are somewhat sensitive; however, please remember that your answers are confidential. [If respondents demonstrate concern during this set of questions, remind them of the material in this introduction and that their responses are confidential. You might also mention: “when the researchers analyze the survey responses they only look at all the survey respondents together. For example, they are interested in if 10% or 20% of households exhibit these traits – not if a particular household answers yes or no.”] , 1. In the last year, has drug use among any household members created any problems in this household? 1-Yes, 2-No (skip to 46) 2. If so, which drugs? a. Alcohol b. Marijuana c. Other drugs 1-Yes, 2-No 3. Has anybody in this household been a victim of a crime in the last 12 months? 1-Yes, 2-No 4. If you are comfortable doing so, could you please tell us the type of crime or crimes. Please select all that apply. 1-Unamed robbery with no assault or physical threat 2 – Unarmed robbery with assault of physical threats 3 - Armed robbery 4 – Assault but not robbery 5- Rape or sexual assault 6-Domestic violence 7-Kidnapping 8-Vandalism 9- Burglary of your home while no one was home 10-Burglary of your home while you were home 119 11-Extortion/blackmail [define extortion: obtain money through force or threats] 12-Bullying 13-Other -98 Do not know -99 Refused to answer 5. Has anybody in this household been charged with a crime in the last 10 years? We do not need to know who specifically. 1-Yes, 2-No 6. If you are comfortable doing so, could you please tell us the type of crime or crimes the household member has been charged with. Please select all that apply. 1-Unamed robbery with no assault or physical threat 2 – Unarmed robbery with assault of physical threats 3 - Armed robbery 4 – Assault but not robbery 5- Rape or sexual assault 6-Domestic violence 7-Kidnapping 8-Vandalism 9- Burglary of your home while no one was home 10-Burglary of your home while you were home 11-Extortion/blackmail [define extortion: obtain money through force or threats] 12-Bullying 13-Other -98 Do not know -99 Refused to answer 7. Has any member of this household ever been a member of a gang? By gang, we mean a group of people who engage in violent and/or criminal acts together. Again, we do not need to know who. 1-Yes, 2-No K. Community (from the CFYR Community Survey) Now, I would like to ask you a few questions about your community. 1. How long have you lived in this community? 1-Less than 6 months, 2-Between 6 months and a year, 3-Between 1-2 years; 4-Between 2-5 years; 5-Between 5-10 years, 6-More than 10 years For the next question, I am going to read you a statement, and I would like for you to answer on a 1 to 7 scale where 1 1 Strongly satisfied, 2,3,4,5,6, 7 Strongly dissatisfied; -98 DK; -99 RA 120 means strongly AGREE and 7 means strongly DISAGREE 2. I have a good bond with others in this neighborhood. For the next two questions I am going to ask you about how satisfied you are with police services provided in your community in the last 12 months. Please rate your level of satisfaction on a 1-7 scale where 1 means strongly SATISFIED and 7 means strongly DISSATISFIED. 3. Police responsiveness to incidents of crime and violence. 1 Strongly satisfied, 2,3,4,5,6, 7 Strongly dissatisfied; -98 DK; -99 RA 4. Approachableness of the police if you require assistance from them. [Clarification if needed: approachableness means that the police make it clear that you can come to them if you need assistance] 1 Strongly satisfied, 2,3,4,5,6, 7 Strongly dissatisfied; -98 DK; -99 RA L. Household characteristics Now, I would like to ask you a few questions about your home. These are questions that are asked on the census. We ask them here to see how our study households differ from the rest of the country. 1. Television 1-Yes, 2-No 2. Computer 1-Yes, 2-No 3. Internet access/connection 1-Yes, 2-No 4. Vehicle (private car, bus, or truck) 1-Yes, 2-No 5. Refrigerator/freezer 1-Yes, 2-No 6. Washing machine 1-Yes, 2-No 7. Stove (gas/electric/solar) 1-Yes, 2-No 8. Microwave 1-Yes, 2-No 9. Telephone (landline) 1-Yes, 2-No 10. Mobile/cellular phone 1-Yes, 2-No 11. Water pump 1-Yes, 2-No 12. Air condition unit 1-Yes, 2-No 13. Water heater 1-Yes, 2-No 121 14. Cable TV/satellite 1-Yes, 2-No 15. Flush toilet 1-Yes, 2-No 16. Boat (with engine or paddle) 1-Yes, 2-No 17. In thinking about the families in XXXXX, How does your household’s income level compare with other households in XXXX? Would you say that you are: 1-Much wealthier, 2-Wealthier, 3-Average, 4- Lower income, 5-Much lower income M. Contact information That basically concludes the survey. As we noted above, we would like to contact you again in one year. We have your basic contact information, but we sometimes have a hard time getting a hold of people after one year, so if you could provide us with some additional contact information it would be very much appreciated. 1. Do you live in more than one place? 1-Yes, 2-No 2. Confirm Address 3. [if 90=yes] Alternative address 4. [if 90=yes] Landline phone number at other address 5. Other parent or guardian cellphone 6. Other family member cellphone 7. Name of other family member 8. Email address Thank you very much for your participation!!! END OF SURVEY [Section N: Questions for enumerator only] 1. Were you able to see any drugs in the house? 1-Yes, 2-No 2. If yes, which drugs? (multiple answer possible) a. Alcohol b. Marijuana c. Other drugs (specify _______) 3. Were you able to smell any drugs in the house? 1-Yes, 2-No 122 4. If yes, which drugs? (multiple answer possible) a. Alcohol b. Marijuana c. Other drugs (specify _______) 5. If yes to question 50 (last question in section I), please elaborate on what you noticed about the respondent that indicates he/she may not have been truthful in the section on XXXX’s behavior. 1- Respondent paused before responding; 2- uncomfortable body language including scratching their head, shrugging, or slouching with arms folded; 3-prior dishonesty; 4- indicating that the enumerator should ask another caregiver about that question; -96 other ________________________ N. End matter (cont) 6. Enter the enumerator ID Numeric 7. Supervisor ID 8. Country 1-St. Lucia, 2-SKN, 3-Guyana, 9. [if Country=1] Community 1-Anse-La-Raye, 2-Castries City, 3-Dennery, 4-Soufriere, 5-Vieux-Fort 10. [if Country=2] Community 1-McKnight/Newtown/The Village, 2-Upper and Lower Cayon, 3-Sandy Point, 4-Rawlins, 5-Bath Village 11. [if Country=3] Community 1-Sophia, 2-East La Penitence, 3-East Ruimveldt, 4-Lodge, 5-Corriverton 12. Enumerator assessment of the survey 1- No concerns, 2-Some concerns but good data quality, 3-Some concerns that could affect data quality 13. [If 2,3] Concerns (select all that apply) 1-Respondent rushing, 2-Respondent potentially not answering honestly, 3-Poor survey conditions, 4-Other individuals listening 14. [If 2,3] Explain any concerns and if they pertain to specific questions (Open ended) 15. Supervisor review 1-No concerns, 2-Corrective action required 16. [If 2] Explain any concerns and corrective action required 17. [If 2] Corrective action completed 1-Yes, 2-No 123 COMMUNITY SURVEY 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 ANNEX III: REGRESSION RESULTS Table 27: Odds ratios with St. Lucia, females, in school, ages 10 to 14, and ethnicity-Black/Afro￾Guyanese as the base GUY SKN Eligibility 0.994 1.042 Weak parental monitoring 0.244*** 0.621*** Critical life events 1.613*** 1.668*** Impulsive risk taking 1.200** 0.999 Guilt neutralization 1.394*** 0.456*** Negative peer influence 0.565*** 2.013*** Peer delinquency 1.286*** 0.786** Family antisocial influence 1.020 1.058 Self-reported delinquency 0.839** 1.138 Risky group behaviors 1.459*** 1.252* Note: significance is * p<.1, ** p<.05, ***p<.01. Table 28: Odds ratios with St. Kitts and Nevis, females, in school, ages 10 to 14, and ethnicity￾Black/Afro-Guyanese as the base GUY SLU Eligibility 0.954 0.960 Weak parental monitoring 0.392*** 1.610*** Critical life events 0.967 0.600*** Impulsive risk taking 1.201** 1.001 Guilt neutralization 3.055*** 2.191*** Negative peer influence 0.281*** 0.497*** Peer delinquency 1.636*** 1.272** Family antisocial influence 0.964 0.946 Self-reported delinquency 0.737*** 0.879 Risky group behaviors 1.166 0.799* Note: significance is * p<.1, ** p<.05, ***p<.01. THRESHOLDS In order to understand whether differences in average scores by country stem from different thresholds or from real country differences, the evaluation team set the same threshold for all three countries on each of the scales. The next two tables include the results. 157 Table 29: Comparing shares of risk on each YSET scale by country, by setting the risk thresholds to be the same Scale Mean for all countries together GUY mean SLU mean SKN mean Weak Parental Monitoring (B) 0.19 0.10 0.29 0.20 B_flag_1 0.27 0.22 0.29 0.33 B_flag_2 0.17 0.13 0.19 0.20 B_flag_3 0.13 0.10 0.15 0.15 Critical Life Events (C) 0.24 0.25 0.20 0.29 C_flag_1 0.45 0.45 0.41 0.54 C_flag_2 0.24 0.25 0.20 0.29 Impulsive risk taking 0.35 0.36 0.34 0.34 DE_flag_1 0.37 0.36 0.34 0.47 DE_flag_2 0.26 0.25 0.24 0.34 Guilt Neutralization 0.40 0.45 0.41 0.24 F_flag_1 0.45 0.45 0.41 0.54 F_flag_2 0.21 0.22 0.18 0.24 Negative Peer Influence (G) 0.18 0.11 0.19 0.32 G_flag_1 0.23 0.16 0.26 0.32 G_flag_2 0.13 0.09 0.14 0.18 Threshold calculation: For scale B, for example, the original Guyana threshold (labeled B_flag_1), the original St. Lucia threshold, (B_flag_2), and the original SKN threshold (B_flag_3) are used uniformly for all the countries. The other scales had only two different thresholds so only two alterative models are estimated for each of them. Country differences remained the same in terms of statistical significance, but the magnitude changed in some cases for the B, C and G scales. For scales DE and F, the differences remained similar between Guyana and St. Lucia, but are reversed for Guyana and SKN (with the same thresholds, SKN youth are now at higher risk than Guyanese youth in 3 out of 4 cases). Those reversals are noted in bold typeface. 158 Table 30: Logistic regressions comparing shares of risk on each YSET scale by country, by setting the risk thresholds to be the same VARIABLE (1) (2) (3) (4) (5) (6) (7) (8) (9) (10) (11) (12) (13) (14) (15) (16) B_flag_1 B_flag_2 B_flag_3 C_flag_1 C_flag_2 DEflag_1 DEflag_2 F_flag_1 F_flag_2 G_flag_1 G_flag_2 2.country 4.106*** 1.436*** 1.489*** 1.613*** 0.620*** 0.681*** 0.620*** 0.834** 0.826** 0.919 0.717*** 0.713*** 0.706*** 1.771*** 1.815*** 1.708*** (0.446) (0.125) (0.151) (0.187) (0.0527) (0.0499) (0.0527) (0.0627) (0.0621) (0.0767) (0.0530) (0.0526) (0.0624) (0.185) (0.166) (0.198) 3.country 2.551*** 1.833*** 1.679*** 1.731*** 1.034 1.179* 1.034 0.832** 1.428*** 1.501*** 0.327*** 1.212** 0.997 3.565*** 2.385*** 2.132*** (0.326) (0.186) (0.200) (0.235) (0.100) (0.103) (0.100) (0.0758) (0.126) (0.143) (0.0317) (0.107) (0.102) (0.401) (0.248) (0.278) Gender 3.370*** 3.059*** 3.186*** 3.513*** 1.331*** 1.185*** 1.331*** 1.126** 1.176*** 1.087 1.300*** 1.299*** 1.177** 1.296*** 1.291*** 1.323*** (0.269) (0.210) (0.263) (0.338) (0.0896) (0.0681) (0.0896) (0.0669) (0.0692) (0.0703) (0.0770) (0.0751) (0.0822) (0.0972) (0.0881) (0.113) Age15 to 17 1.773*** 1.736*** 1.615*** 1.515*** 1.824*** 1.612*** 1.824*** 1.334*** 1.295*** 1.396*** 1.781*** 1.791*** 1.761*** 1.618*** 1.555*** 1.615*** (0.146) (0.127) (0.137) (0.145) (0.133) (0.104) (0.133) (0.0886) (0.0854) (0.0997) (0.118) (0.116) (0.133) (0.131) (0.116) (0.148) Out of School 2.104*** 2.088*** 2.236*** 2.525*** 1.655*** 1.277* 1.655*** 1.521*** 1.533*** 1.524*** 1.677*** 1.688*** 1.672*** 1.279 1.373** 1.316 (0.317) (0.277) (0.322) (0.394) (0.214) (0.161) (0.214) (0.189) (0.191) (0.198) (0.215) (0.218) (0.220) (0.197) (0.193) (0.222) 1.ethnicity 1.589* 1.340 1.538* 1.587* 0.678* 0.769 0.678* 0.985 0.979 1.053 0.774 0.769 0.919 1.099 1.025 1.092 (0.425) (0.283) (0.367) (0.427) (0.146) (0.139) (0.146) (0.181) (0.180) (0.212) (0.141) (0.140) (0.195) (0.295) (0.244) (0.327) 2.ethnicity 0.652** 0.704*** 0.648*** 0.651** 0.500*** 0.522*** 0.500*** 0.655*** 0.646*** 0.718*** 0.597*** 0.593*** 0.608*** 0.720** 0.702** 0.806 (0.121) (0.0929) (0.106) (0.124) (0.0641) (0.0547) (0.0641) (0.0712) (0.0701) (0.0876) (0.0627) (0.0620) (0.0792) (0.120) (0.101) (0.146) 3.ethnicity 1.074 0.894 0.936 0.981 0.852 0.880 0.852 1.017 0.986 1.063 0.799* 0.774** 0.817 1.061 0.973 1.020 (0.200) (0.134) (0.166) (0.199) (0.118) (0.105) (0.118) (0.124) (0.120) (0.142) (0.0984) (0.0937) (0.120) (0.179) (0.149) (0.197) Note: significance is * p<.1, ** p<.05, ***p<.01. Weak Parental Monitoring (Scale B) covers variables (1)-(4) | Critical Life Events (Scale C) covers variables (5)-(7) Impulsive Risk Taking (Scale DE) covers variables (8)-(10) | Guilt Neutralization (Scale F) cover variables (11)-(13) Negative Peer Influence (Scale G) covers variables (14)-(16) 159 ANNEX IV: SUPPORTING TABLES AND FIGURES MEASURES OF INCOME An asset index was calculated from the caregiver survey through two methods, one based on a simple count of a dichotomic measure of how many household goods or services are found in the home, and a factor index generated through a polychoric analysis. It turns out that both methods generate a relatively similar rank ordering of the socioeconomic status of households in the three countries. The variables included are whether the household has a flush toilet, television, computer, internet connection, refrigerator, washing machine, stove, microwave, telephone, cellular phone, water pump, hair dryer, heater, satellite or cable television, vehicle and a boat. Many of these household goods are widespread, while others are rare within the caregiver sample. Only three households had all 16 items, and only two of them reported not having any of them at all. The median household has eight of these items. In the polychoric index four of these items had to be dropped (boat, vehicle, heater, hair dryer) because their infrequent occurrence hindered the calculation of a correlation matrix. Figure 55 shows a scatterplot suggesting that the index calculated either with the count of items of the polychoric method yields very similar rankings across households (the correlation is 0.94). Figure 55: Household socioeconomic ranking according to assets reported in caregiver survey (n=947) 160 This index cannot be readily compared with the socioeconomic status of non-eligible households in the countries, but it provides a metric that may rank households at the baseline according to the degree of deprivation and disadvantage that youth may experience relative to other youth. This ranking is relatively objective (since a home either has or does not have an asset, and it is unlikely that respondents will misreport their existence) and very different from subjective measures of well-being. The caregiver survey in fact included a question regarding whether the respondent believed his or her income to be higher or lower than other members of their community. While the respondents that reported much lower income also showed a much lower number of assets (5.7), and those with lower income also reported slightly lower assets than the median (7.4), it turns out that those saying they had average income reported nine assets on average, while those that perceived themselves to be wealthier or much wealthier than their neighbors reported 8.1 and 8.4 average assets. Hence the subjective measure of income is not a very reliable indicator, particularly for those reporting higher incomes. RELIGION The following figures provide a breakdown in reported religious affiliation among caregivers across the three countries. Figure 56: Share of caregiver self-reported religious affiliation, in St. Lucia (n=413) Note: no respondents selected the options Hindu, Methodist or Muslim. 161 Figure 57: Share of caregiver self-reported religious affiliation, in St. Kitts and Nevis (n=155) Note: no respondents selected the options of Muslim or Evangelical. Figure 58: Share of caregiver self-reported religious affiliation, in Guyana (n=377) 162 Figure 59: Caregiver reported crimes that household members have been charged with in the last ten years (n=76) 163 Figure 60: Caregiver reported household crime victimization in the last 12 months (n=42) 164 Figure 61: Shares of matches and mismatches of youth behavior, by type of primary caregiver 165 ANNEX V: INDEX CONSTRUCTION Principal component analysis (PCA) is a data reduction method: from a group of correlated variables, it produces a smaller set of components that give a good representation of the main characteristics of the original data. These components are a linear combination of the original variables that represent the maximum possible variance between them but are uncorrelated to each other (whereas there may be a high degree of correlation among the original variables). Described mathematically, given an original set of n correlated variables, PCA produces uncorrelated, orthogonal components, where each component is a linear combination of the initial variables. For example, from a set of variables X1 through to Xn, 1st Principal Component = a11X1 + a12X2 + … + a1nXn 2nd Principal Component= a21X1 + a22X2 + … + a2nXn … Zth Principal Component = aZ1X1 + aZ2X2 + … + aZnXn where aZn is the coefficient, or weight, for the Zth principal component and the nth variable. Each component has both magnitude and direction The first component accounts for maximum possible variance in the data. Subsequent components account for information not captured by the first and are uncorrelated with the first component (and each other), with diminishing explanatory power of variance among the original variables. Therefore, the first component often can reasonably be used on its own as an index score that gives information on the underlying variables. PCA does not assume the existence of a group of common factors driving the variation in the data (whereas factor analysis does). It does, however, assume that the underlying data is normally distributed and calculates associations between variables using Pearson correlations. This means that PCA on its own is best suited for continuous data. Conversely, the data from which the indices in the Community Context section of this report are built are ordinal, categorical data. PCA with polychoric correlations solves this problem by assuming that each ordinal variable is a representation of an underlying, inferred “latent variable” that is both continuous and normally-distributed, and produces principal components under this assumption. When polychoric correlations are used in variables with many categories, the method assumes that a discrete amount is actually a continuous variable. In the case of the Likert scales or categorical dichotomic variables, the assumption is more likely to follow when the realizations are not particularly skewed or lopsided in their distribution. The table in this Annex shows the variables from the Community Survey from which each index in the Community Context section was formed, with the factor loadings of the first polychoric principal component for each of them. The table shows the Eigenvalue of the first component (by convention, only 166 components with Eigenvalues higher than 1 are usually retained). The Eigenvalue provides a measure of the variance explained by the first component compared to the variance that would be explained by a single variable in the estimation. The “variance explained” column gives the actual percentage of variance explained by the component. The reported loadings are correlations, providing a sense of how closely associated any given variable is with the estimated factor: Table 31: PCA with polychoric correlations Index Name 1st Principal Component N Eigenvalue Variance explained # Variable Factor Loadings PERCEPTION OF CRIME PREVALENCE crimeprev 3336 3.07066 0.4387 B20 vand 0.3849 B21 drugs 0.3063 B22 robbed 0.4576 B23 sex_assault 0.3842 B24 (recode) rc_lev_crime 0.2348 B25 safe 0.4083 B26 safe_dark 0.4234 SOCIAL CAPITAL I soccap1 4084 1.30642 0.4355 B1 sblive_n_comm 0.3023 B2 (recode) rc_com_trust 0.6737 B4 fight 0.6744 SOCIAL CAPITAL II soccap2 331 3.14447 0.5241 B36 needs3 0.4673 B37 belong 0.4726 B38 say 0.2846 B39 ninflu 0.2773 B40 connect 0.4856 B41 bond 0.4057 ATTITUDES TOWARDS THE LAW lawatt 3998 1.59103 0.2652 B5 (recode) rc_law 0.5118 B6 (recode) rc_lawbreak 0.4629 B7 lawobey -0.0085 B8 (recode) rc_lawtrust 0.5657 B9 (recode) rc_comcrome 0.3946 B10 (recode) rc_bendlaw 0.2190 VIEW OF THE POLICE police 3788 2.13604 0.534 B11 polresp 0.5595 B13 ppatrols 0.5303 B14 appolice 0.5748 B27 (recode) rc_police_call 0.2746 RESOURCES FOR YOUTH resources 2394 2.58237 0.5165 B12 prog_unenroll 0.4567 B15 scprog 0.4188 B16 rescare 0.3546 B18 prosonprog 0.4894 B19 vicsupport 0.5007 SELF-EFFICACY selfeff 4494 3.30377 0.4720 D1 (recode) rc_control_life 0.3938 D2 (recode) rc_solut_prob 0.4209 D3 (recode) rc_change_life 0.4500 D4 (recode) rc_feel_helpless 0.4076 D5 (recode) rc_fate_other 0.3659 D6 determ_life 0.2827 D7 determ_comp 0.2919 167 ANNEX VI: GENDER DISAGGREGATIONS YSET SCALE B: WEAK PARENTAL MONITORING Figure 62: Responses to select questions on parental monitoring, by gender Figure 63: Responses to select questions on parental monitoring, by gender 168 SCALE C: CRITICAL LIFE EVENTS Figure 64: Responses to questions on critical life events, by gender 169 SCALE DE: IMPULSIVE RISK TAKING Figure 65: Responses to questions on impulsive risk taking, by gender Figure 66: Responses to questions on impulsive risk taking, by gender 170 SCALE F: GUILT NEUTRALIZATION Figure 67: Responses on select questions on guilt neutralization, by gender Figure 68: Responses on select questions on guilt neutralization, by gender 171 SCALE G: NEGATIVE PEER INFLUENCE Figure 69: Responses on negative peer influence, by gender Figure 70: Responses on negative peer influence, by gender 172 SCALE H: PEER DELIQUENCY Figure 71: Responses to questions on peer delinquency, by gender 173 SCALE T: FAMILY ANTI-SOCIAL INFLUENCE Figure 72: Responses on family antisocial influence questions, by gender Figure 73: Responses on family antisocial influence question, by gender 174 SCALE IJ: SELF-REPORTED DELIQUENCY Figure 74: Responses on self-reported delinquency, by gender 175 SCALE FRD: GROUP RISKY BEHAVIORS Figure 75: Responses on risky group behavior, by gender 176 CAREGIVER SURVEY Figure 76: Responses to questions on family leadership, by caregiver’s gender n=104 Male, 767 Female Figure 77: Responses to questions on parenting, by caregiver’s gender 177 Figure 78: Responses to questions on parenting, by caregiver’s gender Figure 79: Number of days in a typical week caregivers report select behavior, by caregiver’s gender 178 Figure 80: Number of days in a typical week caregivers report select behavior, by caregiver’s gender Figure 81: Responses to select questions on caregiver presence, by caregiver’s gender 179 Figure 82: Responses to select questions on caregiver presence, by caregiver’s gender Figure 83: Responses to select questions on caregiver presence, by caregiver’s gender 180 Figure 84: Responses to select questions on caregiver presence, by caregiver’s gender Figure 85: Responses to select questions on caregiver presence, by caregiver’s gender 181 Figure 86: Responses to select questions on household drug problems, by caregiver’s gender Figure 87: Responses to question on crime in the last 12 months, by caregiver’s gender 182 Figure 88: Responses to question on crime in the last 12 months, by caregiver’s gender 183 Figure 89: Responses to question on crime in the last 12 months, by caregiver’s gender 184 Figure 90: Responses to question on crime in the last 12 months, by caregiver’s gender 185 Figure 91: Responses to question on gang membership, by caregiver’s gender Figure 92: Responses to select question on community, by caregiver’s gender 186 Figure 93: Responses to select question on community, by caregiver’s gender Figure 94: Responses to questions on police perception, by caregiver’s gender 187 REFERENCES Abt, Thomas and Christopher Winship. 2016. What Works in Reducing Community Violence: A Meta￾Review and Field-Study for the Northern Triangle. Berk-Seligson, Susan, Diana Orcés, Georgina Pizzolitto, Mitchell A. Seligson and Carole J. Wilson. 2014. Impact Evaluation of USAID’s Community-Based Crime and Violence Prevention Approach in Central America: Regional Report for El Salvador, Guatemala, Honduras and Panama. Cahill, M., Jannetta, J. Tiry, E., Lowry, S., Becker-Cohen, M. Paddock, E., Serakos, M., Park, L., Hennigan, K. 2015. Evaluation of the Los Angeles Gang Reduction and Youth Development. Program Year 4 Evaluation Report. Washington: Urban Institute. https://www.urban.org/sites/default/files/publication/77956/2000622-Evaluation-of-the-Los￾Angeles-Gang-Reduction-and-Youth-Development-Program-Year-4-Evaluation-Report.pdf Caribbean Development Bank. 2018. 2017 Caribbean Economic Review and 2018 Economic Outlook. Barbados: Caribbean Development Bank. https://issuu.com/caribank/docs/cdb_2017_caribbean_economic_review_?e=21431045/5817262 5 CARICOM Commission on Youth Development. 2010. Eye on the Future: Investing in Youth Now for Tomorrow’s Community. CFYR. 2017. Family Counselor Training Manual – Draft. Family Counselor Training - St. Lucia. November 27 – December 1, 2017. Chioda, Laura. 2017. Stop the Violence in Latin America: A Look at Prevention From Cradle to Adulthood. Latin American Development Forum, World Bank Group. https://openknowledge.worldbank.org/bitstream/handle/10986/25920/210664ov.pdf Creative Associates International. 2015. Sistematización de la Experiencia Piloto del Programa de Prevención y Oportunidades: Trabajando para el Entendimiento Familiar (PROPONTE). Hennigan, K. M., Maxson, C., Sloane, D. C., Kolnick, K. A., and Vindel, F. (2013). “Identifying high-risk youth for secondary gang prevention,” Journal of Crime and Justice, 37 (1): 104–28. Henggeler, Scott W. and Ashli J. Sheidow. 2012. “Empirically Supported Family-Based Treatments for Conduct Disorder and Delinquency in Adolescents.” Journal of Marital Family Theory 38(1). Jaitman, Laura and Roberto Guerrero Compeán. 2015. Closing Knowledge Gaps: Toward Evidence-Based Crime Prevention Policies in Latin America and the Caribbean. Inter-American Development Bank. Katz, Charles and Lidia Nuño. 2016. Prevalence and Patterns of Troublesome Youth Groups in the Caribbean: Implications for Policy and Practice. Final Report. USAID. 188 Littell, J.H., M. Popa, and B. Forsythe. 2005. “Multisystemic therapy for social, emotional, and behavioral problems in youth aged 10-17.” Cochrane Database of Systematic Reviews 4: 1–51. Lösel, F. and A. Beelmann. 2003. “Effects of child skills training in preventing antisocial behavior: a systematic review of randomized evaluations.” Annals of the American Academy of Political and Social Science 587, Assessing Systematic Evidence in Crime and Justice: Methodological Concerns and Empirical Outcomes: 84–109. Social Impact. 2013. Eastern and Southern Caribbean Youth Assessment (ESCYA). Final Report. September. Sutton, Heather and Inder Ruprah. 2017. Restoring Paradise in the Caribbean: Combatting Violence with Numbers. Inter-American Development Bank. Thornberry, T., B. Kearley, D. C. Gottfredson, M. P. Slothower, D. N. Devlin, and J. J. Fader. 2018. “Reducing Crime Among Youth at Risk for Gang Involvement: A Randomized Trial.” Criminology & Public Policy 17(4). UNICEF. 2017. Innocenti Research Briefs on Adolescent Wellbeing. Link: https://www.unicef￾irc.org/adolescent-research-methods/ U.S. Agency for International Development 1300 Pennsylvania Avenue NW Washington, DC 20523