August 2019 An Evaluation of Proponte Más: A Honduran Secondary Prevention Program Technical Report AN EVALUATION OF PROPONTE MÁS: A HONDURAN SECONDARY PREVENTION PROGRAM TECHNICAL REPORT USAID PROJECT: Award No. AID 522-TO-16-00001 AUTHORS: Charles M. Katz, PhD Hyunjung Cheon, M.S. Scott H. Decker, PhD CENTER FOR VIOLENCE PREVENTION AND COMMUNITY SAFETY ARIZONA STATE UNIVERSITY Phoenix, Arizona, USA August 2019 ACKNOWLEDGEMENTS: This document was made possible through the generous support of the people of the United States of America through the United States Agency for International Development under Award No. AID 522-TO-16-00001. The contents of this document are the sole responsibility of the authors and do not necessarily reflect the views of the United States Government, Proponte Más or ASU. The authors would like to thank Robyn Braverman and Guillermo Céspedes for their leadership and support throughout the project and Axel Rivera for all of his assistance on matters related to data collection and interpretation. We would also like to thank Eric Hedberg for his methodological and statistical advice and Cher Stuewe-Portnoff for her copy editing of the manuscript. UNA EVALUACIÓN DE PROPONTE MÁS: UN PROGRAMA HONDUREÑO DE PREVENCIÓN SECUNDARIA 1 CONTENTS INTRODUCTION....................................................................................................................................................... 3 RISK FACTOR APPROACHES TO UNDERSTANDING PROBLEM BEHAVIOR.................................... 4 THE USE OF FAMILY-BASED INTERVENTIONS TO RESPOND TO PROBLEM BEHAVIOR............................................................................................................................................................... 5 THE NEED FOR EVIDENCE-BASED PROGRAMMING IN CENTRAL AMERICA AND THE CARIBBEAN ....................................................................................................................................... 6 THE PRESENT STUDY............................................................................................................................................. 7 THE SETTING ........................................................................................................................................................ 7 ORIGINS OF PROPONTE MÁS......................................................................................................................... 8 PROGRAMMATIC APPROACH.......................................................................................................................... 9 TECHNICAL DESIGN................................................................................................................................... 9 THE IMC DIAGNOSTIC TOOL.................................................................................................................. 10 DATA AND METHODS........................................................................................................................................ 15 DATA.................................................................................................................................................................. 18 MEASURES............................................................................................................................................................... 19 PROCESS MEASURES.................................................................................................................................... 19 OUTCOME MEASURES ............................................................................................................................... 19 INDEPENDENT VARIABLE.......................................................................................................................... 24 CONTROL VARIABLES................................................................................................................................. 24 ANALYSIS................................................................................................................................................................. 24 POWER ANALYSIS......................................................................................................................................... 24 ANALYSIS PLAN............................................................................................................................................. 24 PROCESS EVALUATION RESULTS..................................................................................................................... 25 REFERRAL................................................................................................................................................................ 25 ELIGIBILITY ............................................................................................................................................................. 26 RETENTION........................................................................................................................................................... 27 DOSAGE .................................................................................................................................................................. 28 IMPACT EVALUATION RESULTS........................................................................................................................ 29 PART 1: CHANGE IN RISK STATUS ................................................................................................................. 29 CHANGE IN PROGRAM ELIGIBILITY ..................................................................................................... 29 CHANGE IN DIAGNOSTIC SCALE SCORES........................................................................................ 29 PART 2: DIRECT EFFECTS................................................................................................................................... 31 OUTCOMES FOR CHANGE IN FAMILY ADAPTABILITY AND COHESION............................... 31 OUTCOMES FOR CHANGES IN REVISED IMC RISK AND PROTECTIVE FACTOR SCORES .......................................................................................................................................... 32 OUTCOMES FOR CHANGES IN DELINQUENCY............................................................................. 36 INFORME TÉCNICO 2 PART 3: INDIRECT EFFECTS .............................................................................................................................. 38 CHANGE IN OVERALL RISK FACTOR SCORE AS A FUNCTION OF CHANGE IN FAMILY ADAPTABILITY AND COHESION................................................................... 39 CHANGE IN DELINQUENCY AS A FUNCTION OF CHANGE IN RISK AND PROTECTIVE FACTORS............................................................................................................................... 39 PART 4: SPLIT-SAMPLE ANALYSES..................................................................................................................... 41 BY SEX: FEMALE VS. MALE........................................................................................................................... 41 BY AGE: 12 YEARS OLD AND YOUNGER VS. 13 YEARS OLD AND OLDER................................ 46 BY GANG MEMBERSHIP: NON-GANG VS. GANG ............................................................................. 51 CONCLUSIONS.......................................................................................................................................................... 55 REFERENCES ............................................................................................................................................................... 58 APPENDIX A................................................................................................................................................................ 63 APPENDIX B: KEY TERMS ..................................................................................................................................... 71 UNA EVALUACIÓN DE PROPONTE MÁS: UN PROGRAMA HONDUREÑO DE PREVENCIÓN SECUNDARIA 3 INTRODUCTION The use of risk factors coincides closely with the application of public health models to the understanding of a wide variety of forms of behavior. This has become a versatile conceptual platform, in no small part due to its direct applicability to policy and program questions and the emergence of what is known as “prevention science.” The versatility of risk factor approaches can be demonstrated in a number of ways. First, the range of behaviors that such a methodology has been applied to is robust. Diseases such as heart attacks, strokes, diabetes and hypertension can be understood and responded to with risk factor approaches. In addition to health-related issues, concerns about achievement, whether in school, employment sectors or relationships, can be understood with the application of risk factors. Of particular value for the current evaluation, a risk factor approach can be applied to the understanding of behaviors with deleterious consequences–for instance, drug and alcohol abuse, risky sexual behaviors, and involvement in violence, crime and delinquency. Indeed, risk factor approaches to involvement in crime and delinquency have proliferated across the past three decades, and they now are a major theoretical and policy orientation in the understanding of violence, gangs, property crime and other forms of illegal and antisocial behaviors (Maguire, Wells, and Katz, 2011). In addition to the vitality of the risk factor approach, as evidenced by its widespread substantive focus, the use of this model is not linked with a specific country or continent. Examples of research, policy or practice involving risk factors can be found in North America (the United States and Canada), Europe and the United Kingdom (England and Scotland, Belgium, Cyprus, Germany, Netherlands, Norway), Australia, and Central and South America (Brazil, Honduras, El Salvador, Colombia). Within the US, risk factor approaches have been used in tribal groups, at the city, county and state levels, within detention and penal institutions, and in programs that serve girls, children of incarcerated parents, and pregnant teens. The Center for Disease Control and Department of Justice (Ritter, Simon, and Mahendra, 2014), as well as the Substance Abuse and Mental Health Services Administration have applied the risk factor model to the multiple complex issues they each deal with. The approach is meant to articulate well with current best practices in re￾entry and prevention and intervention, and it is used consistently with “risk/needs/responsivity” approaches to dealing with delinquents, offenders, and inmates returning from prison. The current evaluation has been undertaken to assess the impact of Proponte Más’ family based intervention on youth and families (i.e., Result 1), a promising intervention designed to deal with crime and delinquency by addressing prominent risk factors for problem behavior. Proponte Más was in part developed based on and inspired by the Los Angeles’ Gang Reduction Youth Development model (GRYD), an evidence-based risk reduction program. The comprehensive nature of GRYD made it a particularly strong candidate for replication in part or whole. Note that this sort of programming is not common in Latin American settings; to our knowledge, ours is the first assessment of the impact of an intervention program focused on risk factor reduction in Central America. In fact, although many policymakers and researchers have emphasized the scope and magnitude of the violence problem in Central America, little research has been conducted on what may effectively address the problem in the region. Several recent reports have reviewed the literature on the state of research and evaluation pertaining to violence and the response to violence in Central America, but these reports have essentially just shed light on how very few high-quality evaluations of violence reduction programs exist (Abt and Winship, 2016; Jaitman and Guerrero Compeán, 2015). Recent literature reviews on risk factor based programs reported that there have been no systematic reviews or meta-analyses of evaluations of violence reduction programs in the region; they concluded that this is the result of very little experimental or high quality quasi-experimental research on violence reduction having been conducted in Central America (Abt and Winship, 2016). Collectively, INFORME TÉCNICO 4 these reports (Abt and Winship, 2016; Jaitman and Guerrero Compeán, 2015; Santiso et al., 2017; Muggah and Aguirre, 2013) identify only two experimental studies on violence reduction: one on the effectiveness of police patrol on reducing violence in Colombia (García, Mejía, and Ortega, 2013) and the other on the effectiveness of electronic monitoring on recidivism in Argentina (Di Tella and Schargrodsky, 2013). In the sections below, we present our foundation for the current evaluation as well as our results. We begin by reviewing risk factor approaches to understanding youth problem behaviors, including crime and delinquency, and family-based intervention programs previously used in Central America. Next, we describe our understanding of the context in which the Proponte Más program was implemented, including the particular municipalities where it was introduced. We then present our evaluation methodology with its rigorous study design, a randomized control trial (RCT).1 We conclude with a discussion of evaluation results and their implications. RISK FACTOR APPROACHES TO UNDERSTANDING PROBLEM BEHAVIOR The risk factor approach is applicable to a number of different domains. This versatility is one reason for its widespread use in dealing with crime and delinquency–it is grounded in the focus of research and programmatic attention on some selected combination of individuals, families, peers, schools and neighborhoods. Considerable research attention is being given to the development and use of sophisticated measures of risk and protective factors, applying solid psychometric measurement methods. This includes careful examinations of the construct and predictive validity of measures of risk factors in low- and middle-income countries 1 An RCT is regarded as the gold standard in evaluation design. Despite this, it is seldom used owing to the difficulty of implementation and the reluctance of program administrators to deny treatment to individuals who, it is believed, would benefit from such programming. One recent study indicated that the majority of the public favors the universal implementation of non-researched, non-tested policies and does not approve of randomized experiments. For example, about 50% of respondents reported that randomization is inappropriate or that it is unethical to randomize treatment assignment to determine what works (Meyer et al., 2019). (Murray et al., 2018). Although a disproportionate number of the studies being examined are from China, oriented towards public health and the use of medical diagnostics, researchers are finding considerable overlap in the measurement qualities of the scales associated with risk and protective factors in diverse countries. (Notable exceptions are measures of early physical health on delinquency.) Similarly, Hawkins, Catalano, and Miller (1992) have identified a number of studies in international contexts that used a risk and protective factor approach for identifying not only risk, but also its amelioration. Among the strengths of the risk factor approach is that it identifies protective factors in addition to factors that may promote harm. This balanced approach provides opportunities for interventions that build on strengths of individuals, families, institutions and communities that can promote success. Such a balanced approach helps to avoid the stigma often attendant to working in poor or disadvantaged communities. It creates a greater sense of hope for individuals and communities, as well as for service providers. In addition, working to enhance community strengths and resilience reduces the magnitude of the challenges faced by individuals who live and work in these communities. Considerable progress has been made in the study of these factors in the United States, Canada and Europe. Several recent meta-analyses of more than 50 studies concluded that the relationship between risk factors and involvement in delinquency was invariant across the collection of studies (Assink et al., 2015; Jolliffe et al., 2017). That is, there was a strong positive association between the number of risk factors associated with youth and their involvement in delinquency. Although variations in the types of risk factors placing a youth at risk are substantial, academics and practitioners are generally in consensus that risk factors for delinquency and crime are consistently associated with at least five domains: neighborhood, school, family, peer and individual.2 Individual factors such as gender and age have a 2 Each domain is independent, but the risk factors associated with each are not mutually exclusive. UNA EVALUACIÓN DE PROPONTE MÁS: UN PROGRAMA HONDUREÑO DE PREVENCIÓN SECUNDARIA 5 strong influence on delinquency, while important family risk factors include poor family management practices and history of parental criminal behavior. Peer risk factors include having delinquent peer associations and membership in a gang. School risk factors associated with delinquency include poor academic performance and poor school attendance, among others. Neighborhood risk factors such as neighborhood disadvantage and neighborhood levels of crime are also associated with delinquency. The past three decades of research on delinquency prevention and intervention have been plagued by the failure to address primary and secondary needs of youth and to implement successful programming. This points to a mismatch between the needs of youth served and the services that many programs have delivered. This is likely due to a fundamental failure to understand the needs of at-risk youth, including those already involved in delinquency or crime. The reasons for this are widely debated; often, the impulse simply to “do something” and reliance on outdated intervention models are cited as primary causes. Regardless, this situation calls for a data-driven and data-led approach to programming, characterized by careful assessment across multiple domains of behavior, including both risks and resiliency or protective factors. Such assessments would more accurately characterize both the challenges that youth face and the strengths they bring to the task of transitioning from adolescence to adulthood. The key is to institute data￾driven methods for identifying a youth’s problems (e.g., deviance, delinquency, school achievement, substance abuse) and individual and family strengths, and then tailoring interventions responsive to those specific conditions. Data for problem identification can come from any number of sources, such as secondary records from schools, neighborhood groups, family members and, most importantly, the youth themselves. This type of information can provide a solid foundation for building effective interventions. This risk factor approach to date has been most closely aligned with efforts in the United States. It remains to be determined whether and how well such an approach (adapted to regional culture and structures) might translate to low- and middle-income nations THE USE OF FAMILY-BASED INTERVENTIONS TO RESPOND TO PROBLEM BEHAVIOR The family has become a key point of entry and intervention for responding to youth violence and other problem behaviors. Prior research has documented that parents and their children bidirectionally influence family culture, management, communication and structure. On one hand, poor parenting skills are associated with negative child outcomes, such as alcohol and drug use, delinquency, school problems, and other high￾risk behaviors (Weisman and Montgomery, 2019; Knerr, Gardner, and Cluver, 2013). On the other hand, child aggression is associated with hostility and conflict between parents or parents and children within the home, which increases harsh and negative parenting and promotes a reciprocal cycle of aggression and hostility (Knerr, Gardner, and Cluver, 2013). The family is seen as the primary means of child socialization and, as such, it can serve as either a risk or protective factor in youth development (Simons et al., 1998). It is generally accepted that dysfunctional families model and provide opportunities for problem behavior, while functional families model and provide opportunities for positive and prosocial behavior (Development Services Group, 2014). Therefore, family-based interventions in general have become a common strategy for preventing and intervening in youth problem behavior, but are rare in Honduras. Today, a wide variety of family-based intervention programs are available. Most focus on identifying and building upon family strengths to address youth misbehavior, developing coping skills and strategies, providing skills-building training for parents and children, identifying patterns of family interaction and setting goals for improving these, and motivating children and parents to reframe problems and act with positive enthusiasm (Development Services Group, 2014). Several family-based interventions have been evaluated in developed nations and were found to have positive impacts when addressing issues such as violence (Maalouf and Campello, 2014), gangs (Thornberry et al., 2018), drug use (Hartnett et al., 2017), and teen suicide and depression (Diamond et al., 2010). The body of literature is deep enough that multiple systematic literature reviews and meta- INFORME TÉCNICO 6 analyses have been completed, substantiating the positive effects of family-based intervention on youth outcomes such as delinquency (Farrington and Welsh, 2003). Family-based intervention programs are increasingly implemented in developing nations to address systemic social and health problems (Maalouf and Campello, 2014). This has, in part, been a consequence of their successful implementation with high-risk participants and effectiveness in achieving desired outcomes elsewhere (Knerr, Gardner, and Cluver, 2013). The programs have rarely been evaluated in developing nations in general or in Central American nations, specifically. The United Nations Office on Drugs and Crime conducted one of the few examinations of family-based intervention in low- and middle-income countries, examining SFP and Families and Schools Together (FAST) programs and their impact on violence in nine low and middle-income nations, including Honduras. No control group was used; however, pre/post-test data suggested both programs were effective for reducing violence, demonstrating strong potential for use in low and middle income nations. The evaluations were necessarily limited by the lack of a control group and a small convenience-based sample (n=188 across nine nations). Regardless, the authors concluded that family-based interventions were a promising strategy for developing nations. They recommended conducted a RCT as the next essential step for determining the programs’ likely effectiveness in low- to middle-income nations (Maalouf and Campello, 2014). THE NEED FOR EVIDENCE-BASED PROGRAMMING IN CENTRAL AMERICA AND THE CARIBBEAN Over the last two decades, violence has grown in a number of countries in Central America, including El Salvador, Guatemala, Honduras. Some of this violence is associated with proximate causes, but much of it has deeper roots and is imbedded in distal causes associated with inequality, economic distress, culture, and institutional weaknesses (Rodgers and Baird, 2014). When combined with the relative instability of several nations in the Central American region, it is clear from the review above that there are many risk factors and not enough protective factors. This strongly underscores the need for evidence-based prevention and intervention throughout the area. Honduras is an especially important site for such efforts given its geographic and economic position in the region, and its historically high levels of violence. Violence serves to disrupt the socializing power of parents, neighborhoods, schools, and institutions such as the labor market and family. Violence reduces labour market opportunities of those living in violent neighbourhoods because it limits resident’s access to jobs and limits the capacity of neighbourhoods to support and generate local business. Community violence also erodes family functioning through its negative effect on family communication and family conflict. It also increases family fear of victimization, results in increased familial isolation, and reduces the financial resources available to the family. Youth exposed to violence are also less connected school, which results in poorer school performance and increased likelihood of dropping out of school. Individuals living in high violence communities also experience reduced trust and cooperation with social institutions and the limited resources that are available are expended on the police, courts, and corrections rather than human development and support (Moser and van Bronkhort, 1999; Vincent, 2019). An intervention that balances risk and protective factors, and built upon a solid understanding of them, could be an important stabilizing influence in the lives of Honduran youth. Unfortunately, little research has been conducted in Central America where delinquency and violence are high; of the research that has been conducted, results have been inconsistent. Along with the lack of analysis on the causes and correlates of problem behavior in Central America, there is a dearth of knowledge about what works in response to the risks faced by youth in these nations (Abt and Winship, 2016; Jaitman and Guerrero Compean, 2015). The lack of evidence￾based programming (i.e., prevention, intervention, and suppression) in Central America stems from multiple sources. First, Central America generally lacks the strong criminological foundations needed for understanding its patterns of criminal justice and delinquency. Second, the history of programming UNA EVALUACIÓN DE PROPONTE MÁS: UN PROGRAMA HONDUREÑO DE PREVENCIÓN SECUNDARIA 7 in the region is quite checkered, as the typical response to most problems of crime and disorder has been based in suppression. That is, crackdowns by police (and occasionally the military) that include round-ups, extensive use of incarceration, and surveillance have been the predominant crime control efforts in the face of growing crime and violence. A focus on intervention and prevention has emerged in the past twenty years (Alvarado, Muggah, and Aguirre, 2015), but there is still a lack of balance between suppression-based and prevention/intervention-based efforts. Third, much of the intervention/ prevention effort has its locus outside the region. Often such efforts are led by groups that provide development funding, such as USAID, UNDP, OAS and the World Bank (Muggah and Aguirre, 2013). In fact, external agencies provide about 70% of the funding for citizen security in the region. In those few instances where organizations with sophisticated research designs have conducted data-driven, intervention/prevention-focused programming, few if any were required to assess their programs’ effectiveness. Of more than 1,350 violence reduction programs sponsored by international development agencies since the late 1990s, more than 57% were not evaluated at all, repeatedly missing out on the chance to identify best practices in violence reduction. (See Alvarado et al., 2015). Large international development organizations often do not require implementers to participate in evaluations to determine what works. As such, the present study represents an opportunity to assess the results of a data-driven, locally focused intervention program with a rigorous research design. THE PRESENT STUDY This report contains the evaluation results of the implementation of Result 1 of Proponte Más, a secondary prevention program sponsored by USAID in Honduras. The report contains five sections. The first section discusses the program’s setting, origins and family-based model. The next section describes the research design, including sampling and recruitment, data, measures and analysis. The third and fourth sections present the results of our process and impact evaluations, and the last section discusses the implications of those results. THE SETTING Over the past several years, Honduras has experienced some of the highest rates of homicide in the world. In 2011, that rate was 86.5 homicides per 100,000 population. The homicide problem has begun to subside substantially, but as recently as 2017, the nation still ranked as one of the most violent in the world with 43.6 homicides per 100,000 population (https://iudpas.unah.edu. hn/observatorio-de-la-violencia/boletines-del￾observatorio-2/boletines-nacionales/cite). In 2014, USAID completed the Honduras Country Development Cooperation Strategy, calling for resources to target the densely populated urban communities at greatest risk for violence. In Honduras, these included 14 intervention zones comprising 216 communities located in the two largest municipalities: Tegucigalpa, the nation’s capital and largest city, and San Pedro Sula, the Honduran industrial center and second largest city. It also included three other municipalities: Choloma, La Ceiba and Tela. As shown in exhibit 1, each of the five municipalities has had a homicide rate above the national average since 2015. INFORME TÉCNICO 8 ORIGINS OF PROPONTE MÁS Proponte Más is an evidence-informed secondary prevention program. Its roots are found in several of the most prominent responses to delinquency and other problem behaviors. The Comprehensive Strategy developed by the Office of Juvenile Justice and Delinquency Prevention (OJJDP) influenced the development of the precursor to Proponte Más. The Gang Reduction and Youth Development (GRYD) response to gangs in Los Angeles, founded on principles established in the OJJDP comprehensive model, was a milestone in the development of risk reduction programs. GRYD took the Comprehensive Strategy to a new level, integrating youth development as a key component in the response to gangs (Kraus et al., 2017). By doing so, GRYD promoted the development of prosocial opportunities and skills as an integral part of a successful response to gangs and delinquency. Proponte Más leveraged the Los Angeles experience with GRYD by incorporating youth development and family strengthening as central elements of its response to at-risk youth in Honduras. In 2012, the City of Los Angeles and USAID signed an agreement that led to their sharing violence prevention and intervention best practices; and in in 2013, a pilot test in Honduras was carried out to demonstrate proof of concept. (known as Proponte). Subsequently USAID awarded funding in December 2015, and Proponte Más was operational by January 2016. Shortly thereafter, regional offices opened, staff received training, and potential participants were referred to the program. YSET-I assessments were completed in September 2016 with the first cohort of participants. A post-hoc evaluation of the first cohort of Proponte Más program participant data showed that the program for secondary group participants was implemented with a high degree of fidelity but fell short of full implementation with tertiary program participants. Findings of the impact evaluation, which relied on regression discontinuity, suggested that the program had a modest but significant effect on reducing risk factors and delinquency (Katz et al., 2017). Based on lessons learned during the first cohort, program plans for the second cohort were refined, and Proponte Más staff worked with the evaluation team to develop and implement a more rigorous evaluation process applying the principles of a randomized control design. Referrals for second cohort program participation began in August 2017, risk factor assessments were completed by February 2018, and program implementation took place between March and September 2018. IMC-R assessments were completed in September 2018. The results of the evaluation of the second round of program implementation (second cohort) are provided in this report. In the next section, we discuss the program’s approach, technical design and activities. YSET/IMC Instrument Glossary YSET-I/IMC-I. The first YSET/IMC taken by youth following referral. Used to determine program eligibility, and used by the evaluation team as the pretest. YSET-R/IMC-R. The second YSET/IMC taken by youth. Administered 6 months following program implementation. Used by the evaluation team as the post-test. Exhibit 1: Homicide rates per 100,000 inhabitants by year and municipality Municipality 2013 2014 2015 2016 2017 La Ceiba 140.7 95.6 157.25 120.83 181.5 Tela 94.7 74.2 93.47 48.7 100.3 San Pedro Sula 193.4 143.8 173.6 107.02 166.4 Choloma 68.7 61.5 78.3 92.7 94.5 Distrito Central 86.0 81.1 88.2 82.3 99.9 Nationwide 73.64 67.17 59.44 59.1 43.6 Source: https://app-iudpas.unah.edu.hn/participacionciudadana/Denuncias/mapa_oficial UNA EVALUACIÓN DE PROPONTE MÁS: UN PROGRAMA HONDUREÑO DE PREVENCIÓN SECUNDARIA 9 PROGRAMMATIC APPROACH The primary goal of Proponte Más is to reduce the risk factors for problem behavior associated with youth ages 8 to 17 in Honduras. Proponte Más’ approach to attain this goal is structured around the achievement of five results: Result 1. Increased the number of youth and families receiving secondary prevention services, Result 2. Increased the number of eligible youth and families accessing additional support services in targeted locations, Result 3. Cadre of family counselors established in targeted communities, Result 4. Alternative justice measures strengthened, and Result 5. Community-based secondary prevention model established. The program’s approach is closely linked to the IMC beta diagnostic instrument, which was used (formerly known as the YSET) to identify youth who, with their families, were eligible to receive violence prevention services in targeted areas. The IMC beta relied on nine risk factor scales, described below. Family counselors, certified in the administration of the IMC beta and family interventions, were assigned to the youth and their families. The Proponte Más model posits that it is necessary to work with the entire family, to provide individual and group interventions and to link members to community services, for the purpose of reducing risk factors and increasing protective factors for the youth and their families. Family-based intervention, based on a theory of change, attempts to modify relational sequences, family dynamics and family cohesion, using behaviors as a lever to create cooperation and change. TECHNICAL DESIGN The Proponte Más technical design provides youth and their families with six months of family-based interventions, divided into seven phases (see exhibit 2). Youth and their families are guided through each phase by the assigned family counselor, who emphasizes achievement of specific goals determined by the participants’ IMC beta assessment and family context. Each phase takes about 30 days to complete. Phase 1: Referral and Collaboration: includes the behavioral description of the problem identified by the referral source, if there is one; the behavioral description of the caretaking signing the informed consent; the information generated by the IMC beta diagnostic and; the information generated by the FACES diagnostic. to identify the youth’s risk factors. Youth who are possess 0 to 3 risk factors are assigned to a primary (no- or low￾risk) group, while those with 4 to 9 risk factors are assigned to a secondary (moderate or high-risk) group Phase 2 Building Agreements: includes an agreement with the family that identifies behaviors that the family and the counselor will work on. Phase 3: Redefining: incudes shifting the emphasis of the intervention from one that requires an individual problem formulation and solution to one that includes the horizontal and vertical family. Phase 4: Celebrating Changes: includes, a celebration of efforts that ideally bring together as many members of the horizontal and vertical family as possible; Phase 5 Integrating: Includes incorporation of emotional resources and program resources that the leadership identifies. Phase 6: Next Level Agreements: Includes meeting with the leadership of the family and establishing a set of rules that will guide the process of supervision in the family. Phase 7: Re Evaluation: Includes a re evaluation of the index youth using the IMC, and an evaluation of the family using FACES. At this time treatment ends. INFORME TÉCNICO 10 As an intervention based on family systems theory and practice, the program’s strong theory of change identifies the family as the key change agent for moderating risk factors. Cespedes and Herz (2011: 9) state that the theory of change is predicated on the prior work of Bowen (1993), Kerr and Bowen (1988), Fisch, Weakland, and Segal (1982), Minuchin and Fishman (1981), and Walsh (2003, 2006, 2009), all of whom… …encourage[d] multigenerational family “connectedness;” reinforcing parental/ caretaker authority; using of positive and effective problem-solving skills at the individual, family, and community levels; and helping youth self-differentiate from the gang culture through alternative, positive activities and positive connections to positive, supportive family members and/or other adults… Therefore, stronger interventions are those in which family members engage in changing the behaviors and risk factors of youth. The goal is to shift the way the family functions such that it reduces embeddedness with delinquent peers and addresses other risk factors. Because the family is viewed as the “medicine” or crucible of change, it is crucial that Proponte Más counselors accurately assess family strengths and weaknesses as they lay out a treatment plan. This points to the critical role of the IMC. At the heart of the intervention is the goal of creating long-term, vertical, perhaps even multi-generational family cohesion. THE IMC DIAGNOSTIC TOOL The IMC emerged from the Gang Risk of Entry Factors (GREF) and Youth Services Eligibility (YSET) assessment tools, which were originally developed by Karen Hennigan and her colleagues in Los Angeles. The GREF was first used in the GRYD (the secondary gang prevention program that addresses high-risk youth in Los Angeles.) The assessment goal was to identify youth at high risk for gang joining. A systematic review of the gang literature was performed to identify “a subset of factors that have consistently been empirically related to gang affiliation…favoring evidence from studies with rigorous research designs” (Hennigan et al., 2014: 109). The risk assessment tool’s architects relied on two systematic literature reviews to identify pertinent risk factors for gang joining. The first was Klein and Maxson’s (2006) review of gang research from the United States, Exhibit 2: Phases of Family- Centered Interventions APPLICATION OF IMC DIAGNOSTIC INTERVENTION STRATEGIES DOSAGE Establishing Agreements (month 1 of service) Redefining (Service month 2) Celebrating Changes (Service month 3) Integrating (Month 4 of service) Next Level Agreements (Month 5 of service) Re - Evaluating (Month of service) NOT ELEGIBLE ELEGIBLE Referral & Collaboration TRANSFER INTERVENTION OTHER PROGRAMS HORIZONTAL STRATEGY: Problem Salving Strategies VERTICAL STRATEGY: Multi Generational-Strenghs Based Genograms Meeting of Strategic Team (1 for Phase) Family Meetings (2 for Phase) Individual Meetings (1 for Phase) All interventions are conducted through a family systems theory and practice lens. IDENTIFICATION OF POTENTIAL PARTICIPANTS APPLICATION OF IMC ABILITY TOOL, FOR ADOLESCENTS AND YOUTH PHASE 1 PHASE 2 PHASE 3 PHASE 4 PHASE 5 PHASE 6 PHASE 7 UNA EVALUACIÓN DE PROPONTE MÁS: UN PROGRAMA HONDUREÑO DE PREVENCIÓN SECUNDARIA 11 Canada and Europe; the second was Krohn and Thornberry’s (2008) review of longitudinal gang research in about a half a dozen American cities, Montreal, Canada, and Bergen, Norway. The literature reviews suggested seven risk factors for gang association: (1) cumulative exposure to stressful life events, (2) problem behavior associated with impulsivity, risk taking, and oppositional, aggressive or externalizing behavior, (3) delinquent beliefs, (4) weak parental supervision, (5) negative peer influence, (6) peer involvement in delinquency, and (7) early involvement in delinquency. Hennigan et al. (2014: 110) emphasized that no single risk factor predicted gang joining; rather, it was the accumulation of multiple risk factors that was associated with gang joining. Based on these findings, the research team created scales that included items that had previously been used in the United States, Canada, and Europe to operationalize these same constructs (see Klein and Maxson, 2006; Krohn and Thornberry, 2008). The instrument was initially pilot tested with youth in Los Angeles to assess each item’s readability. Adaptions were made to improve the respondent’s comprehension of each item (Hennigan et al., 2014). Hennigan et al. (2015) later tested the validity of the YSET using a high-risk sample of 11- to 16-year-olds, again in Los Angeles. The analysis indicated that all males identified as high risk (i.e., possessing four or more risk factors) using the YSET had also self-reported being a gang member. Likewise, 81 percent of male former gang members and 74 percent of males who associated with a gang were identified as being at high risk. (The YSET was also found to be a valuable tool to assess relative levels of risk among females, although the relationship was not as strong, due in part to the relatively small number of female gang members in the sample.). Following the first cohort of program implementation in Honduras, the YSET was revised to include additional items, discussed further below, and was renamed the IMC. IMC stands for Instrumento de Medicion de Comportamientos or in English, Behavior Measurement Instrument. Proponte Más used the IMC to identify at-risk youth and their families, who would receive prevention services in the targeted communities. Family counselors, YSET certified, administered the risk assessment tool (assuring that youth were assessed by counselors other than the one who would later be assigned to their family). The YSET relied on nine risk factor scales related to those identified earlier by Hennigan et al. (2014). The scales include antisocial tendencies (7 items), weak parental supervision (5 items), critical life events (7 items), impulsive risk taking (4 items), neutralization of guilt (6 items), negative peer influence (3 items), peer delinquency (5 items), influence of gangs in the family (2 items), and crime and substance use (16 items). Each risk factor scale was constructed using Proponte Más’s scale development protocol (Hennigan, 2017). Items included in each scale, response categories and descriptive statistics are presented below (exhibit 3). The core premise of the Proponte Más’ secondary prevention through the Prevention and Intervention Family Systems Model is that youth and their families will be selected for program participation based on level of risk, as identified with the IMC. Youth (and their families) are assigned to one of two levels: the primary risk group includes those with 0 to 3 risk factors, and the secondary risk group includes those with 4 to 9 risk factors. INFORME TÉCNICO 12 Exhibit 3: Scales and items distribution (n=4495) Factors and Items # of items Range Mean SD Response Categories Antisocial Tendencies 7 I am kind to others 1-5 2.03 1.21 1=always…5=never I respect the feelings of others 1-5 1.84 1.20 1=always…5=never I get angry easily 1-5 3.39 1.49 1=never…5=always I am obedient 1-5 2.67 1.32 1=always…5=never I threaten others to get what I want 1-5 1.39 0.94 1=never…5=always People “blame me” for lying or cheating 1-5 2.53 1.52 1=never…5=always I take things that don’t belong to me 1-5 1.30 0.77 1=never…5=always Weak Parental Supervision 5 When I go out, I let my parents or guardians know where I am going 1-5 1.91 1.36 1=always…5=never My parents or guardians ask me where I am going when I leave the house 1-5 1.46 1.03 1=always…5=never My parents or guardians know who I am with when I’m not at home or school 1-5 1.93 1.39 1=always…5=never My parents or guardians know who my friends are 1-5 1.84 1.30 1=always…5=never I feel that my parents or guardians care about what I do 1-5 1.61 1.18 1=always…5=never Critical Life Events 9 In the last year, have you... …failed a grade in school? 0-1 0.19 0.39 0=no, 1=yes …been expelled or suspended from your school due to disciplinary reasons? 0-1 0.09 0.28 0=no, 1=yes …had a girlfriend/boyfriend for the first time this year? 0-1 0.35 0.48 0=no, 1=yes …broken up with/ended a relationship with a boyfriend/ girlfriend or been broken up with by a boyfriend/ girlfriend? 0-1 0.30 0.46 0=no, 1=yes …fought or had a problem with a friend? 0-1 0.53 0.50 0=no, 1=yes …tried “hanging out” with a new friend group? 0-1 0.64 0.48 0=no, 1=yes …felt forced to abandon school for any reason? 0-1 0.26 0.44 0=no, 1=yes …has someone close to you died or been seriously hurt due to an accident or illness? 0-1 0.48 0.50 0=no, 1=yes …has someone close to you died (or was murdered) due to violence? 0-1 0.26 0.44 0=no, 1=yes Impulsive Risk Taking 4 Sometimes I like to do dangerous activities for fun 1-5 2.59 1.26 1=strongly disagree…5=strongly agree Sometimes I find it exciting to do things that could get me in trouble 1-5 2.59 1.23 1=strongly disagree…5=strongly agree I frequently do things without thinking if I’ll get in trouble or not 1-5 2.72 1.26 1=strongly disagree…5=strongly agree I like to have fun when I can, even if I’ll get in trouble for doing them later on 1-5 2.96 1.30 1=strongly disagree…5=strongly agree UNA EVALUACIÓN DE PROPONTE MÁS: UN PROGRAMA HONDUREÑO DE PREVENCIÓN SECUNDARIA 13 Factors and Items # of items Range Mean SD Response Categories Neutralization of Guilt 6 It is okay to lie if it keeps my friends from getting in trouble with their parents or with the police 1-5 2.55 1.16 1=strongly disagree…5=strongly agree It is okay to lie to someone to keep myself from getting in trouble with them 1-5 2.50 1.14 1=strongly disagree…5=strongly agree It is okay to steal something if someone is rich and can easily replace it 1-5 1.96 0.81 1=strongly disagree…5=strongly agree It is okay to steal small items from a store without paying because stores have a lot of money and this does not affect them 1-5 1.91 0.80 1=strongly disagree…5=strongly agree It is okay to hit others if others hit me first 1-5 3.03 1.31 1=strongly disagree…5=strongly agree It is okay to hit people if it’s for my own defense 1-5 3.39 1.26 1=strongly disagree…5=strongly agree Negative Peer Influence 3 If your friends were getting in trouble in your home, would you continue being their friend? 1-5 2.23 0.76 1=definitely no… 5=definitely yes If your friends were getting in trouble at school, would you continue being their friend? 1-5 2.19 0.72 1=definitely no… 5=definitely yes If your friends were getting in trouble with the police, would you continue being their friend? 1-5 1.95 0.55 1=definitely no… 5=definitely yes Peer Delinquency a 5 During the last six months, how many friends have… …stolen something? 1-5 1.43 0.83 1=none…5=all …attacked someone? 1-5 1.42 0.87 1=none…5=all …sold marijuana or other illegal drugs? 1-5 1.09 0.43 1=none…5=all …used illegal drugs? 1-5 1.21 0.67 1=none…5=all …belong to or have joined a gang or “mara”? 1-5 1.09 0.47 1=none…5=all Influence of Gangs in the Family 2 Including all of the people that you consider part of your family; how many family members think that you will most likely join a gang one day? 0-4 0.39 0.98 0=0, 1=1, 2=2, 3=3, 4= 4 or more Currently, how many of your family members are in a gang? 0-4 0.32 0.82 0=0, 1=1, 2=2, 3=3, 4= 4 or more INFORME TÉCNICO 14 Factors and Items # of items Range Mean SD Response Categories Crime and Substance Abuse 16 In the last six months… ...consumed alcohol or smoked cigarettes? 0-1 0.14 0.35 0=no, 1=yes ...used marijuana or other illegal drugs? 0-1 0.04 0.21 0=no, 1=yes …skipped class? 0-1 0.17 0.38 0=no, 1=yes …avoided paying for a movie, taxi or bus? 0-1 0.12 0.33 0=no, 1=yes …broken or destroyed something on purpose that was not yours? 0-1 0.13 0.33 0=no, 1=yes …carried a hidden weapon around for protection? 0-1 0.02 0.16 0=no, 1=yes …illegally painted a wall or building – done “graffiti?” 0-1 0.08 0.28 0=no, 1=yes …stolen something of low value? 0-1 0.13 0.33 0=no, 1=yes …stolen something of high value? 0-1 0.01 0.11 0=no, 1=yes …entered or tried to enter a building to try and steal something? 0-1 0.01 0.09 0=no, 1=yes …hit someone with the intention of hurting them? 0-1 0.12 0.33 0=no, 1=yes …attacked someone with a weapon? 0-1 0.01 0.10 0=no, 1=yes …used a weapon or physical force to get money from someone else? 0-1 0.00 0.06 0=no, 1=yes …participated in fights with youths from other neighborhoods? 0-1 0.08 0.27 0=no, 1=yes …participated in gang fights? 0-1 0.01 0.11 0=no, 1=yes …sold marijuana or other drugs/helped others sell drugs? 0-1 0.01 0.10 0=no, 1=yes a = responses for those reported that they do not have a group of friends were coded “none” for all the items in peer delinquency factor. MEASUREMENT AND SCORING OF IMC FOR PROGRAM ELIGIBILITY Data collected through Proponte, the 2013 pilot test demonstrating the programs proof of concept, were used to determine the cut point for the respondents’ level of risk (United States Agency for International Development, 2016). The cut points for each scale, by age, are presented in exhibit 4. In the pilot study, cut points reflected each respondent’s relative level of risk compared with other respondents enrolled in the study (n=327). Respondents who scored from the 1st to the 25th percentile of a scale were labeled as OK. Respondents scoring from the 26th to the 49th percentile were labeled as close. Those scoring from the 50th to the 70th percentile was assigned a 1; those from the 71st to the 89th percentile were assigned a 2; those from the 90th to the 99th percentile were assigned a 3, and those in the greater than 99th percentile were coded as 4. Each scale then was recoded into a dichotomous variable. Those who were labeled OK or close were categorized as not at risk (0). Those at or above the 50th percentile, coded as 1 or higher, were categorized as at risk. An additive scale was created that represented the respondents’ cumulative level of risk, ranging from 0 to 9. As noted above, respondents identified with the IMC as not at risk (0-3 risk factors) were placed in the primary group (i.e., referred to the program, but assessed as no- or low-risk, therefore ineligible to receive services). Primary group members’ individual demographic data were collected and included in selected analyses. Respondents identified as having four or more risk factors were categorized as at risk, and as such were eligible for services; they were placed in the secondary group. UNA EVALUACIÓN DE PROPONTE MÁS: UN PROGRAMA HONDUREÑO DE PREVENCIÓN SECUNDARIA 15 Exhibit 4: Cut points by risk factor Cut Point Risk Factors Risk Level Age 12 and younger (n=2,317) Age 13 and older (n=2,178) Antisocial Tendencies Not at risk 7 thru 16 7 thru 18 At risk 17 and over 19 and over Weak Parental Supervision Not at risk 5 thru 11 5 thru 11 At risk 12 and over 12 and over Critical Life Events Not at risk 0 thru 3 0 thru 4 At risk 4 and over 5 and over Impulsive Risk Taking Not at risk 4 thru 12 4 thru 13 At risk 13 and over 14 and over Neutralization of Guilt Not at risk 6 thru 16 6 thru 17 At risk 17 and over 18 and over Negative Peer Influence Not at risk 3 thru 6 3 thru 6 At risk 7 and over 7 and over Peer Delinquency Not at risk 5 thru 7 5 thru 7 At risk 8 and over 8 and over Family Gang Influence Not at risk 0 thru 1 0 thru 1 At risk 2 and over 2 and over Crime and Substance Abuse Not at risk 0 thru 2 0 thru 3 At risk 3 and over 4 and over Note: The age selection process was determined at baseline (pretreatment period). DATA AND METHODS Youth who lived in the fourteen intervention zones located within five municipalities in Honduras were referred to the program and then assessed with the IMC, after which their eligibility for programming was determined by Proponte Más. Of 4,495 youth assessed, 944 were found though the IMC to be at risk for four or more risk factors, qualifying them to receive program services. Within each intervention zone, as previously noted, the 944 eligible youth were then randomly assigned to the treatment group or the control group. A total of 463 youth with their families were assigned to the treatment group; 481 were assigned to the control group. Of those assigned to treatment, 33 dropped out before the program started and another 58 dropped out during program participation, leaving a total of 372 (80.3%) for analysis. Of the 481youth assigned to the control group, 75 dropped out prior to the post-test, leaving 406 (84.4%) for analysis. (See exhibit 5.) INFORME TÉCNICO 16 Exhibit 5: CONSORT Flow Diagram Randomized (n=944) ENROLLMENT ALLOCATION FOLLOW-UP ANALYSIS Assessed for eligibility (n=4,495) Allocated to treatment group (n=463) Received allocated intervention (n= 430) Did not receive allocated intervention (n= 33); Reasons: Changed mind prior to start (n=13), Medical problem (n=1), Not enough time to participate (n=2), moved out of intervention zone (n=17). Discontinued intervention (n=58) Reasons: Changed mind (n=16) Not enough time to participate (n=14), migrated out of country (n=9), moved out of intervention zone (n=11), medical problem (n=3), other (n=5) Analysed (n=372) Excluded from analysis (give reasons) (n=0) Allocated to control group (n=481) Lost to follow-up (n=75) Reasons: Changed mind (n=18), Not enough time (n=8), migrated out of country (n=11), other (n=8), not located (n=4), moved out of zone (n=26). Analysed (n=406) Excluded from analysis (give reasons) (n=0) Excluded (n=3,551) Not meeting inclusion criteria (n=3,383) Declined to participate (n=168) About 66% of the treatment group and 62% of the control group were male; the mean age of males in both groups was about 12 years old. Roughly, 87% of treatment youth and 84% of control group youth were enrolled in school. Less than 80% of youth in both groups had both parents present in the home; 18.59% of treatment youth and 19.7% of the control group lived with a single mother or female figure. Only a small number of youth in the treatment and control groups lived with a single father or no parent at all. We compared the treatment and control groups in terms of gender, age, school status, parental presence and neighborhood of residence, and found no significance differences between the groups (see exhibit 6). Exhibit 6. Demographic Characteristics of treatment and control groups (n=778) Treatment Group (T0 ; n=371) Control Group (C0 ; n=406) n % n % sig. Gender Male 246 66.13 251 61.82 Female 126 33.87 155 38.18 Agea (mean & s.d.) 12.3(2.5) 12.5(2.5) Currently in school 323 86.83 341 83.99 UNA EVALUACIÓN DE PROPONTE MÁS: UN PROGRAMA HONDUREÑO DE PREVENCIÓN SECUNDARIA 17 Treatment Group (T0 ; n=371) Control Group (C0 ; n=406) n % n % sig. Parental presence Mother and father/female and male figure 291 78.45 314 77.34 Single mother/female figure 69 18.59 80 19.70 Single father/male figure 9 2.42 8 1.97 No parent/adult guidance 2 0.54 4 0.99 Zone San Miguel 25 6.72 25 6.16 Carrizal 30 8.06 33 8.13 Predregal 18 4.84 20 4.93 Villanueva 17 4.57 21 5.17 Chamelecon 30 8.06 27 6.65 Rivera Hernandez 21 5.65 24 5.91 Satelite-Medina 34 9.14 27 6.65 Lopez-Arellano 30 8.06 35 8.62 Choloma Centro 15 4.03 19 4.68 Bonitillo 56 15.05 59 14.53 Las Mercedes 55 14.78 74 18.23 Zona A: El Centro Poli-deportivo 6 1.61 7 1.72 Zona B: 4E - SJ - LB - BV 31 8.33 30 7.39 Zona C: Tornabe 4 1.08 5 1.23 *p < .05; ** p < .01; *** p < .001 We also examined differences in the raw pretest diagnostic risk factor scores for the treatment and control group youth (exhibit 7). No statistically significant differences between the treatment and control groups were found with respect to their individual risk factor scores or their overall average scores. The effect size of the differences was small as well, with the effect size of each measure below .15. Exhibit 7. Pre-Test Diagnostic Risk Factor Scores for Treatment and Control Groups (n=778) Treatment Group (T0 ; n=371) Control Group (C0 ; n=406) Diff a Effect size b Mean(sd) Mean(sd) (T0 -C0 ) Sig. (g) Antisocial Tendencies 43.4 (14.7) 42.4 (14.9) 1.0 0.07 Weak Parental Supervision 34.7 (23.0) 34.9 (22.7) -0.2 -0.01 Critical Life Events 42.9 (20.0) 44.1 (20.7) -1.2 -0.06 Impulsive Risk Taking 67.6 (18.4) 65.1 (19.6) 2.5 0.14 Neutralization of Guilt 54.7 (14.3) 55.5 (13.9) -0.8 -0.05 Negative Peer Influence 35.6 (16.5) 36.7 (17.1) -1.1 -0.07 Peer Delinquency 14.9 (14.7) 15.3 (16.7) -0.4 -0.02 INFORME TÉCNICO 18 DATA Data for this study were drawn primarily from the IMC, FACES IV, and fidelity databases. The IMC database contained pretest (IMC-I) and post-test (IMC-R) data for each participant, including the referral source, IMC administration dates, ID number, eligibility status, demographic characteristics, and responses to items in the risk and protective factors instrument. (The IMC-I was administered just prior to program implementation; the IMC-R was administered six months after initiation of programming.) In particular, the IMC contained 173 items that measured 38 risk and protective factors within four domains (i.e., community, school, family, peer/ individual). We examined the scales for reliability and with a series of tests and found that these scales were internally reliable and valid for youths in Honduras. The results of these analyses are available from the authors upon request.3 The FACES IV (hereafter referred to as FACES) instrument, our second data source, contained 84 items that measured six scales tapping family cohesion and flexibility dimensions. The theoretical framework of the instrument is based on the Circumplex Model that posits three dimensions of family function: cohesion, family flexibility, and communication. Cohesion is defined as “the emotional bonding that family members have toward one another” (Olson, 2011: 65). Family flexibility is defined as “the quality and expression of leadership and organization, role relationship, and relationship rules and negotiations” (Olson, 2011: 65). Communication 3 For details on the the instrument and its psychometric properties see Katz, Cheon, Zheng (2019). is defined as “the positive communication skills utilized in the couple or family system” (Olson, 2011: 65). Balanced levels of cohesion and flexibility are hypothesized as the highest form of family function, and unbalanced levels of cohesion and flexibility are hypothesized as the lowest form of family function. The measurement and scoring of each scale are detailed in the FACES manual. The six scales include three cohesion scales that measure enmeshed, balanced cohesion and disengaged, and three flexibility scales that measure chaotic, balanced flexibility and rigid. Specifics of the scales are proprietary and are not permitted to be detailed in this report, per Proponte Más agreement with the creators of FACES. FACES data were collected from one parent or guardian of each treatment and control youth, prior to and again after program implementation. The third data source contained information on program fidelity. Throughout the program, family counselors input data regarding the intervention. This included information related to the date and time of meetings with the family and youth. Data on family composition, family goals and assignments, family problems addressed, and plans for future meetings were recorded. Likewise, data related to individual meetings with the at￾risk youth were recorded in the database; these included information on problems and solutions applied by youth, defining solutions, tasks prepared for, and designation of a family member who would monitor the youth’s effort. These data were used for the process evaluation. Treatment Group (T0 ; n=371) Control Group (C0 ; n=406) Diff a Effect size b Mean(sd) Mean(sd) (T0 -C0 ) Sig. (g) Family Gang Influence 21.2 (23.8) 21.1 (24.7) 0.1 0.01 Crime and Substance Abuse 15.6 (13.8) 16.9 (15.3) -1.3 -0.09 Overall Average Score 36.7 (7.8) 36.9 (7.7) -0.2 -0.02 a. t-test was performed to compare the mean differences between treatment and control group. b. Effect size = Hedges g estimation *p < .05; ** p < .01; *** p < .001 UNA EVALUACIÓN DE PROPONTE MÁS: UN PROGRAMA HONDUREÑO DE PREVENCIÓN SECUNDARIA 19 MEASURES PROCESS MEASURES We reviewed the database maintained by Proponte Más and created several process measures. Data were captured that measured the number of youths referred to Proponte Más for programming. The referral data included the source (i.e., type of person, organization) of the referral. IMC eligibility results were entered into the implementation database; these included factors for which the youth was at risk and sociodemographic characteristics. A measure on program drop out was also included along with, if applicable, the reason the youth and their family dropped out. Last, four measures of fidelity from the database were used. Our first measure of fidelity was time—that is, the number of minutes per meeting that the counselor reported spending with the family. The number of minutes were summated for a total time measure made up of the time the counselor spent in 12 family meetings, six individual meetings with the youth, and five strategic team meetings in phases 2 through 7. Our second measure of fidelity was the number of assignments given by the counselor and which the youth or their family agreed to accomplish between meetings. The total number of assignments was a count variable made up of the number of assignments for family meetings and individual meetings. Our third measure of fidelity was the number of assignments that were completed. Following each meeting, the counselor documented the number of assignments that had been completed among those that had been assigned in the previous meeting. The total number of completed assignments was a count variable made up of the number of completed assignments from family meetings and individual meetings. The fourth measure of fidelity was the average assignment completion ratio, which was calculated by dividing the number of completed assignments by the number assigned. OUTCOME MEASURES Several primary and secondary outcome measures were used in the present study. First, our primary outcome measures, which are the focus of Proponte Más programming, are measures of risk and protective factors obtained through the IMC instrument. Second, several secondary outcome measures were also obtained through the FACES instrument, which included measures of family cohesion, flexibility, and family function dimensions. Third, a number of self-reported measures of delinquency were obtained through the IMC and were used as the outcome measures. These measures are discussed below. MEASURES OF RISK AND PROTECTIVE FACTORS Our outcome measures consisted of 34 risk and protective factors collected at pre- and post-testing using the IMC.4 They represented three domains: community, family and peer/ individual. Exhibit 9 displays items and their respective response distributions. The community domain contained five risk factor scales (i.e., transitions and mobility, low neighborhood attachment, community disorganization, laws and norms favorable to drug use, and perceived availability of drugs) and two protective factor scales (i.e., opportunities for prosocial involvement and rewards for prosocial involvement). The family domain included six risk factor scales (i.e., family history of antisocial behavior, parental attitudes favorable toward drug use, poor family management, family conflict, weak parental supervision, and family gang influence) and three protective factor scales (i.e., attachment, opportunities for prosocial involvement, and rewards for prosocial involvement). The peer/ individual domain consisted of 14 risk factor scales (i.e., rebelliousness, rewards for antisocial involvement, favorable attitudes toward drug use, favorable attitudes toward antisocial behavior, perceived risks of drug use, friends’ use of drugs, interaction with antisocial peers, intentions to use, antisocial tendencies, critical life events, 4 Initially 38 risk and protective factors were available for analysis. We excluded four risk and protective factors associated with the school domain because some youth were not currently enrolled in school. We also dropped six items in the family history of antisocial behavior scale because these items are not time varying and would not show meaningful changes between the pre/post-treatment periods. INFORME TÉCNICO 20 impulsive risk taking, neutralization of guilt, negative peer influence, and peer delinquency) and four protective factor scales (i.e., belief in the moral order, rewards for prosocial involvement, interaction with prosocial peers, and social skills). All item responses were converted to 0 to 100; item scores were averaged to create the scale score. Higher risk scale scores indicated higher risk, and higher protective scale scores indicated higher protection. Further, we created an overall average score for each domain using average scale scores: overall community domain average, overall family domain average, and overall peer/individual domain average. For these measures, we reverse coded the protective scale scores, with higher scores being interpreted as higher risk. For example, overall community domain average score was created by summing the four risk factor scale scores (i.e., transitions and mobility, low neighborhood attachment, community disorganization, and perceived availability of drugs) and reverse coded the two protective factor scale scores (i.e., opportunities for prosocial involvement and rewards for prosocial involvement).5 We then divided this score by the number of scales within the domain, which is seven. Last, we created an overall risk factor average score by summing all the risk (and reverse coded) protective factors and dividing by the number of factors (n=34). MEASURES OF FAMILY ADAPTABILITY AND COHESION. We relied on eight scales from the FACES IV instrument: cohesion, flexibility, disengaged, enmeshed, rigid, chaos, family communication and family satisfaction (exhibit 8). Each scale used the summated score of items. Higher scores on the balanced cohesion and balanced flexibility scales and the family communication and family satisfaction scales indicate healthier family functioning. Higher scores on the disengaged, enmeshed, rigid and chaotic scales indicate problematic family functioning. 5 Reverse coding was used for consistency in the interpretation of the directionality of findings across factors. Exhibit 8. Items and responses for FACES IV at pre-treatment (n=777) Scales # of Items Range Mean SD Response Categories Balanced scales: Balanced cohesion 7 7-35 26.32 3.95 Family members are involved in each other’s lives. 1-5 3.81 0.82 1=Strongly disagree … 5=Strongly agree Family members feel very close to each other. 1-5 3.63 1.00 Family members are supportive of each other during difficult times. 1-5 3.97 0.90 Family members consult other family members on important decisions. 1-5 3.72 0.99 Family members like to spend some of their free time with each other. 1-5 3.86 0.88 Although family members have individual interests, they still participate in family activities. 1-5 3.86 0.77 Our family has a good balance of separateness and closeness. 1-5 3.47 0.97 UNA EVALUACIÓN DE PROPONTE MÁS: UN PROGRAMA HONDUREÑO DE PREVENCIÓN SECUNDARIA 21 Scales # of Items Range Mean SD Response Categories Balanced flexibility 7 7-35 25.46 3.80 Our family tries new ways of dealing with problems. 1-5 3.76 0.88 1=Strongly disagree … 5=Strongly agree Parents equally share leadership in our family. 1-5 3.50 1.11 Discipline is fair in our family. 1-5 3.88 0.86 My family is able to adjust to change when necessary. 1-5 3.68 0.90 We shift household responsibilities from person to person. 1-5 3.08 1.16 We have clear rules and roles in our family. 1-5 3.71 0.97 When problems arise, we compromise. 1-5 3.85 0.90 Unbalanced scales: Disengaged 7 7-35 20.78 3.89 We get along better with people outside our family than inside. 1-5 2.94 1.13 1=Strongly disagree … 5=Strongly agree Family members seem to avoid contact with each other when at home. 1-5 2.64 1.08 Family members know very little about the friends of other family members. 1-5 3.28 1.05 Family members are on their own when there is a problem to be solved. 1-5 2.90 1.14 Our family seldom does things together. 1-5 2.96 1.12 Family members seldom depend on each other. 1-5 3.03 1.08 Family members mainly operate independently. 1-5 3.02 1.07 Enmeshed 7 7-35 22.18 3.13 We spend too much time together. 1-5 3.51 1.02 1=Strongly disagree … 5=Strongly agree Family members feel pressured to spend most free time together. 1-5 2.55 1.04 Family members are too dependent on each other. 1-5 3.49 0.97 Family members have little need for friends outside the family. 1-5 2.93 1.02 We feel too connected to each other. 1-5 3.12 1.07 We resent family members doing things outside the family. 1-5 3.69 1.03 Family members feel guilty if they want to spend time away from the family. 1-5 2.89 1.08 Rigid 7 7-35 24.18 3.54 There are strict consequences for breaking the rules in our family. 1-5 3.36 1.11 1=Strongly disagree … 5=Strongly agree There are clear consequences when a family member does something wrong. 1-5 3.68 0.96 Our family has a rule for almost every possible situation. 1-5 3.45 1.03 Our family is highly organized. 1-5 3.31 1.05 Our family becomes frustrated when there is a change in our plans or routines. 1-5 3.09 1.07 It is important to follow the rules in our family. 1-5 4.12 0.64 Once a decision is made, it is very difficult to modify that decision. 1-5 3.16 1.11 INFORME TÉCNICO 22 Scales # of Items Range Mean SD Response Categories Chaotic 7 7-35 20.05 4.67 We never seem to get organized in our family. 1-5 3.19 1.12 1=Strongly disagree … 5=Strongly agree It is hard to know who the leader is in our family. 1-5 2.49 1.12 Things do not get done in our family. 1-5 3.05 1.08 It is unclear who is responsible for things (chores, activities) in our family. 1-5 2.81 1.17 There is no leadership in our family. 1-5 2.63 1.20 Our family has a hard time keeping track of who does various household tasks. 1-5 3.04 1.16 Our family feels hectic and disorganized. 1-5 2.85 1.16 Family scales: Family communication 10 10-50 35.44 6.35 Family members are satisfied with how they communicate with each other. 1-5 3.48 1.02 1=Strongly disagree … 5=Strongly agree Family members are very good listeners. 1-5 3.66 0.91 Family members express affection to each other. 1-5 3.54 1.01 Family members are able to ask each other for what they want. 1-5 3.68 0.88 Family members can calmly discuss problems with each other. 1-5 3.39 1.05 Family members discuss their ideas and beliefs with each other. 1-5 3.67 0.88 When family members ask questions of each other, they get honest answers. 1-5 3.59 0.93 Family members try to understand each other’s feelings 1-5 3.72 0.88 When angry, family members seldom say negative things about each other. 1-5 3.03 1.11 Family members express their true feelings to each other. 1-5 3.68 0.87 Family satisfaction 10 10-50 30.89 7.50 The degree of closeness between family members. 1-5 3.32 1.04 1=Very Dissatisfied … 5=Extremely Satisfied Your family’s ability to cope with stress. 1-5 2.92 1.03 Your family’s ability to be flexible. 1-5 3.07 0.97 Your family’s ability to share positive experiences. 1-5 3.37 0.97 The quality of communication between family members. 1-5 3.14 1.09 Your family’s ability to resolve conflicts. 1-5 3.09 1.03 The amount of time you spend together as a family. 1-5 3.18 1.07 The way problems are discussed. 1-5 3.02 1.04 The fairness of criticism in your family. 1-5 2.69 1.05 Family members concern for each other. 1-5 3.09 1.11 MEASURES OF DELINQUENCY We also used delinquency measures as secondary outcome measures obtained through the IMC. These included a variety of problem behaviors based on 18 self-reported items (exhibit 9). The IMC-I captured whether the participant was involved in problem behaviors six months UNA EVALUACIÓN DE PROPONTE MÁS: UN PROGRAMA HONDUREÑO DE PREVENCIÓN SECUNDARIA 23 prior to program implementation, and the IMC-R captured whether the participant engaged in problem behaviors during the six months of program implementation. Seven measures of delinquency are variety scores that were created by summing the dichotomous variables of each problem behavior. Specifically, we used three variables indicating violent behavior (e.g., in the last six months, have you hit someone with the purpose of hurting him/ her?), four variables indicating property crime (e.g., in the past six months, have you purposely damaged or destroyed things that do not belong to you?), three variables indicating gang involvement (e.g., in the past six months, have you been a member of a gang?), three variables indicating alcohol and drug use (e.g., in the past, have you used marijuana or other illegal drugs?), two variables indicating drug sales (e.g., in the past six months, have you sold (or helped to sell) marijuana or other illegal drugs?), and two variables indicating weapon carrying (e.g., in the past six months, have you carried a concealed weapon for protection?). The last delinquency measure, truancy, remains as a dichotomous variable since there is one question asking, “in the past six months, have you skipped class?” Further, we created a variety score for overall delinquency by summing all the delinquency measures, ranging from 0 (no involvement in problem behaviors) to 18 (involvement in all 18 problem behaviors).6 Higher scores indicated greater involvement in problem behaviors. 6 Variety scores were used because they have been found to be a reliable and valid measure of crime and delinquency (Hindelang, Hirschi, & Weis, 1981; Bendixen, Endresen, & Olweus, 2003; Sweeten, 2012). Exhibit 9. Items descriptions by each delinquency outcome measure at baseline In the last 6 months… # of Items Range Mean SD Response Categories Violent Behavior 3 0-3 0.35 0.54 ...hit someone with the intention of hurting them? 0-1 0.31 0.46 0=No; 1=Yes ...attacked someone with a weapon? 0-1 0.03 0.16 ...used a weapon or force to get money or goods from someone? 0-1 0.01 0.11 Property Crime 4 0-4 0.65 0.81 ...purposely damaged or destroyed things that do not belong to you? 0-1 0.30 0.46 0=No; 1=Yes ...stolen (or tried to steal) something of little value? 0-1 0.30 0.46 ...stolen (or tried to steal) something of great value? 0-1 0.04 0.19 ...broken in (or tried to break in) somewhere to steal something? 0-1 0.02 0.14 Gang Involvement 3 0-3 0.13 0.48 ...been a member of a gang? in the past 6 months 0-1 0.03 0.18 0=No; 1=Yes ...Is your group of friends in a gang? 0-1 0.05 0.22 Currently, are you in a “crew,” clique, or associated with a gang? 0-1 0.05 0.21 Alcohol/Drug Use 3 0-3 0.47 0.74 ...consumed alcoholic beverages or smoked cigarettes? 0-1 0.33 0.47 0=No; 1=Yes ...used marijuana or other illegal drugs? 0-1 0.13 0.34 ...used cocaine? 0-1 0.01 0.12 Drug Selling 2 0-2 0.11 0.40 ...sold (or helped to sell) marijuana or other illegal drugs? 0-1 0.04 0.20 0=No; 1=Yes ...brought marijuana from one place to another to complete an “order”? 0-1 0.07 0.25 INFORME TÉCNICO 24 INDEPENDENT VARIABLE Our key independent variable of interest is treatment, coded as treatment group (1) and control group (0). CONTROL VARIABLES Pretreatment delinquency scores, pretreatment risk and protective scale scores, and zone are used as control variables for the purpose of increasing statistical power (type 2 error) and reducing the possibility of spurious statistical significance test results (type 1 error) (Hedberg, 2017). ANALYSIS POWER ANALYSIS We obtained data from 778 program￾eligible youth; we randomly assigned 372 youth to the treatment group and 406 to the control group. Our analysis plan included the use of a stratified￾randomized trial, compensating for the fact that youth are nested within 14 zones, for the purpose of increasing statistical precision. Given a desired power of 0.8 and a two-tailed test with alpha= .05, we calculated the minimum detectible effect size in Cohen’s d units (1988). Using G*Power, we calculated a simple random sample minimum detectable effect size based on independent mean comparison tests. Our analyses showed that the minimum detectible effect size of our sample was about .20 standard deviations (SDs). This effect size is without covariates. Using baseline scores in the model reduces the minimum detectable effect size by a factor of sqrt(1-R2), where R2 is the explanatory power of the covariate. For example, if the pretest explains 30 percent of the variation in outcomes, then the minimum detectable effect size reduces to .20*sqrt(1-.3)= .16, producing a more sensitive test. Therefore, our research design allowed us to detect a relatively small effect when compared with those observed in prior evaluations of family-based interventions (Farrington and Welsh, 2003; Dopp, 2016: vi; Weisman and Montgomery, 2019). ANALYSIS PLAN First, we examined direct effects. We examined the distributional qualities of all outcome measures. The pre- and post-treatment equivalence of the treatment and control group members was assessed using a difference-in-differences estimator (Meyer, 1995). The diff-in-diff estimator compares change in outcomes within the treatment group before and after the treatment to the changes in outcomes within the control group. We then corrected for the spurious effects of treatment on our outcome measures by employing a series of regression models. Different regression models were conducted depending on the structure of the outcome measures. Specifically, negative binomial or Poisson regression models with correction of over-dispersion were run for our variety scores of delinquency outcomes (see Lawless, 1987), and logistic regression models were run for binary outcomes (i.e., truancy). The model for each dependent variable included a control variable measuring a pretreatment score of delinquency and fixed effect of 14 geographic areas. In addition, the effect size statistics were calculated using the incidence rate ratio and odds ratio. Second, we examined indirect effect of treatment on overall risk factor score and overall delinquency using structural equation modeling In the last 6 months… # of Items Range Mean SD Response Categories Carrying Weapon 2 0-2 0.10 0.37 ...carried a concealed weapon for protection? 0-1 0.07 0.26 0=No; 1=Yes ...brought weapons from one place to another? 0-1 0.03 0.17 Truancy (...skipped class?) 1 0-1 0.33 0.47 0=No; 1=Yes General Delinquency 18 0-18 2.15 2.23 UNA EVALUACIÓN DE PROPONTE MÁS: UN PROGRAMA HONDUREÑO DE PREVENCIÓN SECUNDARIA 25 (SEM), using Mplus software (Muthén and Muthén, 1998-2017). In particular, in the present context, SEM has the ability to assess direct and indirect effects of treatment on the outcome measures by examining the pathways simultaneously (Gau, 2010). In addition, we used 10,000 bootstrapped samples drawn from the original data to obtain better estimates of standard errors to identify mediation effects. Bootstrapping is a rigorous and powerful method by requiring fewer assumptions about the shape of the sampling distribution of the indirect effect and having high power while maintaining reasonable control over the type I error rate (Hayes, 2013; MacKinnon, Lockwood, & Williams, 2004; Preacher & Hayes, 2008). The coefficients were standardized, and the magnitude of the mediation effects were expressed as the ratio of the indirect to total effect of treatment on the outcome measures (Hayes, 2013). Based on theory and practical evidence, we tested two hypothesized causal models that the variables considered in this study, as displayed in exhibit 10 and exhibit 11. The first model (exhibit 12) suggests a causal effect of treatment on overall risk factor score, partially mediated by family adaption and cohesion. We expected that the treatment would increase family adaption and cohesion, which would turn into a decreased overall risk factor score. It was also expected that the treatment would reduce the post-treatment unbalanced scales (i.e., disengaged, enmeshed, rigid, and chaotic), which would lead to a decrease in overall risk factor score. The second model in exhibit 13 suggested a causal effect of treatment on overall delinquency outcome, and this relationship would be partially mediated by risk and protective factors. It was expected that treatment would reduce the post-treatment risk factor scores that were positively associated with delinquency, which in turn would result in reduced delinquent behavior. It was also expected that the treatment would increase the post-treatment protective factor scores, which would result in decreased delinquency. Exhibit 11. Hypothesized causal pathways between treatment, risk and protective factors, and delinquency Exhibit 10. Hypothesized causal pathways between treatment, FACES, and risk factors FACES 1 FACES II Overall risk factor score 1 Treatment Overall risk factor score I1 +/- +/- - Risk/Protective Factors 1 Treatment Delinquency 11 +/- +/- - Risk/Protective Factors 1I Delinquency 11 PROCESS EVALUATION RESULTS This section of the evaluation provides background on the types of participants served by Proponte Más. We have included a description of the referral and eligibility process and the characteristics of participants. We have also provided details on program retention rates and the amount and types of services provided to youth and their families. REFERRAL The referral and eligibility processes were conducted in stages. In August 2017, 50 family counselors conducted an awareness campaign in the target communities. The campaign sought to inform referral networks, service providers and community organizations about Promote Más, including the following facts: 1. Participation in the IMC diagnostic process requires informed written consent from a parent or legal guardian; INFORME TÉCNICO 26 2. The focus of the intervention is the family system; 3. The target population is voluntary families with youth, 8 to 17 years old, who have been identified by the IMC as at risk. A total of 4,574 youth were referred to the program for participation. Of the 4,574 youth referred to the program, written consent was obtained from the parents or caregivers of 4,495 youth was obtained to conduct the assessment of the youth using the IMC. This process yielded 4,495 youth who were administered the IMC program eligibility screening assessment (exhibit 14). The breakdown by city is as follows: 1,023 (22.8%) youth came from Distrito Central (Tegucigalpa), 1,032 (23.0%) from San Pedro Sula, 1,131 (25.2%) from La Ceiba, 738 (16.4%) from Choloma, and 571 (12.7%) from Tela. Most the youth were referred by their parent(s) or guardians (54.7%) or school (29.7%). The remainder were referred by a USAID-sponsored outreach center (3.0%), other family member (2.9%), a program or institution (2.7%), church (1%), self-referral (0.4%), counselor/ advisor (0.2%), or another source (5.3%). There was a significant difference in type of referral across communities. For example, the majority of youth from the La Ceiba and Tela were referred by a parent or guardian, while youth from the Central District, San Pedro Sula and Choloma were more likely to be referred by school, a parent, or guardian. ELIGIBILITY Of 4,495 youth who were screened for the program, 944 youth (21%) were determined to be at risk and eligible for services; 3,551 were determined to be not at risk, and therefore ineligible (exhibit 12). Program eligibility varied among municipalities: about 32% of the eligible youth resided in La Ceiba, followed by 23.5% in Distrito Central, 20.3% in San Pedro Sula, 12.9% in Choloma, and 10.8% in Tela. Municipality was significantly related to the likelihood of program eligibility. Exhibit 12. Program eligibility by municipality Primary (n=3,551) Secondary (n=944) Total Sample (n=4,495) Municipality*** n % n % n % Distrito Central 801 22.6 222 23.5 1,023 22.8 San Pedro Sula 840 23.7 192 20.3 1,032 23.0 La Ceiba 825 23.2 306 32.4 1,131 25.2 Choloma 616 17.4 122 12.9 738 16.4 Tela 469 13.2 102 10.8 571 12.7 *p<.05; **p<.01; ***p < .001; Pearson chi-square = 41.1408; df = 4 We compared the not-at-risk (primary) and at-risk (secondary) groups, looking for significant variations in eligibility by source of referral to the program (exhibit 13). Referred youth assigned to the at-risk group were more likely than those assigned to the not-at-risk group to be referred by a parent or guardian (56.5% vs. 54.3%) or other family member (4.3% vs. 2.6%). Conversely, youth assigned to the not-at-risk group were slightly more likely than those in the at-risk group to be referred by their school (30.8% vs. 25.4%). No discernable pattern in group assignment by referral source emerged for those who were self-referred or referred by a counselor, church, program/institution, outreach center, or other. UNA EVALUACIÓN DE PROPONTE MÁS: UN PROGRAMA HONDUREÑO DE PREVENCIÓN SECUNDARIA 27 Male youth were more likely than female youth to be determined at risk and assigned to the secondary group, eligible for programming (exhibit 14). At-risk (secondary) youth were more likely to be older than not-at-risk (primary) youth (average age=12.58 years vs. 12.26 years). With respect to school attendance, 83.7% of at-risk youth and 90% of not-at-risk youth were currently in school. No significant differences between the groups were found with respect to place of birth. Exhibit 13. Number of assessed youth by program eligibility and referral source (n=4,495) Primary (n=3,551) Secondary (n=944) Total Sample (n=4,495) Type of Referral** n % n % n % The youth referred him or herself 14 0.4 3 0.3 17 0.4 Parent/guardian 1,928 54.3 533 56.5 2,461 54.7 Other family member 91 2.6 41 4.3 132 2.9 Counselor 6 0.2 3 0.3 9 0.2 Church 38 1.1 8 0.8 46 1.0 Program or institution 89 2.5 31 3.3 120 2.7 School 1,094 30.8 240 25.4 1,334 29.7 Outreach Center 111 3.1 26 2.8 137 3.0 Other 180 5.1 59 6.3 239 5.3 *p<.05; **p<.01; ***p < .001; Pearson chi-square = 21.2612; df = 8 Exhibit 14. Individual characteristics (n=4,495) Primary (n=3,551) Secondary (n=944) Total Sample (nN=4,495) Characteristics n % n % n % Sex** Male 2,081 58.6 605 64.1 2,686 59.8 Female 1,470 41.4 339 35.9 1,809 40.2 Mean Age (SD)1 ** 12.26 (2.6) 12.58 (2.6) 12.33 (2.6) Currently in school*** 3,194 90.0 790 83.7 3,984 88.7 Place of birth Rural 246 6.9 68 7.2 314 7.0 Urban 3,305 93.1 876 92.8 4,181 93.0 1 Range: 8 to 17 years old *p<.05; **p<.01; ***p<.001 RETENTION Analysis of the program’s case tracking data indicated that 778 (82.4%) of 944 youth who were determined to be at risk and eligible for services agreed to participate and completed the six-month program (exhibit 15). We found no significant differences between those who completed the program and those who dropped out with respect to gender and total number of risk factors. Age, school enrollment, and place of birth, however, were significantly related to at￾risk participants dropping out of the program. Youth completing the program were more likely to be younger (average age=12.39) and to be attending school (85.4%) than those who dropped out (average age=13.44; 75.9% attending school). Also, 12.7% of the sample who dropped out of the program resided in rural areas, compared with about 6% of the sample who completed the program. INFORME TÉCNICO 28 DOSAGE We measured the level or “dosage” of services the program delivered for several reasons. First, for purposes of assessment, it is essential to know whether the intervention was fully implemented and whether the program delivered in the field accurately reflected the program’s blueprint. A program’s success or failure in the field cannot be attributed to its interventions unless these have been executed according to plan. Treatment outcomes that are uncontrolled and/or not standardized, regardless of their effect in a given instance, have no predictive value for future program applications. Measurements of service levels can also help detect substandard or incorrect interventions that may have contributed to program failures. Describing and measuring intervention qualities and quantities allows researchers and practitioners to more completely understand the impact of the program and interventions being assessed. The youth in our study spent an average of 1,201.14 minutes (i.e., 20 hours) with their counselors (SD=134.22), of which 712.05 minutes were in family meetings, 277.79 minutes were in individual meetings, and 211.30 minutes were strategic team meetings. Across the program, youth and their families were assigned an average of 29.13 assignments (SD=5.37), 21.9 of which were from family meetings and 7.23 of which were from individual meetings. Participants completed an average of 30.26 assignments (SD=12.87), 22.79 of which were from family meetings and 7.47 of which were from individual meetings. Participants had an average assignment completion ratio of 1.1 across the meeting sessions (SD=0.35), of which 1.13 were from family meetings and 1.04 were from individual meetings. This indicates that, on average, participants accomplished all assignments initially planned and then more. Exhibit 15. Individual characteristics of youth who completed versus dropped out of program (n=944) Total (n=944) Completed (n=778) Dropped out (n=166) Characteristics n % n % n % Sex Male 605 64.1 498 64.0 107 64.5 Female 339 35.9 280 36.0 59 35.5 Mean Age (SD)1 *** 12.58 (2.6) 12.39 (2.5) 13.44 (2.6) Currently in school** 790 83.7 664 85.4 126 75.9 Place of birth** Rural 68 7.2 47 6.0 21 12.7 Urban 876 92.8 731 94.0 145 87.4 Total Number of Risk Factors (SD)2 4.83 (1.2) 4.82 (1.2) 4.87 (1.2) 1 Range: 8 to 17 years old; 2 Range: 0 to 8 Risk Factors *p<.05; **p<.01; ***p<.001 Exhibit 16. Fidelity Dosage Statistics Summary (n=371) Range Mean SD Time Spent with Family Total Minutes Reported 830 - 1615 1201.14 134.22 Family Meeting Minutes 470 - 1031 712.05 109.49 Individual Meeting Minutes 165 - 400 277.79 46.25 Strategic Team Meeting Minutes 110 - 300 211.30 55.01 UNA EVALUACIÓN DE PROPONTE MÁS: UN PROGRAMA HONDUREÑO DE PREVENCIÓN SECUNDARIA 29 IMPACT EVALUATION RESULTS PART 1: CHANGE IN RISK STATUS CHANGE IN PROGRAM ELIGIBILITY We examined whether participants’ risk status changed pre/post-test (exhibit 17). About 75% of participants in the treatment group were no longer eligible for treatment at post-test due to changes in risk status, while approximately 70% of participants in the control group were no longer eligible at post-test. The difference between treatment and control group participants’ changes in eligibility was not statistically significant. Number of Assignments Total Number of Assignments 20 - 57 29.13 5.37 Family Meetings 14 - 46 21.90 3.98 Individual Meetings 5 - 18 7.23 2.43 Number of Completed Assignments Total Number of Completed Assignments 12 - 87 30.26 12.87 Family Meetings 4 - 70 22.79 11.41 Individual Meetings 5 - 19 7.47 2.86 Average Assignment Completion Ratio a Total Number of Completed Assignments 0.6 - 3.1 1.10 0.35 Family Meetings 0.4 - 4.0 1.13 0.48 Individual Meetings 0.8 - 2.4 1.04 0.17 a. About 12% (n=44) of cases in family meeting (also of total) that has zero assignment is removed in average assignment completion ratio calculation. Exhibit 17. Changes in Participants Risk Status from Pre- to Post-test Treatment group (n=372) Control group (n=406) n % n % Remained eligible at post-test 94 25.27 123 30.30 No longer eligible at post-test 278 74.73 283 69.70 Note. Differences in frequencies were tested using chi-squared test. No significant difference between treatment and control group regarding eligibility change (X2 = 2.439, p = 0.118); effect size (d) = -0.11 CHANGE IN DIAGNOSTIC SCALE SCORES We looked at pre/post-test changes in the IMC diagnostic scale scores (exhibit 18). For antisocial tendencies, 50.5% of the treatment group and 42.6% of the control group changed status from at risk to not at risk over the project period. Likewise, over the project period, for weak parental control, 44.1% of the treatment group and 34.5% of the control group changed from at risk to not at risk, and for impulsive risk taking, 55.1% of the treatment group and 37.4% of the control group changed from at risk to not at risk. On the other hand, for critical life events, 32.5% of the treatment group and 42.6% of the control group changed from being at risk to not at risk over the project period. INFORME TÉCNICO 30 Exhibit 18. Changes in IMC Diagnostic Scale Scores from Pre to Post-test No Risk At Risk No Change At Risk No Risk Treatment group (n=372) Control group (n=406) Effect size a Treatment group (n=372) Control group (n=406) Effect size a Treatment group (n=372) Control group (n=406) Effect size a n % n % Sig. n % n % Sig. n % n % Sig. Antisocial Tendencies 17 4.6 29 7.1 0.11 167 44.9 204 50.2 0.11 188 50.5 173 42.6 * -0.16 Weak Parental Supervision 18 4.8 36 8.9 * 0.16 190 51.1 230 56.7 0.11 164 44.1 140 34.5 ** -0.20 Critical Life Events 38 10.2 27 6.7 -0.13 213 57.3 206 50.7 -0.13 121 32.5 173 42.6 ** 0.21 Impulsive Risk Taking 19 5.1 34 8.4 0.13 148 39.8 220 54.2 *** 0.29 205 55.1 152 37.4 *** -0.36 Neutralization of Guilt 22 5.9 27 6.7 0.03 151 40.6 170 41.9 0.03 199 53.5 209 51.5 -0.04 Negative Peer Influence 48 12.9 47 11.6 -0.04 215 57.8 220 54.2 -0.07 109 29.3 139 34.2 0.11 Peer Delinquency 23 6.2 28 6.9 0.03 229 61.6 235 57.9 -0.07 120 32.3 143 35.2 0.06 Family Gang Influence 34 9.1 30 7.4 -0.06 243 65.3 270 66.5 0.02 95 25.5 106 26.1 0.01 Delinquency and Drug Use 15 4.0 10 2.5 -0.09 256 68.8 280 69.0 0.00 101 27.2 116 28.6 0.03 Note. Differences in proportions were tested using chi-squared test. a. Effect size = Cohen’s d estimation *p < .05; ** p < .01; *** p < .001 UNA EVALUACIÓN DE PROPONTE MÁS: UN PROGRAMA HONDUREÑO DE PREVENCIÓN SECUNDARIA 31 PART 2: DIRECT EFFECTS OUTCOMES FOR CHANGE IN FAMILY ADAPTABILITY AND COHESION As noted above, we examined changes in family adaptability and cohesion with a pre/post￾analysis of the treatment and control groups, on FACES IV (exhibit 19). Pretreatment, we observed no significant differences between treatment and control groups. Six months following initiation of treatment, however, the two balanced scales, cohesion and flexibility, had significantly increased for the treatment group compared with the control group (t=3.32, p<.001; t=4.77, p<.001, respectively). Effect sizes of the differences were small, .26 for balanced cohesion and .24 for balanced flexibility. In other words, the two groups’ mean scores differ by .26 standard deviations for balanced cohesion and .24 standard deviations for balanced flexibility. Post-treatment, both family scales, communication and satisfaction, had significantly increased in the treatment group when compared with the control group (t=2.61, p<.01; t=5.21, p<.001, respectively). The increases in both dimensions indicated that families in the treatment group improved to healthier functioning when compared with the families in the control group. The effect sizes of these changes were .14 for family communication and .21 for family satisfaction, suggesting that the program had a small effect on these measures. We also observed large positive improvements in unbalanced scales in the treatment group. After six months, disengaged and chaotic scale scores had significantly decreased for families in the treatment group, indicating that these families had reduced problematic family functioning after treatment (t=4.79, p<.001; t=8.24, p<.001, respectively). The effect size was small: -.27 for disengaged and -.37 for chaotic. The rigid scale score, however, increased slightly for the treatment group when compared with the control group (t=3.15, p<.01; g=.20). Exhibit 19. Pre-Post analysis with treatment and control group on FACES IV (n=658) Pre-test Post-test Treatment group (n=319) Control group (n=339) Diff a Treatment group (n=319) Control group (n=339) Diff a Difference in Differences Mean(sd) Mean(sd) (T0-C0 ) sig. Mean(sd) Mean(sd) (T1-C1 ) sig. |t|(T1-C1) - (T0-C0 ) sig. Effect size a Balanced scales: Balanced cohesion 26.6(3.7) 26.3(4.1) 0.33 28.7(3.5) 27.0(3.9) 1.73 *** 3.32 *** 0.26 Balanced flexibility 25.5(3.7) 25.6(3.6) -0.05 28.0(3.4) 26.2(3.6) 1.85 *** 4.77 *** 0.24 Unbalanced scales: Disengaged 20.7(3.9) 20.7(3.9) -0.04 19.0(4.4) 21.2(4.0) -2.19 *** 4.79 *** -0.27 Enmeshed 22.3(3.0) 22.1(3.2) 0.25 23.4(3.6) 23.2(3.5) 0.22 0.09 0.07 Rigid 24.4(3.4) 24.2(3.5) 0.12 26.3(3.5) 24.9(3.7) 1.34 *** 3.15 ** 0.20 Chaotic 20.0(4.6) 19.7(4.7) 0.29 16.3(4.5) 20.1(4.2) -3.81 *** 8.24 *** -0.37 Family scales: Family communication 35.6(5.9) 35.7(6.6) -0.01 39.5(5.2) 37.7(6.6) 1.75 *** 2.61 ** 0.14 Family satisfaction 30.7(7.2) 31.2(7.6) -0.50 36.2(6.0) 32.5(8.1) 3.69 *** 5.21 *** 0.21 a. t-test was performed to compare the mean differences between treatment and control group. b. Effect size = Hedges g estimation *p < .05; ** p < .01; *** p < .001 INFORME TÉCNICO 32 Exhibit 20 presents the results of regression on FACES IV scales. Regression results were consistent with findings using bivariate analyses. Treatment had a medium and significant impact on balanced scales cohesion and flexibility (effect size of .47), net of pretreatment score and geographical variations (b=1.72, p <.01; b=1.71, p<.01). Treatment had a large negative effect on unbalanced scales disengaged and chaotic (effect size of -.57 and -.91) net of pretreatment score and geographical variations (b=-2.42, p<.001; b=- 4.08, p<.001). Treatment was also associated with a small- to medium-sized significant increase in family scales communication and satisfaction (effect size of .28 and .52), net of pretreatment score and geographical variations (b=1.74, p<.001; b=3.96, p<.001). Exhibit 20: Multivariate Analysis: Regression on FACES IV (n=777) Treatment Pretest score Effect size a b (se) Sig. b (se) Sig. Balanced scales: Balanced cohesion 1.72 (0.41) ** 0.22 (0.04) *** 0.47 Balanced flexibility 1.71 (0.25) *** 0.19 (0.04) *** 0.47 Unbalanced scales: Disengaged -2.42 (0.30) *** 0.27 (0.05) *** -0.57 Enmeshed 0.20 (0.52) 0.16 (0.06) * 0.06 Rigid 1.33 (0.52) * 0.20 (0.06) ** 0.35 Chaotic -4.08 (0.32) *** 0.18 (0.04) ** -0.91 Family scales: Family communication 1.74 (0.36) *** 0.29 (0.04) *** 0.28 Family satisfaction 3.96 (0.84) *** 0.24 (0.04) *** 0.52 Std. Err. adjusted for 14 clusters in Zone a. Effect size = Cohen’s d estimation *p < .05; ** p < .01; *** p < .001 OUTCOMES FOR CHANGES IN REVISED IMC RISK AND PROTECTIVE FACTOR SCORES Exhibit 21 presents our pre/post-test findings on risk and protective factors for the treatment and control groups. Pretreatment, most risk and protective scale scores did not differ significantly between the two groups, with the exception of perceived availability of drugs, parental attitudes favorable toward drug use, and belief in the moral order. Post-treatment, however, a number of risk and protective scale scores differed significantly between the groups, including opportunities for prosocial involvement in the community, family history of antisocial behavior, family conflict, weak parental supervision, opportunities for prosocial family involvement, rewards for prosocial family involvement, overall family domain average, rebelliousness, favorable attitudes toward antisocial behavior, perceived risk and drug use, impulsive risk taking, negative peer influence, and interaction with prosocial peers. Post-treatment, nine risk and protective factors and overall family domain average score showed significant changes for the treatment group when compared with the control group. Among risk factors, weak parental supervision, rebelliousness, antisocial tendencies, and impulsive risk taking significantly decreased (t=3.07, p<.01; t=4.25, p<.001; t=2.04, p<.05; t=3.40, p<.01, respectively) for the treatment group compared with the control group. Among protective factors, opportunities for community prosocial involvement, opportunities for family prosocial UNA EVALUACIÓN DE PROPONTE MÁS: UN PROGRAMA HONDUREÑO DE PREVENCIÓN SECUNDARIA 33 involvement, and interaction with prosocial peers significantly increased (t=1.98, p<.05; t=2.88, p<.01; t=2.46, p<.05, respectively) for the treatment group compared with the control group. Two of the nine factors, parental attitudes favorable toward drug use and individual favorable attitudes toward antisocial behavior, decreased more for the control group compared with treatment group (t=2.90, p<.01 and t=2.41, p<.05). Post-treatment, the overall family domain average score decreased (t=2.01, p<.05) for the treatment group when compared with the control group. Effect sizes were very small for most differences between the groups over time; the greatest change observed was for rebelliousness, with a small effect (g=-0.2). Exhibit 21. Bivariate Analysis: Single Difference in Differences Estimation Using All Predictors (n=777) Pre-test Post-test Difference in Differences Treatment Group (T0 ) Control Group (C0 ) Diff a Treatment Group (T1 ) Control Group (C1 ) Diff a Effect size b Mean(sd) Mean(sd) (T0 -C0 ) Mean(sd) Mean(sd) (T1 -C1 ) |t|(T1 -C1 ) - (T0 -C0 ) (g) Community Risk Factors Transitions and Mobility 21.1(21.4) 19.0(20.9) 2.1 16.3(17.2) 16.3(19.6) 0.0 1.04 0.05 Low Neighborhood Attachment 24.4(32.8) 23.1(28.9) 1.3 23.9(32.6) 20.3(31.8) 3.6 0.70 0.08 Community Disorganization 57.2(21.9) 57.3(22.8) 0.0 56.2(23.0) 55.3(22.8) 0.9 0.40 0.02 Laws and Norms Favorable to Drug Use 23.3(23.0) 24.9(23.7) -1.6 17.3(19.2) 16.8(21.1) 0.5 0.94 -0.03 Perceived Availability of Drugs 37.9(31.4) 43.2(31.9) -5.3* 36.4(34.2) 38.9(38.3) -2.6 0.80 -0.12 Community Protective Factors Opportunities for Prosocial Involvement 61.8(27.0) 61.6(27.5) 0.1 67.4(26.8) 61.8(27.4) 5.6** 1.98* 0.11 Rewards for Prosocial Involvement 30.3(34.1) 31.7(34.5) -1.4 23.9(32.1) 28.6(35.8) -4.7 0.96 -0.09 Overall Community Domain Average 38.8(10.4) 39.2(9.4) -0.3 37.0(10.3) 36.8(10.5) 0.2 0.52 -0.01 Family Risk Factors Family History of Antisocial Behavior 18.8(19.1) 18.3(18.5) 0.5 16.7(19.5) 13.8(17.8) 2.9* 1.25 0.09 Parental Attitudes Favorable Toward Drug Use 12.1(15.2) 14.9(17.2) -2.8** 7.7(11.4) 6.4(11.2) 1.3 2.90** -0.05 Poor Family Management 27.2(22.7) 26.1(22.4) 1.1 15.9(18.9) 18.5(22.3) -2.6 1.67 -0.03 Family Conflict 65.4(33.7) 67.3(32.9) -1.9 47.8(36.8) 55.0(38.5) -7.2** 1.48 -0.13 Weak Parental Supervision 34.7(23.0) 34.9(22.7) -0.2 15.5(17.7) 22.6(24.6) -7.1*** 3.07** -0.15 Family Gang Influence 21.2(23.8) 21.1(24.7) 0.1 14.4(23.7) 13.1(22.4) 1.3 0.51 0.03 Family Protective Factors Attachment 64.8(24.9) 65.5(24.9) -0.7 73.0(23.8) 69.6(27.3) 3.4 1.60 0.05 Opportunities for Prosocial Involvement 68.8(31.6) 72.4(28.8) -3.7 78.9(28.4) 73.8(31.1) 5.1* 2.88** 0.02 Rewards for Prosocial Involvement 66.8(25.6) 66.8(24.1) 0.0 73.2(24.2) 68.2(26.7) 5.0** 1.95 0.10 Overall Family Domain Average 31.0(14.1) 30.9(13.3) 0.1 21.4(14.1) 24.2(15.8) -2.8** 2.01* -0.09 INFORME TÉCNICO 34 Pre-test Post-test Difference in Differences Treatment Group (T0 ) Control Group (C0 ) Diff a Treatment Group (T1 ) Control Group (C1 ) Diff a Effect size b Mean(sd) Mean(sd) (T0 -C0 ) Mean(sd) Mean(sd) (T1 -C1 ) |t|(T1 -C1 ) - (T0 -C0 ) (g) Peer/Individual Risk Factors Rebelliousness 51.3(15.0) 51.1(16.3) 0.2 38.4(16.0) 45.6(20.4) -7.2*** 4.25*** -0.20 Rewards for Antisocial Involvement 15.9(23.4) 17.4(24.8) -1.6 12.9(22.8) 15.5(25.1) -2.6 0.41 -0.09 Favorable Attitudes Toward Drug Use 21.2(22.1) 23.4(23.7) -2.1 13.3(17.7) 12.2(17.9) 1.1 1.54 -0.02 Favorable Attitudes Toward Antisocial Behavior 21.8(20.3) 23.7(22.9) -1.9 13.3(16.1) 10.6(14.8) 2.7* 2.41* 0.02 Perceived Risks of Drug Use 29.7(30.4) 27.8(28.5) 2.0 21.5(26.7) 17.1(24.6) 4.4* 0.88 0.11 Friends’ Use of Drugs 18.7(24.8) 20.5(24.2) -1.7 14.1(22.9) 14.8(26.7) -0.7 0.40 -0.05 Interaction with Antisocial Peer 17.7(17.1) 18.7(19.4) -1.0 14.3(18.2) 11.9(18.0) 2.3 1.83 0.04 Intentions to Use 15.4(21.7) 17.4(24.4) -2.0 13.1(21.9) 13.3(24.0) -0.2 0.79 -0.05 Antisocial Tendencies 43.4(14.7) 42.4(14.9) 1.0 28.3(14.9) 30.4(15.7) -2.1 2.04* -0.03 Critical Life Events 42.9(20.0) 44.1(20.7) -1.2 34.5(21.5) 32.1(19.2) 2.4 1.77 0.03 Impulsive Risk Taking 67.6(18.4) 65.1(19.6) 2.5 43.0(22.4) 48.2(26.6) -5.2** 3.40** -0.05 Neutralization of Guilt 54.7(14.3) 55.5(13.9) -0.8 39.1(15.6) 40.8(16.0) -1.7 0.65 -0.08 Negative Peer Influence 35.6(16.5) 36.7(17.1) -1.1 29.5(17.5) 32.7(19.0) -3.2* 1.16 -0.12 Peer Delinquency 14.9(14.7) 15.3(16.7) -0.4 8.6(14.7) 6.7(13.7) 1.9 1.49 0.05 Peer/Individual Protective Factors Belief in the Moral Order 65.5(23.1) 61.9(23.2) 3.6* 77.2(20.0) 74.6(21.5) 2.6 0.44 0.14 Rewards for Prosocial Involvement 52.4(25.3) 53.5(25.1) -1.0 52.2(24.8) 53.6(26.6) -1.4 0.14 -0.05 Interaction with Prosocial Peers 53.3(29.0) 52.5(28.0) 0.8 56.0(31.2) 47.5(34.9) 8.5*** 2.46* 0.15 Social Skills 66.4(20.9) 66.8(21.6) -0.4 71.7(19.0) 69.9(19.9) 1.9 1.08 0.04 Overall Peer/Individual Domain Average 34.1(8.5) 34.7(8.3) -0.6 25.9(9.8) 27.0(10.8) -1.1 0.46 -0.08 Overall Risk Factor Average Score 34.3(8.0) 34.6(7.8) -0.3 27.0(9.1) 28.3(10.2) -1.2 0.99 -0.09 Note. Protective factors were reversely scored to calculate overall average by each domain. Higher score in each overall average score indicates higher risk. a. t-test was performed to compare the mean differences between treatment and control group. b. Effect size = Hedges’s g estimation *p < .05; ** p < .01; *** p < .001 UNA EVALUACIÓN DE PROPONTE MÁS: UN PROGRAMA HONDUREÑO DE PREVENCIÓN SECUNDARIA 35 Exhibit 22 presents the regression results on the effect of treatment on risk and protective factors. First, in terms of community domain, consistent with bivariate analysis, receiving treatment was significantly related to opportunities for prosocial involvement, net of pretreatment score and geographical variations (b=5.76, p <.001). Second, with regard to family domain, receiving treatment was negatively associated with three risk factors: poor family management, family conflict, and weak parental supervision (b=-2.83, p <.05; b=-6.78, p <.05; b=-6.83, p <.001, respectively). Receiving treatment also was significantly associated with an increase in the score of all family protective factors: attachment, opportunities for prosocial involvement and rewards for prosocial involvement (b=3.56, p <.05; b=5.98, p <.01; b=4.66, p <.01, respectively). In addition, treatment group youth experienced a significant decrease in their overall family domain average score (b=-2.79, p <.01). Last, in terms of peer-individual domain, those who received treatment experienced a significant decline in four peer-individual risk factors: rebelliousness, antisocial tendencies, impulsive risk taking, and negative peer influence (b=-7.18, p <.001; b=-2.43, p <.05; b=-5.59, p <.05; b=-2.97, p <.05, respectively). Treatment was associated with an increase in interactions with prosocial peers (b=8.32, p <.05). Similar to what we found from the previous analysis, however, some factors were significantly related to treatment, but in an unexpected direction: Treatment youth reported more family history of antisocial behavior, favorable attitudes toward antisocial behavior, perceived risks of drug use, and peer delinquency when compared with youth in the control group (b=2.27, p <.05; b=3.02, p <.05; b=4.26, p <.05; b=2.07, p <.01, respectively). Exhibit 22. Multivariate Analysis: Regression on Risk and Protective Factors (n=777) Treatment Pretreatment score Effect size a b (se) Sig. b (se) Sig. Community Risk Factors Transitions and Mobility -0.73 (1.56) 0.34 (0.05) *** 0.00 Low Neighborhood Attachment 3.12 (2.84) 0.26 (0.05) *** 0.11 Community Disorganization 0.86 (2.28) 0.23 (0.02) *** 0.04 Laws and Norms Favorable to Drug Use 0.82 (1.09) 0.22 (0.04) *** 0.02 Perceived Availability of Drugs 0.04 (2.47) 0.50 (0.06) *** -0.07 Community Protective Factors Opportunities for Prosocial Involvement 5.76 (1.19) *** 0.29 (0.05) *** 0.20 Rewards for Prosocial Involvement -4.06 (2.84) 0.29 (0.05) *** -0.13 Overall Community Domain Average 0.20 (1.28) 0.27 (0.05) *** 0.01 Family Risk Factors Family History of Antisocial Behavior 2.27 (1.20) * 0.37 (0.04) *** 0.16 Parental Attitudes Favorable Toward Drug Use 1.58 (0.84) 0.12 (0.03) ** 0.11 Poor Family Management -2.83 (1.15) * 0.29 (0.06) *** -0.11 Family Conflict -6.78 (2.46) * 0.35 (0.04) *** -0.19 Weak Parental Supervision -6.83 (1.31) *** 0.26 (0.04) *** -0.31 Family Gang Influence 1.46 (1.44) 0.34 (0.06) *** 0.06 INFORME TÉCNICO 36 Treatment Pretreatment score Effect size a b (se) Sig. b (se) Sig. Family Protective Factors Attachment 3.56 (1.54) * 0.51 (0.02) *** 0.12 Opportunities for Prosocial Involvement 5.98 (1.96) ** 0.32 (0.06) *** 0.16 Rewards for Prosocial Involvement 4.66 (1.51) ** 0.45 (0.04) *** 0.18 Overall Family Domain Average -2.79 (0.75) ** 0.57 (0.05) *** -0.17 Peer/Individual Risk Factors Rebelliousness -7.18 (1.48) *** 0.12 (0.04) ** -0.38 Rewards for Antisocial Involvement -2.07 (1.47) 0.32 (0.05) *** -0.10 Favorable Attitudes Toward Drug Use 1.46 (1.27) 0.12 (0.03) ** 0.07 Favorable Attitudes Toward Antisocial Behavior 3.02 (1.25) * 0.13 (0.04) * 0.18 Perceived Risks of Drug Use 4.26 (1.51) * 0.19 (0.05) ** 0.18 Friends’ Use of Drugs 0.37 (1.44) 0.48 (0.05) *** -0.02 Interaction with Antisocial Peer 2.90 (1.40) 0.36 (0.07) *** 0.14 Intentions to Use 0.54 (1.45) 0.31 (0.05) *** 0.00 Antisocial Tendencies -2.43 (1.09) * 0.29 (0.03) *** -0.14 Critical Life Events 3.01 (1.42) 0.30 (0.03) *** 0.13 Impulsive Risk Taking -5.59 (1.89) * 0.25 (0.05) *** -0.20 Neutralization of Guilt -1.62 (1.46) 0.11 (0.04) * -0.11 Negative Peer Influence -2.97 (1.34) * 0.19 (0.05) ** -0.17 Peer Delinquency 2.07 (0.61) ** 0.32 (0.07) *** 0.14 Peer/Individual Protective Factors Belief in the Moral Order 1.87 (2.11) 0.21 (0.04) *** 0.12 Rewards for Prosocial Involvement -1.18 (2.08) 0.15 (0.03) *** -0.05 Interaction with Prosocial Peers 8.32 (3.22) * 0.29 (0.05) *** 0.25 Social Skills 1.85 (1.52) 0.17 (0.03) *** 0.09 Overall Peer/Individual Domain Average -0.70 (0.65) 0.45 (0.07) *** -0.09 Overall Risk Factor Average Score -0.96 (0.64) 0.59 (0.07) *** -0.12 Note. Zones are also controlled using fixed effect but are not presented in the table. a. Effect size = Cohen’s d estimation *p < .05; ** p < .01; *** p < .001 OUTCOMES FOR CHANGES IN DELINQUENCY We also examined the impact of the program on delinquency. We began these analyses by conducting a bivariate analysis using a diff-in-diff estimator. None of the differences between the treatment and control groups attained statistical significance across any of the outcome measures (exhibit 23). Although post-treatment truancy was reduced for the treatment group compared with the control group, the difference was not statistically significant. UNA EVALUACIÓN DE PROPONTE MÁS: UN PROGRAMA HONDUREÑO DE PREVENCIÓN SECUNDARIA 37 Exhibit 23. Difference -in-Difference Test using Outcome Measures (n=777) Pre-treatment Post-treatment Difference in Differences b Treatment Group (T0 ) Control Group (C0 ) Diff a Treatment Group (T1 ) Control Group (C1 ) Diff a Effect size d Range Mean(sd) Mean(sd) (T0 -C0 ) Mean(sd) Mean(sd) (T1 -C1 ) z ((T1 -C1 ) - (T0 -C0 )) (g) Violent Behavior 0 - 3 0.35(0.5) 0.34(0.5) 0.01 0.17(0.4) 0.15(0.4) 0.02 0.38 -0.02 Property Crime 0 - 4 0.62(0.8) 0.68(0.8) -0.07 0.26(0.5) 0.23(0.5) 0.03 1.23 -0.03 Gang Involvement 0 - 3 0.13(0.5) 0.13(0.5) 0.00 0.09(0.4) 0.08(0.4) 0.01 0.31 0.02 Alcohol/Drug Use 0 - 3 0.42(0.7) 0.52(0.8) -0.10 0.30(0.6) 0.39(0.7) -0.09 -0.32 0.14 Drug Selling 0 - 2 0.09(0.4) 0.13(0.4) -0.04 0.08(0.3) 0.06(0.3) 0.02 1.51 0.03 Carrying Weapon 0 - 2 0.09(0.4) 0.11(0.4) -0.02 0.05(0.3) 0.06(0.3) -0.01 -0.14 0.05 Truancy c 0 - 1 32.80% 33.74% -0.95 16.67% 21.18% -4.52 -1.06 0.06 Overall Delinquency 0-18 2.03(2.1) 2.26(2.4) -0.23 1.11(1.9) 1.19(1.8) -0.08 0.31 -0.07 a. t-test was performed to compare the mean differences between treatment and control group. b. poisson or negative binomial model is used to account for the distribution of counts. c. chi-square test was performed to compare proportions between treatment and control group, and logistic regression model is used for Diff-in-Diff test to account for the distribution of binary. d. Effect size = Hedges g estimation *p < .05; ** p < .01; *** p < .001 Exhibit 24 shows the results of regression analyses on our delinquency outcomes. Again, we found no significant treatment effect on the seven delinquent behaviors and overall delinquency, net of pretreatment delinquency and geographical variation. Exhibit 24. Multivariate Analysis: Regression on Outcome (n=777) Treatment Pre-treatment delinquency b se Sig. b se Sig. Effect size a Violent Behavior 0.11 (0.18) 0.77 (0.14) *** 1.10 Property Crime 0.17 (0.15) 0.51 (0.08) *** 1.14 Gang involvement 0.53 (0.31) 1.26 (0.13) *** 1.25 Drug Use -0.15 (0.13) 0.88 (0.07) *** 0.77 Drug Selling 0.55 (0.34) 2.06 (0.19) *** 1.42 Carrying Weapon -0.24 (0.32) 1.55 (0.19) *** 0.69 Truancy -0.28 (0.19) 1.04 (0.19) *** 0.76 Overall Delinquency 0.02 (0.07) 0.20 (0.01) *** 0.92 Note. Zones are also controlled using fixed effect but are not presented in the table. a. Effect size = Incidence rate ratios or Odds ratio estimation *p < .05; ** p < .01; *** p < .001 INFORME TÉCNICO 38 PART 3: INDIRECT EFFECTS In this section, we examined the total effect of treatment by examining both direct and indirect effects on outcome measures by testing for mediation effects. A mediated effect indicates that the effect of an independent variable, in this case treatment, on a dependent variable, such as delinquency, is transmitted through a third variable (i.e., family communication), which is called a mediator or intervening variable (Shrout & Bolger, 2002). A direct effect indicates the association of an independent variable (i.e., treatment) with outcome variable net of the indirect paths specified in the model, whereas indirect effect indicates the association of independent variable with the outcome variable path through a mediated variable. We examine for the possibility for indirect effects because the evaluated intervention’s theory of change posits that through family counselors and their subsequent treatment program they can change family adaptability, cohesion, and satisfaction, which in turn will reduce youth risk factors. Furthermore, the risk factor paradigm posits that a reduction in risk factors will necessarily result in a reduction in other problems such as delinquency. We examine indirect effects to assess the efficacy of the program’s theory of change as shown in exhibits 10 and 11. Testing mediation is decomposing an association into components that reveal a possible causal mechanism. Figure 1. Illustrates the simplest example of a path model testing mediation that is a three-variable recursive model where X, Y, and M are observed variables and its effects are presented with arrow lines, then a, b, c, and c’. This model provides two casual paths feeding into a single dependent variable; the independent variable (X) and the mediating variable (M) directly affect the dependent variable (Y), whereas the independent variable (X) directly affects the mediator (M). Mediator is both an outcome of X and a predictor of Y. X Y c d Y Part 1 X Y M c’ a b d d Y M Part 2 The above exhibit shows the total effect (Part 1) and mediated effect (Part 2) of X on Y. When mediation occurs, the c’ path in Part 2 is smaller than the c path in Part 1, as indicated by dashed lines. Residual terms are displayed as d effects. There is a large literature using path models testing mediation effect in social science research. For example, Overbeek and colleagues (2005) tested whether parental attachment and life stress (X) indirectly influenced juvenile delinquency (Y) through the mechanism of emotional disturbance (M). Given this, we hypothesized that the treatment condition (X) will cause the levels of family adaptability/cohesion and risk/protective factors (mediators, M) to change, which in turn will cause change in the outcome of interest (Y). In the section below we first examine the hypothesized causal pathway between treatment, FACES, and overall risk factor score. The first model (see exhibit 10) suggests a causal effect of treatment on overall risk factor score, partially mediated by FACES variables. Specifically, we expected that the treatment would decrease overall risk factor score through increased in cohesion, flexibility, family communication, and family satisfaction and decreased in disengagement, enmeshed, rigidity, and chaotic. The second model (see exhibit 11) tested the hypothesized pathway between treatment, risk and protective factors, and overall delinquency. It suggests a causal effect of treatment on overall delinquency, partially mediated by risk and protective factors. We expected that treatment would decrease overall delinquency through UNA EVALUACIÓN DE PROPONTE MÁS: UN PROGRAMA HONDUREÑO DE PREVENCIÓN SECUNDARIA 39 reduced risk factors (e.g., transitions and mobility, poor family management, and rebelliousness) and increased protective factors (e.g., opportunities for prosocial involvement, family attachment, and interaction with prosocial peers). CHANGE IN OVERALL RISK FACTOR SCORE AS A FUNCTION OF CHANGE IN FAMILY ADAPTABILITY AND COHESION Exhibit 25 presents the structural model for treatment, FACES, and overall risk factor score. This model tests the indirect effect of treatment (e.g., the path from treatment to FACES domains) on overall risk factor average score. The estimated values of the standardized model coefficients, standard errors, ratio of the indirect effect to total effect, and bootstrap confidence intervals for indirect effects are listed in the exhibit. We found no significant indirect effect of treatment on overall risk factor scores through the following factors: balanced cohesion, balanced flexibility, disengagement, enmeshed, rigidity, and chaotic. Both factors in family scales, family communication and family satisfaction, significantly mediated the relationship between treatment and overall risk factor scores. Exhibit 25. SEM Analysis – Risk factors as an outcome Treatment Mediating variables Mediating variables Risk Factors Treatment Risk Factors 95% IC Mediating variables a se Sig. b se Sig. c’ se Sig. Balanced scales: Balanced cohesion 0.22 0.03 *** -0.06 0.03 -0.04 0.03 -0.030; 0.001 Balanced flexibility 0.24 0.03 *** -0.06 0.04 -0.04 0.03 -0.033; 0.002 Unbalanced scales: Disengaged -0.27 0.03 *** 0.04 0.04 -0.04 0.03 -0.032; 0.007 Enmeshed 0.03 0.03 -0.05 0.03 -0.05 0.03 -0.007; 0.002 Rigid 0.18 0.03 *** -0.02 0.03 -0.05 0.03 -0.016; 0.010 Chaotic -0.42 0.03 *** 0.02 0.04 -0.04 0.03 -0.038; 0.021 Family scales: Family communication 0.14 0.03 *** -0.1 0.04 ** -0.04 0.03 -0.029; -0.004 Family satisfaction 0.26 0.03 *** -0.1 0.04 * -0.03 0.03 -0.041; -0.002 CHANGE IN DELINQUENCY AS A FUNCTION OF CHANGE IN RISK AND PROTECTIVE FACTORS Exhibit 26 shows the structural equation models for our overall delinquency outcome measure testing the indirect effect of treatment, the paths from treatment to risk and protective factors to delinquency. The estimated values of the standardized coefficients, standard errors, and bootstrap confidence intervals of indirect effect are listed in the exhibit. With respect to the direct effect of treatment, no significant effect was observed. The results indicated several complete mediation effects. The treatment was indirectly related to overall delinquency through its association with risk and protective factors. (The bootstrap confidence intervals for indirect effects were not containing zero). Within the family domain, treatment significantly decreased overall delinquency by improving family management, reducing family conflict, and increasing parental supervision. Treatment also significantly reduced overall delinquency through enhanced family INFORME TÉCNICO 40 attachment, increasing opportunities for prosocial involvement, and increasing rewards for prosocial involvement. Our findings also illustrated that treatment reduced overall delinquency through reducing the overall family domain risk and protective factor score. Within the peer-individual domain, treatment was significantly associated with reduced overall delinquency through decreased rebelliousness, antisocial tendencies, impulsive risk taking, and negative peer influence. In addition, treatment significantly reduced overall delinquency through increased interaction with prosocial peers. We did observe an unexpected association: Treatment increased overall delinquency through increased family history of antisocial behavior, favorable attitudes towards antisocial behavior, perceived risks of drug use, interaction with antisocial peers, critical life events, and peer delinquency. Exhibit 26. SEM Analysis - Overall delinquency as an outcome (n=777) Treatment -> Mediating variables Mediating variables -> Overall delinquency Treatment -> Overall delinquency 95% CI a (se) Sig. b (se) Sig. c’ (se) Sig. Community Risk Factors Transitions and Mobility -0.02 0.03 0.09 0.06 0.02 0.05 -0.007; 0.003 Low Neighborhood Attachment 0.05 0.04 0.16 0.07 * 0.02 0.05 -0.002; 0.014 Community Disorganization 0.02 0.03 0.19 0.08 * 0.01 0.05 -0.006; 0.012 Laws and Norms Favorable to Drug Use 0.02 0.03 0.24 0.08 ** 0.02 0.05 -0.007; 0.014 Perceived Availability of Drugs -0.00 0.03 0.52 0.13 *** 0.04 0.05 -0.022; 0.023 Community Protective Factors Opportunities for Prosocial Involvement 0.11 0.03 ** -0.03 0.06 0.02 0.06 -0.013; 0.007 Rewards for Prosocial Involvement -0.06 0.03 0.18 0.07 * 0.03 0.06 -0.018; 0.000 Overall Community Domain Average 0.01 0.03 0.41 0.11 *** 0.03 0.05 -0.015; 0.021 Family Risk Factors Family History of Antisocial Behavior 0.07 0.03 * 0.41 0.11 *** 0.04 0.05 0.002; 0.042 Parental Attitudes Favorable Toward Drug Use 0.07 0.04 0.30 0.08 *** -0.01 0.05 0.000; 0.029 Poor Family Management -0.07 0.03 * 0.42 0.11 *** 0.06 0.06 -0.041; -0.001 Family Conflict -0.09 0.03 ** 0.48 0.12 *** 0.05 0.05 -0.056; -0.006 Weak Parental Supervision -0.16 0.03 *** 0.43 0.11 *** 0.10 0.06 -0.074; -0.020 Family Gang Influence 0.03 0.03 0.29 0.10 ** -0.00 0.06 -0.005; 0.021 Family Protective Factors Attachment 0.07 0.03 * -0.35 0.09 *** 0.04 0.05 -0.033; -0.002 Opportunities for Prosocial Involvement 0.10 0.03 ** -0.28 0.09 ** 0.05 0.06 -0.035; -0.005 Rewards for Prosocial Involvement 0.09 0.03 ** -0.34 0.10 *** 0.06 0.06 -0.039; -0.005 Overall Family Domain Average -0.09 0.03 ** 0.68 0.16 *** 0.08 0.05 -0.080; -0.014 UNA EVALUACIÓN DE PROPONTE MÁS: UN PROGRAMA HONDUREÑO DE PREVENCIÓN SECUNDARIA 41 Treatment -> Mediating variables Mediating variables -> Overall delinquency Treatment -> Overall delinquency 95% CI a (se) Sig. b (se) Sig. c’ (se) Sig. Peer/Individual Risk Factors Rebelliousness -0.19 0.03 *** 0.41 0.11 *** 0.11 0.06 -0.085; -0.024 Rewards for Antisocial Involvement -0.04 0.03 0.37 0.10 *** 0.04 0.05 -0.064; 0.013 Favorable Attitudes Toward Drug Use 0.04 0.04 0.42 0.10 *** -0.01 0.05 -0.007; 0.033 Favorable Attitudes Toward Antisocial Behavior 0.09 0.04 ** 0.39 0.10 *** -0.03 0.05 0.005; 0.047 Perceived Risks of Drug Use 0.08 0.04 * 0.32 0.09 *** -0.00 0.05 0.002; 0.033 Friends’ Use of Drugs 0.01 0.03 0.62 0.15 *** 0.02 0.05 -0.049; 0.064 Interaction with Antisocial Peer 0.08 0.03 * 0.47 0.12 *** -0.03 0.05 0.004; 0.054 Intentions to Use 0.01 0.03 0.43 0.11 *** 0.03 0.05 -0.036; 0.024 Antisocial Tendencies -0.08 0.04 * 0.45 0.11 *** 0.05 0.05 -0.049; -0.003 Critical Life Events 0.08 0.03 * 0.46 0.11 *** -0.02 0.05 0.003; 0.047 Impulsive Risk Taking -0.11 0.03 ** 0.58 0.14 *** 0.09 0.06 -0.082; -0.014 Neutralization of Guilt -0.05 0.04 0.56 0.13 *** 0.04 0.05 -0.051; 0.008 Negative Peer Influence -0.08 0.03 * 0.29 0.08 *** 0.05 0.06 -0.032; -0.002 Peer Delinquency 0.07 0.03 * 0.56 0.14 *** -0.05 0.05 0.002; 0.060 Peer/Individual Protective Factors Belief in the Moral Order 0.05 0.04 -0.57 0.13 *** 0.05 0.05 -0.048; 0.008 Rewards for Prosocial Involvement -0.03 0.04 -0.22 0.08 ** 0.01 0.05 -0.006; 0.015 Interaction with Prosocial Peers 0.13 0.03 *** -0.21 0.07 ** 0.05 0.06 -0.034; -0.004 Social Skills 0.05 0.04 -0.48 0.13 *** 0.04 0.05 -0.042; 0.006 Overall Peer/Individual Domain Average -0.04 0.03 0.81 0.20 *** 0.06 0.07 -0.069; 0.018 Overall Risk Factor Average Score -0.05 0.03 0.83 0.19 *** 0.08 0.04 -0.078; 0.006 95% IC = 95% confidence intervals of total indirect effect (10,000 bootstrap replications) *p < .05; ** p < .01; *** p < .001 PART 4: SPLIT-SAMPLE ANALYSES We conducted split sample supplementary analyses to detect the direct and indirect effect of treatment on overall risk factor score and overall delinquency by different groups such as sex, age, and gang membership. In other words, we replicated the SEM analyses that were conducted above using subsamples by splitting the sample by (1) female and male, (2) 12 years old and younger and 13 years old and older, and (3) non-gangs and gangs. For each set of split sample analyses we conducted a first model that examines the hypothesized causal pathway between treatment, FACES, and overall risk factor score by different groups, and a second model that tests the hypothesized pathway between treatment, risk and protective factors, and overall delinquency. BY SEX: FEMALE VS. MALE We conducted additional supplementary SEM analyses by sex. Exhibit 27 presents the INFORME TÉCNICO 42 result of the change in overall risk factor score as a function of change in FACES. For females, treatment significant increased family cohesion, flexibility, rigidity, and satisfaction, and decreased disengagement and chaotic. However, we found no significant indirect effect of treatment on overall risk factor scores through the FACES variables among female participants. For males, cohesion, flexibility, and family communication significantly mediated the relationship between treatment and overall risk factor scores; balanced cohesion, flexibility, and family communication. Treatment significant improved family adaptability and cohesion. In addition, treatment significantly reduced overall risk factor score through an increase in cohesion, flexibility, and family communication among males. With regard to the change in delinquency as a function of change in risk and protective factors (exhibit 28), treatment significantly decreased overall delinquency by increasing parental supervision, increasing opportunities for prosocial involvement, and reducing rebelliousness for both females and males. Among female participants, treatment significantly reduced overall delinquency by reducing family conflict, enhancing family attachment, increasing opportunities for prosocial involvement, reducing the overall family domain risk factor score, decreasing impulsive risk taking, and reducing negative peer influence. We also observed an unexpected association: for females, treatment increased overall delinquency through increased family history of antisocial behavior, perceived risks of drug use, whereas, for males, treatment increased overall delinquency through interaction with antisocial peers, critical life events, and peer delinquency. Exhibit 27. SEM: treatment, FACES, and Overall Risk Factor Average Score by Sex (n=777) Female (n=281) Male (n=496) Treatment Mediating variables Mediating variables Risk Factors Treatment Risk Factors 95% IC Treatment Mediating variables Mediating variables Risk Factors Treatment Risk Factors 95% IC Mediating variables a se Sig. b se Sig. c’ se Sig. a se Sig. b se Sig. c’ (se) Sig. Balanced scales: Balanced cohesion 0.20 0.06 ** 0.25 0.04 *** -0.11 0.04 * -0.052; -0.006 Balanced flexibility 0.17 0.06 ** 0.28 0.04 *** -0.10 0.04 * -0.054; -0.004 Unbalanced scales: Disengaged -1.97 0.50 *** -0.30 0.04 *** Enmeshed Rigid 0.16 0.06 ** 0.20 0.04 *** Chaos -0.38 0.05 *** -0.45 0.04 *** Family scales: Family communication 0.18 0.04 *** -0.10 0.05 * -0.038; -0.001 Family satisfaction 0.25 0.05 *** 0.27 0.04 *** Only significant coefficients are presented. *p < .05; ** p < .01; *** p < .001 UNA EVALUACIÓN DE PROPONTE MÁS: UN PROGRAMA HONDUREÑO DE PREVENCIÓN SECUNDARIA 43 Exhibit 28. SEM Analysis - Treatment, risk/protective factors, and overall delinquency (n=777) Female (n=281) Male (n=496) Treatment Mediating variables Mediating variables Delinquency Treatment Delinquency 95% CI Treatment Mediating variables Mediating variables Delinquency Treatment Delinquency 95% CI Mediating variables a se Sig. b se Sig. c’ se Sig. a se Sig. b se Sig. c’ se Sig. Community Risk Factors Transitions and Mobility Low Neighborhood Attachment 0.21 0.08 ** Community Disorganization Laws and Norms Favorable to Drug Use 0.08 0.04 * Perceived Availability of Drugs 0.30 0.09 ** 0.21 0.10 * Community Protective Factors Opportunities for Prosocial Involvement 0.12 0.06 * 0.13 0.06 * Rewards for Prosocial Involvement -0.15 0.06 * 0.15 0.07 * Overall Community Domain Average 0.19 0.06 ** 0.40 0.11 *** Family Risk Factors Family History of Antisocial Behavior 0.12 0.06 * 0.16 0.05 ** 0.001; 0.035 0.42 0.12 *** Parental Attitudes Favorable Toward Drug Use 0.14 0.05 ** 0.30 0.09 ** Poor Family Management 0.21 0.06 ** 0.41 0.11 *** Family Conflict -0.17 0.06 ** 0.14 0.05 ** -0.046; -0.005 0.19 0.09 * Weak Parental Supervision -0.12 0.05 * 0.24 0.07 *** -0.056; -0.003 -0.19 0.04 *** 0.39 0.11 *** -0.078; -0.016 Family Gang Influence 0.12 0.04 ** 0.27 0.10 * INFORME TÉCNICO 44 Female (n=281) Male (n=496) Treatment Mediating variables Mediating variables Delinquency Treatment Delinquency 95% CI Treatment Mediating variables Mediating variables Delinquency Treatment Delinquency 95% CI Mediating variables a se Sig. b se Sig. c’ se Sig. a se Sig. b se Sig. c’ se Sig. Family Protective Factors Attachment 0.14 0.05 ** -0.14 0.05 ** -0.037; -0.003 -0.39 0.11 *** Opportunities for Prosocial Involvement 0.13 0.06 * -0.09 0.04 * -0.138; -0.001 0.09 0.04 * -0.29 0.10 ** -0.038; 0.000 Rewards for Prosocial Involvement 0.15 0.05 ** -0.39 0.11 ** Overall Family Domain Average -0.13 0.05 ** 0.34 0.10 *** -0.072; -0.005 0.69 0.16 *** Peer/Individual Risk Factors Rebelliousness -0.24 0.06 *** 0.27 0.08 ** -0.097; -0.003 -0.16 0.05 *** 0.39 0.11 *** -0.078; -0.013 Rewards for Antisocial Involvement 0.12 0.055 * 0.42 0.11 *** Favorable Attitudes Toward Drug Use 0.17 0.05 ** 0.40 0.10 *** Favorable Attitudes Toward Antisocial Behavior 0.16 0.05 ** 0.40 0.10 *** Perceived Risks of Drug Use 0.12 0.06 * 0.16 0.05 ** 0.001; 0.038 0.27 0.09 ** Friends’ Use of Drugs 0.23 0.07 ** 0.64 0.17 *** Interaction with Antisocial Peer 0.18 0.05 ** 0.1 0.04 * 0.53 0.14 *** 0.006; 0.074 Intentions to Use 0.15 0.05 ** 0.21 0.09 * Antisocial Tendencies 0.24 0.07 ** 0.44 0.11 *** Critical Life Events 0.18 0.06 ** 0.08 0.04 * 0.50 0.13 *** 0.001; 0.062 Impulsive Risk Taking -0.21 0.06 *** 0.25 0.07 ** -0.406; -0.098 0.56 0.14 *** UNA EVALUACIÓN DE PROPONTE MÁS: UN PROGRAMA HONDUREÑO DE PREVENCIÓN SECUNDARIA 45 Female (n=281) Male (n=496) Treatment Mediating variables Mediating variables Delinquency Treatment Delinquency 95% CI Treatment Mediating variables Mediating variables Delinquency Treatment Delinquency 95% CI Mediating variables a se Sig. b se Sig. c’ se Sig. a se Sig. b se Sig. c’ se Sig. Neutralization of Guilt 0.23 0.07 ** 0.58 0.14 *** Negative Peer Influence -0.13 0.06 * 0.14 0.05 ** -0.035; -0.001 0.26 0.09 ** Peer Delinquency 0.20 0.06 *** 0.14 0.04 ** 0.59 0.15 *** 0.016; 0.102 Peer/Individual Protective Factors Belief in the Moral Order -0.27 0.08 *** -0.57 0.14 *** Rewards for Prosocial Involvement Interaction with Prosocial Peers -0.10 0.04 ** 0.21 0.05 *** Social Skills -0.20 0.08 ** -0.49 0.13 *** Overall Peer/Individual Domain Average 0.29 0.08 *** 0.85 0.22 *** Overall Risk Factor Average Score 0.41 0.12 ** 0.85 0.20 *** Only the significant coefficients are presented. *p < .05; ** p < .01; *** p < .001 INFORME TÉCNICO 46 BY AGE: 12 YEARS OLD AND YOUNGER VS. 13 YEARS OLD AND OLDER We conducted additional supplementary SEM analyses by age (exhibits 29 and 30). In terms of change in overall risk factor score as a function of change in FACES scales, the results were similar for both participants who are 12 years and younger (younger age group) and 13 years and older (older age group). Specifically, treatment significantly increased cohesion, flexibility, rigidity, and family satisfaction, and decreased disengagement and chaotic for both groups, but their indirect effects were not statistically significant. However, for both age groups, family communication significantly mediated the relationship between treatment and overall risk factor scores. With regard to the change in delinquency as a function of change in risk and protective factors, treatment significantly decreased overall delinquency by increasing parental supervision for both age groups. For younger participants, treatment significantly reduced overall delinquency by reducing family conflict, reduced rewards for antisocial involvement and antisocial tendencies. For older participants, treatment significantly reduced overall delinquency by improving family management, family attachment, opportunities for prosocial involvement, rewards for prosocial involvement, interaction with prosocial peers, and social skill, and reducing overall family domain risk and protective score, rebelliousness, impulsive risk taking, negative peer influences, and overall risk factor average score. We also observed an unexpected association: for the younger age group, treatment increased overall delinquency through increased parental attitudes favorable toward drug use, friend’s drug use, interaction with antisocial peer, and peer delinquency, whereas, for older participants, treatment increased overall delinquency through favorable attitudes towards antisocial behavior, and critical life events. UNA EVALUACIÓN DE PROPONTE MÁS: UN PROGRAMA HONDUREÑO DE PREVENCIÓN SECUNDARIA 47 Exhibit 29. SEM: treatment, FACES, and Overall Risk Factor Average Score by Age (n=777) 12 years or younger (n=400) 13 years or older (n=377) Treatment Mediating variables Mediating variables Risk Factors Treatment Risk Factors 95% IC Treatment Mediating variables Mediating variables Risk Factors Treatment Risk Factors 95% IC Mediating variables a se Sig. b se Sig. c’ se Sig. a se Sig. b se Sig. c’ (se) Sig. Balanced scales: Balanced cohesion 0.20 0.05 *** 0.24 0.05 *** Balanced flexibility 0.21 0.05 *** 0.27 0.05 *** Unbalanced scales: Disengaged -0.26 0.05 *** -0.29 0.05 *** Enmeshed -0.10 0.04 * Rigid 0.23 0.05 *** 0.16 0.05 ** -0.09 0.05 * Chaos -0.4 0.04 *** -0.46 0.04 *** Family scales: Family communication 0.14 0.05 ** -0.12 0.06 * -0.603; -0.006 0.13 0.04 ** -0.12 0.06 ** -0.036; -0.001 Family satisfaction 0.24 0.04 *** 0.28 0.04 *** Only the significant coefficients are presented. *p < .05; ** p < .01; *** p < .001 INFORME TÉCNICO 48 Exhibit 30. SEM Analysis - Treatment, risk/protective factors, and overall delinquency (n=777) 12 years or younger (n=400) 13 years or older (n=377) Treatment Mediating variables Mediating variables Delinquency Treatment Delinquency 95% CI Treatment Mediating variables Mediating variables Delinquency Treatment Delinquency 95% CI Mediating variables a se Sig. b se Sig. c’ se Sig. a se Sig. b se Sig. c’ se Sig. Community Risk Factors Transitions and Mobility -0.12 0.05 * 0.10 0.05 * Low Neighborhood Attachment 0.11 0.05 * Community Disorganization 0.21 0.06 *** Laws and Norms Favorable to Drug Use 0.08 0.04 * 0.18 0.08 * Perceived Availability of Drugs 0.17 0.05 *** 0.47 0.13 *** Community Protective Factors Opportunities for Prosocial Involvement 0.15 0.05 ** Rewards for Prosocial Involvement Overall Community Domain Average 0.24 0.07 *** 0.28 0.10 ** Family Risk Factors Family History of Antisocial Behavior 0.19 0.05 *** 0.34 0.12 ** Parental Attitudes Favorable Toward Drug Use 0.12 0.05 * 0.14 0.04 ** 0.001; 0.031 0.27 0.09 ** Poor Family Management -0.17 0.05 *** 0.44 0.12 *** -0.091; -0.016 Family Conflict -0.17 0.05 ** 0.22 0.06 *** 0.09 0.04 * -0.060; -0.011 0.39 0.12 ** Weak Parental Supervision -0.16 0.05 ** 0.16 0.05 ** 0.09 0.04 * -0.038; -0.006 -0.16 0.05 ** 0.42 0.12 *** -0.084; -0.012 Family Gang Influence 0.09 0.04 * 0.23 0.10 * UNA EVALUACIÓN DE PROPONTE MÁS: UN PROGRAMA HONDUREÑO DE PREVENCIÓN SECUNDARIA 49 12 years or younger (n=400) 13 years or older (n=377) Treatment Mediating variables Mediating variables Delinquency Treatment Delinquency 95% CI Treatment Mediating variables Mediating variables Delinquency Treatment Delinquency 95% CI Mediating variables a se Sig. b se Sig. c’ se Sig. a se Sig. b se Sig. c’ se Sig. Family Protective Factors Attachment -0.2 0.05 *** 0.15 0.05 ** -0.21 0.08 * -0.048; -0.003 Opportunities for Prosocial Involvement 0.17 0.05 ** -0.26 0.10 * -0.060; -0.004 Rewards for Prosocial Involvement -0.14 0.05 ** 0.09 0.05 * 0.12 0.05 ** -0.29 0.10 ** -0.051; -0.004 Overall Family Domain Average 0.31 0.07 *** 0.09 0.04 * -0.14 0.05 ** 0.61 0.16 *** -0.110; -0.017 Peer/Individual Risk Factors Rebelliousness 0.12 0.05 * 0.07 0.04 * -0.31 0.06 *** 0.40 0.11 *** -0.138; -0.032 Rewards for Antisocial Involvement -0.11 0.05 * 0.15 0.04 *** 0.08 0.04 * -0.028; -0.001 0.31 0.10 ** Favorable Attitudes Toward Drug Use 0.12 0.04 ** 0.49 0.13 *** Favorable Attitudes Toward Antisocial Behavior 0.15 0.05 ** 0.11 0.05 * 0.41 0.11 *** 0.001; 0.064 Perceived Risks of Drug Use 0.12 0.05 * 0.41 0.12 *** Friends’ Use of Drugs 0.11 0.05 * 0.16 0.04 *** 0.001; 0.037 0.65 0.18 *** Interaction with Antisocial Peer 0.18 0.05 *** 0.19 0.05 *** 0.010; 0.056 0.45 0.13 *** Intentions to Use 0.14 0.04 *** 0.45 0.13 *** Antisocial Tendencies -0.11 0.05 * 0.22 0.05 *** 0.09 0.04 * -0.041; -0.002 0.42 0.12 *** Critical Life Events 0.21 0.06 *** 0.12 0.05 * 0.39 0.11 ** 0.004; 0.063 Impulsive Risk Taking 0.20 0.06 *** -0.19 0.06 ** 0.54 0.14 *** -0.263; -0.066 Neutralization of Guilt 0.23 0.06 *** 0.53 0.14 *** Negative Peer Influence 0.13 0.05 ** -0.11 0.05 * 0.26 0.08 ** -0.043; 0.000 Peer Delinquency 0.16 0.05 ** 0.26 0.07 *** 0.010; 0.071 0.57 0.15 *** INFORME TÉCNICO 50 12 years or younger (n=400) 13 years or older (n=377) Treatment Mediating variables Mediating variables Delinquency Treatment Delinquency 95% CI Treatment Mediating variables Mediating variables Delinquency Treatment Delinquency 95% CI Mediating variables a se Sig. b se Sig. c’ se Sig. a se Sig. b se Sig. c’ se Sig. Peer/Individual Protective Factors Belief in the Moral Order Rewards for Prosocial Involvement -0.17 0.05 *** -0.58 0.15 *** Interaction with Prosocial Peers -0.11 0.04 ** Social Skills 0.14 0.05 ** 0.14 0.05 ** -0.27 0.09 ** -0.054; -0.004 Overall Peer/Individual Domain Average -0.11 0.04 ** 0.13 0.05 * -0.53 0.15 *** -0.095; -0.007 Overall Risk Factor Average Score 0.18 0.05 *** 0.79 0.22 *** Only the significant coefficients are presented. *p < .05; ** p < .01; *** p < .001 UNA EVALUACIÓN DE PROPONTE MÁS: UN PROGRAMA HONDUREÑO DE PREVENCIÓN SECUNDARIA 51 BY GANG MEMBERSHIP: NON-GANG VS. GANG We conducted additional supplementary SEM analyses by gang membership (see exhibits 31 and 32). In terms of change in overall risk factor score as a function of change in FACES scores, the results varied by non-gang members (secondary group) and gang members (tertiary group). In terms of the direct effects of treatment on FACES scores, non-gang members experienced significant increases in cohesion, flexibility, rigidity, family communication, family dissatisfaction, and decreases in disengagement and chaotic as a result of treatment. Gang members also experienced significant increases in family satisfaction and decreases in disengagement and chaotic as a result of treatment. Indirect effects of treatment were also observed. Non-gang members who received treatment experienced significant decreases in their overall risk factor score through improved family communication and family satisfaction. For gang members, treatment significantly decreased the overall risk factor score through a reduced chaotic scale score. With regard to the change in delinquency as a function of change in risk and protective factors, non-gang members who received treatment experienced significant decreases in their overall delinquency through increased in parental supervision, family attachment, family opportunities for prosocial involvement, rewards for prosocial involvement, and interaction with prosocial peers, and decreases in rebelliousness, antisocial tendencies, impulsive risk taking, and negative peer influence. While no indirect effect of treatment was observed for gang members. we observed an unexpected association among non￾gang members: treatment significantly increased overall delinquency through increased interaction with antisocial peer, critical life events, and peer delinquency. Exhibit 31. SEM: treatment, FACES, and Overall Risk Factor Average Score by Gang Membership (n=777) Secondary (n=651) Tertiary (n=126) Treatment Mediating variables Mediating variables Risk Factors Treatment Risk Factors 95% IC Treatment Mediating variables Mediating variables Risk Factors Treatment Risk Factors 95% IC Mediating variables a se Sig. b se Sig. c’ se Sig. a se Sig. b se Sig. c’ se Sig. Balanced scales: Balanced cohesion 0.23 0.04 *** Balanced flexibility 0.25 0.04 *** Unbalanced scales: Disengaged -0.26 0.04 *** -0.38 0.08 *** Enmeshed Rigid 0.19 0.04 *** Chaos -0.42 0.03 *** -0.45 0.08 *** 0.23 0.08 ** -4.595; -0.625 Family scales: Family communication 0.15 0.03 *** -0.12 0.05 ** -0.037; -0.004 Family satisfaction 0.26 0.03 *** -0.11 0.04 * -0.053; -0.007 0.29 0.08 *** Only the significant coefficients are presented. *p < .05; ** p < .01; *** p < .001 INFORME TÉCNICO 52 Exhibit 32. SEM Analysis - Treatment, risk/protective factors, and overall delinquency (n=777) Secondary (n=651) Tertiary (n=126) Treatment Mediating variables Mediating variables Delinquency Treatment Delinquency 95% CI Treatment Mediating variables Mediating variables Delinquency Treatment Delinquency 95% CI Mediating variables a se Sig. b se Sig. c’ se Sig. a se Sig. b se Sig. c’ se Sig. Community Risk Factors Transitions and Mobility Low Neighborhood Attachment 0.29 0.09 ** Community Disorganization 0.28 0.10 ** Laws and Norms Favorable to Drug Use 0.24 0.09 * 0.20 0.09 * Perceived Availability of Drugs 0.55 0.15 *** Community Protective Factors Opportunities for Prosocial Involvement 0.12 0.04 ** Rewards for Prosocial Involvement 0.28 0.10 ** Overall Community Domain Average 0.46 0.13 ** Family Risk Factors Family History of Antisocial Behavior 0.48 0.14 ** Parental Attitudes Favorable Toward Drug Use 0.33 0.10 ** 0.02 0.01 * Poor Family Management 0.49 0.14 *** Family Conflict 0.51 0.15 ** -0.17 0.08 * Weak Parental Supervision -0.17 0.04 *** 0.50 0.14 *** -0.098; -0.021 Family Gang Influence 0.27 0.11 * UNA EVALUACIÓN DE PROPONTE MÁS: UN PROGRAMA HONDUREÑO DE PREVENCIÓN SECUNDARIA 53 Secondary (n=651) Tertiary (n=126) Treatment Mediating variables Mediating variables Delinquency Treatment Delinquency 95% CI Treatment Mediating variables Mediating variables Delinquency Treatment Delinquency 95% CI Mediating variables a se Sig. b se Sig. c’ se Sig. a se Sig. b se Sig. c’ se Sig. Family Protective Factors Attachment 0.08 0.04 * -0.46 0.13 *** -0.049; -0.001 Opportunities for Prosocial Involvement 0.09 0.04 * -0.33 0.11 ** -0.043; -0.003 Rewards for Prosocial Involvement 0.09 0.04 * -0.38 0.12 ** -0.055; -0.003 Overall Family Domain Average Peer/Individual Risk Factors Rebelliousness -0.19 0.04 *** 0.48 0.14 *** -0.103; -0.023 -0.19 0.10 * Rewards for Antisocial Involvement 0.38 0.11 ** 0.01 0.00 * Favorable Attitudes Toward Drug Use 0.35 0.10 *** 0.35 0.16 * Favorable Attitudes Toward Antisocial Behavior 0.02 0.00 *** Perceived Risks of Drug Use 0.28 0.09 ** 0.25 0.12 * Friends’ Use of Drugs 0.60 0.17 *** Interaction with Antisocial Peer 0.09 0.04 * 0.49 0.14 *** 0.004; 0.066 Intentions to Use 0.47 0.13 *** Antisocial Tendencies -0.09 0.04 * 0.49 0.13 *** -0.059; -0.002 Critical Life Events 0.09 0.038 * 0.46 0.13 *** 0.004; 0.055 Impulsive Risk Taking -0.10 0.04 * 0.60 0.16 *** -0.081 :-0.006 Neutralization of Guilt 0.53 0.15 *** Negative Peer Influence -0.11 0.04 ** 0.32 0.10 ** -0.047; -0.005 Peer Delinquency 0.08 0.04 * 0.51 0.14 *** 0.002; 0.061 INFORME TÉCNICO 54 Secondary (n=651) Tertiary (n=126) Treatment Mediating variables Mediating variables Delinquency Treatment Delinquency 95% CI Treatment Mediating variables Mediating variables Delinquency Treatment Delinquency 95% CI Mediating variables a se Sig. b se Sig. c’ se Sig. a se Sig. b se Sig. c’ se Sig. Peer/Individual Protective Factors Belief in the Moral Order -0.62 0.17 *** Rewards for Prosocial Involvement -0.24 0.09 ** Interaction with Prosocial Peers 0.11 0.04 ** -0.29 0.10 ** -0.044; -0.004 0.24 0.09 ** Social Skills -0.52 0.16 ** Overall Peer/Individual Domain Average Overall Risk Factor Average Score 0.79 0.21 *** Only the significant coefficients are presented. *p < .05; ** p < .01; *** p < .001 UNA EVALUACIÓN DE PROPONTE MÁS: UN PROGRAMA HONDUREÑO DE PREVENCIÓN SECUNDARIA 55 CONCLUSIONS In 2016, Arizona State University’s Center for Violence Prevention and Community Safety undertook an evaluation of the Proponte Mas program in Honduras to assess its impact on risk and protective factors associated with delinquency. This report has presented the results of that evaluation. To our knowledge, this randomized control trial has been one of the very few conducted in the Northern Triangle, and is the first to examine the impact of a family-based intervention program on reducing risk factors and their association with subsequent delinquency. Proponte Más began by qualifying youth who lived in fourteen intervention zones located within five municipalities in Honduras and then randomly assigning them to treatment and control groups. The youth were referred to the program by family members, schools and other influential people in their life. They were administered the IMC BETA to determine their eligibility for services; 944 of the 4,495 youth who were assessed met the criteria (4 or more risk factors). Across all intervention zones, a total of 463 eligible youth and their families were assigned to the treatment group, and 481 were assigned to the control group. About 80% (n=372) of the treatment group youth and 84% (n=406) of the control group youth remained in the program through the post-test. Our findings provided strong support for the position that family-based interventions such as Proponte Más can succeed in decreasing risk factors and increasing protective factors, which in turn serves to decrease youth delinquency, in low￾to middle-income nations such as Honduras. Case tracking data indicated that a little over 80% of qualified youth and their families completed the program, whether in the treatment group or the control group, for a relatively low dropout rate. On average, treatment group participants spent more than 20 hours with their counselors: almost 12 hours in family meetings, five in individual meetings, and three in strategic team meetings. Youth and their families completed, on average, all of the 29 assignments made by their counselors. These findings demonstrate that, given an opportunity, youth and their families will commit to improvement, regardless of the many challenges they face when living in a complex environment replete with poverty, violence, poor education and poor living arrangements. The evaluation showed that Proponte Más had a substantial and statistically significant effect on family functioning. For example, we found that receiving treatment had a: medium effect on balanced cohesion (g=0.47) and flexibility (g=0.47), large negative effect on family disengagement (g= -0.57) and chaos (g= -0.91), small- to medium-sized positive impact on family communication (g=0.28) and family satisfaction (g=0.52). Improvements in these dimensions of family functioning have been tied to a large number of benefits. For example, prior research has shown that improvements in balanced family functioning is associated with greater medication compliance among children and better adjustment to chronic illness (Chaney and Peterson, 1989). Family cohesion has been associated with greater success in the use of prenatal care (Kugler, Yeash, and Rumbaugh, 1993), recovery from drug addiction (Kouneski, 2000), and the treatment of depression (Warner, Mufson, and Weissman, 1995). Improved family flexibility has been associated with improved coping behavior, social acceptance and academic success (Kouneski, 2000). Likewise, balanced cohesion and flexibility, family satisfaction and communication, and disengagement, rigidity, chaos and complexity have all been associated with greater rates of aggressive behavior, rule breaking, fighting, assault and other problem behavior (Đurišić, 2018). Therefore, our results suggest that the impact of Proponte Más on family functioning might affect participating families and their children in wide variety of positive ways in the near and distant future. INFORME TÉCNICO 56 Additionally, our analysis showed that Proponte Más successfully reduced the number of risk factors and increased the number of protective factors among treatment group youth. Specifically, the program was associated with significant and meaningful improvements in: opportunities for prosocial involvement in the community, family management, family conflict, parental supervision, family attachment, family opportunities for prosocial involvement, family rewards for prosocial involvement, rebelliousness, antisocial tendencies, impulsive risk taking, and negative peer influence. The effect sizes of these changes were small (ranging from d=0.11 -.38), but were the same as or larger than those found in prior evaluations of the Communities that Care model (odd ratio 1.25), one of the most widely recognized risk factor reduction programs (Feinberg et al., 2007). Mediation effects, although rarely examined in evaluations, are important to understanding the causal mechanisms that result in changed behavior. In the case of the present study, our findings suggest that Proponte Más’ family￾based intervention had a positive impact on the family, which had a positive impact on youth risk and protective factors and delinquency. Our SEM analysis showed that family communication and family satisfaction, as measured through the FACES instrument, accounted for 31% and 39% of the reductions in the total risk and protective score as measured through the IMC BETA. These findings provide support for the program’s theory of change, which posits that providing support to families with at-risk youth can mitigate their overall level of risk and increase their resilience. Stronger families provide youth with the support to identify and respond to problems in ways that reduce future risks and provides them with opportunities to learn prosocial strategies and coping techniques for addressing their problems at school and among their peers. The SEM analyses also revealed that youth participating in Proponte Más, compared with the control group, significantly reduced general levels of self-reported delinquency because of the reduction of risk factors and increase in protective factors. Participation in Proponte Más was associated with improvement in family management, family conflict, parental supervision, family attachment, family opportunities for prosocial involvement, rewards for family prosocial involvement, rebelliousness, antisocial tendencies, impulsive risk taking, peer influence, and interaction with prosocial peers, which caused declines in general levels of self￾reported delinquency. These findings suggest that family-based interventions, such as Proponte Más, conducted inside the home and focused on those who reside in the most dangerous neighborhoods in the world are possible and can be effective. It is important to point out however, that these analyses did reveal an unanticipated pattern with respect to mediated effects within some of the peer-individual risk factors and delinquency. Specifically, our SEM analyses showed that treatment was associated with a significant increase in favorable attitudes toward anti-social behavior, perceived risks of drug use, interactions with anti-social peers, critical life events, and interacting with peers who are delinquent, which was related to an increase in delinquency. We do not have the data to example possible explanations for these findings and could only speculate about this observed relationship. We are not aware of any programmatic reasons (e.g., peer group meetings called for from the intervention) that could result in these findings. Regardless, we believe that additional research is needed on this issue, and future related programming should consider adopting components focused on the reduction of favorable attitudes toward problem behavior and reducing association to problem peers. Our use of a randomized control trial addressed a number of potential issues related to threats to internal validity such as history, maturation, instrumentation, regression to the UNA EVALUACIÓN DE PROPONTE MÁS: UN PROGRAMA HONDUREÑO DE PREVENCIÓN SECUNDARIA 57 mean, selection bias, and mortality.7 Other potential limitations of the study are possible and must be noted, however. First, our methodology could have been limited by a testing effect. A testing effect occurs when a respondent’s pretest or the 7 One potential threat to our study is Regression to the Mean (RTM). Although our research was designed to mitigate the effect of RTM by using a randomized-controlled trial (RCT), at the request of one reviewer we conducted sensitivity analysis to address this potential validity threat. RTM is a universal statistical phenomenon in which outlier values of an outcome measure move towards the center of the distribution (Barnett et al., 2005; Marsden et al., 2011). This may affect results of evaluations for programs that seek to reduce extreme behaviors since, by definition, extreme behaviors may regress to the mean. Our sensitivity analysis follows Linden (2013), where the analyses seek to estimate the expected RTM effect using the within and between subject variance and a bootstrap resampling method to estimate a hypothesis test. For our analysis, we used the user written rtmci command in Stata 15 (StataCorp LP, College Station, Texas, USA). We used this procedure to help detect any possible within-subject variability. We found that the observed reductions in outcome measures (i.e., overall delinquency and overall risk factor average score) withstood the threat from RTM. Specifically, among participants above the cutoff in overall delinquency in the treatment group, the expected reduction from RTM is 1.45 (95% CI 1.12- 1.78). The actual reduction was 2.43 (95% CI 1.80 - 3.06); note that the confidence intervals do not overlap. For the overall risk factor average score, the expected RTM reduction is 4.99 (95% CI 4.15 - 5.82) among participants those with above the cutoff in the overall risk factor average score, whereas an actual reduction was 10.44 (95% CI 8.32 - 12.56); note that the confidence intervals do not overlap. Since the confidence interval for the predicted RTM effect did not overlap with its actual RTM, we affirm that our analyses and findings have good internal validity and were not contaminated by RTM effect. Related to the above, and at the request of the same reviewer, we conducted supplemental analysis to examine the possibility for spillover effects. Spillover effects occur when the treatment spills over into the non-treated population, and the non-treated population is affected by the treatment. In this evaluation, members of the treatment group could have shared their experiences with the control group, who then could have incorporated what they learned into their family’s routines and behaviors. The intervention did not provide any formal mechanism for this to occur, but one hypothesis might be that treatment and control group participants could have had contact with one another through routine neighborhood contract and relationships. Since the total number of parameters are large, we estimated the pooled spillover effect by comparing control units with one or more treated peers to other units in a same community (for more detail, see Vazquez-Bare, 2017). We also used clustered sandwich estimator by zone to obtain correct standard errors. We found no evidence of any spillover effects. None of the 51 outcome measures (i.e., FACES scales, risk and protective factors, and overall delinquency score) displayed evidence of the control group being positively influenced by the treatment group. intervention itself influences their responses to a post-test. Given that the counselors (albeit not their own counselors) conducted the follow-up post-test, respondents may have been influenced by social desirability to demonstrate change. Second, our findings were limited to self-report data, with no official or administrative data included that could have validated our findings derived from self-reports. Future research using alternative data sources is needed to examine the robustness of our findings and determine whether they are valid. Third, further longitudinal analyses are required. Our findings are based on a six-month follow￾up period. Evaluations testing the sustainability of the effect following program implementation are needed. Last, we encourage Proponte Más in the future to broaden the number and types of outcome measures that are used to evaluate the success of the program. Previous research has indicated that risk factor reduction and protective factor enhancement is associated with a broad number of outcomes; Proponte Más should consider the relevance of their model on issues such as immigration, public health and education. Last, the findings of the evaluation should not be generalized to other populations. Other geographic areas with different cultures, customs and practices might not respond similarly to the intervention, particularly without adaptation specific to those regions. Likewise, our findings are not reflective of populations other than those who live in high-risk neighborhoods and who exhibit a high number of risk factors. We do not know whether the program would be effective with those from lower risk neighborhoods, who present fewer risk factors, or who are institutionalized. Programs with similar family-based intervention components are currently being implemented in El Salvador, Saint Kitts and Nevis, Saint Lucia and Guyana. Results from these evaluations should provide further evidence on the impact of the program and assessments of its generalizability to other populations. Future research is needed to further assess the effectiveness of the family-based intervention program implemented by Proponte Más. It is important to highlight that fact that the intervention most likely has utility beyond INFORME TÉCNICO 58 delinquency. First, we recommend that future researchers examine the impact of the program on outcomes related to immigration, school, physical health, mental health and employment. For example, the evidence on the connection between strengthening family systems and the management of chronic illness suggests that the model might be useful to the field of family systems medicine (Rolland, 1999) and the field of addictions (Stanton and Todd, 1982). Understanding how such family￾based interventions impact these outcomes would provide further evidence on the possible applications of such an approach and what types of problems it is most helpful in responding to. Second, future research that employs alternative outcome measures that allow for triangulation is also needed. For example, the present study relied exclusively on self-report data from caretakers and youth. Administrative, police and other types of data would be useful to confirm data obtained through self-reporting. Third, we also recommend that the program be evaluated over a longer period. Although the results of these analyses suggest effectiveness over a short period (6 months), a longer follow up using a randomized control trial would be useful. A follow-up period of at least ten years would be necessary to fully understand the direction of the effect on multiple outcomes (e.g., crime, employment, mental health, immigration, physical health, education). This research would be useful for fully understanding the long-term consequences of risk reduction programming and its potential benefits across the life course and would assist development agencies invest their available funds with greater efficacy. In conclusion, to our knowledge, the current evaluation has been one of the few randomized control trials to be conducted in Central America, and the first to examine the effect of a family-based intervention program on risk factor reduction and delinquency. It suggests that RCTs and family-based interventions can be effectively carried out in the most violent and at-risk communities in the Western Hemisphere. Further, although future research is needed to confirm these results, it suggests that Proponte Más is an effective family based intervention that strengthens families, reduces risk factors and lowers delinquency. We recommend that Proponte Más should be recognized and promoted for its work in the implementation of family systems based approach and experimental evaluation and its capacity for providing technical assistance in other countries. REFERENCES Abt, T. & Winship, C. (2016). What works in reducing community violence: A meta￾review and field study for the northern triangle. USAID: Washington DC. Alvarado, N., Muggah, R., & Aguirre, K. (2015). Some key good and bad takeaways from a citizen security mapping tool for Latin America. International Development Bank. https:// blogs.iadb.org/seguridad-ciudadana/en/ some-key-good-and-bad-takeaways-from￾a-citizen-security-mapping-tool-for-latin￾america/ Assink, M., van der Put, C. E., Hoeve, M., de Vries, S. L., Stams, G. J. J., & Oort, F. J. (2015). Risk factors for persistent delinquent behavior among juveniles: A meta-analytic review. Clinical Psychology Review, 42, 47-61. Baker, Peter (2019, February 15). Trump Declares a National Emergency, and Provokes a Constitutional Clash. New York Times. Found at https://www.nytimes. com/2019/02/15/us/politics/national￾emergency-trump.html Barnett, A. G., Van Der Pols, J. C., & Dobson, A. J. (2005). Regression to the mean: What it is and how to deal with it.International Journal of Epidemiology, 34(1), 215-220. Bendixen, M., Endresen, I. M., & Olweus, D. (2003). Variety and frequency scales of antisocial involvement: Which one is better? Legal and Criminological Psychology, 8(2), 135-150. Bowen, M. (1993). Family Therapy in Clinical Practice. Lanham, MD: Jason Aronson. UNA EVALUACIÓN DE PROPONTE MÁS: UN PROGRAMA HONDUREÑO DE PREVENCIÓN SECUNDARIA 59 Céspedes, G., & Herz, D. (2011). The City of Los Angeles Mayor’s Office of Gang Reduction and Youth Development (GRYD) Comprehensive Strategy. Unpublished report. Chaney, J. M., & Peterson, L. (1989). Family variables and disease management in juvenile rheumatoid arthritis. Journal of Pediatric Psychology, 14, 389-403. Cohen, J. (1988). Statistical Power Analysis for the Behavioral Sciences (2nd ed.). Hillsdale, NJ: Erlbaum. Development Services Group, Inc. (2014). Family Therapy. Literature review. Washington, D.C.: Office of Juvenile Justice and Delinquency Prevention. h t t p s : / / www.ojjdp.gov/mpg/litreviews/Family_ Therapy.pdf Diamond, G. S., Wintersteen, M. B., Brown, G. K., Diamond, G. M., Gallop, R., Shelef, K., & Levy, S. (2010). Attachment-based family therapy for adolescents with suicidal ideation: A randomized controlled trial. Journal of the American Academy of Child & Adolescent Psychiatry, 49(2), 122-131. Di Tella, R., & Schargrodsky, E. 2013. Criminal recidivism after prison and electronic monitoring. Journal of Political Economy. 121(1): 28–73. Dopp, A. R. (2016). Family-based treatments for serious juvenile offenders: A multilevel meta-analysis. Dissertation, University of Missouri. Đurišić, M. (2018). Relations and communication in families of children with externalizing behavior problems. Research in Pedagogy, 8(2), 111-120. Evans, G. W., Li, D., & Whipple, S. S. (2013). Cumulative risk and child development. Psychological Bulletin, 139(6), 1342. Farrington, D. P., & Welsh, B. C. (2003). Family￾based prevention of offending: A meta￾analysis. Australian & New Zealand Journal of Criminology, 36(2), 127-151. Feinberg, M. E., Greenberg, M. T., Osgood, D. W., Sartorius, J., & Bontempo, D. (2007). Effects of the Communities That Care model in Pennsylvania on youth risk and problem behaviors. Prevention Science, 8(4), 261-270. Fisch, R., Weakland, J.H., and Segal, L. (1982). The Tactics of Change. San Francisco, CA: Wiley, John, and Sons. Foreign Aid Explorer (2019). Honduras. Found at https://explorer.usaid.gov/cd/HND on June 21, 2019. García, J., Mejía, D., and Ortega, D. 2013. Police reform, training and crime: Experimental evidence from Colombia’s Plan Cuadrantes. Documento CEDE. Gau, J. M. (2010). Basic principles and practices of structural equation modeling in criminal justice and criminology research. Journal of Criminal Justice Education, 21(2), 136-151. Gao, G. (2014, August 11). 5 facts about Honduras and immigration. Factank: New in the Numbers. Found at: https://www. pewresearch.org/fact-tank/2014/08/11/5- facts-about-honduras-and-immigration/ Hartnett, D., Carr, A., Hamilton, E., & O’Reilly, G. (2017). The effectiveness of functional family therapy for adolescent behavioral and substance misuse problems: A meta￾analysis. Family Process, 56(3), 607-619. Hawkins, J. D., Catalano, R. F., & Miller, J. Y. (1992). Risk and protective factors for alcohol and other drug problems in adolescence and early adulthood: implications for substance abuse prevention. Psychological Bulletin, 112(1), 64. Hayes, A. F. (2013). Introduction to Mediation, Moderation, and Conditional Process Analysis. New York: Guilford Press. Hedberg, E. C. (2017). Introduction to Power Analysis: Two-group Studies (Vol. 176). Sage Publications. INFORME TÉCNICO 60 Hennigan, K. M., Maxson, C. L., Sloane, D. C., Kolnick, K. A., & Vindel, F. (2014). Identifying high-risk youth for secondary gang prevention. Journal of Crime and Justice, 37(1), 104-128. Hennigan, K. M., Kolnick, K. A., Vindel, F., & Maxson, C. L. (2015). Targeting youth at risk for gang involvement: Validation of a gang risk assessment to support individualized secondary prevention. Children and Youth Services Review, 56, 86-96. Hennigan, K. M. (2017). Manual for YSET Scoring, Program Eligibility, and Cut Point Calibration Secondary Violence Prevention Approach in Honduras. Los Angeles: University of Southern California. Hill, K. G., Howell, J. C., Hawkins, J. D., & Battin￾Pearson, S. R. (1999). Childhood risk factors for adolescent gang membership: Results from the Seattle Social Development Project. Journal of Research in Crime and Delinquency, 36(3), 300-322. Hindelang, M. J., Hirschi, T., & Weis, J. G. (1981). Measuring Delinquency. Beverly Hills, CA: Sage. Jaitman, L., & Guerrero Compeán, R. (2015). Closing knowledge gaps: Toward evidence-based crime prevention policies in Latin America and the Caribbean. Inter-American Development Bank. Jolliffe, D., Farrington, D. P., Piquero, A. R., Loeber, R., & Hill, K. G. (2017). Systematic review of early risk factors for life-course￾persistent, adolescence-limited, and late￾onset offenders in prospective longitudinal studies. Aggression and Violent Behavior, 33, 15-23. Katz, C. M., & Fox, A. M. (2010). Risk and protective factors associated with gang-involved youth in Trinidad and Tobago. Revista Panamericana de Salud Pública, 27, 187-202. Katz, C. Hedberg, E., Decker, S., Nuno, L., and Daniels, A. (2017). An Interim Evaluation of Proponte Más: A Secondary Prevention Program. Phoenix: Arizona State University. Kerr, M, and Bowen, M. (1988). Family evaluation: An approach based on Bowen Theory. New York: Norton. Klein, M. W., & Maxson, C. L. (2006).Gang Structures, Crime Patterns, and Police Responses. Los Angeles: Social Science Research Institute, University of Southern California. Knerr, W., Gardner, F., & Cluver, L. (2013). Improving positive parenting skills and reducing harsh and abusive parenting in low-and middle-income countries: A systematic review. Prevention Science, 14(4), 352-363. Kouneski, E. F. (2000). The family circumplex model, FACES II, and FACES III: Overview of research and applications. Recuperado el, 1, 04-09. Kugler, J. P., Yeash, J., & Rumbaugh, P. C. (1993). The impact of sociodemographic, health care system, and family function variables on prenatal care and utilization in a military setting. The Journal of Family Practice, 37, 143-147. Kraus, M, Chan, K., Martin, A., Park, L. Leap, J., Rivas, L., Manos, K., Hennigan, K., and Kolnick, K. (2017). GRYD Gang Prevention 2017 Evaluation Report, Los Angeles, CA: California State-LA. Krohn, M. D., & Thornberry, T. P. (2008). Longitudinal perspectives on adolescent street gangs. In The long view of crime: A synthesis of longitudinal research (pp. 128-160). New York, NY: Springer. Kumpfer, K. L., Whiteside, H. O., Greene, J. A., & Allen, K. C. (2010). Effectiveness outcomes of four age versions of the Strengthening Families Program in statewide field sites. Group dynamics: Theory, research, and practice, 14(3), 211. Kumpfer, K. L., & Magalhães, C. (2018). Strengthening Families Program: An evidence-based family intervention for parents of high-risk children and adolescents. Journal of Child & Adolescent Substance Abuse, 27(3), 174-179. UNA EVALUACIÓN DE PROPONTE MÁS: UN PROGRAMA HONDUREÑO DE PREVENCIÓN SECUNDARIA 61 Lawless, J. F. (1987). Negative binomial and mixed Poisson regression. The Canadian Journal of Statistics, 15(3), 209-225. Linden, A. (2013). Assessing regression to the mean effects in health care initiatives. BMC medical research methodology, 13(1), 119. Maalouf, W., & Campello, G. (2014). The influence of family skills programmes on violence indicators: Experience from a multi-site project of the United Nations Office on Drugs and Crime in low and middle income countries. Aggression and Violent Behavior, 19(6), 616-624. MacKinnon, D. P., Fairchild, A. J., & Fritz, M. S. (2007). Mediation analysis. Annual Review of Psychology, 58, 593-614. MacKinnon, D. P., Lockwood, C. M., & Williams, J. (2004). Confidence limits for the indirect effect: Distribution of the product and resampling methods. Multivariate Behavioral Research, 39, 99–128. Maguire, E. R., Wells, W., & Katz, C. M. (2011). Measuring community risk and protective factors for adolescent problem behaviors: Evidence from a developing nation. Journal of Research in Crime and Delinquency, 48(4), 594-620. Marsden, J., Eastwood, B., Wright, C., Bradbury, C., Knight, J., & Hammond, P. (2011). How best to measure change in evaluations of treatment for substance use disorder. Addiction, 106(2), 294-302. Meyer, B. D. (1995). Natural and Quasi-experiments in economics. Journal of Business and Economic Statistics, 13(2), 151–161. Meyer, M. N., Heck, P. R., Holtzman, G. S., Anderson, S. M., Cai, W., Watts, D. J., & Chabris, C. F. (2019). Objecting to experiments that compare two unobjectionable policies or treatments. Proceedings of the National Academy of Sciences, 116(22), 10723- 10728. Minuchin, S. and Fishman, H.C. (1981). Family therapy techniques. Boston, MA: Harvard University Press. Moosavi, L. (2019). A Friendly Critique of ‘Asian Criminology’ and ‘Southern Criminology’. The British Journal of Criminology, 59(2), 257-275. Muggah, R., & Aguirre, K. (2013). Mapping citizen security interventions in Latin America: reviewing the evidence. NOREF Report, October. Murray, J., Shenderovich, Y., Gardner, F., Mikton, C., Derzon, J. H., Liu, J., & Eisner, M. (2018). Risk factors for antisocial behavior in low￾and middle-income countries: a systematic review of longitudinal studies. Crime and Justice, 47(1), 255-364.Muthén, L.K. & Muthén, B.O. (1998-2017). Mplus User’s Guide (8th ed.). Los Angeles, CA: Muthén & Muthén. Olate, R., Salas-Wright, C., & Vaughn, M. G. (2012). Predictors of violence and delinquency among high risk youth and youth gang members in San Salvador, El Salvador. International Social Work, 55(3), 383-401. Olson, D. (2011). FACES IV and the circumplex model: Validation study. Journal of marital and family therapy, 37(1), 64-80. Preacher, K. J., & Hayes, A. F. (2008). Asymptotic and resampling strategies for assessing and comparing indirect effects in multiple mediator models. Behavior Research Methods, 40, 879–891. Re, Gregg (2019, June 18th). State Department cuts new funds to El Salvador, Guatemala and Honduras, pending immigration fixes. Foxnews.com. Found at https:// www.foxnews.com/politics/el-salvador￾guatemala-honduras-funds-state￾department-immigration-fixes Ritter, N., Simon, T. R., & Mahendra, R. R. (2014). Changing course: Keeping kids out of gangs. NIJ Journal, 273, 16-27. INFORME TÉCNICO 62 Rodgers, Dennis, & Baird, Adam. (2014, July 22). Understanding Gangs in Contemporary Latin America. Forthcoming in Scott H. Decker & David C. Pyrooz (eds.), Handbook of Gangs and Gang Responses, New York: Wiley, 2015. Available at SSRN: https:// ssrn.com/abstract=2469973 Santiso, Carlos, Alvarado, N., Villa, K., Marmolejo, L., Restrepo, A., Posadas, A., Beliz, G., Olcese, M., & Gonzalez, M. (2017). Citizen Security and Justice Sector Framework Document. Citizen Security and Justice Sector Framework Document: Innovation in Citizen Services Division. International Development Bank. Found at http:// idbdocs.iadb.org/wsdocs/getdocument. aspx?docnum=EZSHARE-1023060505-26 Savignac, J. (2009). Families, youth and delinquency: the state of knowledge, and family-based juvenile delinquency prevention programs. Ottawa, Ontario: National Crime Prevention Centre. Shrout, P. E., & Bolger, N. (2002). Mediation in experimental and nonexperimental studies: New procedures and recommendations. Psychological Methods, 7(4), 422-445. Simons, R. L., Johnson, C., Conger, R. D., & Elder Jr, G. (1998). A test of latent trait versus life-course perspectives on the stability of adolescent antisocial behavior. Criminology, 36(2), 217-244. StataCorp. (2017). Stata statistical software: release 15. College Station, TX: StataCorp LLC. Sweeten, G. (2012). Scaling criminal offending. Journal of Quantitative Criminology, 28, 533- 557. Thornberry, T. P., Kearley, B., Gottfredson, D. C., Slothower, M. P., Devlin, D. N., & Fader, J. J. (2018). Reducing crime among youth at risk for gang involvement: A randomized trial. Criminology & Public Policy, 17(4), 953-989. Vazquez-Bare, G. (2017). Identification and estimation of spillover effects in randomized experiments. arXiv preprint arXiv:1711.02745. Walsh, Froma. (2003). Normal Family Processes: Growing Diversity & Complexity, Third Edition. New York: The Guilford Press. Walsh, Froma. (2006). Strengthening Family Resilience, Second Edition. New York: The Guilford Press, 33. Walsh, Froma. (2009). Spiritual Resources in Family Therapy, Second Edition. New York: The Guilford Press. Warner, V., Mufson, L., & Weissman, M. M. (1995). Offspring at high and low risk for depression and anxiety. Journal of the American Academy of Child and Adolescent Psychiatry, 34, 786-797. Weisman, C. B., & Montgomery, P. (2019). Functional Family Therapy (FFT) for behavior disordered youth aged 10–18: An overview of reviews. Research on Social Work Practice, 29(3), 333-346. Webb, V. J., Nuño, L. E., & Katz, C. M. (2016). Influence of risk and protective factors on school￾aged youth involvement with gangs, guns, crime and delinquency: Findings from the El Salvador Youth Survey. Phoenix, AZ: Center for Violence Prevention and Community Safety, Arizona State University. UNA EVALUACIÓN DE PROPONTE MÁS: UN PROGRAMA HONDUREÑO DE PREVENCIÓN SECUNDARIA 63 APPENDIX A Items descriptions by each risk and protective factors at baseline # of Items Range Mean SD Response Categories Community Risk Factors Transitions and Mobility 4 Have you changed homes in the past 6 months? 0-1 0.22 0.42 0=No; 1=Yes Have you changed schools (including changing from elementary to middle or middle to high school) in the past six months? 0-1 0.19 0.40 0=No; 1=Yes How many times have you changed schools (including changing from elementary to middle or middle to high school) since kindergarten? 0-4 0.78 0.82 0=Never…4=7 or more times How many times have you changed homes since kindergarten? 0-4 0.75 0.98 0=Never…4=7 or more times Low Neighborhood Attachment 2 I’d like to get out of my neighborhood 1-4 1.88 1.27 1=No!...4=Yes! If I had to move, I would miss the neighborhood I now live in 1-4 1.54 1.03 1=Yes!...4=No! Community Disorganization 10 How much do each of the following statements describe your neighborhood? Crime and/or drug selling 1-4 2.56 1.36 1=No!...4=Yes! Fights 1-4 3.01 1.27 1=No!...4=Yes! Lots of empty or abandoned buildings 1-4 2.82 1.33 1=No!...4=Yes! Lots of graffiti 1-4 2.30 1.34 1=No!...4=Yes! Please indicate how much of a problem each of the following is in your neighborhood. Run down or poorly kept buildings in your neighborhood 0-2 0.87 0.81 0=Not a problem… 2=A big problem Graffiti on buildings and fences in your neighborhood 0-2 0.82 0.83 0=Not a problem… 2=A big problem Hearing gunshots in your neighborhood 0-2 1.40 0.76 0=Not a problem… 2=A big problem Cars traveling too fast throughout the streets of your neighborhood 0-2 1.35 0.79 0=Not a problem… 2=A big problem Gangs in your neighborhood 0-2 1.36 0.82 0=Not a problem… 2=A big problem Cars without plates 0-2 1.20 0.86 0=Not a problem… 2=A big problem INFORME TÉCNICO 64 # of Items Range Mean SD Response Categories Laws and Norms Favorable to Drug Use 5 If a kid drank some beer, wine or hard liquor (for example, vodka, whiskey, or gin) in your neighborhood would he or she be caught by the police?* 1-4 2.18 1.34 1=Yes!...4=No! If a kid smoked marijuana in your neighborhood would he or she be caught by the police?* 1-4 1.74 1.17 1=Yes!...4=No! If a kid carried a handgun in your neighborhood would he or she be caught by the police?* 1-4 1.51 0.99 1=Yes!...4=No! How wrong would most adults (over 21) in your neighborhood think it is for kids your age… …to use marijuana? 1-4 1.52 0.87 1=Very wrong… 4=Not wrong at all …to drink alcohol? 1-4 1.66 0.92 1=Very wrong… 4=Not wrong at all Perceived Availability of Drugs 4 If you wanted to get a drug like cocaine, heroin, or amphetamine, how easy would it be for you to get some? 1-4 2.03 1.27 1=Very hard… 4=Very easy If you wanted to get some marijuana, how easy would it be for you to get some? 1-4 2.33 1.32 1=Very hard… 4=Very easy If you wanted to get a handgun, how easy would it be for you to get one? 1-4 1.70 1.09 1=Very hard… 4=Very easy If you wanted to get some beer, wine or hard liquor (for example, vodka, whiskey or gin), how easy would it be for you to get some? 1-4 2.83 1.30 1=Very hard… 4=Very easy Community Protective Factors Opportunities for Prosocial Involvement 5 There are lots of adults in my neighborhood I could talk to about something important* 1-4 3.03 1.29 1=No!...4=Yes! Which of the following activities for people your age are available in your community? Sports teamsb 0-1 0.81 0.39 0=No/I don’t know; 1=Yes Religious youth groupsb 0-1 0.72 0.45 0=No/I don’t know; 1=Yes Service clubsb 0-1 0.49 0.50 0=No/I don’t know; 1=Yes Cultural organizationsb 0-1 0.39 0.49 0=No/I don’t know; 1=Yes Rewards for Prosocial Involvement 3 My neighbors notice when I am doing a good job and let me know about it 1-4 2.07 1.32 1=No!...4=Yes! There are people in my neighborhood who are proud of me when I do something well 1-4 1.92 1.24 1=No!...4=Yes! There are people in my neighborhood who encourage me to do my best 1-4 1.80 1.19 1=No!...4=Yes! UNA EVALUACIÓN DE PROPONTE MÁS: UN PROGRAMA HONDUREÑO DE PREVENCIÓN SECUNDARIA 65 # of Items Range Mean SD Response Categories Family Risk Factors Family History of Antisocial Behavior 4 About how many adults (over 21) have you known personally who in the past six months have… … used marijuana, crack, cocaine, or other drugs? 1-5 1.54 0.96 1=None…5=5 or more adults …sold or dealt drugs? 1-5 1.16 0.54 1=None…5=5 or more adults …done other things that could get them in trouble with the police like stealing, selling stolen goods, mugging or assaulting others, etc. 1-5 1.24 0.64 1=None…5=5 or more adults … gotten drunk or high? 1-5 1.95 1.17 1=None…5=5 or more adults Parental Attitudes Favorable Toward Drug Use 5 How wrong do your parents feel it would be for you to… …smoke marijuana? 1-4 1.21 0.50 1=Very wrong… 4=Not wrong at all …steal something worth more than $5? 1-4 1.26 0.57 1=Very wrong… 4=Not wrong at all …draw graffiti, or write things or draw pictures on buildings or other property (without the owner’s permission)? 1-4 1.46 0.74 1=Very wrong… 4=Not wrong at all …pick a fight with someone? 1-4 1.68 0.84 1=Very wrong… 4=Not wrong at all …drink beer, wine or hard liquor (for example, vodka, whiskey or gin) regularly (at least once or twice a month)? 1-4 1.42 0.71 1=Very wrong… 4=Not wrong at all Poor Family Management 8 The rules in my family are clear 1-4 1.96 1.24 1=Yes!...4=No! My parents ask if I’ve gotten my homework done 1-4 1.57 1.08 1=Yes!...4=No! When I am not at home, one of my parents knows where I am and who I am with 1-4 2.00 1.28 1=Yes!...4=No! Would your parents know if you did not come home on time? 1-4 1.58 1.06 1=Yes!...4=No! My family has clear rules about alcohol and drug use 1-4 2.09 1.38 1=Yes!...4=No! If you drank some beer or wine or hard liquor (for example, vodka, whiskey or gin) without your parents’ permission, would you be caught by your parents? 1-4 1.79 1.23 1=Yes!...4=No! If you carried a handgun without your parents’ permission, would you be caught by your parents? 1-4 1.58 1.10 1=Yes!...4=No! If you skipped school, would you be caught by your parents? 1-4 1.83 1.22 1=Yes!...4=No! Family Conflict 3 We argue about the same things in my family over and over. 1-4 3.02 1.23 1=No!...4=Yes! People in my family have serious arguments 1-4 2.98 1.28 1=No!...4=Yes! People in my family often insult or yell at each other 1-4 2.97 1.27 1=No!...4=Yes! INFORME TÉCNICO 66 # of Items Range Mean SD Response Categories Weak Parental Supervision 5 When I go out, I let my parents or guardians know where I am going 1-5 2.79 1.55 1=Always… 5=Never My parents or guardians ask me where I am going when I leave the house 1-5 1.91 1.33 1=Always… 5=Never My parents or guardians know who I am with when I’m not at home or school 1-5 2.72 1.56 1=Always… 5=Never My parents or guardians know who my friends are 1-5 2.41 1.48 1=Always… 5=Never I feel that my parents or guardians care about what I do 1-5 2.12 1.43 1=Always… 5=Never Family Gang Influence 2 Including all of the people that you consider part of your family, how many family members think that you will most likely join a gang one day? 1-5 2.03 1.47 1=0…5=4 or more Currently, how many of your family members are in a gang? 1-5 1.66 1.14 1=0…5=4 or more Family Protective Factors Attachment 8 Do you feel very close to your mother? 1-4 3.51 1.04 1=No!...4=Yes! Do you share your thoughts and feelings with your mother? 1-4 3.15 1.24 1=No!...4=Yes! Do you feel close to your father? 1-4 2.71 1.41 1=No!...4=Yes! Do you share your thoughts and feelings with your father? 1-4 2.49 1.41 1=No!...4=Yes! You can talk to your parents about anything. 1-5 3.26 1.45 1=Never… 5=Always Your parents make you feel trusted. 1-5 3.53 1.43 1=Never… 5=Always You depend upon your parents for advice and guidance. 1-5 3.76 1.36 1=Never… 5=Always Your parents praise you when you do well. 1-5 3.83 1.34 1=Never… 5=Always Opportunities for Prosocial Involvement 3 If I had a personal problem, I could ask my mom or dad for help. 1-4 3.50 0.97 1=No!...4=Yes! My parents give me lots of chances to do fun things with them. 1-4 3.07 1.24 1=No!...4=Yes! My parents ask me what I think before most family decisions affecting me are made. 1-4 2.78 1.34 1=No!...4=Yes! Rewards for Prosocial Involvement 4 Do you enjoy spending time with your mother? 1-4 3.64 0.88 1=No!...4=Yes! Do you enjoy spending time with your father? 1-4 2.94 1.35 1=No!...4=Yes! My parents notice when I am doing a good job and let me know about it. 1-4 2.88 1.10 1=Never…4=All the times How often do your parents tell you they’re proud of you for something you’ve done? 1-4 2.55 1.09 1=Never…4=All the times UNA EVALUACIÓN DE PROPONTE MÁS: UN PROGRAMA HONDUREÑO DE PREVENCIÓN SECUNDARIA 67 # of Items Range Mean SD Response Categories Peer/Individual Risk Factors Rebelliousness 3 I like to see how much I can get away with. 1-4 2.58 0.71 1=Very false… 4=Very true I ignore rules that get in my way. 1-4 2.59 0.68 1=Very false… 4=Very true I do the opposite of what people tell me, just to get them mad. 1-4 2.44 0.73 1=Very false… 4=Very true Rewards for Antisocial Involvement 3 What are the chances you would be seen as cool if you… …began drinking alcoholic beverages regularly, that is, at least once or twice a month? 1-5 1.78 1.17 1=No or very little chance… 5=Very good chance …smoked marijuana? 1-5 1.58 1.09 1=No or very little chance… 5=Very good chance …carried a handgun? 1-5 1.64 1.20 1=No or very little chance… 5=Very good chance Favorable Attitudes Toward Drug Use 3 How wrong do you think it is for someone your age to… …drink beer, wine or hard liquor (for example, vodka, whiskey or gin) regularly, that is, at least once or twice a month? 1-4 1.99 1.04 1=Very wrong… 4=Not wrong at all …smoke marijuana? 1-4 1.58 0.89 1=Very wrong… 4=Not wrong at all …use heroin, cocaine, amphetamines or another illegal drug? 1-4 1.43 0.81 1=Very wrong… 4=Not wrong at all Favorable Attitudes Toward Antisocial Behavior 5 How wrong do you think it is for someone your age to… …take a handgun to school? 1-4 1.45 0.78 1=Very wrong… 4=Not wrong at all …steal something worth more than $5? 1-4 1.60 0.86 1=Very wrong… 4=Not wrong at all …pick a fight with someone? 1-4 2.02 0.97 1=Very wrong… 4=Not wrong at all …attack someone with the idea of seriously hurting them? 1-4 1.70 0.91 1=Very wrong… 4=Not wrong at all …stay away from school all day when their parents think they are at school? 1-4 1.64 0.93 1=Very wrong… 4=Not wrong at all Perceived Risks of Drug Use: How much do you think people risk harming themselves (physically or in other ways) if they… 3 try marijuana once or twice? 1-4 1.90 1.14 1=Great risk… 4=No risk smoke marijuana regularly (once or twice a week)? 1-4 1.84 1.07 1=Great risk… 4=No risk take one or two drinks of an alcoholic beverage (beer, wine, liquor) nearly every day? 1-4 1.84 1.03 1=Great risk… 4=No risk INFORME TÉCNICO 68 # of Items Range Mean SD Response Categories Friends’ Use of Drugs 3 In the past 6 months, how many of your best friends have… …used marijuana? 0-4 0.73 1.24 0=None of my friends… 4=4 of my friends …used heroin, cocaine, amphetamines, or other illegal drugs? 0-4 0.26 0.78 0=None of my friends… 4=4 of my friends …tried beer, wine or hard liquor (for example, vodka, whiskey or gin) when their parents didn’t know about it? 0-4 1.37 1.59 0=None of my friends… 4=4 of my friends Interaction with Antisocial Peer 4 In the past 6 months, how many of your best friends have… …been suspended from school? 0-4 0.99 1.22 0=None of my friends… 4=4 of my friends …carried a handgun? 0-4 0.18 0.62 0=None of my friends… 4=4 of my friends …been arrested? 0-4 0.41 1.00 0=None of my friends… 4=4 of my friends …dropped out of school? 0-4 1.33 1.38 0=None of my friends… 4=4 of my friends Intentions to Use 2 When I am an adult I will smoke marijuana. 1-4 1.27 0.66 1=No!...4=Yes! When I am an adult I will drink beer, wine, or liquor. 1-4 1.72 0.99 1=No!...4=Yes! Antisocial Tendencies 7 I am kind to others 1-5 2.48 1.30 1=Always… 5=Never I respect the feelings of others 1-5 2.42 1.41 1=Always… 5=Never I get angry easily 1-5 4.02 1.32 1=Never… 5=Always I am obedient 1-5 3.34 1.24 1=Always… 5=Never I threaten others to get what I want 1-5 1.79 1.23 1=Never… 5=Always People “blame me” for lying or cheating 1-5 3.30 1.48 1=Never… 5=Always I take things that don’t belong to me 1-5 1.64 1.03 1=Never… 5=Always UNA EVALUACIÓN DE PROPONTE MÁS: UN PROGRAMA HONDUREÑO DE PREVENCIÓN SECUNDARIA 69 # of Items Range Mean SD Response Categories Critical Life Events 7 In the last year, have you... …failed a grade in school? 0-1 0.29 0.45 0=No; 1=Yes …been expelled or suspended from your school due to disciplinary reasons? 0-1 0.16 0.37 0=No; 1=Yes …broken up with/ended a relationship with a boyfriend/ girlfriend or been broken up with by a boyfriend/girlfriend? 0-1 0.45 0.50 0=No; 1=Yes …fought or had a problem with a friend? 0-1 0.75 0.44 0=No; 1=Yes …felt forced to abandon school for any reason? 0-1 0.43 0.50 0=No; 1=Yes …has someone close to you died or been seriously hurt due to an accident or illness? 0-1 0.58 0.49 0=No; 1=Yes …has someone close to you died (or was murdered) due to violence? 0-1 0.39 0.49 0=No; 1=Yes Impulsive Risk Taking 4 Sometimes I like to do dangerous activities for fun 1-5 3.57 1.18 1=Strongly disagree… 5=Strongly agree Sometimes I find it exciting to do things that could get me in trouble 1-5 3.58 1.12 1=Strongly disagree… 5=Strongly agree I frequently do things without thinking if I’ll get in trouble or not 1-5 3.58 1.19 1=Strongly disagree… 5=Strongly agree I like to have fun when I can, even if I’ll get in trouble for doing them later on 1-5 3.88 1.04 1=Strongly disagree… 5=Strongly agree Neutralization of Guilt 6 It is okay to lie if it keeps my friends from getting in trouble with their parents or with the police 1-5 3.31 1.16 1=Strongly disagree… 5=Strongly agree It is okay to lie to someone to keep myself from getting in trouble with them 1-5 3.24 1.18 1=Strongly disagree… 5=Strongly agree It is okay to steal something if someone is rich and can easily replace it 1-5 2.35 1.03 1=Strongly disagree… 5=Strongly agree It is okay to steal small items from a store without paying because stores have a lot of money and this does not affect them 1-5 2.29 1.06 1=Strongly disagree… 5=Strongly agree It is okay to hit others if others hit me first 1-5 3.92 1.09 1=Strongly disagree… 5=Strongly agree It is okay to hit people if it’s for my own defense 1-5 4.12 0.95 1=Strongly disagree… 5=Strongly agree INFORME TÉCNICO 70 # of Items Range Mean SD Response Categories Negative Peer Influence 3 If your friends were getting in trouble in your home, would you continue being their friend? 1-5 2.60 0.89 1=No, definitely not…5=Yes, definitely If your friends were getting in trouble at school, would you continue being their friend? 1-5 2.58 0.89 1=No, definitely not…5=Yes, definitely If your friends were getting in trouble with the police, would you continue being their friend? 1-5 2.16 0.74 1=No, definitely not…5=Yes, definitely Peer Delinquency 5 During the last six months, how many friends have…a …stolen something? 1-5 1.95 1.12 1=None…5=All of them …attacked someone? 1-5 2.01 1.22 1=None…5=All of them …sold marijuana or other illegal drugs? 1-5 1.24 0.70 1=None…5=All of them …used illegal drugs? 1-5 1.56 1.05 1=None…5=All of them …belong to or have joined a gang or “mara”? 1-5 1.27 0.79 1=None…5=All of them Peer/Individual Protective Factors Belief in the Moral Order 4 It is important to be honest with your parents, even if they become upset or you get punished. 1-4 3.45 0.92 1=No!...4=Yes! I think sometimes it’s okay to cheat at school. 1-4 2.92 1.32 1=Yes!...4=No! I think it is okay to take something without asking if you can get away with it. 1-4 3.28 1.15 1=Yes!...4=No! It is all right to beat up people if they start the fight. 1-4 1.99 1.29 1=Yes!...4=No! Rewards for Prosocial Involvement 4 What are the chances you would be seen as cool if you… … worked hard at school? 1-5 3.15 1.34 1=No or very little chance… 5=Very good chance …defended someone who was being verbally abused at school? 1-5 2.96 1.46 1=No or very little chance… 5=Very good chance …regularly volunteered to do community service? 1-5 3.03 1.44 1=No or very little chance… 5=Very good chance …made a commitment to stay drug-free? 1-5 3.34 1.61 1=No or very little chance… 5=Very good chance UNA EVALUACIÓN DE PROPONTE MÁS: UN PROGRAMA HONDUREÑO DE PREVENCIÓN SECUNDARIA 71 # of Items Range Mean SD Response Categories Interaction with Prosocial Peers 5 In the past 6 months, how many of your best friends have… …participated in clubs, organizations or activities at school? 0-4 2.33 1.64 0=None of my friends… 4=4 of my friends …made a commitment to stay drug-free? 0-4 1.67 1.76 0=None of my friends… 4=4 of my friends …liked school? 0-4 2.36 1.63 0=None of my friends… 4=4 of my friends …regularly attended religious services? 0-4 1.92 1.57 0=None of my friends… 4=4 of my friends …tried to do well in school? 0-4 2.31 1.59 0=None of my friends… 4=4 of my friends Social Skills 3 It’s 8:00 on a weeknight and you are about to go over to a friend’s home when your mother asks you where you are going. You say, “Oh, just going to go hang out with some friends.” She says, “No, you’ll just get into trouble if you go out. Stay home tonight.” What would you do now? 1-4 3.12 0.90 1=Leave the house anyway… 4=Explain what you are going to do with your friend, tell your mom or dad when you’d get home, and ask if you can go out You are visiting another part of town, and you don’t know any of the people your age there. You are walking down the street, and some teenager you don’t know is walking toward you. He is about your size, and as he is about to pass you, he deliberately bumps you and you almost lose your balance. What would you say or do? 1-4 3.06 1.01 1=Push the person back… 4=Say “Excuse me” and keep on walking You are at a party at someone’s house, and one of your friends offers you a drink containing alcohol. What do you say or do? 1-4 2.81 0.90 1=Drink it… 4=Tell your friend “No thanks, I don’t drink” and suggest that you and your friend go and do something else. APPENDIX B: KEY TERMS Balanced: Balanced levels of cohesion and flexibility are most conducive to healthy family functioning. Chi-square test: A test the null hypothesis that two categorical variables are independent. Cohesion: The emotional bonding that family members have toward one another. Communication: the positive communication skills utilized in the couple or family system. Effect size: A quantitative measure of the magnitude of a phenomenon. A larger value indicates a stronger effect, with the main exception being if the effect size is an odds ratio. INFORME TÉCNICO 72 Family Flexibility: the quality and expression of leadership and organization, role relationship, and relationship rules an negotiations Flexibility: the about of change in family leadership, role relationship and relationship rules. Reliability: The overall consistency of a measure. A measure with high reliability would produce consistent results under similar conditions. Significance: Risk we are willing to take in rejecting a true null hypothesis. For example, if we select a significance level of .05, we are willing to be incorrect .05 or 5% of the time. T-test: Statistical test used to test several hypotheses including the differences between two means. Unbalanced: Unbalanced levels cohesion and flexibility are associated with problematic family functioning Validity: The overall accuracy of a measure. Validity of an assessment is the degree to which it measures what it is supposed to measure.