This document was produced for review by the United States Agency for International Development. It was prepared by LINC under Contract # 7200AA20D00017, Task Order #72053222F00002. FINAL PERFORMANCE EVALUATION OF THE LOCAL PARTNER DEVELOPMENT ACTIVITY (LPD): FINAL EVALUATION REPORT JAMIE BARNETT/FHI 360 Submitted by: LINC LLC 1100 15th St NW, Washington, DC 20005 Website: www.linclocal.org Email: info@linclocal.org This Final Evaluation of the Local Partner Development Activity (LPD) was conducted by a team of international and local researchers fielded and managed by LINC, namely Mario Martinez (Team Leader - LINC), Kate Marple-Cantrell (Assistant Team Leader/Evaluation Specialist - The Cloudburst Group), Zahra Miller (Evaluation Subject Matter Specialist - The Cloudburst Group), Simon Conté (Research Specialist - The Cloudburst Group), Makayla Barker (Research Assistant - The Cloudburst Group), and Carolyne Njihia (Task Order Manager - LINC). USAID.GOV FINAL PERFORMANCE EVALUATION OF THE LOCAL PARTNER DEVELOPMENT ACTIVITY | 1 CONTENTS ABBREVIATIONS & ACRONYMS 2 ABSTRACT 3 EXECUTIVE SUMMARY 4 INTRODUCTION 9 BACKGROUND 9 EVALUATION QUESTIONS 11 METHODOLOGY 11 LIMITATIONS 14 FINDINGS AND CONCLUSIONS 15 RECOMMENDATIONS 36 ANNEX A: STATEMENT OF WORK 40 ANNEX B: DATA COLLECTION AND ANALYSIS TOOLS 51 ANNEX C: DATA AND INFORMATION SOURCES 87 ANNEX D: EVALUATION TEAM QUALIFICATIONS AND EXPERIENCE 91 ANNEX E: SELECT YOUTH SURVEY FINDINGS 94 ANNEX F: SIGNED DISCLOSURE OF CONFLICT OF INTEREST FORMS 106 2 | FINAL PERFORMANCE EVALUATION OF THE LOCAL PARTNER DEVELOPMENT ACTIVITY (LPD) USAID.GOV ABBREVIATIONS & ACRONYMS CBSI Caribbean Basin Security Initiative CBT Cognitive behavioral therapy CDCS Country development cooperation strategy CSO Civil society organization EQ Evaluation question ESQ Evaluation sub-question FGD Focus group discussion IRB Institutional review board KII Key informant interview LPD Local Partner Development Activity MoNS Ministry of National Security PSE Private sector engagement USAID United States Agency for International Development USAID.GOV FINAL PERFORMANCE EVALUATION OF THE LOCAL PARTNER DEVELOPMENT ACTIVITY | 3 ABSTRACT USAID commissioned a final performance evaluation of the Local Partner Development (LPD) Activity (2017–2023) implemented by FHI 360. LPD has built the capacity of Jamaican civil society organizations (CSOs), the private sector, and government entities to prevent crime and violence among at-risk youth at medium and high-risk levels through evidence-based strategies. Between November 2022 and January 2023, the evaluation team conducted a document review, 42 key informant interviews with a range of activity stakeholders including the national government, grantees, CSOs, and private sector organizations. In addition, 8 focus group discussions and 149 structured interviews, were conducted with targeted youth participants. Key areas of inquiry included assessing the extent to which LPD improved resilience among targeted youth, the extent of the activity’s capacity-strengthening, and the extent of private sector engagement. The evaluation found that LPD’s integrated model with life skills training, vocational training/apprenticeships, cognitive behavioral therapy, and other components was key to increasing the resilience of youth in a short period of time, despite external challenges related to the COVID-19 pandemic, gaps in grantee capacity, and limited time and resources for interventions. Awareness and engagement workshops, specialized training, facilitating collaboration among stakeholders, and the study tour to Mexico enabled successful implementation. Private sector engagement was a critical component of the activity, although there are potential barriers to sustainability of the program outcomes more generally, including funding, willingness to employ at-risk youth, and high program attrition. Recommendations focused on training potential partners on evidence-based approaches, sustaining engagement with CSOs, the government, and the private sector, and strengthening connections with at-risk youth, including program graduates. 4 | FINAL PERFORMANCE EVALUATION OF THE LOCAL PARTNER DEVELOPMENT ACTIVITY (LPD) USAID.GOV EXECUTIVE SUMMARY EVALUATION PURPOSE USAID commissioned a final performance evaluation of the Local Partner Development (LPD) activity to: (1) determine the extent to which the LPD activity’s strategic approach improved the resilience of targeted youth, their families, and communities to crime and violence; (2) assess the degree to which targeted local organizations are able to implement evidence-based programming to improve activity outcomes; and (3) examine the extent to which private sector engagement (PSE) may improve the sustainability of youth crime and violence prevention interventions. LINC and its partner, The Cloudburst Group, led the evaluation. The evaluation period of performance was September 2022– March 2023. LPD is a USAID/Jamaica activity funded through the Caribbean Basin Security Initiative (CBSI) of the U.S. Department of State. Evaluation findings will inform broader CBSI, Department of State, and USAID programming, as well as that of other units, such as the Department of State’s Bureau of International Narcotics and Law Enforcement Affairs, that receive CBSI funding. BACKGROUND USAID/Jamaica has been supporting Jamaica to reduce crime and violence in communities through initiatives that improve community and law enforcement cooperation, reduce corruption in the public and private sectors, and increase economic opportunities and skills for vulnerable populations. The LPD activity was one of those initiatives, with a budget of $16 million and a period of performance of just over six years (February 2017–June 2023). LPD was implemented by prime partner FHI 360. For the first 2.5 years of implementation, LPD’s main objective was to strengthen the capacity of Jamaican civil society organizations (CSOs) and government entities that partner with USAID/Jamaica’s Office of Citizen Security to become effective and sustainable actors to improve citizen security and social cohesion. In 2019, LPD’s primary focus changed to strengthening the capacity of targeted Jamaican CSOs and government entities that support them to become more effective in preventing crime and violence among youth at medium and high risk levels by increasing awareness and implementation of evidence-based approaches and enhancing collaboration among the government, the private sector, and civil society. Since September 2019, LPD has been working to make Jamaican institutions more effective in advancing collaborative, evidence-based youth crime and violence prevention strategies. LPD Phase 2 worked in selected high-risk geographic areas applying a place-based approach that has proven to be more effective than a more geographically dispersed crime and violence prevention approach. This final performance evaluation focuses on Phase 2. EVALUATION QUESTIONS USAID/Jamaica, in consultation with USAID/Washington, selected the following evaluation questions, taking into account LPD’s objectives within the context of USAID/Jamaica’s CDCS, CBSI goals, and Agency priorities. USAID.GOV FINAL PERFORMANCE EVALUATION OF THE LOCAL PARTNER DEVELOPMENT ACTIVITY | 5 1. To what extent has the LPD activity improved the resilience of targeted youth and their families to crime and violence? 2. In what ways has LPD built the capacity of local government and non-government organizations to implement evidence-based interventions? 3. To what extent has LPD integrated private sector engagement to address the risk and resilience factors faced by targeted youth, families, and communities? METHODOLOGY The evaluation team applied a rigorous mixed-methods approach to evaluate the LPD activity. The final performance evaluation methodology focused on validating the theory of change; specifically, if and how LPD activities resulted in the improved resilience to crime and violence of targeted youth most at risk (medium to high risk) and their families, increased capacity of local government and non-government organizations to implement evidence-based interventions, and increased collaboration with businesses to improve the sustainability of youth crime and violence prevention interventions. To understand how and why program results were achieved or not, the evaluation team gathered qualitative evidence through 42 key informant interviews (KIIs) with activity stakeholders including national government, grantees, CSOs, and private sector organizations, and 8 focus group discussions (FGDs) with youth participants. Additionally, the team drew quantitative perception data from a non￾random, purposive sample survey of targeted youth and their families (n=149) who benefited from the program, and from monitoring and evaluation data and program reporting documents. Data collection took place between November 2022 and January 2023. The quantitative and qualitative evidence was then triangulated to produce findings, conclusions, and recommendations. LIMITATIONS General challenges such as limited time and availability of proposed respondents affected data collection. It was also difficult to ascertain respondents’ willingness to openly and honestly discuss issues and share information and insights. Because of the nature of the activity design (the non-random selection of youth participants and implementation of interventions and the absence of a reasonable control population), it was not possible to evaluate causal relationships between the interventions and the observed outcomes. In addition, qualitative findings were not intended to be representative of all organizations that received USAID support, as they were drawn from only three locations, and may therefore be skewed or biased toward certain experiences. The evaluation team experienced a number of difficulties mobilizing at-risk youth because of the nature of the demographic group targeted by the activity (some of whom are young men in conflict with the law). The evaluation team coordinated closely with LPD partner organizations and case managers (contacts known and trusted by the target respondents) to reach out to potential respondents. However, because of challenges locating and mobilizing activity graduates, current participants are overrepresented in the survey sample. Low literacy among youth participants also affected engagement with youth, potentially causing them to struggle to understand complex questions or to read survey questions. This challenge was most pronounced during FGDs with youth participants, who largely used Jamaican Patois in discussions. To help address communication issues, the evaluation team used a Jamaican co-facilitator during FGDs with youth participants to facilitate the discussion and clarify linguistic differences. 6 | FINAL PERFORMANCE EVALUATION OF THE LOCAL PARTNER DEVELOPMENT ACTIVITY (LPD) USAID.GOV FINDINGS AND CONCLUSIONS EVALUATION QUESTION 1 LPD’s integrated model with life skills training, vocational training/apprenticeships, cognitive behavior therapy (CBT), and other components was key to increasing the resilience of youth in a short period of time. It is important that this integrated approach incorporated individualized elements such as case management, mentorship, and other support tailored to specific challenges youth face. Participants strongly endorsed this sustained, integrated approach and highly praised all of the core components. Another core component of LPD’s success in increasing youth resilience was the activity’s willingness to adapt over time by adding components that emerged as necessary (such as stipends), sequencing components to enable sustained participation, and scaling up components that proved successful (such as case management/mentorship). Identified activity gaps included gaps in programming because of external circumstances such as the COVID-19 pandemic (tertiary prevention) and gaps in capacity (coordination). Additionally, the limited timeframe for interventions to effect major changes in the lives of program participants was a significant challenge. These gaps represent opportunities for future programming to continue to make sustained, system-level efforts. EVALUATION QUESTION 2 Through awareness and engagement workshops, specialized training, facilitating collaboration among stakeholders, and the study tour to Mexico, LPD successfully increased awareness and understanding of evidence-based interventions and motivated local government and non-government organizations to implement these interventions to reduce crime and violence among youth. Most respondents perceived that evidence-based interventions were effective in reducing youth risk factors. Limited budget and staff to manage LPD sub-grants was a major internal factor faced by LPD’s implementing partners that limited product creation and the achievement of outcomes. A combination of external factors, including insufficient grant budget, a short timeframe to produce results, the COVID-19 pandemic, limited coordination among implementers, donor budget limitations, youth priority on jobs (not psychosocial support), and low parental engagement limited activity results and brought into question the interventions’ sustainability. LPD’s strong convening power appears to have been a key enabling factor for successful implementation of evidence-based approaches. LPD cultivated ownership of activities among local government and non￾government organizations. The high quality of the capacity-strengthening activities—awareness and technical workshops, specialized training, and the study tour to Mexico—were key enabling factors for adopting evidence-based programming among implementing partners. LPD’s evidence-based approach is likely to be sustained by only a few implementing partners that have reported that they are already implementing or planning to implement this approach. The majority of interviewees confirmed that they will not be able to continue applying the evidence-based approach at all after LPD closes out because they lack the financial resources to sustain these costly approaches. USAID.GOV FINAL PERFORMANCE EVALUATION OF THE LOCAL PARTNER DEVELOPMENT ACTIVITY | 7 EVALUATION QUESTION 3 By integrating private sector partners into the development and implementation of its interventions, including multi-sector learning, collaboration, and mutual reinforcement, LPD promoted private sector partners’ capacity to contribute to youth crime and violence prevention. PSE was neither an afterthought nor an exclusive focus of LPD’s strategy, but rather a critical component of an integrated, holistic approach combining proven, evidence-based intervention strategies. LPD used an integrated approach to PSE youth interventions, combining employment-oriented activities such as apprenticeships and business opportunity training with CBT, anger management, drug counseling, and other life skills-oriented training. Based on feedback from youth program participants and KII participants, who cited the decreased attrition of programs using integrated interventions, an integrated approach to youth and crime violence prevention for PSE seems more likely to resonate with youth and break the cycle of self-defeating behaviors than programs focusing on a single intervention. Private sector partners identified a variety of potential barriers to greater participation in youth crime and violence prevention, including funding, willingness to employ at-risk youth, and high program attrition. Private sector organizations are increasingly interested in youth development and the challenges presented by youth crime and violence; however, they are less prepared than other types of stakeholders to work in this sphere. For PSE to fully contribute to sustainable crime and violence prevention, private sector capacity-strengthening in social development, programming approaches, project management, and skills in connecting with and navigating the donor community is essential. Sustained funding is also a crucial factor for recruiting and maintaining private sector support for youth crime and violence prevention. RECOMMENDATIONS Because this final performance evaluation report has been prepared near the end of LPD, recommendations are expected to inform design and implementation of future programming in Jamaica aimed at assisting youth at medium and high-risk levels as a means to reduce crime and violence at the secondary and tertiary levels. The key recommendations are as follows: • Few organizations in Jamaica possess the technical and organizational capacity to work with at-risk youth at the secondary and tertiary levels. USAID/Jamaica should consider training potential implementing partners/grantees in evidence-based approaches and award grants to organizations that meet minimum levels of technical and organizational capacity. • Activities such as managing grants, providing stipends to program participants, and, especially, psychosocial support are costly. USAID/Jamaica should consider commissioning a cost-effectiveness study to analyze and determine the optimum level of support needed for technical and psychosocial support to youth to effectively reduce their risk factors. • Future interventions should continue to coordinate as much as possible with the Government of Jamaica and across donors/partners (including CSOs and the private sector) to maximize program effectiveness. • Future interventions should increase engagement of youths’ family members to further improve reduction in risk factors and youth resilience to crime and violence. 8 | FINAL PERFORMANCE EVALUATION OF THE LOCAL PARTNER DEVELOPMENT ACTIVITY (LPD) USAID.GOV • Future interventions should consider involving youth in activity design, rather than engaging them midway through when issues have already arisen. • Future programming with unattached youth should focus on permanent job placement opportunities (or apprenticeships with the opportunity for permanent placement) rather than short-term internships. • Future program should share activity results and M&E data with grantees and government stakeholders to help them incorporate lessons learned into their programming. • Programming should include substantial outreach to the general business community to sensitize them to the challenges facing at-risk youth, the related social problems for the communities in which the private sector organizations work, and the benefits of working with at-risk youth. USAID.GOV FINAL PERFORMANCE EVALUATION OF THE LOCAL PARTNER DEVELOPMENT ACTIVITY | 9 INTRODUCTION As LPD neared the end of its period of performance, USAID commissioned this performance evaluation to: (1) determine the extent to which the LPD activity’s strategic approach improved the resilience of targeted youth, their families, and communities to crime and violence; (2) assess the degree to which targeted local organizations are able to implement evidence-based programming to improve activity outcomes; and (3) examine the extent to which private sector engagement may improve the sustainability of youth crime and violence prevention interventions. LINC and its partner, The Cloudburst Group, led the evaluation. The evaluation period of performance was September 2022– March 2023. LPD is an activity under the Caribbean Basin Security Initiative of the U.S. Department of State. Findings, conclusions, and recommendations from the evaluation will be instructive for broader CBSI programming considerations and for the Department of State, USAID, and other units, such as the Department of State’s Bureau of International Narcotics and Law Enforcement Affairs, that receive CBSI funding. BACKGROUND USAID/Jamaica has been supporting Jamaica to reduce crime and violence in communities through initiatives that improve community and law enforcement cooperation, reduce corruption in the public and private sectors, and increase economic opportunities and skills for vulnerable populations. The LPD activity was one of those initiatives, with a budget of $16 million and a period of performance of just over six years (February 2017–June 2023).1 LPD was implemented by prime contractor FHI 360. For the first 2.5 years of implementation, LPD’s main objective was to build the capacity of Jamaican CSOs and government entities that partner with USAID/Jamaica’s Office of Citizen Security to become effective and sustainable actors to improve citizen security and social cohesion. In 2019, LPD’s primary focus changed to strengthening the capacity of targeted Jamaican CSOs and government entities that support them to become more effective in preventing crime and violence among youth at medium and high risk levels by increasing awareness and implementation of evidence-based approaches and enhancing collaboration among the government, the private sector, and civil society. Exhibit 1 presents the different components of the LPD activity before and after the focus changed in September 2019 (Amendment 4 to the Agreement). Exhibit 1. Components of the LPD activity during Phase 1 and Phase 2 Component 1: Strengthen the capacity of targeted local entities Component 1: Strengthening targeted CSOs to implement secondary and tertiary crime and violence prevention strategies Component 2: Support social enterprises through a business incubator model Component 2: Building private sector partnerships for crime prevention 1 Original life of activity and budget were five years and $15,000,000. The Agreement was amended to extend the end date from February 2022 to June 2023 and increase the budget to $16,000,000. PHASE 1 FEBRUARY 2017–SEPTEMBER 2019 PHASE 2 OCTOBER 2019–JUNE 2023 10 | FINAL PERFORMANCE EVALUATION OF THE LOCAL PARTNER DEVELOPMENT ACTIVITY (LPD) USAID.GOV Component 3: Improve the enabling environment for civil society organizations and social enterprises Component 3: Strengthening the capacity of key Government of Jamaica entities to implement and sustain effective secondary and tertiary prevention initiatives Component 4: Intentional collaboration between civil society, government, and the private sector Since September 2019, LPD has been working to achieve the following, updated results: Jamaican institutions will become more effective in advancing collaborative, evidence-based youth crime and violence prevention strategies; CSOs and key public and private sector partners will be better able to mobilize and sustain targeted and effective secondary and tertiary prevention. To achieve these results, LPD Phase 2 worked in selected high-risk geographic areas applying a place￾based approach that has proven to be more effective than a more geographically dispersed crime and violence prevention approach. This final performance evaluation focuses on LPD Phase 2. THEORY OF CHANGE As reflected in the theory of change, LPD engaged partner organizations in working directly with the targeted group through sub-grants that encouraged the use of evidence-based strategies targeting the social, economic, and psychosocial drivers of crime and violence and improved youth identification. Exhibit 2. Theory of change, LPD2 IF LPD: THEN: SO THAT: 1. Builds key stakeholders’ awareness of evidence-based prevention approaches and improves their capacity to target youth at higher risk of perpetration of crime and violence, including youth in conflict with the law or in formal institutional settings, AND 2. Empowers local governments, civil society organizations, and community members to advocate for critical resources and reforms, LPD’s partners, through USAID funding, will 1. Holistically and cost-effectively target the social, economic, and psychosocial drivers of crime and violence and provide effective services to the most at-risk youth, 2. Create the conditions, services, and opportunities for youth in the justice system to rehabilitate, reintegrate, and avoid re-engagement with crime and violence, 3. Apply evidence-based strategies, including through government partners’ accessing and generating reliable crime and violence data, and test the efficacy of new approaches, and 4. Elevate local/public pressure in support of civil society’s ability to address needs and grievances, strengthening feedback loops to hold governments accountable to resource crime and violence prevention 1. At-risk youth experience reduced risk factors and become more resilient to becoming victims or perpetrators of crime 2. Families and communities access services, infrastructure, and support that address physical, emotional, and psychosocial needs and the complex drivers/risk factors of crime and violence, 3. Schools support students to resolve conflict nonviolently, find healthy avenues for a sense of belonging, build self-efficacy, improve academic performance, and provide workforce development opportunities, 4. Youth in conflict with the law more effectively rehabilitate and reintegrate into society, and 5. Community members, civil society organizations, and local governments will effectively advocate for resources and reforms to improve crime and violence prevention and address key community needs. 2 LPD was extended from the original date (February 2017) to February 28, 2023 in September 2019, and then further extended to June 30, 2023 (Annual Report FY 2022, p. 1). USAID.GOV FINAL PERFORMANCE EVALUATION OF THE LOCAL PARTNER DEVELOPMENT ACTIVITY | 11 EVALUATION QUESTIONS USAID/Jamaica, in consultation with USAID/Washington, selected the evaluation questions and sub￾questions listed below in order of priority. The questions also take into account LPD’s objectives within the context of USAID/Jamaica’s CDCS, CBSI goals, and Agency priorities. 1. To what extent has the LPD activity improved the resilience of targeted youth and their families to crime and violence? 1a. What interventions were most effective for beneficiaries? 1b. From the perspective of beneficiaries, what were the areas that could be improved? 1c. Discuss gaps and interventions that were least effective. How could the interventions be improved? 2. In what ways has LPD built the capacity of local government and non-government organizations to implement evidence-based interventions? 2a. What are the main barriers (internal and/or external) to local organizations adopting evidence-based approaches? 2b. What are the enabling factors that allow organizations to successfully adopt evidence-based programming? 2c. What evidence exists that beneficiary organizations are likely to continue to implement these approaches? 3. To what extent has LPD integrated private sector engagement to address the risk and resilience factors faced by targeted youth, families, and communities? 3a. What PSE approaches are most useful in youth crime and violence prevention? What are the potential barriers to greater private sector participation in this area? 3b. To what extent can PSE approaches help build sustainability in youth crime and violence prevention activities in Jamaica? 3c. What are the enabling factors that allow private sector organizations to support youth crime and violence prevention activities? 3d. What evidence exists that private sector organizations are likely to continue to provide support in this area? The evaluation team discussed the unexpected conditions or challenges the activity experienced during implementation and how LPD adapted to or resolved those challenges. METHODOLOGY The evaluation team applied a rigorous mixed-methods approach to evaluate the LPD activity. The final performance evaluation methodology focused on validating the theory of change; specifically, if and how 12 | FINAL PERFORMANCE EVALUATION OF THE LOCAL PARTNER DEVELOPMENT ACTIVITY (LPD) USAID.GOV LPD activities resulted in the improved resilience to crime and violence of targeted youth most at risk (medium to high risk) and their families, increased capacity of local government and non-government organizations to implement evidence-based interventions, and increased collaboration with businesses to improve the sustainability of youth crime and violence prevention interventions. To understand how and why program results were achieved or not, the evaluation team gathered qualitative evidence through KIIs and FGDs. Additionally, the team drew quantitative perception data from a non-random, purposive sample survey of youth most at risk who benefited from the program and from monitoring and evaluation data and program reporting documents. The quantitative and qualitative evidence was then triangulated to produce findings, conclusions, and recommendations. The methods used for the performance evaluation are described below. Annex B and C present additional details on the data sources and the data collection instruments, respectively. DOCUMENT REVIEW: The evaluation team conducted a desk review of primary and secondary sources, program documents (e.g., activity implementation plans; monitoring, evaluation, learning, and planning documents; and quarterly and annual reports), select LPD analytical products such as assessments, Government of Jamaica legislation and policy documents, donor analyses, public opinion survey reports, and policy/academic think pieces. The evaluation team paid particular attention to the modifications made to the LPD activity in 2019 to ensure that the evaluation design accounted for this shift in objectives. The evaluation team employed content analysis and fidelity analysis to understand how the LPD activity performed against implementation outcomes. The document review informed the evaluation design and culminated in the evaluation design report with the final research questions and sub-questions and the data collection instruments. KIIS AND FGDS: Qualitative interviews formed the heart of the evaluation approach. Complementing the broad input from the survey and document review, the evaluation team conducted 42 in-depth interviews with 57 key informants and facilitated 8 FGDs with 52 youth activity participants across all three sites. The evaluation team conducted KIIs and FGDs with local and national stakeholders and activity participants over three weeks between November 28 and December 16, 2022 through online interviews and in-person fieldwork. The evaluation team developed guides for semi-structured interviews and FGDs that were specific to the type of respondent, were comprehensive, and addressed the evaluation and learning questions. SITE VISITS: Site visits were preceded by a series of KIIs and/or FGDs in the capital city of Kingston with representatives of LPD, the national government, other donors, and national and international organizations. Members of the evaluation team then visited two additional LPD activity sites, St. James and St. Catherine, to collect data. At each of these locations, the evaluation team conducted interviews with CSOs and participating private sector organizations and carried out surveys with targeted youth and their families. Fieldwork also included briefings with USAID/Jamaica at the start and the end of the fieldwork. PERCEPTION SURVEY: Because of the inclusion of minors among the youth activity participants targeted for the survey, the evaluation team sought and received institutional review board approval prior to data collection. Data collection for this component included 45-minute, in-person quantitative perception surveys with targeted youth and their families in the three selected LPD activity sites. The survey largely addressed Evaluation Question 1 on changes in the resilience of targeted youth and their USAID.GOV FINAL PERFORMANCE EVALUATION OF THE LOCAL PARTNER DEVELOPMENT ACTIVITY | 13 families to crime and violence by roughly assessing youth risk for involvement in crime and violence at project close and collecting information about perceptions of services received. The team selected targeted LPD youth and their families using a stratified sampling approach. In each selected evaluation site, the survey attempted to contact all participants who received LPD’s youth risk factor assessment and families who participated in LPD parenting classes. The evaluation team believed this approach was necessary (instead of choosing a random sample of respondents to survey) because of the anticipated high non-response rate because of the demographic characteristics of the target respondents. The evaluation team interviewed 149 survey participants stratified by category of respondent (based on services received, e.g., risk assessment, therapy), sex, and age. The age of survey participants ranged from 14 to 34. Enumerators, who were trained in the tool and electronic data collection using mobile devices, administered the survey. The evaluation team triangulated survey data with analysis of LPD records such as program baseline and endline risk assessments to arrive at the richest possible picture of outcomes experienced by youth. SAMPLE SELECTION AND CRITERIA To ensure that all expected results were covered in this evaluation, the evaluation team collected data from all categories of stakeholders that were engaged in the activity across all components and results. This enabled the evaluation team to gather evidence on whether the results were achieved or not and how and why they were achieved or not. To gather qualitative data from all stakeholders, the evaluation team chose informants by applying a non-probability purposive sampling approach. Across the KIIs, the evaluation team interviewed 13 CSO representatives assisted by LPD and providing services to youth, 11 Government of Jamaica representatives working in areas of crime and violence prevention, 10 private sector representatives that worked with LPD and were providers of services to youth, 7 LPD staff, I USAID representative, and 1 other youth crime and violence prevention donor-funded project. Exhibit 3. Overview of sex breakdown of participants across data sources KIIS FGDS YOUTH SURVEY Females 37 14 45 Males 20 38 104 Total 57 52 149 14 | FINAL PERFORMANCE EVALUATION OF THE LOCAL PARTNER DEVELOPMENT ACTIVITY (LPD) USAID.GOV Exhibit 4. Overview of KII interviews CSO GOJ PRIVATE SECTOR LPD STAFF USAID OTHER YOUTH CRIME & VIOLENCE PREVENTION DONOR-FUNDED PROJECT TOTAL KIIs held 13 10 10 7 1 1 42 KII participants 22 15 11 7 1 1 57 The youth survey received 149 responses. Thirty-nine percent of respondents lived in Kingston (58), 39 percent lived in St. James (n=57), and 22 percent lived in St. Catherine (n=32). Most survey respondents (86 percent, n=126) were 18 or older; only 14 percent (n=20) of respondents were minors. Seventy percent of respondents (n=103) were male and 30 percent (n=44) were female. Most respondents (82 percent, n=120) were currently receiving services. The majority of respondents lived with their parents (66 percent, n=97) and did not have children (67 percent, n=98). Annex E contains additional youth survey results tables, including disaggregation by sex and age. LIMITATIONS There were some logistical challenges during data collection, and institutional review board approval for the youth survey was uncertain until the first week of fieldwork. However, the evaluation team was still able to nearly meet the target number of people interviewed in KIIs through consistent mobilization efforts across the stakeholder groups. Organizing KIIs was difficult because the contact information for some organizations and entities was either missing or generic, which delayed identification, contact, and scheduling of KIIs with the relevant people. Other organizations delayed confirming their interview times because of competing priorities in their schedules, and some of the confirmed interviews had to be rescheduled throughout the data collection period. There may have been a bias with the FGDs because activity participants who agreed to participate and took the time to be present were likely either currently participating in the activity and/or were those who held a mostly positive view of the LPD activity. Interviewers therefore explicitly asked about components of the activity that could be improved and probed for adverse experiences to facilitate balanced discussions. Furthermore, because of the nature of the activity design (the non-random selection of youth participants and implementation of interventions and the absence of a reasonable control population), it was not possible to evaluate causal relationships between the interventions and the observed outcomes, though correlations are still informative. Additionally, as with any primarily qualitative analysis, the primary limitation is that the analysis and conclusions may be skewed by the biases and experiences of the selected interviewees, who were drawn from only three field sites and not guaranteed to be representative of all organizations that USAID.GOV FINAL PERFORMANCE EVALUATION OF THE LOCAL PARTNER DEVELOPMENT ACTIVITY | 15 received USAID support. Evaluators attempted to mitigate this potential bias by capturing and triangulating the widest view possible via the youth perception survey. In addition to general risks such as limited time and the availability of proposed respondents, another critical aspect of the quality of the analysis was the willingness of respondents to openly and honestly discuss issues and share information and insights. The evaluation team made every effort to build a strong rapport with respondents by explaining the purpose of the study and choosing a private and comfortable environment to complete the interview. However, if respondents were concerned about how results would be used, despite the full disclosures and guarantees given in the consent statement before data collection, it is possible that they were not fully forthcoming in their answers. The youth survey faced two principal challenges. First, the evaluation team experienced a number of difficulties mobilizing respondents because of the nature of the demographic group targeted by the activity, some of whom are young men in conflict with the law. Because these respondents do not respond to unsolicited phone calls and change their phone numbers often, the evaluation team coordinated closely with LPD partner organizations and case managers (contacts known and trusted by the target respondents) to reach out to potential respondents. The evaluation team attempted to mitigate these challenges by traveling to central locations in survey respondents’ communities to interview them and by offering a small incentive for their participation. However, because of challenges locating and mobilizing activity graduates, current participants are overrepresented in the survey sample.3 Although data collection was extended by two more weeks beyond the initial three that had been planned to account for the limited number of activity participants organizations were able to mobilize each day, the data collection team was not able to meet the target of 210 survey responses. However, the number of responses reached (149) was still largely sufficient to identify trends and patterns among activity participants’ answers. The evaluation team experienced particular challenges mobilizing youth under 18 and families of LPD participants because of the limited number of active LPD programs with youth under 18 at the time of data collection. The second main challenge the youth survey faced was low literacy among youth participants, potentially causing them to struggle to understand complex questions or to read survey questions. To address this challenge, enumerators administered the survey orally to all targeted youth and were trained to explain the meaning of the survey question to respondents if needed. The evaluation team also piloted the survey instrument before launch to make sure all questions were understandable and appropriate. FINDINGS AND CONCLUSIONS EVALUATION QUESTION 1: TO WHAT EXTENT HAS THE LPD ACTIVITY IMPROVED THE RESILIENCE OF TARGETED YOUTH AND THEIR FAMILIES TO CRIME AND VIOLENCE? QUALITATIVE FINDINGS 3 The sample was restricted to program participants who were enrolled for at least six months to ensure that they had sufficient exposure to assess their experience in the program and hopefully gain benefits from it. 16 | FINAL PERFORMANCE EVALUATION OF THE LOCAL PARTNER DEVELOPMENT ACTIVITY (LPD) USAID.GOV LPD expected its activities with youth would improve participating youths’ resilience to crime and violence by providing them with various types of support to get and keep jobs, get back to school, or create income-generating activities. The majority of interview respondents from CSOs, the private sector, and the Government of Jamaica believed that LPD’s approach has been effective in reducing secondary and tertiary crime and violence. Indeed, some grantees and participants reported anecdotal reductions in crime, violence, and shootings in their neighborhoods since the start of the activity, though the activity’s scale is likely not sufficient to affect overall parish crime statistics. One interviewee remarked: “It’s a Godsend program. We hope there are more programs like this in other communities. Other youths and communities could be beneficiaries of programs like this. [We] need broader spread of programs. LPD was an excellent program and we wish it would continue and be more widespread. [It] can impact the level of crime and violence in society, and it has. We are just hoping other funding agencies will take a leaf out of this LPD booklet. It’s really a success story.” – Grantee In FGDs, activity participants credited LPD for providing an effective approach for youth crime and violence prevention (e.g., by providing economic opportunities and helping youth overcome negative behaviors such as drug/alcohol use and anger issues). Responses to LPD activities were overwhelmingly positive and respondents often described the program as “life-changing.” QUANTITATIVE FINDINGS According to the youth survey, 71 percent of respondents (n=103) were totally or highly satisfied with the program, and only 2 percent (n=3) were dissatisfied. As shown in Exhibit 5, female respondents and minor respondents reported significantly higher program satisfaction than male respondents and adult respondents. However, the small sample sizes for these subgroups make it difficult to determine why these differences exist. There did not seem to be significant differences by subgroup in the overall amount of support received, but women and minors were less likely to have received mentoring (though the difference is only statistically significant for women), potentially suggesting that additional refinement to the program components aimed at men, including the mentorship intervention components (as detailed in the Findings section), could improve overall program satisfaction. Generally, these results likely indicate that these subgroups (women and minors) have different underlying characteristics (for example, female respondents were significantly more likely than male respondents to have children: 64 percent [n=28], versus 20 percent [n=21]) and experienced the program and its benefits differently. It will be important to ensure the continuation of customized support approaches that take these differences into account. USAID.GOV FINAL PERFORMANCE EVALUATION OF THE LOCAL PARTNER DEVELOPMENT ACTIVITY | 17 Exhibit 5. Survey respondent program satisfaction by gender and age4 The primary quantitative measure employed to measure resilience to crime and violence was tracking youth risk assessment scores over time. Youth in private sector programs were risk screened, while all youth in other programs received a risk assessment at program entry.5 After completing the program, some youth received risk re-assessments, as detailed in footnote 5. When the risk for involvement in crime and violence as measured by the tool decreased over time, this was considered evidence of improved resilience. According to the data available to the evaluation team, 12 percent of youth (n=129) had been risk re-assessed as of the data collection, and 64 percent (n=83) of those youth showed decreased risk. Female participants had only slightly higher initial risk assessment scores compared to males (2.6 versus 2.4), but they were significantly more likely to show reduced risk factors at re￾assessment (76 percent [n=32] versus 59 percent [n=51]). Activity participants who received case management and those who completed more than 80 percent of the program were also significantly more likely to be re-assessed as having lower risk for involvement in crime and violence, as shown in Exhibit 6. Interestingly, activity participants who participated in group counseling were not more likely than other participants to show decreased risk assessment scores. Re-assessments are ongoing; the cumulative activity target is for 197 youth to be found during re-assessment to have reduced crime and violence risk factors.6 4 The p-value indicates whether the estimated relationship is statistically significant, or probably not due to chance. Critical values for the p-values are generally set to three different “alpha levels,” .01, .05, and .1. If a p-value is below the alpha level of .1, it means the effects are statistically significant. 5 According to the monitoring and evaluation (M&E) data available to the evaluation team, the total number of youth who have been risk assessed is 1,037. It is the evaluation team’s understanding that, according to the LPD M&E plan, a target has not been set for number of youth to be re-assessed, only a target number of youth found to have reduced risk factors that drive crime and violence. According to the 2022 M&E Plan and the 2022 Annual Report, LPD exceeded targets in 2021 (target was 35; actual was 63) but not in 2022 (target was 102; actual was 59). The target number in 2023 is 15. 6 LPD Annual Report 2022, p. 28. 18 | FINAL PERFORMANCE EVALUATION OF THE LOCAL PARTNER DEVELOPMENT ACTIVITY (LPD) USAID.GOV Exhibit 6. Percent of youth with decreased risk by intervention category CONCLUSIONS LPD’s model for secondary and tertiary crime and violence prevention appears promising and powerful. Across data sources (KIIs, FGDs, program data) there is evidence that LPD contributed to reduced risk for crime and violence among participating youth, potentially making communities that were part of this activity safer. Additionally, all data collected from participants (through FGDs and the youth survey) show high confidence in and satisfaction with the programs supported by LPD. EVALUATION SUB-QUESTION 1A: WHAT INTERVENTIONS WERE MOST EFFECTIVE FOR BENEFICIARIES? FINDINGS LIFE SKILLS: Across the KIIs, FGDs, and youth survey, respondents perceived life skills training and vocational training to be the most effective interventions. LPD identified that activity participants’ social networks often did not instill the life and employability skills needed on the pathway to employment, and designed life skills training that included topics like conflict resolution and management, communication skills, listening skills, money management, and coping with grief and loss due to violence, as well as activities like weekly workshops and interactive activities, to improve participants’ social skills. It was important for the activity to begin with these life skills because they are foundational skills that allow participants to derive the most benefit from job skills training and to navigate the professional world. VOCATIONAL TRAINING AND JOB/INTERNSHIP/APPRENTICESHIP PLACEMENT: LPD designed vocational training or business skills training (as appropriate to the specific program) to give youth tangible skills to make them more employable and potentially receive certification in a trade or skill. In many instances, this vocational training was tied to a short-term or permanent placement in the USAID.GOV FINAL PERFORMANCE EVALUATION OF THE LOCAL PARTNER DEVELOPMENT ACTIVITY | 19 role participants were being trained to perform, giving them practical experience applying the concepts learned in training, and in some cases, a pathway to permanent employment. In other LPD-supported programs, training focused on microbusiness skills and was paired with assistance starting microenterprises. Based on program M&E data, 468 targeted youths have been trained to date (cumulative 2020–2022 target is 689).7 In the FGDs, youth expressed that the support they valued most highly was the technical training and job placement support as a means to bring about a better life. Grantees and partners reported positive changes in youth who completed this training: “[The] guys… who did entrepreneurial training seemed to be the most stable in terms of their ability to organize and stay more focused… That group ‘seems to be’ on the right track so far—they had the intention of making it better, not just for themselves. Coming out of that group, they wanted to spearhead a football camp for the children so that was an unexpected outcome of the entrepreneurial training that they initiated a giving back component.” – Grantee MENTORSHIP AND CASE MANAGEMENT: Case management, mentorship, and social worker support were also very commonly cited by interviewees and focus group participants as essential to participant success. While the specific structure and content of this support varied across programs, case managers often developed a customized case plan based on needs that emerged from the participant’s risk assessment. The case manager or a mentor worked with participants on an ongoing basis to support their progress and help address issues that arose during their time in the program such as by providing support with administrative tasks or referrals to other services. According to the survey, 70 percent (n=73) of youth who worked with a case manager reported that they were totally or highly satisfied with the experience, and three respondents reported that they were dissatisfied. Survey respondents especially appreciated that mentors were a caring presence that would keep in touch with them regularly via WhatsApp, phone calls, or text messages. As with the overall rates of satisfaction, female respondents and minor respondents reported significantly higher satisfaction with their case managers. There were instances recounted by survey respondents where their mentor or case manager did not appear to have sufficient time to provide the needed support, so it has been a good practice to limit the number of youth assisted by psychologists/social workers/case managers to only 25 for better results. Additionally, LPD training activities did not train mentors on working with at-risk youth to sensitize them to specific considerations for working with this population; this could be added in future programming. CBT AND OTHER PSYCHOLOGICAL SERVICES: A core program success highlighted by interviewees was the successful introduction by LPD of CBT and other psychological services to the intervention model. LPD designed this work to provide underlying psychological support (where needed) to maximize participants’ chances of success in the life skills and professional program components. Although the activity tried both individual and group appointment models, the group model gained the most traction because of its efficiency, and participants responded well to peer 7 According to LPD’s 2022 M&E Plan and 2022 Annual Report, LPD exceeded the target number of youth trained in social and leadership skills for the first time in 2022 (target was 187; actual was 245). 20 | FINAL PERFORMANCE EVALUATION OF THE LOCAL PARTNER DEVELOPMENT ACTIVITY (LPD) USAID.GOV support in sessions. Several providers mentioned observing attitudinal shifts in CBT participants. For example, one grantee said: “I believe [the CBT curriculum] was effective. This is supported by post-assessment data. Could also see this in terms of the quality of sessions—more robust discussion. I didn’t expect a total transformation, but I think the curriculum was effective to contribute to creating behavior change. In our adolescent cohort, we started with 10 participants and 9 completed. They were all high school dropouts. Eight of the nine were back in school at the end of the three months, and have remained in school since. [CBT] had an impact on how they saw themselves, and on controlling angry impulses…. For older guys, I saw some similar changes…. In terms of behavior, I see some changes, sometimes marginal, including in the pre- and post-tests. [The change] is more in terms of what was said. I can see in the depth of discussions that behavior change is taking place. They gave a lot of heartfelt testimonials at the end.”’ – Grantee At-risk youth assisted by LPD expressed gratitude for having these opportunities to change their behavior and mindset for a better life. Survey and focus group participants also thought that CBT was extremely beneficial and called CBT group sessions a safe place where they could open up and not be judged. They particularly credited CBT with helping them learn how to manage their anger, develop self￾control, improve interpersonal relationship skills, and use meditation and relaxation techniques. Some credited the program for keeping them out of prison. One participant explained: “The majority of us are quick to react, because if something was to happen like someone threw a bottle at us, we would be quick to retaliate and get violent. The whole, stop, take a deep breath, and observe then proceed…. It used to be just one way we always used to act; CBT is the best thing.” – FGD Respondent LPD also measured improvements for CBT participants using pre- and post-tests to measure participants’ attitudes before treatment and after and capture growth or change that took place. According to the pre- and post-test results for four CBT programs that were completed and made available to the evaluation team, 62 percent of participants (n=24) reported less impulsiveness following CBT and just under two-fifths of participants reported decreased aggression (38 percent, n=15) and increased pro-social behaviors (38 percent, n=15). RISK ASSESSMENTS: Interviewees highly praised the risk assessment methodology LPD employed to identify eligible youth and better understand their needs. This approach aimed to increase the effectiveness of interventions through better targeting and gathering information that allowed for a customized approach to meet underlying individual needs. One interviewee explained that the ability to identify high-risk youth had helped prioritize them for emergency intervention faster, though the Government of Jamaica still faces constraints that mean that there will not always be services available, even in critical cases: “There is a greater attitude now within the Government of Jamaica that once a child meets specific criteria, they need immediate attention. In terms of giving our children access to these interventions, this is a work in progress that is being worked through with the Ministry of Education and Youth and other counterparts including non-Government of Jamaica entities.” – Government of Jamaica Stakeholder USAID.GOV FINAL PERFORMANCE EVALUATION OF THE LOCAL PARTNER DEVELOPMENT ACTIVITY | 21 OTHER SUPPORT: Interviewees also praised the customized support LPD provided to address other specific underlying issues. This included referrals to substance abuse counseling for youth who were assessed to have drug problems and literacy or numeracy training for youth with gaps in their education that were affecting their ability to participate in professional activities. Though not widespread within the activity, parenting classes, either for parents of participating youth or for youth who were parents, and joint activities/bonding exercises involving parents and children were strongly supported by both parents and youth participating in LPD activities. Parents described learning how to communicate better with their children, manage their own anger, demonstrate respect and love, and encourage positive behavior. They credited the program with making them better parents and transforming their relationships with their children, and described the joy of being spontaneously hugged and kissed by their adult child for the first time. ADAPTATION: LPD made a number of adaptations based on experience to increase program effectiveness (e.g., case management, adaptations to CBT model, stipend for participation). LPD made these adaptations to combat client attrition, which was an issue at the beginning of LPD, and to expand components that proved to be successful. One interviewee explained, “[There were] things that were not considered [at program design]—lack of morale, lack of interest. Kudos to the LPD team who huddled together, looked at issues, and accepted and implemented recommendations and saved the project in the long run. They involved parents, which was a good thing. The engagement of case managers was the best decision of the program and it literally saved the program. A stipend was given to get to school, but it is not enough for them to still live.” – Grantee CONCLUSIONS LPD’s integrated model with life skills training, vocational training/apprenticeships, CBT, and other components was key to increasing the resilience of youth in a short period of time (6–9 months). It is important that this integrated approach incorporated individualized elements such as case management, mentorship, and other support tailored to specific challenges youth faced. Participants strongly endorsed this sustained, integrated approach and highly praised all of the core components. Another core component of LPD’s success in increasing youth resilience was the activity’s willingness to adapt over time by adding components that emerged as necessary (such as stipends), sequencing components to incentivize sustained participation, and scaling up components that proved successful (such as case management/mentorship). However, it is possible that some of these adaptations would not have been necessary if LPD had incorporated youth perspectives earlier in the activity development process; this is a strategy that future activities could test. EVALUATION SUB-QUESTION 1B: FROM THE PERSPECTIVE OF BENEFICIARIES, WHAT ARE THE AREAS THAT COULD BE IMPROVED? FINDINGS LOW STIPENDS: Many interviewees identified stipends as a major area for improvement. In many cases, the stipend was only sufficient for them to travel to program activities, not to support themselves while there. In some cases, the stipend did not even cover the full transportation costs. Some programs had to be adapted to offer different training because participants could not afford to reach the intended 22 | FINAL PERFORMANCE EVALUATION OF THE LOCAL PARTNER DEVELOPMENT ACTIVITY (LPD) USAID.GOV training; in other programs, youth were not able to attend in-person meetings with their mentors because travel was not paid. Many of the youth who dropped out of the program did so because they needed to work despite remaining interested in completing the training. One stakeholder described: “When you go into a parish like St. James where one of the biggest challenges is lottery scamming, and at the center of lottery scamming is money, you are trying to identify at-risk youth from those same communities where they see their peers ‘living the life.’ But you say to them that you are trying to preserve their life and give them an opportunity—‘come and engage in a six-month training and we try to place you afterwards’—they still need to survive in the meantime while [they] also [need] support to attend the interventions.” – Grantee SHORT PROGRAM TIMEFRAMES: Youth survey respondents also desired a longer period of program support. Some described their disappointment that the program ended by saying it felt like “broken promises.” Interviewees echoed this desire. They explained that the very short intervention timeframe—sometimes consisting of one or two sessions on a certain component like counseling or follow-ups—was insufficient to support transformation in youth who may have spent years struggling with underlying issues. Grantees at times felt compelled to try to continue these components despite lacking the program budget to cover them because, as one described, “you have to find a way to help; you cannot just walk away.” The short program timeframe also created prioritization challenges for case managers who had to make difficult decisions about how to triage service referrals for high-risk clients who would not be able to receive all of the support they needed in the allotted time. A case manager noted: “There are several interventions happening at the same time, so it is difficult for the case manager to know what services the client must get based on the short timeframe, especially given the context of them being high risk. They will likely need remedial support, psychological support, and a social worker, and need to upskill them so they can be more marketable and better able to resist a life of crime. They need all these services, but based on the short timeframe, it is hard to prioritize which ones to do.” CONCLUSIONS While LPD was able to accomplish a remarkable amount using several successful approaches and did important work to adapt in response to challenges, including expanding the use of stipends, limited stipend amounts and short program timeframes still presented challenges for clients and participants and likely dampened program effects. EVALUATION SUB-QUESTION 1C: DISCUSS GAPS AND INTERVENTIONS THAT WERE LEAST EFFECTIVE. HOW COULD THE INTERVENTIONS BE IMPROVED? FINDINGS LIMITED TERTIARY PREVENTION: The youth reintegration component of the activity largely did not go forward because partners were not allowed to enter remand centers as a result of COVID-19, so it is too early to ascertain its effectiveness. Some programs were able to recruit a limited number of young people who had left remand centers and were re-entering society and train officers online, but the CBT intervention in remand centers is still stalled aside from training the main psychologist in the system. USAID.GOV FINAL PERFORMANCE EVALUATION OF THE LOCAL PARTNER DEVELOPMENT ACTIVITY | 23 The activity also faced difficulty tracking youth once they left facilities because the Department of Correctional Services did not release their contact information to partners. NEED FOR ENHANCED CLIENT MANAGEMENT SYSTEM AND BETTER DATA SHARING: Several grantees noted a lack of a shared client management system across partners and suggested that having one would help track youth and target appropriate resources to them. One grantee explained that they built an internal client management system for their own program administration needs with other funds, but as of this report’s drafting, there does not appear to be a platform for organizations to track at-risk youths’ receipt of services across organizations and government entities. Interviewees suggested that the Government of Jamaica should manage this system and indicated that the Ministry of National Security may be looking into creating something like this, but the effort is complicated by the lack of a national ID number in Jamaica. Grantees also expressed a desire for additional data sharing by LPD on what interventions the activity is finding to be most successful to inform their programming. LIMITED COORDINATION BETWEEN PARTNERS AND STAKEHOLDERS: LPD used several strategies to facilitate coordination among stakeholders and partners, such as the co-design workshops at the beginning of the activity. Nevertheless, a desire for greater coordination in their work was a major theme across interviewees. Some grantees noted that they did not always achieve the anticipated level of coordination, such as through the referral system, largely because of capacity challenges at the organizations. One grantee explained, “We were supposed to have this referral system… and in a few cases it did work… but I don't think at the level that we had all anticipated. I don't know that that is a reflection necessarily on LPD. I think that's probably just a cultural issue that we have in Jamaica that we don't effectively coordinate. So I wouldn’t place that—it’s more of a recommendation than a criticism of the program because again, I think it’s bigger than LPD, it’s a systematic problem. But I definitely applaud LPD for their efforts to try and have that done, having core partners coming together co-designing the program and implementing simultaneously. It’s just that we weren’t able to have the level of coordination across our activities. I think that could be strengthened in some ways.” – Grantee In addition, grantee and LPD respondents both spoke about a desire for greater leadership and coordination from Government of Jamaica stakeholders. Specific challenges included limited coordination with CSOs providing similar services, lack of budget allocation to support interventions (e.g., reintegration), and communication challenges because of short staffing. Stakeholders explained that, ideally, Government of Jamaica partners would identify their support needs and coordinate donors to work on different areas, but that this level of leadership was not present during LPD. Collaboration across entities engaging in different approaches to prevent crime and violence, such as police and peacebuilders, is also an area where future activities could aim to make progress. CONSTRAINED TIMEFRAME: A key factor that likely limits the ability of LPD and this evaluation to observe activity benefits is the constrained activity timeframe. Several interviewees noted that two years is not enough time to expect to see sustained transformative change. This kind of work requires long￾term planning and resource allocation that provides enough flexibility that activities can be easily modified as realities change on the ground. Additionally, the key indicator used to measure resilience (reduction of risk factors) does not fully measure resilience, since the sustainability of changes is unknown and other levels of risk (family, community, justice) remain. 24 | FINAL PERFORMANCE EVALUATION OF THE LOCAL PARTNER DEVELOPMENT ACTIVITY (LPD) USAID.GOV CONCLUSIONS Identified activity gaps included gaps in programming because of external circumstances such as the COVID-19 pandemic (tertiary prevention) and gaps in capacity (coordination). Additionally, the limited timeframe for interventions to effect major changes in the lives of program participants was a significant challenge. These gaps represent opportunities for future programming to continue to make sustained, systemic efforts. EVALUATION QUESTION 2. IN WHAT WAYS HAS LPD BUILT THE CAPACITY OF LOCAL GOVERNMENT AND NON-GOVERNMENT ORGANIZATIONS TO IMPLEMENT EVIDENCE￾BASED INTERVENTIONS? FINDINGS WORKSHOPS: The majority of respondents from the Government of Jamaica, CSOs, and the private sector confirmed that the LPD activity had invited them to participate in workshops and presentations about the objectives, scope, and implementing mechanisms of the activity’s work. Participants reported learning about evidence-based tools to identify and assess youth and best practices and evidence-based approaches to implement youth crime and violence prevention at the secondary and tertiary levels. Most respondents emphasized the importance of LPD’s convening power and reputation, which they said led to high participation in the events. The majority of respondents from government agencies, civil society, and the private sector also highlighted the professionalism and strong technical capacity of LPD staff and confirmed that the training and workshops were key elements in motivating them to adopt evidence-based approaches and tools. Respondents reported that the workshops motivated several private sector participants to collaborate with other implementing partners to provide apprenticeships, internships, temporary jobs, and even full-time permanent jobs to participating youth. SPECIALIZED TRAINING SESSIONS: All sub-awardee evaluation respondents confirmed that LPD had provided training on LPD’s evidence-based approach, including CBT, case management, and other specialized approaches. However, they also reported that not all organizations or entities administered the activity’s two-part risk assessment, the Jamaica Risk Assessment, themselves, despite receiving training on it; some interviewees mentioned that the assessment and re-assessment were conducted by LPD staff or consultants hired by LPD. The evaluation team confirmed that one private sector implementing partner provided a training-of￾trainers approach with the Department of Correctional Services. The Department confirmed this and conveyed that the staff trained to train others had relocated within the Ministry of National Security or found another job and that it would be helpful to do this training-of-trainers on an annual basis because of high staff turnover. Evaluators did not find evidence that the training-of-trainers’ support included follow-up to support implementation. COLLABORATION AND CO-DESIGN: Participants highly valued LPD’s coordination of stakeholder collaboration and program co-design. However, most respondents indicated that coordination among the organizations that were awarded funds to implement their youth support plans decreased over time. The implementing partners were supposed to meet frequently, but because of non-LPD work backlogs in each organization, coordination meetings became scarce over time. Some interviewees also mentioned that the culture of working collaboratively is low in the Jamaican context. USAID.GOV FINAL PERFORMANCE EVALUATION OF THE LOCAL PARTNER DEVELOPMENT ACTIVITY | 25 STUDY TOUR OF MEXICO: Eleven CSO, private sector, and Government of Jamaica organizations traveled to Mexico to learn about the risk assessment tool and CBT methodology. All interviewees that participated in the study tour reported learning that the CBT model had been highly effective in reducing crime and violence in the Mexican communities they visited with similar economic and social conditions to those of the Jamaican communities where LPD planned to work. They also agreed that the learning experience in Mexico helped them believe in the importance of evidence-based approaches, such as the use of a youth targeting tool and CBT, and the engagement of the private sector in reducing crime and violence. They reported that the experience was highly motivational for them to be active LPD implementing partners. CONCLUSIONS Through awareness and engagement workshops, specialized training, facilitating collaboration among stakeholders, and the study tour to Mexico, LPD successfully increased awareness and understanding of evidence-based interventions and motivated local government and non-government organizations to implement these interventions to reduce crime and violence among youth. Most respondents perceived that evidence-based interventions were effective in reducing youth risk factors. EVALUATION SUB-QUESTION 2A: WHAT ARE THE MAIN BARRIERS (INTERNAL AND/OR EXTERNAL) TO LOCAL ORGANIZATIONS ADOPTING EVIDENCE-BASED APPROACHES? FINDINGS Internal barriers LACK OF FULL-TIME STAFF TO ADMINISTER THE LPD GRANT: All respondents said that their non-LPD work competed for their time with LPD activities and reporting and documentation requirements. This resulted in them not meeting with other organizations as frequently as expected and delays in preparing and delivering reports and documentation to LPD. Existing work limited the ability of staff assigned to LPD activities within implementing organizations to attend coordination meetings, meet deadlines to prepare project/program documentation and deliverables, and attend capacity-strengthening activities. All interviewees agreed that they did not have enough budget to assign full-time staff to implement the activities of the sub-grants. INSUFFICIENT INSTITUTIONAL BUDGET TO PROVIDE COMPLEMENTARY SUPPORT TO PARTICIPANT YOUTH: The majority of interviewees recognized the soundness and relevance of LPD’s evidence-based interventions; however, they also remarked that the activity was very expensive and that there were occasions when additional financial support was needed beyond that of the sub￾grant. Moreover, there were some activities that needed more money than what was provided by LPD under the sub-grant mechanism. For example, the stipend provided to participant youth for transportation was not enough and prevented some participants from showing up to the training sessions; implementing organizations did not have the resources to complement the stipend provided by the sub-grant. Likewise, some youth needed support to feed themselves or their children and tended to use the stipend for that purpose, leaving the organizations with insufficient funds to meet their other needs and improve their attendance. In the case of internships, one respondent said that the funds were not enough to support youth to attend internships for a long period of time—such as six months or 26 | FINAL PERFORMANCE EVALUATION OF THE LOCAL PARTNER DEVELOPMENT ACTIVITY (LPD) USAID.GOV longer—even though they knew that the longer they spent in one internship, the more likely they were to get a job. The evaluation team received the same feedback from LPD staff, who also pointed out that the evidence-based approach was very expensive, primarily because of the case management approach, which requires highly qualified and high-salary professionals. All interviewees consulted on the budget issue agreed that they did not have sufficient institutional financial resources to assign full-time staff to implement the sub-grant activities and predicted that this would likely limit the sustainability of the evidence-based approach, despite their willingness to implement it. The majority of respondents indicated that it was unlikely that they would be able to continue implementing this approach after LPD’s closeout and only a few said they would continue implementing the model holistically, as discussed under Evaluation Sub-Question 2c. External barriers SHORT TIMEFRAME TO GET RESULTS: The majority of interviewees agreed that the timeframe to aid participating youth was too short to reduce risk factors identified by the risk assessment tool. While respondents agreed that more time was needed, there was no agreement on the optimal time. Some mentioned timeframes from 2–5 years. Some examples of the feedback included: • “Two years or less will never be enough for this kind of work, it requires 5 years at a minimum.” • Several case managers and social workers who provided direct support to youth indicated that two counseling sessions for a high-risk youth would not do anything and mentioned that youth would need a minimum of six sessions. • One respondent indicated that for the August Town and Grange Hill interventions, they needed more time to effect the real change they were looking for because behavior change takes time; for instance, psychosocial support for three months is not enough. COVID-19 PANDEMIC-RELATED DELAYS: The COVID-19 pandemic generated significant delays in the implementation of LPD activities. For instance, the adaptation of the CBT ROLE Model of Mexico to the Jamaican context was largely delayed from 2020 to 2021. In one location, one of its three interventions was canceled because the Ministry of Health and Wellness and the Jamaica Defense Force Army used its facility for their COVID-19 response. Attendance at some face-to-face events was halved because of social distancing imposed during the pandemic, and the number of training sessions had to be increased to accommodate smaller groups. Implementing partners were unable to directly support delivery of assistance to high-risk youth at the tertiary risk level in some areas, although one private sector partner indicated that they carried out a training-of-trainers event with the Department of Correctional Services during this time. Although LPD’s end date was extended from February 2022 to February 2023, and then again to June 2023, most respondents felt they would have been able to assist more youth and provide more support sessions without the negative effects of the pandemic. Respondents praised LPD’s responsiveness and adaptability in addressing these challenges. Adaptations included extending deliverable deadlines, requesting no-cost extensions to sub-grants with the support of LPD staff, and realignment of budget line items to reflect new costs, such as buying tablets for online USAID.GOV FINAL PERFORMANCE EVALUATION OF THE LOCAL PARTNER DEVELOPMENT ACTIVITY | 27 instruction. LPD took advantage of the intensive interaction and coordination with local community members to create confidence and engagement of local communities in LPD activities. LIMITED COORDINATION AMONG IMPLEMENTERS: Evaluators found that most respondents appreciated the coordination strategy applied by LPD to support collaborative action planning and co￾design. However, although initially there were frequent coordination meetings, after a few months, implementing partners met less frequently, according to various respondents. Respondents suggested that this was because organizations did not like to work together, coordinating everybody’s time was a challenge, and organizations did not have enough time to attend meetings. According to one respondent, “It didn't work the way it was planned, and LPD should have been coordinating to make it work.” GRANT BUDGET LIMITATIONS: The majority of respondents opined that LPD’s evidence-based approach was expensive, particularly the case management component. Respondents pointed out that delivery of specialized services to youth such as life skills and vocational training and psychosocial counseling requires hiring professionals who are not only scarce but also expensive. Most respondents believed that this might have affected the number and duration of some services. For example, respondents pointed to the low number of counseling sessions, the stipend for internships lasting six months not being allowed, the insufficient stipend for participant transportation, insufficient budget to hire full-time administrative staff to manage the grants, and lack of budget for project personnel transportation. One respondent indicated that the implementers themselves did not benefit financially and had expected at least some administrative support. This situation prevented implementing organizations from fully committing to the activity, and one interviewee even mentioned that for this reason one agency had pulled out of the Collaborative Action program in St. Catherine. Most interviewees stated that they would not be able to continue applying the evidence-based approach if they did not receive funds from donors or other sources. LIMITED NUMBER OF ORGANIZATIONS WORKING WITH AT-RISK YOUTH: At the beginning of the activity, LPD realized that there were very few organizations working with at-risk youth and even fewer working with at-risk youth at medium to high risk levels. Evidence indicates that the organizations that were working with at-risk youth were not using the risk assessment to target youth based on the level of risk, a gap that required intense capacity-strengthening with potential implementers. YOUTH FOCUSED EXCLUSIVELY ON GETTING EMPLOYMENT: Evidence from FGDs with youth participants indicates that they entered the program primarily to get a job, and this was confirmed by most key informant interviews. Some interviewees reported that youth ended up not receiving the social part of the program, a key component to effectively reducing youth risk factors. Interviewees suggested that this was because of a common mindset among youth that focused exclusively on finding a job and that they did not value life skills training and psychological support. LIMITED PARENT OR CAREGIVER ENGAGEMENT: Parenting or caregiver support was not mentioned by organizational respondents as a generalized program service. Those that confirmed that they provide this kind of assistance recognized that full involvement of parents/caregivers is critical for reducing crime and violence among youth. However, they pointed out that low levels of financial resources forced some parents/caregivers to prioritize attending work rather than participating in family counseling sessions, limiting LPD’s impact. 28 | FINAL PERFORMANCE EVALUATION OF THE LOCAL PARTNER DEVELOPMENT ACTIVITY (LPD) USAID.GOV CONCURRENT INTERVENTIONS: All respondents, including case managers, social workers, and psychologists agreed that each youth, given their high risk profile, had to receive many interventions to improve their resilience to crime and violence. One case manager mentioned that they had to handle many youth interventions at the same time—for example, in August Town and Grange Hill—and that they had to prioritize and limit the number of services provided to youth. Other respondents mentioned that the number of counseling sessions, for example, were not enough to really produce a behavioral change. In general, respondents reported that program participants did not get all the assistance they needed based on their risk assessment results. One respondent said that participating youth “will likely need remedial support, psychological support, and a social worker, and need to upskill them so they can be more marketable and better able to resist a life of crime—they need all these services but based on the short timeframe, [it was] hard to prioritize which ones to do.” FLARE-UPS OF VIOLENCE: In some implementing communities, activity participants were victims of violence, leading to delays, cancellations, or relocation of activities. CONCLUSIONS There were several key barriers or limitations that affected the achievement of better results. Limited budget and staff to manage LPD sub-grants was a major internal factor that limited the achievement of products and outcomes. A combination of external factors, including insufficient grant budget, a short timeframe to produce results, the COVID-19 pandemic, low coordination among implementers, donor budget limitations, youth prioritizing jobs instead of psychosocial support, and low parental engagement limited activity results and brought into question the interventions’ sustainability. EVALUATION SUB-QUESTION 2B: WHAT ARE THE ENABLING FACTORS THAT ALLOW ORGANIZATIONS TO SUCCESSFULLY ADOPT EVIDENCE-BASED PROGRAMMING? FINDINGS AWARENESS OF THE EFFECTIVENESS OF EVIDENCE-BASED APPROACHES: LPD was successful in raising awareness and understanding of evidence-based approaches to reduce crime and violence among most at-risk youth at the secondary and tertiary levels. Most interviewees from CSOs, private sector foundations, and Government of Jamaica entities confirmed that they participated in awareness events facilitated by LPD prior to becoming grantees of LPD. During transfer knowledge events, they learned best practices to interface with at-risk youth, how these approaches should be implemented, and how effective the approaches are. Most implementing partners remarked that their experience confirmed that the approach applied by LPD has proven effective, particularly the risk screening and risk assessment and re-assessment tools, which have provided evidence of reduced levels of risk among participating youth. Evaluation data found high desire among LPD’s implementing partners to continue applying the evidence￾based approach to reduce crime and violence. However, implementation depends on being awarded funds from external sources such as the Government of Jamaica or national and/or international donor organizations. Interviewees agreed that evidence-based approaches are effective at reducing risk factors of youth participating in the activity, but also that the model is expensive, and most organizations do not have the USAID.GOV FINAL PERFORMANCE EVALUATION OF THE LOCAL PARTNER DEVELOPMENT ACTIVITY | 29 resources to implement the approach to its full potential. For instance, deploying the approach requires full-time staff including psychologists and case managers, stipends for participants, and coaches and mentors. STUDY TOUR TO MEXICO: The evaluation confirmed that study tour participants found this activity very valuable. Participants acknowledged that the tour helped increase the buy-in of key government entities, private sector businesses, and CSOs. All study tour participants indicated that they learned about best practices and evidence-based approaches for dealing effectively with youth at higher risk of getting involved in crime and violence, measuring progress toward reducing risk factors, implementing individual treatment plans, engaging the private sector in combating at-risk youth crime and violence, and delivery of behavioral support, including counseling and therapy. As a consequence, the tour broadened their perspective of how to introduce evidence-based tools in Jamaica, and enabled several organizations to successfully apply for LPD sub-grants to implement evidence-based programming. LPD also decided to adopt the Mexican model of CBT that participants learned about on the tour. OWNERSHIP GENERATED BY LPD AMONG IMPLEMENTING PARTNERS: Collaboratively co￾designing program activities and LPD’s flexibility in adjusting program activities generated ownership among implementing organizations by allowing them to develop programs that were aligned with their institutional interests. LPD brought together groups of potential implementing partners—including CSOs, private sector foundations, and community members—to identify and design programs in targeted geographic areas such as Clarendon and St. Catherine. Numerous respondents conveyed their belief that this approach was a highly effective strategy to motivate implementing partner organizations to participate in LPD and work to achieve the intended results. Some respondents also mentioned that they used their own resources to support participating youth. For instance, in some cases, stipends, counseling, and support in obtaining personal identification numbers for taxes, bank accounts, etc. that were not budgeted for under the sub-grant were partially covered by the implementing organizations. PROFESSIONALISM, FLEXIBILITY, AND COMMITMENT OF THE LPD TEAM: Most respondents highlighted the expertise, flexibility, and commitment of LPD’s technical staff as a motivational factor that supported the successful implementation of evidence-based programming. Some respondents indicated that the convening power of LPD, based on its reputation and expertise, enabled the program to convince interested organizations to implement evidence-based programming under LPD. Respondents noted that LPD staff were highly flexible in providing needed organizational support for implementing partners, including training staff on monitoring and evaluation and using the newly created case management system to report on progress toward results. The evaluation team confirmed that during the initial socialization and sensitization phase of the activity and during implementation, the LPD team effectively conveyed its expertise on evidence-based approaches to reducing crime and violence among high-risk youth, a key factor that helped organizations decide to become involved in the activity. One respondent said, “In terms of project design and responding to the needs of the target population, they have been responsive and creative to the benefit of the program. The team is a good blend of professionals in terms of what is needed. Worked well together to troubleshoot and work out challenges on the ground during implementation.” 30 | FINAL PERFORMANCE EVALUATION OF THE LOCAL PARTNER DEVELOPMENT ACTIVITY (LPD) USAID.GOV CONCLUSIONS LPD’s strong convening power appears to have been a key enabling factor for successful implementation of evidence-based approaches. LPD cultivated ownership of activities among local government and non￾government organizations. The high quality of the capacity-strengthening activities—awareness and technical workshops, specialized training, and the study tour to Mexico—were key enabling factors for adopting evidence-based programming among implementing partners. EVALUATION SUB-QUESTION 2C: WHAT EVIDENCE EXISTS THAT BENEFICIARY ORGANIZATIONS ARE LIKELY TO CONTINUE TO IMPLEMENT THESE APPROACHES? FINDINGS Respondents told evaluators that they have the capacity and willingness to continue partially applying evidence-based approaches, but do not have the resources, strategies, or plans to allocate dedicated budget to apply it fully. For example, some indicated that they would at least continue to use the risk assessment tool to measure results toward reduction of risk factors among assisted youth, but not necessarily other aspects of the approach they have been using during LPD implementation. Some interviewees, particularly case managers, believed that most organizations do not have the institutional resources to fully implement LPD’s evidence-based approaches. However, a few private sector, CSO, and Government of Jamaica implementing partners reported that they plan to continue or have started using the whole evidence-based approach, including providing CBT services to clients and providing technical training on evidence-based approaches to other partners: One private sector foundation, for example, is initiating work with two initiatives; one of them is funded by the European Union and the other is receiving support from a private sector company (not identified), and will reportedly replicate the LPD approach model. One CSO will apply the evidence￾based approach under the new Social Transformation and Renewal (STAR) project funded by the Private Sector Organization of Jamaica in conjunction with the Jamaica Constabulary Force. The CSO “will provide technical expertise with a focus on evidence-based approaches to violence prevention utilizing a public health approach” with the aim of bringing about social transformation in Jamaica’s low-resource communities. One Government of Jamaica ministry is also part of the STAR project and will be able to apply the evidence-based approach in working in schools to reduce crime and violence. In October 2022, this ministry identified nine high schools in Grange Hill to conduct a CBT methodology pilot and then extend its use to all Jamaican high schools. CONCLUSIONS LPD’s evidence-based approach is likely to be sustained by only a few implementing partners that have reported that they are already implementing or planning to implement this approach in programs or projects funded by donors or other financial supporters. The rest of the interviewees said that they do not have the budget to implement the evidence-based approaches in a holistic manner unless they receive funds from external sources to do so. Some do plan to continue using the Jamaica Risk Assessment, which is a key component of the model, because they consider it an effective way to better target youth and reduce crime and violence. The majority of USAID.GOV FINAL PERFORMANCE EVALUATION OF THE LOCAL PARTNER DEVELOPMENT ACTIVITY | 31 interviewees confirmed that they will not be able to continue applying the evidence-based approaches at all after LPD closes out because they lack the financial resources to sustain these costly approaches. EVALUATION QUESTION 3: TO WHAT EXTENT HAS LPD INTEGRATED PRIVATE SECTOR ENGAGEMENT TO ADDRESS THE RISK AND RESILIENCE FACTORS FACED BY TARGETED YOUTH, FAMILIES, AND COMMUNITIES? FINDINGS INCLUSION IN TRAINING: As evidenced by both desk review documents and KIIs, LPD employed a collaborative approach to private sector engagement. For example, both the Clarendon Collaborative Action Project and St. Catherine Collaborative Action Project used multistakeholder groups drawn from civil society, the private sector, and government entities to jointly participate in workshops, training activities, strategy sessions, and program development.8 Additionally, the Mexico study tour included three private sector foundations. COMPLEMENTARY STRENGTHS OF PRIVATE SECTOR PARTNERS: Youth activity participants identified lack of employment opportunities as a primary concern, making employment-based interventions a natural programming activity and private sector organizations logical partners because of their capacity to provide both vocational training and short- and long-term opportunities for employment. LPD enabled holistic, integrated PSE approaches that leveraged a combination of evidence￾based interventions by providing capacity-strengthening for private sector partners, such as training in life skills and CBT interventions, enhancing the probability of youth participants’ overall success. CONCLUSIONS LPD substantially integrated private sector engagement to address the risk and resilience factors faced by targeted youth, families, and communities. By integrating private sector partners into the development and implementation of its interventions, including muti-sector learning, collaboration, and mutual reinforcement, LPD promoted private sector partners’ capacity to contribute to youth crime and violence prevention. PSE was neither an afterthought nor an exclusive focus of LPD’s strategy, but a critical component of an integrated, holistic approach combining proven, evidence-based intervention strategies. EVALUATION SUB-QUESTION 3A: WHAT PSE APPROACHES ARE MOST USEFUL IN YOUTH CRIME AND VIOLENCE PREVENTION? WHAT ARE THE POTENTIAL BARRIERS TO GREATER PRIVATE SECTOR PARTICIPATION IN THIS AREA? FINDINGS Approaches most useful in youth crime and violence prevention INTEGRATED APPROACH TO PSE YOUTH INTERVENTIONS: Youth participants identified apprenticeship programs, business opportunity training, and other employment-oriented activities as their primary motivation for initially participating in LPD programs, overwhelmingly citing the lack of 8 Jamaica Local Partner Development, Annual Report: Fiscal Year 2020. 32 | FINAL PERFORMANCE EVALUATION OF THE LOCAL PARTNER DEVELOPMENT ACTIVITY (LPD) USAID.GOV employment opportunities as one of their most significant obstacles. Once in the programs, youth participants also highly praised the broad range of individualized and sustained interventions, such as CBT, life skills training, mentorship, anger management training, and drug counseling, that were integrated with the employment-oriented activities. CBT, in particular, was widely praised by youth participants in almost every FGD, with numerous participants describing how the program changed their outlook on life, their relationships with their families, and their ability to identify and avoid behaviors that previously led to negative outcomes. Additionally, as noted in the analysis for Evaluation Question 1, KIIs and survey data suggest that an integrated approach to youth interventions led to better outcomes. For example, while FGD participants frequently cited employment-based activities as their primary motivation for participating in LPD activities, once in the program, they overwhelmingly credited CBT and life skills training for providing a healthier and more constructive approach to relationships and life, and for preventing them from slipping back into counterproductive behaviors. COLLABORATION AND CAPACITY-STRENGTHENING FOR PRIVATE SECTOR PARTNERS: Private sector partners engaged by LPD frequently expressed appreciation for the collaborative approach and the skills, knowledge, and capacity that will remain even after the cessation of LPD activities. In particular, private sector partners favorably mentioned the co-creation process, the ability to work with and learn from CSOs and other private sector partners through workshops and discussions, and the assistance provided by LPD in navigating and understanding the donor-funded administrative, financial, and regulatory environment. Private sector partners also credited LPD with significantly enhancing their understanding of at-risk youth and how to engage with them and better target them based on the level of risk they have at the point of enrollment in the activity. For example, several private sector partners reported that the Jamaica Risk Assessment tool is now embedded in their approach to youth crime and violence prevention. PROVISION OF EMPLOYMENT-BASED OPPORTUNITIES: As noted in Evaluation Sub-Question 1a, vocational training and access to improved opportunities for employment are key motivating factors for at-risk youth. Youth participants in FGDs frequently cited lack of employment as one of their most significant challenges and identified employment-based opportunities as the primary benefit of LPD’s program. LPD’s private sector partners served as the primary mechanism for providing employment￾based opportunities to activity participants. Potential barriers to greater private sector participation LACK OF ONGOING FUNDING: While some private sector partners indicated an ongoing commitment to youth crime and violence prevention activities, many indicated that without funding they would be unable to continue the kind of integrated, multi-faceted approach to interventions used by LPD. Such interventions are resource intensive, and even larger private sector partners may not be able to provide sufficient financial support for participants to support themselves and their families during the program without taking time away from programming to work. LACK OF UNDERSTANDING/WILLINGNESS TO WORK WITH AT-RISK YOUTH: Private sector organizations often have little understanding of youth crime and violence prevention, while others are reluctant to engage with at-risk youth, particularly those deemed to be high risk. Some private sector organizations are concerned about working with youth viewed as potentially having gang/criminal backgrounds, and who may be unreliable or difficult to work with. For example, one grantee noted: USAID.GOV FINAL PERFORMANCE EVALUATION OF THE LOCAL PARTNER DEVELOPMENT ACTIVITY | 33 “[There is] small to no willingness of the private sector to collaborate with at-risk youth. They are not that interested in hiring youth with a ‘rap sheet.’ Closer to the end of the project some youth got small positions in supermarkets or hardware as Christmas jobs but there wasn’t a lot of assistance there in terms of apprenticeships, so it was difficult…. Bigger organizations—they are not ready to take on youth from certain communities.” LOW AVAILABILITY OF EMPLOYMENT OPPORTUNITIES: Youth participants cited the desire to obtain employment opportunities as their primary motivation but some respondents reported that, overall, private sector organizations were unwilling to take on large groups of at-risk youth. This is particularly problematic after apprenticeships or training activities have concluded, as participants often have difficulty obtaining ongoing employment even with their new certifications or skills. ATTRITION: Some private sector organizations identified significant attrition in programs for at-risk youth. For example, one organization specified that out of 138 referrals, only 50 completed Phase 1 training, of which only 11 completed Phase 2 mentorship and business support—a completion rate of approximately 8 percent. CONCLUSIONS LPD used an integrated approach to PSE youth interventions, combining employment-oriented activities such as apprenticeships and business opportunity training with CBT, anger management, drug counseling, and other life skills-oriented training. Based on feedback from youth program participants and the decreased attrition of programs using integrated interventions, an integrated approach to youth and crime violence prevention for PSE seems more likely to resonate with youth and break the cycle of self-defeating behaviors than programs focusing on a single intervention. Private sector partners identified a variety of potential barriers to greater participation in youth crime and violence prevention, including funding, willingness to employ at-risk youth, and high program attrition. However, most of these limitations can be mitigated by strategies identified in the Recommendations section. EVALUATION SUB-QUESTION 3B–D ESQ3B. TO WHAT EXTENT CAN PSE APPROACHES HELP BUILD SUSTAINABILITY IN YOUTH CRIME AND VIOLENCE PREVENTION ACTIVITIES IN JAMAICA? ESQ3C. WHAT ARE THE ENABLING FACTORS THAT ALLOW PRIVATE SECTOR ORGANIZATIONS TO SUPPORT YOUTH CRIME AND VIOLENCE PREVENTION ACTIVITIES? ESQ3D. WHAT EVIDENCE EXISTS THAT PRIVATE SECTOR ORGANIZATIONS ARE LIKELY TO CONTINUE TO PROVIDE SUPPORT IN THIS AREA? This section presents findings for Evaluation Sub-Questions 3b–d by addressing topics related to PSE and sustainability that are relevant to all three sub-questions. FINDINGS 34 | FINAL PERFORMANCE EVALUATION OF THE LOCAL PARTNER DEVELOPMENT ACTIVITY (LPD) USAID.GOV PRIVATE SECTOR MOTIVATION TO ADDRESS COMMUNITY CRIME AND VIOLENCE: While several respondents noted reluctance by some private sector organizations to work with at-risk youth, most private sector and CSO KII respondents observed increasing private sector interest both in community programming generally and in youth crime and violence prevention activities specifically. The number of private sector foundations is increasing, in part because of greater acceptance of corporate social responsibility. In particular, KII participants noted that private sector organizations’ links to the communities in which they are located and greater awareness of social challenges are motivation for their interest in engaging in youth crime and violence prevention. In many instances, private sector organizations are already engaged in community development and have existing ties to community-based organizations, such as community development committees. As ongoing members of the community, and with employees living and working in affected areas, private sector organizations can serve as long-term allies to promote youth crime and violence prevention in their communities, contributing to the sustainability of these efforts. Private sector organizations are also, to some extent, motivated by increasing concern about the impact of crime and violence on GDP and Jamaica’s economic growth and stability. Additionally, some KII participants observed that private sector organizations are particularly results-oriented, and the successful approach employed by LPD resonated with private sector partners and encouraged them to continue it. As one private sector partner stated, “As we speak, we have been exploring the possibility of having a part 2 and are planning to do a second program.” The same partner also stated, “If something is evidence based, and it is working, it should be replicated. The point of a pilot is to test efficacy. If it works it should be scaled up.” PRIVATE SECTOR CAPACITY-STRENGTHENING: The evaluation team received consistently positive feedback on LPD’s approach to increasing private sector capacity to engage in this area, helping to build sustainability through their PSE activities. While KII participants positively noted increasing interest and commitment from the private sector to engage in youth crime and violence prevention, they also identified an ongoing need for private sector capacity-strengthening and skills training. Most private sector organizations have limited familiarity with social development, and KII participants positively cited LPD’s efforts to build private sector capacity, including development of substantive knowledge, program management skills, and skills for working with the donor community. Private sector partners frequently noted in KIIs that the skills and knowledge they derived from LPD programs will enable their participation in youth crime and violence prevention in the future, helping to build locally driven, sustainable capacity. For example, one grantee noted: “CBT has now become one of our pillars because of LPD support, we have been trained and we have people on staff who can train others in CBT. They also provided support in master trainers in life skills, which will also add to sustainability, because we can then train others. It’s allowing us to have greater impact, reach, and scale in terms of the delivery of our life skills program. And the assessment is something that we have. It’s become part of general operations in any program that we do now. We do screening and assessments today to make sure that we are on target.” MULTISTAKEHOLDER COORDINATION AND COOPERATION: LPD used joint activities and planning with private sector and CSO partners to promote an integrated approach to youth crime and violence prevention through multistakeholder collaboration and coordination. For instance, the USAID.GOV FINAL PERFORMANCE EVALUATION OF THE LOCAL PARTNER DEVELOPMENT ACTIVITY | 35 Collaborative Action Projects for Clarendon and St. Catherine demonstrated the benefits of effective coordination between the private sector and other stakeholders. KII participants described how multistakeholder programming benefits from the inherent advantages possessed by private sector partners in conducting employment-oriented activities, while private sector partners benefit from the skills and knowledge training gained from working with CSOs in designing and implementing social development programs. Accordingly, several KII participants identified the importance of effective coordination in the area of youth crime and violence prevention as an enabling factor for private sector participation. This includes effective coordination within the private sector itself and between the private sector and civil society. One KII participant from the Government of Jamaica observed: “I think there could be greater impact with more cohesion among the private sector. We are a culture of scarce benefits, which hurts cohesion. To get as many youth as possible in the program we need better coordination in the private sector rather than competition. Also, the U.S. Embassy needs to help organizations stop working in isolation. Jamaica gets lots of funding, lots has already been tried, but we need greater open discussion and coordination to align efforts and reduce competition.” SUSTAINED FUNDING AND PROGRAMMING: In conjunction with the need for effective coordination, some KII participants expressed frustration with the limited nature of programs and identified sustained funding as an enabling factor for private sector participation. KII participants noted that impact in the area of youth crime and violence prevention will only be realized through long-term programming, and this will only be possible through ongoing support from the donor community. Several private sector partners praised the funding provided by LPD and identified financial support they received as “crucial” for their participation. While larger private sector participants expressed an intent in KIIs to continue at least some aspects of their youth crime and violence prevention programs, the majority of private sector participants indicated that ongoing funding would be necessary for them to continue programming. CONCLUSIONS The private sector is increasingly interested in youth development and the challenges presented by youth crime and violence, and with sufficient capacity-strengthening and funding, is likely to remain involved in youth crime and violence prevention. PSE approaches can be extremely helpful in building sustainability in youth crime and violence prevention in Jamaica. Private sector organizations play an essential role in providing much-needed employment￾based activities and employment opportunities for at-risk youth, and should be integrated into a comprehensive, collaborative approach to youth crime and violence prevention in conjunction with civil society, government agencies, and the donor community. Private sector organizations also have existing ties to the communities in which they operate and are sensitive to the particular social challenges facing their communities. However, they are less prepared than other types of stakeholders to work in this sphere, so for PSE to fully contribute to sustainable crime and violence prevention, private sector capacity-strengthening in social development, programming approaches, project management, and skills in connecting with and navigating the donor community is essential. Sustained funding is also a crucial factor for recruiting and maintaining private sector support for youth crime and violence prevention; without funding from LPD, most private sector partners would not have engaged in youth crime and violence prevention. 36 | FINAL PERFORMANCE EVALUATION OF THE LOCAL PARTNER DEVELOPMENT ACTIVITY (LPD) USAID.GOV RECOMMENDATIONS Because this final performance evaluation report has been prepared near the end of LPD, the recommendations below are targeted at USAID/Jamaica. These recommendations are expected to inform design and implementation of future programming in Jamaica aimed at assisting youth at medium and high risk levels as a means to reduce crime and violence at the secondary and tertiary levels. Train potential implementing partners in evidence-based approaches and award grants to organizations that meet minimum levels of technical and organizational capacity. Because few organizations in Jamaica possess the technical and organizational capacity to work with at￾risk youth at the secondary and tertiary levels, and most organizations are not equipped with full-time staff with the technical expertise needed, such as case managers, psychologists, project coordinators, and mentors, USAID/Jamaica should design programs that include two phases. First, it should train potential implementing partners in evidence-based approaches to reducing youth crime and violence to equip organizations with full-time staff able to implement the LPD integrated model. Second, it should award grants to trained organizations that meet minimum technical and organizational capacity standards to implement these approaches. By doing so, USAID will ensure that more organizations have the capacity to work with at-risk youth more effectively. Consider commissioning a cost-effectiveness study to analyze and determine the optimum level of support needed for technical and psychosocial support to youth to effectively reduce their risk factors. Insufficient budget was a major barrier identified in the evaluation. Activities such as managing grants, providing stipends to program participants, and, especially, psychosocial support are costly, and many organizations working to reduce youth crime and violence lack sufficient funds to cover these costs. Studying the cost effectiveness of these various interventions would enable stakeholders to understand how to allocate their resources most effectively. Sustain funding as part of a long-term approach to youth crime and violence prevention. Consistent funding was identified as an enabling factor for ongoing participation by private sector organizations. While many private sector organizations are interested in participating in youth crime and violence prevention programs, they are reluctant to do so unless staffing, program participant stipends, and other costs are covered. Future interventions should continue to coordinate as much as possible with the Government of Jamaica and across donors/partners (including CSOs and the private sector) to maximize program effectiveness. Effective coordination between stakeholders was identified as a need across stakeholder types and was an enabling factor for ongoing private sector participation. Approaches that promote collaboration between the private sector, civil society, and government agency partners foster a common understanding of youth crime and violence prevention and facilitate effective coordination of programming. For example, greater coordination can help future partners target specific participant groups. Strategies that employ collaboration and mutual learning also help build private sector capacity USAID.GOV FINAL PERFORMANCE EVALUATION OF THE LOCAL PARTNER DEVELOPMENT ACTIVITY | 37 to continue work in this area by transferring knowledge and skills related to social development, program management, and working in the donor-funded environment. Increase engagement of youth family members to further improve reduction in risk factors and youth resilience to crime and violence. Though LPD programming in this area was limited, family members of at-risk youth (generally family members of minors) often described the parenting classes they participated in as transformative. Expanding this type of support in an integrated youth risk reduction model increases its chances of effectiveness by addressing additional dimensions of risk for involvement in crime and violence beyond the individual, and could be provided either through complementary programming (e.g., USAID’s Positive Pathways activity) for which participants receive referrals or through direct inclusion in a future activity model. Focus future programming with unattached youth on permanent job placement opportunities (or apprenticeships with the opportunity for permanent placement) rather than short-term internships. Internships are most appropriate and successful in providing youth enrolled in school with a first employment experience. Respondents identified increased access to employment opportunities as a significant motivation for youth participation in interventions, but they also identified a lack of actual long-term employment opportunities as part of LPD programming as an obstacle. Ongoing employment support beyond their initial training and apprenticeships would help the long-term success of youth program participants. Continue including case management/mentorship and CBT components in future interventions. The case management/mentorship and CBT components should be continued in future programming because they address underlying issues to decrease dropouts. Government of Jamaica entities should also institutionalize them to increase sustainability. Consider involving youth in activity design, rather than engaging them midway through when issues have already arisen. LPD’s efforts to include partners and stakeholders in program co-creation and to solicit feedback from youth were laudable and successful. The next step in demonstrating this commitment to incorporating knowledge gained through youths’ lived experience to improve programming is to expand consultation with youth at the beginning of the activity. Though there may be logistical or timeline challenges to accomplishing this, expanding the co-design process could improve participant buy-in and avoid some issues in implementation. Build in sufficient resources for youth crime prevention programming. Some of the elements of LPD’s youth crime prevention programming that participants considered most important in reducing youth crime and violence were also some of the programs’ most cost-intensive elements, such as case managers and stipends. Respondents often pointed out that funding for these programmatic elements was insufficient. USAID should ensure that program funding is sufficient to hire 38 | FINAL PERFORMANCE EVALUATION OF THE LOCAL PARTNER DEVELOPMENT ACTIVITY (LPD) USAID.GOV enough case managers and cover participants’ costs—including supporting their families, if deemed necessary in special cases—during the program design and budgeting stage to ensure that participants fully attend programming and complete their individual treatment plans. For instance, stipends and meals should be considered an essential component of all activities and must be sufficient to adequately compensate participants’ attendance-related costs. However, future programming should avoid making stipends so high that they become an “incentive for attendance.” As a complementary approach to alleviate the lack of funds, future programs should consider combining program activities with support to youth to find a part-time job that could help participants earn some money to cover attendance costs and increase the rate of graduated youth, something that would require private sector engagement in program design, given the challenges LPD has experienced with securing short-term internships. Continue the individualized, integrated approach to youth interventions. These early results of LPD’s integrated approach showed promising decreases in risk for youth involvement in crime and violence and earned praise from both participants and grantees who saw the activity making a difference in their communities. Current and future programming should build on this success by adopting and refining LPD’s comprehensive, customized, and sustained model of support for medium- and high-risk youth. Specifically, programming for youth crime and violence prevention should continue to combine employment-oriented activities with CBT, anger management, mentorships, drug counseling, and other life skills-oriented training. Expand youth assistance programs into other hotspot areas. Because many of the benefits of this work are individual or highly local, USAID should consider expanding the geographic reach of future activities to maximize programmatic outcomes. Build sustainability through ongoing connections with program graduates. Integrating program graduates into future interventions (for example, as mentors for future classes/activities) could be a powerful tool to support graduates’ progress and provide current program participants with advocates who understand youth challenges firsthand. Plan for a longer program timeline to realize full benefits. LPD’s timeframe to assess and re-assess at-risk youth was short (six months for clients below 18 years old, and nine months for clients 18 years or older) and should be extended in future programming to have more significant reduction in risk levels. Grants should also include a period of youth mentoring and/or monitoring after the end of the case treatment plans to support transition and monitor for sustainability of changes. Because the timeframe for assessing and re-assessing at-risk youth was set by the Ministry of National Security, any changes to those timelines should be agreed upon with this ministry. Expand work to share data on results of activities and best practices, so that a wider community of stakeholders has timely access to USAID-funded evidence on what works. Grantees and government stakeholders expressed desire for more timely access to M&E data to help them incorporate lessons learned into their programming. Investments in access to information can also USAID.GOV FINAL PERFORMANCE EVALUATION OF THE LOCAL PARTNER DEVELOPMENT ACTIVITY | 39 help sustain learning after the activity ends. For example, making resources on programming for organizations serving youth available electronically through an online platform can be a low-cost way to sustain learning gained through activities such as LPD. PSE programming for youth crime and violence prevention should continue to use a collaborative approach, along with outreach to the general business community that leverages their ties to the local community. Ongoing collaboration should use co-creation processes, management support, skills training, and other capacity-strengthening activities that enhance private sector partners’ knowledge and skills in youth crime and violence prevention. Because private sector organizations have limited experience and knowledge in social development, collaborative approaches that strengthen their capacity and encourage collaboration with civil society increase their ability to effectively engage in youth crime and violence prevention. Additionally, PSE programming should sensitize the business community to the challenges facing at-risk youth, the related social problems for the communities in which the private sector organizations work, and the benefits of working with at-risk youth. Finally, in many instances, private sector organizations are already engaged in community development and have existing ties to community-based organizations, such as community development committees. As ongoing members of the community, and with employees living and working in affected areas, private sector organizations can serve as long-term allies to promote youth crime and violence prevention, contributing to the sustainability of these efforts. 40 | FINAL PERFORMANCE EVALUATION OF THE LOCAL PARTNER DEVELOPMENT ACTIVITY (LPD) USAID.GOV ANNEX A: STATEMENT OF WORK ACTIVITY TITLE Final Performance Evaluation of the Local Partner Development (LPD) activity. PURPOSE OF THE EVALUATION USAID’s Automated Directives System (ADS) Chapter 20119 establishes Agency Program Cycle guidance and procedures, including those for learning from performance monitoring, evaluations, and other relevant sources of information to make course corrections as needed and inform future programming. According to the policy, “[e]valuation is the systematic collection and analysis of information about the characteristics and outcomes of strategies, projects, and activities conducted as a basis for judgments to improve effectiveness and timed to inform decisions about current and future programming. Evaluation is distinct from assessment or an informal review of projects.” Further, “[t]he purpose of evaluations is twofold: to ensure accountability to stakeholders and to learn to improve development outcomes.” The purpose of this performance evaluation is threefold to: (1) determine the extent to which the LPD activity’s strategic approach improved the resilience of targeted youth, their families and communities to crime and violence; (2) assess the degree to which targeted local organizations are able to implement evidence-based programming to improve activity outcomes; and (3) examine the extent to which private sector engagement may improve the sustainability of youth crime and violence prevention interventions.10 Findings and recommendations from the evaluation will be instructive for broader Caribbean Basin Security Initiative (CBSI) programming considerations (see Section III for more information). AUDIENCE AND INTENDED USES The primary evaluation stakeholders are: 1. USAID/Jamaica 2. Ministry of National Security, including agencies and departments such as the Department of Correctional Services 3. Ministry of Justice 4. Ministry of Education and Youth 5. The Planning Institute of Jamaica 9 ADS 201 can be found at https://www.usaid.gov/ads/policy/200/201 10 USAID’s definition of Private Sector Engagement (PSE) may be found in the PSE policy at: https://www.usaid.gov/work￾usaid/private-sector-engagement/policy USAID.GOV FINAL PERFORMANCE EVALUATION OF THE LOCAL PARTNER DEVELOPMENT ACTIVITY | 41 6. Civil Society Organizations, including community and faith-based organizations, engaged in youth crime prevention programming 7. Private Sector Organization of Jamaica and select member organizations engaged in youth crime prevention programming LPD is an activity under the CBSI of the U.S. Department of State (DOS). The DOS, USAID, and other agencies such as the Bureau of International Narcotics and Law Enforcement (INL) that receive CBSI funding will take a particular interest in any lessons for CBSI success. USAID/Jamaica will disseminate evaluation findings to secondary audiences, such as LAC missions with similar programs and with USAID’s Bureau for Development, Democracy, and Innovation (DDI) and Bureau for Conflict-Prevention and Stabilization for sharing in other regions. The Mission will use the findings of the evaluation report to inform the development of future capacity building and youth crime and violence prevention programming. Other international development partners such as the UK Foreign Commonwealth and Development Office, Global Affairs Canada, and the European Union that implement citizen security activities would also be interested in the findings. SUMMARY INFORMATION Strategy/Project/Activity Name Local Partner Development Activity USAID Office Office of Citizen Security, USAID/Jamaica Implementer(s) FHI 360 Cooperative Agreement # AID-532-LA-17-00001 Total Estimated Ceiling of the Evaluated Activity (TEC) $16,000,000 Life of Activity February 2017 – February 2023 Active Geographic Regions See Section C4. B Development Objective(s) (DOs) DO 2: Youth Crime and Violence Prevented in Targeted Communities (CDCS 2020-2025) DO: Threats to the environment and citizen vulnerability reduced (CDCS 2013-2019) Required evaluation? Yes External or internal evaluation? External 42 | FINAL PERFORMANCE EVALUATION OF THE LOCAL PARTNER DEVELOPMENT ACTIVITY (LPD) USAID.GOV BACKGROUND DESCRIPTION OF THE PROBLEM AND CONTEXT Jamaica is an upper middle-income country with a population of approximately 2.8 million. The country boasts a relatively high life expectancy and a high literacy rate. Its Journey to Self- Reliance Roadmap education quality score (0.45) is slightly above the average (0.40) and shows room for further improvement.11 Despite these gains, the country is confronted with challenges that impede its development, including a high homicide rate. High levels of crime and violence continue to be a major concern of the government and citizens of Jamaica—threatening many industries and placing a heavy burden on the health and criminal justice systems. In an October 12, 2017, Jamaica Gleaner newspaper article, the then Minister of National Security estimated the direct cost of violence to the Jamaican economy to be 5 percent of GDP or J$68B (U$535M). For many years, Jamaica has experienced high and persistent levels of violence placing the country in the top ten countries with the highest homicide rates. In-Sight Crime’s 2021 Homicide Roundup reported that Jamaica had the highest homicide rate in Latin America and the Caribbean at 49.4 deaths per 100,000 people in 2021. Notwithstanding additional crime-fighting measures put in place by the Government of Jamaica (GOJ), the number of murders continues to increase. Data from the Jamaica Constabulary Force indicates that there were 1,463 murders in 2021, representing a 10 percent increase over 2020.12 The country’s 0.54 score out of 1.0 on the 2022 USAID Road Map’s “Safety and Security” metric confirms that crime and violence is still a pressing concern. In 2021, Jamaica scored 0.57 for safety and security. In Jamaica, youth are most often the perpetrators and victims of crime and violence. Youth are arrested, jailed and murdered at twice the rate of the general population. An IDB study13 on crime and violence in Jamaica profiles most victims of homicide as, “male, young, uneducated, and poor.” The study found that, “In 2013, 90 percent of all victims were male” and 51 percent were under the age of 35. Per USAID/Jamaica’s Gender Analysis, the frequency of physical violence for males averaged 2,799 per year over the 2007-2012 period and the violence frequency for females averaged 311 during the same period. Many young men possess criminal records that present obstacles to accessing legitimate social services, jobs, and accompanying opportunities that keep them confined in the perpetual cycle of a criminal and violent lifestyle. Those with a criminal history face difficulty getting jobs in the formal sector as many employers require a criminal background check for employment. The GOJ has implemented several strategies to reduce the number of homicides and increase cooperation with the police. The Ministry of National Security (MNS) ended its major intervention—the Citizen Security and Justice Program (CSJP) in September 2019—and has launched a new Citizen Security Plan (Plan Secure Jamaica)14, which seeks to intensify and deepen interventions in targeted 11 See FY 2022 Jamaica Country Roadmap Score of 0.45 on Education Quality (https://selfreliance.usaid.gov/country/jamaica) 12 https://jcf.gov.jm/stats/ - April 25, 2022 13 Inter-American Bank (IDB) Series on Crime and Violence in the Caribbean: Crime and Violence in Jamaica, 2016 14 See JIS report of the Minister of National Security presentation on Plan Secure Jamaica - https://jis.gov.jm/govt￾committed-to-plan-secure-jamaica-dr-chang/ USAID.GOV FINAL PERFORMANCE EVALUATION OF THE LOCAL PARTNER DEVELOPMENT ACTIVITY | 43 hotspot communities and address the challenges identified, including trust levels between police and citizens and impunity among public officials. Despite Jamaica consistently scoring very high (0.96) on the USAID Road Map’s “Civil Society Capacity” metric, USAID’s work with civil society organizations (CSOs) in citizen security has shown that they tend to have low capacity to manage donor funding and implement data-driven programs. In addition, they have inadequate human resources, low funding and often rely on donor funds, and there is little collaboration and coordination between CSOs. Findings from a 2016 Jamaica Local Capacity Development mapping exercise reveal that although Jamaica has a vibrant and active civil society its overall capacity and structure are still weak and fragmented, with limited genuine collaboration and coordination. It is USAID’s experience that non- governmental organizations (NGOs) that work on youth issues tend to focus on primary interventions that target the general population rather than secondary and tertiary interventions targeting youth who have serious behavioral issues, have come into contact with the law, or have a police record. These findings and observations have influenced USAID/Jamaica’s decision to build the capacity of local CSOs and the private sector to effectively work with youth who are most at risk of getting involved in violence and crime, that is, secondary and tertiary crime prevention. LINKS TO JAMAICA CDCS AND CARIBBEAN BASIN SECURITY INITIATIVE The CBSI was launched after the 5th Summit of the Americas in April 2009 in response to rising crime (high homicide rates),15 the security threat posed by Transnational Organized Crime, and the need to secure the United States’ third border. It brings together members of the Caribbean Community and the Dominican Republic to collaborate on regional security with the United States as a partner and provides funding for activities aimed at crime and violence prevention. In 2017, USAID started the implementation of the Local Partner Development activity with CBSI funds. The activity was implemented across two USAID Country Development Cooperation Strategies (CDCSs): 1. 2013-2019 CDCS (see Figure 1) - Development Objective “Threats to the Environment and Citizen Vulnerability Reduced” through Intermediate Result (IR) 2: “Violence in Community Renewal Programme (CRP) Communities Reduced” and Sub IR 2.3: “Institutional Capacity of Community Based Organizations (CBOs), Civil Society Organizations (CSO), Non￾Governmental Organizations (NGOs) and Ministries, Department and Agencies (MDAs) Improved.” 2. 2020-2025 CDCS (see Figure 2) - DO 2: Youth crime and violence prevented in targeted communities through IR 2.1 “Resilience of youth improved to prevent crime and violence” and IR 2.2 “Institutions' effective response to growing crime and violence.” The relevant sub-IRs are: − Sub-IR 2.1.1- Community-level Violence Prevention strengthened to increase resilience factors and reduce risk factors that drive youth involvement in crime and violence − Sub-IR 2.1.2 - Child Justice System improved to rehabilitate and reintegrate youth in conflict with the law back into their communities 15 UNDOC reported that the Caribbean had the highest homicide rate of any other region in the world (30 per 100,000, 2007) 44 | FINAL PERFORMANCE EVALUATION OF THE LOCAL PARTNER DEVELOPMENT ACTIVITY (LPD) USAID.GOV − Sub-IR 2.2.1 - Criminal Justice System strengthened to effectively process crime and violence cases − Sub-IR 2.2.2 - Institutions Capacity to Advance Crime and Violence Prevention Increased Figure 1: USAID/Jamaica’s Results Framework (Source: Jamaica CDCS 2013-2019) USAID.GOV FINAL PERFORMANCE EVALUATION OF THE LOCAL PARTNER DEVELOPMENT ACTIVITY | 45 Figure 2: USAID/Jamaica’s Results Framework (Source: Jamaica CDCS 2020-2025 The Local Partner Development activity also supports the 2020 CBSI Strategic Framework, which has two IRs: 1. IR 2.1: Resilience of Youth Improved to Prevent Crime and Violence Under this IR, USAID supports efforts that help targeted communities increase resilience factors and reduce risk factors that drive youth involvement in crime and violence. 2. IR 2.2: Institutions’ Effective Response to Growing Crime and Violence Through IR 2.2, USAID partners with GOJ institutions to strengthen restorative justice practices and implement alternative dispute resolution methods and practices in order to reduce court backlogs at the parish and community levels. Partnerships with the GOJ, civil society and the private sector implement crime prevention activities and seek to strengthen interventions to build trust and confidence at the community level among the JCF and citizens. CBSI has been codified under the Caribbean-United States Framework for Security Cooperation, the Caribbean-United States Declaration of Principles, and the Caribbean-United States Plan of Action. When the Framework was updated in November 2017, it emphasized that improving citizen security requires a coordinated multi-national and multi-sector approach including 46 | FINAL PERFORMANCE EVALUATION OF THE LOCAL PARTNER DEVELOPMENT ACTIVITY (LPD) USAID.GOV prevention, institutional reforms, and information sharing. When the Plan of Action was also updated in November 2017, the governments committed to implementing and strengthening the collection and use of reliable data as a basis for policies and decision-making. DESCRIPTION OF THE INTERVENTION TO BE EVALUATED AND THEORY OF CHANGE LPD 2017-2019 The LPD activity began as a general organizational strengthening program working with local partners across several sectors. After two years, the activity transitioned to strengthening institutions that work in the youth crime and violence prevention sector and re-align with the CBSI Framework. For the first two years of implementation, the main objective of the LPD was to build the capacity of Jamaican Nongovernmental Organizations (NGOs), Social Enterprises (SEs), Community Based Organizations (CBOs), Civil Society Organizations (CSOs), and government entities that partner with USAID/Jamaica’s Office of Citizen Security (OCS) to become effective and sustainable actors to improve citizen security and social cohesion. LPD then had four main components: 1. Component 1: Strengthen the Capacity of Targeted Local Entities; 2. Component 2: Support Social Enterprises through a Business Incubator Model; 3. Component 3: Improve the Enabling Environment for Civil Society Organizations and Social Enterprises; and 4. Component 4: Intentional Collaboration Between Civil Society, Government, and the Private Sector. LPD 2019-2023 To better align with the revised CBSI Framework and USAID’s CDCS, from 2019 onwards, the primary focus of the LPD changed to strengthening the capacity of targeted Jamaican CSOs and government entities that support them to become more effective in preventing crime and violence, by increasing collaboration between government, business, and civil society. LPD works to achieve the following result: Jamaican institutions will become more effective in advancing collaborative, evidence- based youth crime and violence prevention strategies. CSOs and key public and private partners will be better able to mobilize and USAID.GOV FINAL PERFORMANCE EVALUATION OF THE LOCAL PARTNER DEVELOPMENT ACTIVITY | 47 sustain targeted and effective secondary and tertiary prevention. LPD’s theory of change hypothesizes that: IF LPD: 1. Builds key stakeholders’ awareness of evidence-based prevention approaches and improves their capacity to target youth at higher risk of perpetration of crime and violence, including youth in conflict with the law or in formal institutional settings, AND 2. Empowers local governments, civil society organizations and community members to advocate for critical resources and reforms, THEN LPD’s partners, through USAID funding, will 1. Holistically and cost-effectively target the social, economic and psycho-social drivers of crime and violence and provide effective services to the most at-risk youth, and 2. Create the conditions, services and opportunities for youth in the justice system to rehabilitate, reintegrate and avoid re-engagement with crime/violence, 3. Apply evidence-based strategies, including through government partners’ accessing and generating reliable crime and violence data, and test the efficacy of new approaches 4. Elevate local/public pressure in support civil society’s ability to address needs and grievances, strengthening feedback loops to hold governments accountable to resource crime and violence prevention 48 | FINAL PERFORMANCE EVALUATION OF THE LOCAL PARTNER DEVELOPMENT ACTIVITY (LPD) USAID.GOV SO THAT primarily: 5. At-risk youth experience reduced risk factors and become more resilient to becoming victims or perpetrators of crime 6. Families and communities access services and infrastructure and supports that address physical, emotional, and psycho-social needs and the complex drivers/risks factors of crime and violence 7. Schools support students to resolve conflict nonviolently, find healthy avenues for a sense of belonging, build self-efficacy, improve academic performance, and provide workforce development opportunities 8. Youth in conflict with the law more effectively rehabilitate and reintegrate into society 9. Community members, civil society organizations, and local governments will effectively advocate for resources and reforms to improve crime and violence prevention and address key community needs LPD focuses on three components, namely: 1. Component 1 activities strengthen the efforts of CSOs that work on youth crime and violence prevention, allowing them to become more effective actors who are able to provide sustainable services that positively impact youth crime and violence in Jamaica. 2. Component 2 activities build the capacity of private sector organizations in secondary and tertiary crime prevention, while involving the sector in strategic efforts to prevent and reduce crime and insecurity. 3. Component 3 activities work with Government of Jamaica (GOJ) institutions that directly work in the area of crime and violence to enable them to design and sustain effective crime prevention strategies. COVID-19 RESPONSE In response to the novel Coronavirus (COVID-19), LPD’s award was modified in September 2020 to include $1 million to provide immediate relief to persons impacted by the disease over 12 months. COVID-19 Response activities are in line with Component 1. The specific goal of this intervention was to strengthen the ability of LPD partners to effectively respond to social, economic, and environmental disruptions of individuals most affected by the pandemic. TARGET POPULATION AND GEOGRAPHIC AREA Youth crime and violence interventions are most effective when they focus on locations with the highest risk and the persons most at risk of participating in or experiencing violence. Therefore, one of the primary reasons for the realignment of the LPD was to focus on: 1. hot-spot or “opportunity” communities where crime and violence are the highest; 2. the demographic groups of “opportunity youth” (e.g., youth in detention) that are most at￾risk for involvement in crime and violence; and/or USAID.GOV FINAL PERFORMANCE EVALUATION OF THE LOCAL PARTNER DEVELOPMENT ACTIVITY | 49 3. specific enabling environment issues (macro-level governance issues and systems) that must be addressed for effective crime and violence prevention. LPD is required to work with partners to identify high risk places, people, and behaviors most closely associated with violence and leverage resources and align them into areas where they will be most effective, for example, juvenile correction facilities and ‘hot-spot’ communities. LPD is expected to coordinate closely with the GOJ’s Ministry of National Security and the Planning Institute of Jamaica (PIOJ), which serves as the coordinating entity for the Community Renewal Programme (CRP) to identify target communities. Under the MNS, there is a list of the 25 priority communities based on the Jamaica Constabulary Force’s data focusing on the communities with the highest murder rates. Similarly, the CRP accounts for the top 100 most vulnerable communities. The target communities are located in the parishes of Kingston, St. Andrew, St. Catherine, St. James and Clarendon. The core LPD activity is implemented in select communities within these parishes. The COVID-19 intervention (see below) was much broader than these parishes. ACTIVITY IMPLEMENTATION The activity implementation period runs from February 2017 to February 2023. As stated above, the first phase of the LPD activity (2017-2019) focused on general capacity development of USAID/Jamaica local partners. Since October 2019, activity implementation has been more closely aligned with the CBSI Framework and builds the capacity of local partners that implement youth crime and violence prevention programs. Since March 2020, the COVID-19 pandemic has adversely impacted implementation resulting in missed milestones and deliverables, the complete redesign of some interventions, and the inability to engage with some beneficiaries. The evaluation team shall review all activity documents made available by USAID or FHI 360, as well as any other sources relevant to the evaluation, such as government or international data. USAID will provide the evaluation team with the following list of available information sources to be examined by the evaluation team: 1. Cooperative agreement and modifications 2. LPD progress reports 3. Annual work plans 4. LPD Monitoring, Evaluation, and Learning (MEL) Plans 5. Studies or assessment completed or commissioned by LPD or its sub-partners Prior to conducting field work, the evaluation team will review existing literature and data, including the documents provided by USAID. EVALUATION QUESTIONS In keeping with the evaluation purpose described in Section C.2, USAID/Jamaica, in consultation with USAID/Washington, has selected the questions listed below. The questions below are listed in order of priority and take into account LPD’s objectives within the context of USAID/Jamaica’s CDCS, CBSI 50 | FINAL PERFORMANCE EVALUATION OF THE LOCAL PARTNER DEVELOPMENT ACTIVITY (LPD) USAID.GOV goals, and agency priorities (see Section C.4). The sub- questions elaborate on the main questions; answering them should assist with, and provide support for, answering the top-level questions. To what extent has the LPD activity improved the resilience of targeted youth and their families to crime and violence? a. What interventions were most effective for beneficiaries? b. From the perspective of beneficiaries, what are the areas that could be improved? Discuss gaps and interventions that were least effective. How could the interventions be improved? c. In what ways has LPD built the capacity of local government and non-government organizations to implement evidence-based interventions? − What are the main barriers (internal and/or external) to local organizations adopting evidence-based approaches? − What are the enabling factors that allow organizations to successfully adopt evidence-based programming? − What evidence exists that beneficiary organizations are likely to continue to implement these approaches? − To what extent has LPD integrated PSE to address the risk and resilience factors faced by targeted youth, families, and communities? − What PSE approaches are most useful in youth crime and violence prevention? What are the potential barriers to greater private sector participation in this area? − To what extent can PSE approaches help build sustainability in youth crime and violence prevention activities in Jamaica? − What are the enabling factors that allow private sector organizations to support youth crime and violence prevention activities? − What evidence exists that private sector organizations are likely to continue to provide support in this area? In answering these questions, the evaluator shall explicitly discuss the unexpected conditions or challenges that were experienced by the activity during implementation and how the implementing partner adapted to or resolved those challenges. The Evaluation contractor shall incorporate gender concerns in the evaluation methodology (including data collection and analysis) and disaggregate data by sex, age and geographic location (for example, rural vs. urban). The evaluation team must integrate this analysis into the evaluation’s overall findings, conclusions, and recommendations. USAID.GOV FINAL PERFORMANCE EVALUATION OF THE LOCAL PARTNER DEVELOPMENT ACTIVITY | 51 ANNEX B: DATA COLLECTION AND ANALYSIS TOOLS KII SEMI-STRUCTURED GUIDES USAID 1. How have you interacted with the LPD Activity and what was your role? How would you describe those interactions? 2. What are your impressions of LPD in promoting the capacity of Jamaica government, CSO, and private sector organizations to implement youth crime and violence prevention strategies? 3. Overall, what aspects of LPD’s activities worked well? a. (Probe for program development, training, communication and collaboration, support, overall management) 4. Overall, what aspects of LPD’s activities could have been improved? a. (Probe for program development, training, communication and collaboration, support, overall management) 5. How did COVID impact LDP’s activities? a. What adaptations were made in response to COVID? b. How effective and responsive was LPD in facilitating adaptations to COVID? 6. What other challenges did LPD face? How did LPD adapt to those challenges? 7. What should be donor strategic priorities in this regard? What programming should be considered? a. What mix and intensity of different types of activities would be appropriate for the Jamaican context going forward? 8. What recommendations can be made to ensure greater impact on building Jamaica government, CSO, and private sector capacity to implement youth crime and violence prevention strategies? 9. Are there good examples of youth crime and violence prevention strategies from other countries that could be introduced into Jamaica? 10. Are there any gender-related issues that have been impacted, positively or negatively, by LPD activity? What about issues related to persons with disabilities? What about other groups of at-risk youth? 11. Are there any other views/persons within the US Government that we should seek out to interview? 12. Beyond what we’ve discussed, what else do you think I should know about LPD and their effectiveness in promoting youth crime and violence prevention strategies? FHI 360 (LPD TECHNICAL STAFF) 1. Can you briefly describe how you selected the CSOs, government entities and private sector organizations to implement LPD activities during phase 1 and phase 2? 2. What kind of technical support did you provide to those organizations? a. Probe to what extent and how the OCA tool was used to plan technical support for CSOs, GoJ and private sector? Were there other assessment tools? If yes, please describe them. 52 | FINAL PERFORMANCE EVALUATION OF THE LOCAL PARTNER DEVELOPMENT ACTIVITY (LPD) USAID.GOV 3. To what extent LPD partners (CSOs, GoJ entities and private sector) have contributed to rehabilitate, reintegrate and avoid re-engagement of youth in crime and violence? What has worked well? What has not worked well? 4. In what ways and how did LPD strengthen CSOs, GoJ and private sector capacity to generate and use evidence to design effective activities to overcome secondary and tertiary risks and reduce youth crime and violence? 5. To what extent the organizations benefited by LPD have improved their capacity to target youth at higher risk of getting involved in crime and violence, those in conflict with the law or those already in a formal institutional setting (detainees, center of rehabilitation/reintegration, etc.)? 6. What type of services were provided to youth at higher risk of perpetration of crime and violence, including youth in conflict with the law or in formal institutional settings? 7. In what ways and to what extent, LPD improved youth resilience to crime and violence? How do you measure youth resilience to crime and violence? a. (Probe for use of Youth Risk Factor Assessment -YRFA- tool) 8. In your opinion, what social, economic and psycho-social drivers of crime and violence are the most challenging? What services have worked well to address those drivers and reduce crime and violence? a. (Probe for the use of M&E data to measure improvements; use of YRFA tool) 9. To what extent LPD has achieved the expected results as per the M&E plan and in line with the ToC? 10. In your opinion, is a 5-6-year period sufficient to achieve the outcomes/results as per the theory of change? Are there improvements that can be made in this regard? 11. Overall, what aspects of LPD’s activities could be improved for future similar programs? a. (Probe for program development, training, communication and collaboration, support, overall management) 12. Overall, are LPD’s partners likely to continue working in the activities to reduce crime and violence after the Activity ends? Why? GOVERNMENT OF JAMAICA Before getting into the interview would you please describe the work you and your organization have done with LPD? 1. What role does your organization play with regard to youth crime and violence prevention? a. What target groups are your activities aimed at? (e.g., males/females; unattached youth; youth offenders, youth victims, parents etc.) b. What guidelines, policies or laws inform these youth crime prevention activities? c. In your opinion, does your organization have enough resources to successfully address youth crime and violence prevention? (e.g., database or other form of shared data, support services or formal referral mechanisms with other MDAs, etc.) d. What types of data do you collect on activities geared towards youth crime and violence prevention? What do you officially report on and who gets this report? 2. Can you describe the work done with LPD regarding institutional capacity building to work with youths at secondary and tertiary risk level? a. Did you receive assistance to design and implement evidence-based approaches to work with youth at-risk? If yes, please describe. USAID.GOV FINAL PERFORMANCE EVALUATION OF THE LOCAL PARTNER DEVELOPMENT ACTIVITY | 53 b. What improvements has this assistance made to program implementers? What improvements have this made to program participants or other stakeholders? c. Overall, what interventions with LPD worked well and what did not work well? d. What challenges you experienced working with LPD? e. What areas can be improved? 3. In what ways has LPD built the capacity of local government and non-government organizations to implement evidence-based interventions? What are the barriers or enabling factors? 4. Can you describe how you engaged the private sector (PSE) to reduce the risk factors and improve resilience factors faced by targeted youth, families, and communities? 5. What are the perspectives to continue working to reduce crime and violence in Jamaica using the experience gained by working with LPD? CSOs, Private sector and GoJ? 6. Beyond what we’ve discussed, what else do you think I should know about LPD and their effectiveness in promoting youth crime and violence prevention strategies? PRIVATE SECTOR ORGANIZATIONS Before getting into the interview would you please describe the work you and your organization have done with LPD? 1. Can you describe what capacity building support you/your organization received from LPD? Did you receive assistance to design and implement evidence-based approaches to work with youth at￾risk? If yes, please describe. (Probe: OCA, YRFA, M&E, other) 2. Why your organization decided to work with LPD in secondary and tertiary youth crime and violence prevention? a. In what ways LPD engaged your organization and the private sector in general (whether as services providers to youths at risk or business (individual, association or chamber of commerce) providing opportunities of apprenticeship, internship or employment? b. In your opinion, what ways have proven most effective and why? 3. What services did you provide to youths and their families under the LPD activity? 4. Based on the results achieved so far, with support of LPD, to what extent your organization or LPD has: a. Implemented evidence-based activities or programs to reduce secondary and tertiary crime and violence? b. Reduced the participation of youth in secondary and tertiary youth crime and violence in the community (improved resilience of youth and their families to youth crime and violence)? c. Engaged the private sector in the crime and violence prevention activities and how this can improve the sustainability of the activities? What are the major barriers or enabling factors to engage private sector and for sustainability? 5. Overall, what aspects of your interaction with LPD worked well or not worked well? (interventions most effective or less effective) a. (Probe for what can be improved) 6. What challenges you experienced working with LPD? a. Probe for recommendations for improvements, what could be done to address these challenges. How effective and responsive was LPD in facilitating adaptations to COVID? 7. To what extent the work done with LPD is sustainable? 54 | FINAL PERFORMANCE EVALUATION OF THE LOCAL PARTNER DEVELOPMENT ACTIVITY (LPD) USAID.GOV a. (Probe for How can the experience gained from working with LPD be used to continue work aimed at reducing secondary and tertiary youth crime and violence in Jamaica?) 8. Beyond what we’ve discussed, what else do you think I should know about LPD and their effectiveness in promoting youth crime and violence prevention strategies? CIVIL SOCIETY ORGANIZATIONS Before getting into the interview would you please describe the work you and your organization have done with LPD? 1. Can you describe what capacity building support you/your organization received from LPD? Did you receive assistance to design and implement evidence-based approaches to work with youth at￾risk? If yes, please describe. a. (Probe: OCA, OPI, YRFA, M&E, other) 2. Why your organization decided to work with LPD 2 in secondary and tertiary youth crime and violence prevention? 3. What services did you provide to youths and their families before LPD and then under the LPD activity? 4. Based on the results achieved so far, with the support of LPD, to what extent your organization has: a. Implemented evidence-based activities or programs to reduce secondary and tertiary crime and violence? b. Reduced the participation of youth in secondary and tertiary youth crime and violence in the community (improved resilience of youth and their families to youth crime and violence)? c. Engaged the private sector in crime and violence prevention activities and how this can improve the sustainability of the activities? 5. How effective and responsive was LPD in facilitating adaptations to COVID? 6. Overall, what aspects of your interaction with LPD worked well or not worked well? (interventions most effective or less effective) a. (Probe for what can be improved) 7. What challenges have you experienced working with LPD? a. (Probe for recommendations for improvements, what could be done to address these challenges) 8. To what extent the work done with LPD is sustainable? a. (Probe for How can the experience gained from working with LPD be used to continue work aimed at reducing secondary and tertiary youth crime and violence in Jamaica?) 9. Beyond what we’ve discussed, what else do you think I should know about LPD and their effectiveness in promoting youth crime and violence prevention strategies? CASE MANAGERS 1. Can you please tell us about your experience with and knowledge about youth at-risk before LPD or any similar job? 2. Would you please tell us when did you start working with LPD Activity and describe the work (tasks, activities, etc.,) you have been implementing? 3. Did you receive any technical training to do your duties with youth-at-risk? If yes, what specific training you received? USAID.GOV FINAL PERFORMANCE EVALUATION OF THE LOCAL PARTNER DEVELOPMENT ACTIVITY | 55 4. Can you describe how you interacted with the CSOs, Private Sector organizations and GoJ and what were the objectives of these interactions? 5. What are the expected results of your work regarding reducing the risk factors of youth? Is there a personal performance evaluation Plan with specific targets for your position -e.g., number of youths assisted, number of organizations contacted, etc.? 6. Did the LPD activity have a case management system to track the work done by case managers? What type of registries did you prepare? How did you know that the youths assisted are more resilient to crime and violence? 7. What are the main challenges you found in your activities? What measures, if any, were implemented to address them? 8. Do you have any recommendations to increase the effectiveness in reducing risk factors of youth at secondary and tertiary levels? 9. In your opinion, the LPD’s approach to assist youth at risk at the secondary and tertiary levels is sustainable? a. (Probe for the availability of resources, political will, commitment, ownership, etc.) 10. Is there something that we have not discussed that you think we need to know to inform our evaluation report? INTERNATIONAL COMMUNITY 1. Have you or your organization ever interacted with LPD representatives and what was the situation? Please describe that interaction. 2. Are you familiar with LPD’s activities to strengthen Jamaican government, CSO, and private sector capacity to implement youth crime and violence prevention strategies? · a. If so, what are your impressions of those activities? b. Can you recall positive/good results or shortcomings of LPD regarding its youth crime and violence prevention activities? Any external factors that hindered the achievements? c. What are the key weaknesses or problems that you are aware of in addressing youth crime and violence prevention in Jamaica? 3. What should be donor strategic priorities in this regard? What programming should be considered? a. What outcome/result for the Jamaican context going forward? b. What recommendations can be made to ensure a greater impact on building Jamaica government, CSO, and private sector capacity to implement youth crime and violence prevention strategies? c. Are there good examples of youth crime and violence prevention strategies from other countries that could be introduced into Jamaica? SAMPLE FGD ISSUES: PROGRAM BENEFICIARIES/FAMILIES 1. How did you learn about the PROGRAM NAME? 2. Why did you decide to participate in the program? 3. Please describe your experience with the program. 4. Overall, what were the most important benefits of the program? 5. What do you think the program could have done better? 56 | FINAL PERFORMANCE EVALUATION OF THE LOCAL PARTNER DEVELOPMENT ACTIVITY (LPD) USAID.GOV 6. Overall, how well do you think the program has improved conditions and opportunities for youths and families? 7. Do you know anyone else involved in a youth-oriented program and, if so, which program and what was their experience? 8. Have you ever participated in another youth-oriented program? How were the experiences similar or different? 9. What do you think are the primary factors contributing to youth crime and violence? 10. What steps do you think could be taken to help prevent youth crime and violence? 11. What else do you think I should know about youth crime and violence in Jamaica? YOUTH SURVEY PRE-INTERVIEW QUESTIONS TO BE COMPLETED BY ENUMERATOR Parish Kingston/St. Andrew St. Catherine St. James Other, specify Community Name [List of towns will be preloaded and filtered based on parish selection] Enumerator Name (List TK) Respondent type: Beneficiary youth (18 and over) Beneficiary youth (under 18) Family member of beneficiary youth Unique ID [Select from list] Respondent Name [Select from list] USAID.GOV FINAL PERFORMANCE EVALUATION OF THE LOCAL PARTNER DEVELOPMENT ACTIVITY | 57 BACKGROUND INFORMATION What is your age? (In completed years) [Text fill, numbers 10-35 only] Do you still live with your parent(s)/guardian(s)? Yes No Prefer not to respond How many people in total live in your household at this time? (Including yourself) [Text fill, numbers only] How many children under the age of 13 live in your household? [Text fill, numbers only] Who do you live with? Ask if do not live with parent(s)/guardian(s) and household size > 1. Other family Roommates Significant other Other, specify Other, please specify [Text fill] Do you live in …? The capital A large/medium town/community A small town/community A rural area Do you consider yourself Black, Indian, White, Chinese, Mixed, or of another race? If respondent says Afro-Jamaican, mark Black. White Indian Black Mixed 58 | FINAL PERFORMANCE EVALUATION OF THE LOCAL PARTNER DEVELOPMENT ACTIVITY (LPD) USAID.GOV Chinese Other, specify Don’t know Prefer not to respond Other, please specify [Text fill] What is your marital status? Ask if age 16 or older. Single Married Common law marriage (Living together) Divorced Separated Widowed Civil union Don’t know Prefer not to respond Do you have children? Yes No Don’t know Prefer not to respond If yes, do you provide financial support to your child(ren)? Yes No Don’t know Prefer not to respond USAID.GOV FINAL PERFORMANCE EVALUATION OF THE LOCAL PARTNER DEVELOPMENT ACTIVITY | 59 MODULE 1: RECEIPT OF SERVICES What types of supportive services have you received in the last four years? None Food and sanitation supply vouchers Book vouchers, backpacks, and/or school supplies Case management Therapy/counselling (including CBT) Substance mis-use counselling Mentorship Life skills training Psychosocial support for GBV survivors School reintegration support Job skills training / vocational training Apprenticeship placement Microenterprise support Youth risk screening Literacy/numeracy training Other, specify Don’t know Prefer not to respond Other, please specify [Text fill] What types of supportive services are you currently receiving? [Select from list filtered to include services ever received above] Other, please specify [Text fill] 60 | FINAL PERFORMANCE EVALUATION OF THE LOCAL PARTNER DEVELOPMENT ACTIVITY (LPD) USAID.GOV What organization(s) provided the service(s) that you received? [Select from preloaded list of LPD CSOs or other, specify] Other, please specify [Text fill] What supportive services have you found most helpful? [Select from list filtered to include services ever received above] Are there any services that you did not find helpful? [Select from list filtered to include services ever received above] If yes, why not? [Text fill] Did you experience any challenges in accessing any of these services? Yes No Don’t know Prefer not to respond If yes, what were the challenges? [Text fill] Did you experience any benefits from your participation in this program? Yes No Don’t know Prefer not to respond If yes, what are the benefits? Obtained a full-time job Obtained a higher paying job Was able to attend or stay in school Reduced substance use Other, specify USAID.GOV FINAL PERFORMANCE EVALUATION OF THE LOCAL PARTNER DEVELOPMENT ACTIVITY | 61 Other, please specify [Text fill] Please rate your level of satisfaction with the services you received. Totally satisfied Highly satisfied Satisfied Partially Satisfied Dissatisfied If case management or mentorship, how would you rate your satisfaction with your case manager/mentor? Totally satisfied Highly satisfied Satisfied Partially Satisfied Dissatisfied Why did you select this response? Please explain and provide examples. [Text fill] Do you have any suggestions for areas that could be improved? What are they? [Text fill] MODULE 2: JUSTICE SYSTEM INVOLVEMENT Have you ever been arrested? Yes No Don’t know Prefer not to respond If yes, what is the year of your first arrest? [Text fill, numbers 1987-2022 only] 62 | FINAL PERFORMANCE EVALUATION OF THE LOCAL PARTNER DEVELOPMENT ACTIVITY (LPD) USAID.GOV If yes, what type of offense have you been arrested for? Assault Sexual assault Theft Burglary Selling drugs Destruction of property Other, specify Other, please specify [Text fill] Have you been arrested in connection with a violent assaultive offense in the last 5 years (since 2017)? Yes No Don’t know Prefer not to respond If yes, how many times have you been arrested in connection with a violent assaultive offense in the last 5 years (since 2017)? [Text fill, numbers only] If yes, have you been arrested in connection with a violent assaultive offense in the last year (2022)? Yes No Don’t know Prefer not to respond Have you been arrested in connection with a nonviolent offense in the last 5 years (since 2017)? This includes theft, burglary, drug selling, and destruction of property. Yes No USAID.GOV FINAL PERFORMANCE EVALUATION OF THE LOCAL PARTNER DEVELOPMENT ACTIVITY | 63 Don’t know Prefer not to respond If yes, how many times have you been arrested in connection with a nonviolent assaultive offense in the last 5 years (since 2017)? [Text fill, numbers only] Have you spent any time in jail or prison? This includes youth detention centers. Yes No Don’t know Prefer not to respond If yes, what was the last year that you spent any time in jail or prison? [Text fill, numbers only] Are you currently involved with the justice system? This includes any current interaction with the criminal justice system as a defendant. Yes No Don’t know Prefer not to respond MODULE 3: MENTAL HEALTH AND BEHAVIOR Have you ever been diagnosed with a mental illness? Enumerator: This includes diagnoses for psychotic disorders, major mood disorders, other mental disorders, and any previous or current diagnoses of antisocial, psychopathic, or dissocial personality disorders (do not read list aloud). No previous or current diagnoses Previous diagnoses Current diagnoses Don’t know Prefer not to respond Do you have a history of suicide or self-harm attempts? No history 64 | FINAL PERFORMANCE EVALUATION OF THE LOCAL PARTNER DEVELOPMENT ACTIVITY (LPD) USAID.GOV Yes, injuries without suicidal intent Yes, suicide attempt(s) Have you ever been diagnosed with an educational disability (slow learner)? Yes No Don’t know Prefer not to respond Have you ever experienced any of the following symptoms? Select all that apply. None Hyperactivity (including Attention Deficit Hyperactivity Disorder [(ADHD]) Extreme restlessness Don’t know Prefer not to respond MODULE 4: BEHAVIOR For the next questions, I will ask whether you experience certain emotional reactions, regularly, sometimes, or never. Do you keep things in until you finally explode with anger? Regularly Sometimes Never Don’t know Do you have a tendency to take your anger out on someone other than the person you're angry with? When a problem arises between you and someone else, do you discuss it without losing control of your emotions? USAID.GOV FINAL PERFORMANCE EVALUATION OF THE LOCAL PARTNER DEVELOPMENT ACTIVITY | 65 Are you satisfied with the way you settle differences with others? Prefer not to respond Do you tend to feel very guilty or bad after getting angry? When you get angry, do people around you feel threatened or frightened? When you get angry, have you ever damaged property? Yes No Don’t know Prefer not to respond If yes, how severe was the property damage? Very severe Severe Not severe How much do you agree or disagree with the following statements? I have a reserved and cautious attitude toward life. Agree Strongly Agree Some Disagree Some Disagree Strongly Don’t know Prefer not to respond I have trouble controlling my impulses. I generally seek new and exciting experiences and sensations. When I am very happy, I can’t seem to stop myself from doing things that can have bad consequences. I have trouble resisting my cravings (for food, cigarettes, etc.). I'll try anything once. I tend to give up easily. 66 | FINAL PERFORMANCE EVALUATION OF THE LOCAL PARTNER DEVELOPMENT ACTIVITY (LPD) USAID.GOV I am not one of those people who blurt out things without thinking. I often get involved in things I later wish I could get out of. When I feel bad, I will often do things I later regret in order to make myself feel better now. I enjoy taking risks. When I am upset, I often act without thinking. It is hard for me to resist acting on my feelings. Others are shocked or worried about the things I do when I am feeling very excited. When I feel rejected, I will often say things that I later regret. I welcome new and exciting experiences and sensations, even if they are a little frightening and unconventional. I always keep my feelings under control. In the heat of an argument, I will often say things that I later regret. When I am really excited, I tend not to think of the consequences of my actions. MODULE 5: RISK FOR INVOLVEMENT IN CRIME AND VIOLENCE Have you been a victim of any type of crime in the past 12 months? That is, have you been a victim of robbery, burglary, assault, fraud, blackmail, extortion, violent threats, or any other type of crime in the past 12 months? Yes No Don’t know USAID.GOV FINAL PERFORMANCE EVALUATION OF THE LOCAL PARTNER DEVELOPMENT ACTIVITY | 67 Prefer not to respond How many days in the last 30 days did you carry a weapon, such as a gun or knife? [Text fill, numbers 0-30 only] If ever carry a weapon, have you used the weapon on someone or to threaten someone? Yes No Don’t know Prefer not to respond Have you ever been associated with a gang? Yes No Don’t know Prefer not to respond What proportion of your close friends have a history of criminal justice system involvement? All Most Some Few None Don’t know Prefer not to respond Do your parents or guardians have an adult criminal history? Yes No Don’t know Prefer not to respond 68 | FINAL PERFORMANCE EVALUATION OF THE LOCAL PARTNER DEVELOPMENT ACTIVITY (LPD) USAID.GOV How would you describe the discipline, supervision, and support that you receive from your parent(s) or guardian(s)? Not applicable (no discipline/ supervision/support received from parent(s)/guardian(s)) Adequate Sometimes inconsistent Overly strict Overly permissive Don’t know Prefer not to respond Have you ever experienced conflict and violence in the home? This includes witnessing or experiencing pushing, hitting, slapping, grabbing, throwing objects, marital discord, etc. Enumerator: Occasional violence or discord means two or less violent events in the previous year. No violence Violence/discord is occasional Violence/discord is severe and/or regular Don’t know Prefer not to respond For each of the following items, please specify if the statement is Not True, Somewhat True or Certainly True. I try to be nice to other people. I care about their feelings. Not true Somewhat true Certainly true Don’t know Prefer not to respond I usually share with others, for example CDs/video games, games, and food. I am helpful if someone is hurt, upset or feeling ill. I am kind to younger children. I often offer to help others (parents, teachers, children). USAID.GOV FINAL PERFORMANCE EVALUATION OF THE LOCAL PARTNER DEVELOPMENT ACTIVITY | 69 There are lots of adults in my neighborhood I could talk to about something important. To what extent do you agree or disagree with this statement? [Number on scale from 1, 'not at all' to 7, 'a lot'] MODULE 6: ATTITUDES For each of the following items, please specify if the statement is Not True, Somewhat True or Certainly True. It’s O. K. to fight if the other guy says bad enough things about you or your family. Not true Somewhat true It’s O. K. to skip school every once in a while. Certainly true It’s O. K. to steal something from someone who is rich and can easily replace it. It’s O. K. to lie to someone if it will keep you out of trouble with them. For each of the following items, please specify if you engage in this thinking never, a little, or a lot. When you get mad, do you sometimes imagine hitting or hurting the other person? Never A little Do you ever daydream about people getting killed? A lot To what extent do you trust the justice system? [Number on scale from 1, 'not at all' to 7, 'a lot'] To what extent do you trust the local government? [Number on scale from 1, 'not at all' to 7, 'a lot'] To what extent do you trust the police in your neighborhood? [Number on scale from 1, 'not at all' to 7, 'a lot'] 70 | FINAL PERFORMANCE EVALUATION OF THE LOCAL PARTNER DEVELOPMENT ACTIVITY (LPD) USAID.GOV I am going to read some statements about the relationships of couples and I am going to ask you to indicate whether you agree or disagree. A man has a right to discipline his partner with physical violence. Strongly agree Agree Neither agree nor disagree Disagree Strongly disagree Don't know Prefer not to respond Physical violence between members of a couple is a private matter and should be handled by the couple or close family. Employers discriminate against people with past criminal behavior. To what extent do you agree or disagree with this statement? [Number on scale from 1, 'not at all' to 7, 'a lot'] MODULE 7: EDUCATION What is your highest level of education completed? None Kindergarten Grade 1 Grade 2 Grade 3 Grade 4 Grade 5 Grade 6 Grade 7 Grade 8 Grade 9 USAID.GOV FINAL PERFORMANCE EVALUATION OF THE LOCAL PARTNER DEVELOPMENT ACTIVITY | 71 Grade 10 Grade 11 Grade 12 High school diploma Post-secondary education (incomplete or complete) Vocational training Are you currently enrolled in school? Yes No Don’t know Prefer not to respond If yes, about how often did you attend school in the last month? Never Rarely Absent more often than present Present more often than absent Almost all the time All the time If yes, thinking back over the past year in school, how often did you try to do your best work in school? All of the time Most of the time Some of the time Rarely Never Don’t know 72 | FINAL PERFORMANCE EVALUATION OF THE LOCAL PARTNER DEVELOPMENT ACTIVITY (LPD) USAID.GOV Prefer not to respond In the past year, how many of your best friends have tried to do well in school? All Most Some Few None Don’t know Prefer not to respond 1. If yes, at school, do you do the following things? No Maybe Yes Organize a group of students in order to achieve changes at school. Take school problems or concerns to my teacher or principal. Teach things to other students in my school. Help keep the classroom and school clean. Serve on the student government at my school. Encourage my parent(s) to come to PTA meetings. Was your school education affected due to the COVID-19 pandemic? No, because I did not attend school for another reason USAID.GOV FINAL PERFORMANCE EVALUATION OF THE LOCAL PARTNER DEVELOPMENT ACTIVITY | 73 No, my classes continued normally Yes, I went to virtual or remote/online classes Yes, I switched to a combination of virtual/online and in-person classes (mixed learning) Yes, I cut all ties with the school Are you currently enrolled in vocational training (including apprenticeships)? Yes No Don’t know Prefer not to respond If yes, about how often did you attend training in the last month? Never Rarely Absent more often than present Present more often than absent Almost all the time All the time MODULE 8: INCOME AND EMPLOYMENT Are you currently employed? Yes, full time Yes, part time No Don’t know Prefer not to respond If yes, about how often did you attend work in the last month? Never 74 | FINAL PERFORMANCE EVALUATION OF THE LOCAL PARTNER DEVELOPMENT ACTIVITY (LPD) USAID.GOV Rarely Absent more often than present Present more often than absent Almost all the time All the time What is the total monthly income in your household? In JMD. Between 0 and $30,000 Between $31,001 and $40,000 Between $40,001 and $60,000 Between $60,001 and $80,000 More than $80,000 Don’t know Prefer not to respond Do you think that your current economic situation is better, the same or worse than it was twelve months ago? Better Same Worse In the past three months, because of a lack of money or other resources, did your household ever run out of food? Yes No Don’t know Prefer not to respond Do you or anyone in the home where you live have any of the following? Select all that apply. . Refrigerator . Landline/residential telephone (not cellular) . Washing machine . Microwave oven . Computer, laptop, tablet, or iPad Yes No Don’t know Prefer not to respond USAID.GOV FINAL PERFORMANCE EVALUATION OF THE LOCAL PARTNER DEVELOPMENT ACTIVITY | 75 . Broadband/Wi-Fi internet service in your home . Internet from your home including from phone or tablet . Flat screen/panel TV . Cable or satellite television service MODULE 9: ALCOHOL AND DRUG USE How often in the past year have you used an illegal drug or used a prescription medication for nonmedical reasons? Never Less than monthly Monthly Weekly Daily or almost daily How often in the past year have you used marijuana? Never Less than monthly Monthly Weekly Daily or almost daily In the past year, how often have you had six or more alcoholic drinks/beverages on one occasion? Never Less than monthly Monthly Weekly Daily or almost daily 76 | FINAL PERFORMANCE EVALUATION OF THE LOCAL PARTNER DEVELOPMENT ACTIVITY (LPD) USAID.GOV Are you currently experiencing physical, social, or legal problems because of your alcohol or drug use? Yes No Don’t know Prefer not to respond In the past year have you experienced any violence associated with alcohol or drug use? Yes No Don’t know Prefer not to respond CONCLUSION Thank you. This concludes our survey. Respondent name First name: Last name: [Text fill] FAMILY SURVEY PRE-INTERVIEW QUESTIONS TO BE COMPLETED BY ENUMERATOR Parish Kingston/St. Andrew St. Catherine St. James Other, specify USAID.GOV FINAL PERFORMANCE EVALUATION OF THE LOCAL PARTNER DEVELOPMENT ACTIVITY | 77 Community Name [Text fill] Enumerator Name (List TK) Youth Beneficiary name [Select from list of preloaded names] UID [Text fill, numbers only] Respondent name [Text fill] Background information What is your gender identity? Enumerator observation Male Female Other What is your relationship to the beneficiary youth? Parent Sibling Extended family (aunt, uncle, grandparent) Other, specify Other, please specify [Text fill] Do you consider yourself Black, Indian, White, Chinese, Mixed, or of another race? If respondent says Afro-Jamaican, mark Black. White Indian Black Mixed Chinese 78 | FINAL PERFORMANCE EVALUATION OF THE LOCAL PARTNER DEVELOPMENT ACTIVITY (LPD) USAID.GOV Other, specify Don’t know Prefer not to respond Other, please specify [Text fill] What is your marital status? Ask if age 16 or older. Single Married Common law marriage (Living together) Divorced Separated Widowed Civil union Don’t know Prefer not to respond MODULE 1: RECEIPT OF SERVICES What types of supportive services have you and/or your child received in the last four years? None Food and sanitation supply vouchers Book vouchers, backpacks, and/or school supplies Therapy/counselling Substance mis-use counselling Mentorship Life skills training USAID.GOV FINAL PERFORMANCE EVALUATION OF THE LOCAL PARTNER DEVELOPMENT ACTIVITY | 79 Psychosocial support for GBV survivors School reintegration support Job skills training / vocational training Apprenticeship placement Microenterprise support Youth risk screening Literacy/numeracy training Parenting classes Other, specify Don’t know Prefer not to respond Other, please specify [Text fill] What types of supportive services are you or your child currently receiving? None Food and sanitation supply vouchers Book vouchers, backpacks, and/or school supplies Therapy/counselling Substance mis-use counselling Mentorship Life skills training Psychosocial support for GBV survivors School reintegration support Job skills training / vocational training 80 | FINAL PERFORMANCE EVALUATION OF THE LOCAL PARTNER DEVELOPMENT ACTIVITY (LPD) USAID.GOV Apprenticeship placement Microenterprise support Youth risk screening Literacy/numeracy training Parenting classes Other, specify Don’t know Prefer not to respond Other, please specify [Text fill] What supportive services have you found most helpful? [Select from list filtered to include services ever received above] Are there any services that you did not find helpful? [Select from list filtered to include services ever received above] If yes, why not? [Text fill] What organization(s) provided the services that you and/or your child received? [Select from list of LPD partner organizations or other, specify] Other, please specify [Text fill] Did your child experience any challenges in accessing any of these services? Yes No Don’t know Prefer not to respond If yes, what were the challenges? [Text fill] USAID.GOV FINAL PERFORMANCE EVALUATION OF THE LOCAL PARTNER DEVELOPMENT ACTIVITY | 81 Please rate your level of satisfaction with the services you and/or your child received. Totally satisfied Highly satisfied Satisfied Partially Satisfied Dissatisfied Do you have any suggestions for areas that could be improved? What are they? [Text fill] MODULE 2: RISK FOR INVOLVEMENT IN CRIME AND VIOLENCE Have you been a victim of any type of crime in the past 12 months? That is, have you been a victim of robbery, burglary, assault, fraud, blackmail, extortion, violent threats, or any other type of crime in the past 12 months? Yes No Don’t know Prefer not to respond Do you have an adult criminal history? Yes No Don’t know Prefer not to respond How would you describe the discipline, supervision, and support that you provide to your child? Not applicable (no discipline/ supervision/support received from parent(s)/guardian(s)) Adequate Sometimes inconsistent Overly strict Overly permissive 82 | FINAL PERFORMANCE EVALUATION OF THE LOCAL PARTNER DEVELOPMENT ACTIVITY (LPD) USAID.GOV Have you ever experienced conflict and violence in the home? This includes witnessing or experiencing pushing, hitting, slapping, grabbing, throwing objects, marital discord, etc. No violence Violence/discord is occasional Violence/discord is severe and/or regular MODULE 3: ATTITUDES For each of the following items, please specify if the statement is Not True, Somewhat True or Certainly True. It’s O. K. to fight if the other guy says bad enough things about you or your family. Not true Somewhat true It’s O. K. to skip school every once in a while. Certainly true It’s O. K. to steal something from someone who is rich and can easily replace it. It’s O. K. to lie to someone if it will keep you out of trouble with them. To what extent do you trust the justice system? [Number on scale from 1, 'not at all' to 7, 'a lot'] To what extent do you trust the local government? [Number on scale from 1, 'not at all' to 7, 'a lot'] To what extent do you trust the police in your neighborhood? [Number on scale from 1, 'not at all' to 7, 'a lot'] I am going to read some statements about the relationships of couples and I am going to ask you to indicate whether you agree or disagree. A man has a right to discipline his partner with physical violence. Strongly agree USAID.GOV FINAL PERFORMANCE EVALUATION OF THE LOCAL PARTNER DEVELOPMENT ACTIVITY | 83 Physical violence between members of a couple is a private matter and should be handled by the couple or close family. Agree Neither agree nor disagree Disagree Strongly disagree Don't know Prefer not to respond Employers discriminate against people with past criminal behavior. To what extent do you agree or disagree with this statement? [Number on scale from 1, 'not at all' to 7, 'a lot'] MODULE 4: EDUCATION What is your highest level of education completed? None Kindergarten Grade 1 Grade 2 Grade 3 Grade 4 Grade 5 Grade 6 Grade 7 Grade 8 Grade 9 Grade 10 Grade 11 Grade 12 84 | FINAL PERFORMANCE EVALUATION OF THE LOCAL PARTNER DEVELOPMENT ACTIVITY (LPD) USAID.GOV High school diploma Post-secondary education (incomplete or complete) Was your child’s school education affected due to the COVID-19 pandemic? No, because child did not attend school for another reason No, my child’s classes continued normally Yes, child went to virtual or remote/online classes Yes, child switched to a combination of virtual/online and in-person classes (mixed learning) Yes, child cut all ties with the school MODULE 5: INCOME AND EMPLOYMENT Are you currently employed? Yes, full time Yes, part time No Don’t know Prefer not to respond If yes, about how often did you attend work in the last month? Never Rarely Absent more often than present Present more often than absent Almost all the time All the time What is the total monthly income in your household? Between 0 and $30,000 USAID.GOV FINAL PERFORMANCE EVALUATION OF THE LOCAL PARTNER DEVELOPMENT ACTIVITY | 85 Between $31,001 and $40,000 Between $40,001 and $60,000 Between $60,001 and $80,000 More than $80,000 Don’t know Prefer not to respond Do you think that your current economic situation is better, the same or worse than it was twelve months ago? Better Same Worse In the past three months, because of a lack of money or other resources, did your household ever run out of food? Yes No Don’t know Prefer not to respond Do you or anyone in the home where you live have any of the following? Select all that apply. . Refrigerator . Landline/residential telephone (not cellular) . Washing machine . Microwave oven . Computer, laptop, tablet, or iPad • Broadband/Wi-Fi___33 internet service in your home . Internet from your home including from phone or tablet . Flat screen/panel TV . Cable or satellite television service Yes No Don’t know Prefer not to respond 86 | FINAL PERFORMANCE EVALUATION OF THE LOCAL PARTNER DEVELOPMENT ACTIVITY (LPD) USAID.GOV CONCLUSION Respondent phone number [Text fill, integers only] Thank you. This concludes our survey. [ENUMERATOR ANSWER] What proportion of the questions do you feel the respondent had difficulty answering? All Most Some A few None [ENUMERATOR ANSWER] What was the respondent's reaction to the interview? Very positive Somewhat positive Neutral Somewhat negative Very negative [ENUMERATOR ANSWER] Any additional comments? [Text fill] USAID.GOV FINAL PERFORMANCE EVALUATION OF THE LOCAL PARTNER DEVELOPMENT ACTIVITY | 87 ANNEX C: DATA AND INFORMATION SOURCES DOCUMENT REVIEW WORK PLANS 1. Jamaica Local Partner Development Annual Work Plan: Fiscal Year 2021 (October 1, 2020 - September 30, 2021) 2. Jamaica Local Partner Development Annual Work Plan: Fiscal Year 2020 (October 1, 2019 - September 30, 2020) 3. Jamaica Local Partner Development Annual Work Plan: Fiscal Year 2019 (October 1, 2018 - September 30, 2019) 4. Jamaica Local Partner Development Annual Work Plan: Fiscal Year 2018 (October 1, 2017 - September 30, 2018) 5. Jamaica Local Partner Development Annual Work Plan: Fiscal Year 2017/2018 (October 1, 2017 - September 30, 2018) 6. Jamaica Local Partner Development Semi Annual Work Plan: Fiscal Year 2017 (February 24, 2017 - September 30, 2017) 7. Jamaica Local Partner Development Annual Work Plan: Fiscal Year 2016/2017 (February 24, 2017 - September 30, 2017) COOPERATIVE AGREEMENT AND MODIFICATIONS 8. Jamaica Local Partner Development Cooperative Agreement No. AID-532-LA-17-00001 (February 24, 2017) 9. Jamaica Local Partner Development Activity Modification of Assistance Number 1 (October 17, 2017) 10. Jamaica Local Partner Development Activity Modification of Assistance Number 3 (March 26, 2019) 11. Jamaica Local Partner Development Activity Modification of Assistance Number 4 (September, 2019) 12. Jamaica Local Partner Development Activity Modification of Assistance Number 6 (September, 2020) 13. Jamaica Local Partner Development Activity Modification of Assistance Number 7 (December 23, 2021) 14. Jamaica Local Partner Development COVID-19 Response Annual Report (October 1 - 2020 - September 30 - 2021) 88 | FINAL PERFORMANCE EVALUATION OF THE LOCAL PARTNER DEVELOPMENT ACTIVITY (LPD) USAID.GOV ANNUAL REPORTS 15. Jamaica Local Partner Development Annual Work Plan: Fiscal Year 2022 (October 1, 2021 - September 30, 2022) 16. Jamaica Local Partner Development Annual Report: Fiscal Year 2021 (October 1, 2020, September 20, 2021) 17. Jamaica Local Partner Development Annual Report Fiscal Year 2020 (October 1, 2019 - September 30, 2020) 18. Jamaica Local Partner Development Annual Report: Fiscal Year 2019 (October 1, 2018 - September 30, 2019) QUARTERLY REPORTS 19. Jamaica Local Partner Development Quarterly Report: Fiscal Year 2022, Quarter 3 (April 1, 2022 - June 30, 2022) 20. Jamaica Local Partner Development Quarterly Report: Fiscal Year 2022, Quarter 2 (January 1, 2022 - March 31, 2022) 21. Jamaica Local Partner Development Quarterly Report: Fiscal Year 2022, Quarter 1 (October 1, 2021 - December 31, 2021) 22. Jamaica Local Partner Development Quarterly Report: Fiscal Year 2021, Quarter 3 (April 1, 2021 - June 30, 2021) 23. Jamaica Local Partner Development Quarterly Report: Fiscal Year 2021, Quarter 2 (January 1, 2021 - March 31, 2022) 24. Jamaica Local Partner Development Quarterly Report: Fiscal Year 2021, Quarter 1 (October 1, 2020 - December 31, 2020) 25. Jamaica Local Partner Development Quarterly Report: Fiscal Year 2020, Quarter 3 (April 1, 2020 - June 30, 2020) 26. Jamaica Local Partner Development Quarterly Report Fiscal Year 2020, Quarter 2 (January 1, 2020 - March 31, 2020) 27. Jamaica Local Partner Development Quarterly Report Fiscal Year 2020, Quarter 1 (October 1, 2019 - December 31, 2019) 28. Jamaica Local Partner Development Quarterly Report: Fiscal Year 2019, Quarter 3 (April 1, 2019 - June 30, 2019) 29. Jamaica Local Partner Development Quarterly Report: Fiscal Year 2019, Quarter 2 (January 1, 2019 - March 31, 2019) 30. Jamaica Local Partner Development Quarterly Report: Fiscal Year 2019, Quarter 1 (October 1, 2018 - December 31, 2018) USAID.GOV FINAL PERFORMANCE EVALUATION OF THE LOCAL PARTNER DEVELOPMENT ACTIVITY | 89 31. Jamaica Local Partner Development Quarterly Report: Fiscal Year 2018, Quarter 3 (April 1, 2018 - June 30, 3018) 32. Jamaica Local Partner Development Quarterly Report: Fiscal Year 2018, Quarter 2 (January 1, 2018 - March 31, 2018) 33. Jamaica Local Partner Development Quarterly Report: Fiscal Year 2018, Quarter 1 (October 1, 2017 - December 31, 2017) 34. Jamaica Local Partner Development Quarterly Report: Fiscal Year 2016/2017, Quarter 3 (April 1, 2017 - June 30, 2017) 35. Jamaica Local Partner Development Quarterly Report: Fiscal Year 2016/2017, Quarter 2 (February 24, 2017 - March 31, 2017) MONITORING AND EVALUATION PLANS 36. Jamaica Local Partner Development Activity Monitoring, Evaluation and Learning Plan (October 1, 2021 September 30, 2022) 37. Jamaica Local Partner Development Activity Monitoring, Evaluation and Learning Plan (October 1, 2020 - September 30, 2021) 38. Jamaica Local Partner Development Activity Monitoring Evaluation and Learning Plan Fiscal Year 2020 (October 1, 2019 - September 30, 2019) 39. Jamaica Local Partner Development Activity Monitoring Evaluation and Learning Plan Fiscal Year 2019 (October 1, 2018 - September 20, 2019) 40. Jamaica Local Partner Development Activity Monitoring Evaluation and Learning Plan Fiscal Year 2017/2018 (October 1, 2017 - September 30, 2018) 41. Jamaica Local Partner Development Activity Semi-Annual Monitoring Evaluation and Learning Plan Fiscal Year 2017 (February 24, 2017 - September 30, 2017) OTHER 42. Jamaica Local Partner Development Gender and Social Inclusion Analysis (June, 2017) 43. Jamaica Local Partner Development List of Communities 44. Blurred Lines and Tough Choices: Jamaica’s Complex Legal Environment for Civil Society Organizations and Social Enterprises and Implications for Reforms: An initial assessment for discussion among Stakeholders (June, 2018) 45. Jamaica Local Partner Development Info Sheet (2017 - 2022) 46. Local Partner Development Phase 2 Info Sheet 90 | FINAL PERFORMANCE EVALUATION OF THE LOCAL PARTNER DEVELOPMENT ACTIVITY (LPD) USAID.GOV LIST OF STAKEHOLDER TYPES THAT PARTICIPATED IN KIIS USAID FHI 360 – LPD STAFF GOVERNMENT OF JAMAICA CIVIL SOCIETY ORGANIZATIONS PRIVATE SECTOR ORGANIZATIONS INTERNATIONAL COMMUNITY USAID.GOV FINAL PERFORMANCE EVALUATION OF THE LOCAL PARTNER DEVELOPMENT ACTIVITY | 91 ANNEX D: EVALUATION TEAM QUALIFICATIONS AND EXPERIENCE QUALIFICATION REQUIREMENT SPECIFIC EVIDENCE OF QUALIFICATION (BRIEF) Mario Martinez, Team Leader, will oversee the design and facilitation of the initial meeting, manage the document review, develop data collection instruments, lead KIIs/FGDs, and lead the analysis and drafting for the Final Evaluation Report and Presentation. Mr. Martinez will supervise the work of the entire team and provide quality assurance review of deliverables. Master’s level degree in international development or a social science Master of Science in Applied Economics and Graduate Diploma in Financial and Macroeconomic Planning (PontÍfica Universidad Católica de Chile); Master of Science in Economic Development Planning (Universidad Nacional Autónoma de Honduras) Experience leading an evaluation team or professional experience coordinating and leading teams. Served as Evaluation Team Leader on multiple prior evaluations in Jamaica, El Salvador, Honduras, Macedonia, Liberia, Burkina Faso, Paraguay, among other countries. Experience with mixed methods in the past 5 years – undertaking field quantitative and qualitative data collection through interviews, surveys, or focus groups for either performance or impact evaluations – demonstrated through written performance or impact evaluation reports authored or co- authored by the proposed Team Leader. Lead and authored two multi-country final evaluations funded by USDOL in 2019-2020 on the Youth Pathways Central America (El Salvador and Honduras) program and on its Country Level Engagement and Assistance to Reduce Child Labor (CLEAR) II program for combatting combat child labor, involved MEL work in seven countries (Belize, Panama, Honduras, Jamaica, Nepal, Liberia and Burkina Faso). These evaluations utilized literature and project document review, individual and small group KIIs and FGDs; quantitative analysis of administrative data and reports by international organizations; and review of secondary data using time series and statistical and regression analysis. Mr. Martinez also co-authored the US State Department Reducing Violence & Homicide through Access to Justice in Honduras evaluation report and the Inter-American Development Bank Productive Development for Increased Trade Support Program in El Salvador Final Performance and Impact Evaluation. In addition to literature and project document review, individual and small group KIIs and FGDs; quantitative analysis of administrative data and reports by international organizations, the latter evaluation also utilized ex-post Cost-Benefit Analysis (CBA). Direct knowledge and/or experience working with USAID rules, evaluation policy, regulations, and procedures Longstanding experience with USAID, including serving as MEL Director in El Salvador for a USAID-funded project, as team lead for an assessment of youth at risk for USAID/Honduras, nine years as a project manager for USAID/El Salvador, and also as an expert and team leader on multiple USAID research and evaluation projects and fidelity with USAID regulations and evaluation policy. Exceptional organizational, analytical, writing, and presentation skills. Extensive experience with analytical approaches on performance and impact evaluations (experimental, quasi- experimental, non- experimental, mixed-methods). Drafted informative reports and presentations communicating performance, impact, lessons learned, best practices and success stories of projects for different audiences including implementors, stakeholders, beneficiaries, and donors. 92 | FINAL PERFORMANCE EVALUATION OF THE LOCAL PARTNER DEVELOPMENT ACTIVITY (LPD) USAID.GOV Kate Marple-Cantrell, Assistant Team Leader/Evaluation Specialist, will support the Team Leader in the design and facilitation of the initial meeting; contribute to the document and data review; develop data collection instruments; lead KIIs/FGDs in Phase 2; contribute to Field Work Exit Briefing; and conduct the analysis and draft and revise the Final Evaluation Report and Presentation under the guidance of the Team Leader. Master’s level degree in a social science or international development with at least 10 years of technical knowledge and experience in a relevant field (e.g., democracy and governance, anti- corruption programming, etc.). Master of City Planning and MA in International Area Studies (University of California Berkeley); 10+ years of experience in relevant international development fields such as governance and capacity building, with eight of those years conducting rigorous mixed methods evaluations for USAID and the Millennium Challenge Corporation (MCC). Demonstrated experience with mixed methods in the past 5 years, undertaking field quantitative and qualitative data collection through interviews, surveys, or focus groups for either performance or impact evaluations. This experience must be demonstrated through written performance or impact evaluation reports authored or co-authored by the proposed Assistant Team Leader. Deep experience designing and implementing multiple performance and impact evaluations in diverse contexts in the past five years, including coauthoring five evaluation reports. For example, in 2018 for MCC in Mongolia, Ms. Marple-Cantrell was the lead author for the Registry Systems Process Study Performance Evaluation (2017- 2020) findings report. This rigorous pre-post performance evaluation utilized surveys, structured interviews, semi- structured interviews, FGDs, and transcription. She also served as lead author on a rigorous performance evaluation/small-N impact evaluation of the Community Land Protection Program in Liberia with a quasi- experimental design drawing from surveys, structured interviews, FGDs, and transcription. Finally, for an evaluation of USAID/Malawi’s Local Government Accountability and Performance Activity, she employed surveys and semi-structured interviews. Active evaluation team participation with responsibility for collecting field data on at least three rigorous evaluations in the past five years. Designs and executes flexible, efficient, and varied data collection methods to deliver timely and effective analysis. This includes active team participation on eight rigorous evaluations in the last five years: in Liberia (2017 – surveys, structured interviews, FGDs, transcription), Mongolia (2018 – surveys, structured interviews, semi-structured interviews, FGDs, transcription), Mozambique - 2019-2022 (structured interviews, semi-structured interviews, transcription), Ukraine (2020 – surveys, semi-structured interviews, FGDs), Belarus (2020 – surveys, semi- structured interviews), Malawi (2020 – surveys, semi- structured interviews), Burkina Faso (2022 – surveys, structured interviews, semi- structured interviews, transcription), and the United States (2019-2022 – surveys, FGDs, transcription). Zahra Miller, Evaluation Subject Matter Specialist, will support the Team Leader in the design and facilitation of the initial meeting; contribute to the document and data review; develop data collection instruments; lead KIIs/FGDs, contribute to Field Work Exit Briefing; and contribute to the analysis and draft the Final Evaluation Report and Presentation under the guidance of the Team Leader. Additionally, she will be responsible for assisting in coordinating all evaluation deliverables, and providing overall administrative and logistical support to the team. Master’s level degree with at least 6 years of technical knowledge and experience in a relevant field. MBA, University of Massachusetts Amherst (pending); MPH, University of Liverpool; 10+ years of experience in program evaluation, research, and data management in relevant technical areas such as NGO capacity building, youth engagement, childhood education, child trafficking prevention, and community development. Prior rigorous evaluation experience or prior experience in research studies involving survey data collection, data analysis, etc. Ms. Miller served on the evaluation team of the British Council Jamaica Evaluation of Core Skills Training Programme (10/2019 - 09/2020). This evaluation methodology included teacher progress surveys, key informant interviews, focus group discussions, and classroom observation. She has conducted evaluations on child diversion and trafficking prevention, and core skills training for UNICEF, Winrock International, and the British Council, among others. USAID.GOV FINAL PERFORMANCE EVALUATION OF THE LOCAL PARTNER DEVELOPMENT ACTIVITY | 93 NON-KEY TECHNICAL STAFF SUMMARY PERSON/ROLE QUALIFICATIONS Simon Conté, Global Development Research Specialist Role: Support the Team Leader, Assistant Team Leader/ Evaluation Specialist, and Evaluation Subject Matter Specialist with the document review, Conducting and coding KIIs, and contributing to the report. Mr. Simon Conté has over 20 years of experience conceptualizing, developing, and implementing innovative research projects, assessment tools, and knowledge products on rule of law, democracy and governance, and anticorruption. As an expert in the justice sector, including the Access to Justice Assessment Tool, the Detention Procedures Assessment Tool, the Prosecutorial Reform Index, and the ICCPR Index, among others. Mr. Conté has led or contributed to the development of numerous methodologies that evaluate and explore issues related to the fair and equitable treatment of youth in the criminal justice system. With deep experience in research and evaluation and international technical assistance, Mr. Conté served as the lead assessor for seven country assessments and authored over 60 legislative analyses, country assessments, and research papers on rule of law reform, democracy and governance, and human rights Makayla Barker, Junior Analyst, Global Development Role: Provide research and analysis support, contributing to the desk review, coding qualitative interview notes, cleaning quantitative data, and analyzing secondary data sources. Assist with administrative tasks. Ms. Makayla Barker is a quantitative and qualitative researcher with experience providing data management support across the data lifecycle for USAID. Her data management competencies include survey design, data collection, and data cleaning and analysis using statistical programs (R, STATA, SPSS). Ms. Barker has considerable experience providing support to evaluation research through desk reviews, survey programming and testing, qualitative notetaking and analysis, and supporting briefing meetings. 94 | FINAL PERFORMANCE EVALUATION OF THE LOCAL PARTNER DEVELOPMENT ACTIVITY (LPD) USAID.GOV ANNEX E: SELECT YOUTH SURVEY FINDINGS 1. Selected results charts 60 40% 57 38% 32 21% A1. Parish Kingston/ St. Andrew St. James St. Catherine 1 .6% 12 14.7% 126 84.5% A4. Respondent Type Family member of beneficiary youth Youth beneficiary (under 18) Youth beneficiary (18 and older) USAID.GOV FINAL PERFORMANCE EVALUATION OF THE LOCAL PARTNER DEVELOPMENT ACTIVITY | 95 16 11% 131 89% C.7 Did you expierence any challenges in accessing any of these services? Yes No 147 93% 11 7% C.9 Did you expierence any benefits from your participation in this program? Yes No 96 | FINAL PERFORMANCE EVALUATION OF THE LOCAL PARTNER DEVELOPMENT ACTIVITY (LPD) USAID.GOV 25 4 36 20 73 Obtained a full time job Obtained a higher paying job Was able to attend or stay in school Reduced substance use Other C.10 What are the benefits that you recieved from the program? 54 37% 50 34% 33 22% 7 4% 3 2% C.11 Please rate your level of satisfaction with the services you received Totally Satisfied Highly Satisfied Satisfied Partially Satisfied Dissatisfied USAID.GOV FINAL PERFORMANCE EVALUATION OF THE LOCAL PARTNER DEVELOPMENT ACTIVITY | 97 45 44% 28 26% 19 18% 9 9% 3 3% C.13 How would you rate your satisfaction with your case manager/mentor? Totally Satisfied Highly Satisfied Satisfied Partially Satisfied Dissatisfied 19 13% 129 87% G.1 Have you been a victim of any type of crime in the last 12 months? Yes No 98 | FINAL PERFORMANCE EVALUATION OF THE LOCAL PARTNER DEVELOPMENT ACTIVITY (LPD) USAID.GOV 20 14% 128 86% I.2 Are you currently enroled in school? Yes No 1 2 7 3 7 Rarely Absent more than present Present more than absent Almost all the time All the time I.3 About how often did you attend school in the last month? USAID.GOV FINAL PERFORMANCE EVALUATION OF THE LOCAL PARTNER DEVELOPMENT ACTIVITY | 99 58 39% 89 61% I.13 Are you enrolled in vocational training? Yes No 1 4 20 10 23 Never Absent more than present Present more than absent Almost all the time All the time I.14 About how often did you attend training in the last month? 100 | FINAL PERFORMANCE EVALUATION OF THE LOCAL PARTNER DEVELOPMENT ACTIVITY (LPD) USAID.GOV 69 46% 30 20% 27 18% 22 15% J.1 Are you currently employed? No Yes, Part time Yes, Full time Yes, Self-Employed 2 3 8 4 12 Rarely Absent more than present Present more than absent Almost all the time All the time J.2 If yes, about how often did you attend work in the last month? USAID.GOV FINAL PERFORMANCE EVALUATION OF THE LOCAL PARTNER DEVELOPMENT ACTIVITY | 101 28 20% 31 22% 83 58% J.4 Do you think that your current economic situation is better, the same, or worse than it was 12 months ago? Worse Same Better 102 | FINAL PERFORMANCE EVALUATION OF THE LOCAL PARTNER DEVELOPMENT ACTIVITY (LPD) USAID.GOV 2. Full results tables Table 1: Youth Survey Descriptive Statistics, Part 1 (Binary and Continuous Variables) Mean Male Female Over18 Under18 Min Max N B.4 What is your age? (In completed years) 22.32 22 23.87 23.07 16.19 14 34 148 B.5 Do you still live with your parent(s)/guardian(s)? .66 .68 .62 .62 1 0 1 148 B.6 How many people in total live in your household at this time? 4.63 4.25 5.05 4.66 4.38 1 15 148 B.7 How many children under the age of 13 live in your household? 1.23 .93 1.74 1.23 1.25 0 8 148 B.12 Do you have children? .33 .19 .64 .36 .13 0 1 148 B.13 Do you provide financial support to your child(ren)? .98 1 .96 .98 1 0 1 49 D.1 Have you ever been arrested? .34 .41 .23 .36 .19 0 1 148 D.9 Have you spent any time in jail or prison? .57 .59 .56 .56 .67 0 1 51 D.11 Are you currently involved with the justice system? .1 .12 0 .08 .33 0 1 51 E.3 Have you ever been diagnosed with an educational disability (slow learner)? .18 .17 .13 .16 .31 0 1 148 F.1 - F.6: 0=Never, 1=Sometimes, 2=Regularly F.1 Do you keep things in until you finally explode with anger? .92 .84 1.13 .92 .94 0 2 148 F.2 Do you have a tendency to take your anger out on someone other than the pers .43 .32 .72 .4 .69 0 2 148 F.3 When a problem arises between you and someone else, do you discuss it without .99 1.05 .79 .99 1 0 2 147 F.4 Are you satisfied with the way you settle differences with others? 1.35 1.43 1.16 1.38 1.06 0 2 147 F.5 Do you tend to feel very guilty or bad after getting angry? 1.01 .94 1.15 1.02 1 0 2 147 F.6 When you get angry, do people around you feel threatened or frightened? .64 .62 .58 .63 .73 0 2 141 F.7 When you get angry, have you ever damaged property? .22 .23 .23 .21 .25 0 1 148 F.9 - F.27 1 = Disagree Strongly, 5 = Agree Strongly F.9 I have a reserved and cautious attitude toward life 3.18 3.29 2.92 3.22 2.87 1 4 145 F.10 I have trouble controlling my impulses. 2.09 2 2.26 2.06 2.31 1 4 145 F.11 I generally seek new and exciting experiences and sensations. 3.02 2.94 3.1 3 3.2 1 4 145 F.12 When I’m happy I can’t stop myself from doing things with bad consequences 1.99 1.95 2.03 1.94 2.38 1 4 143 F.13 I have trouble resisting my cravings (for food, cigarettes, etc.). 2.29 2.3 2.28 2.31 2.06 1 4 147 F.14 I’ll try anything once. 2.39 2.32 2.51 2.37 2.63 1 4 147 F.22 Others are shocked by things I do when I’m excited 2.21 2.24 2.05 2.2 2.25 1 4 145 F.23 When I feel rejected, I will often say things that I later regret. 2.24 1.96 2.79 2.13 3.13 1 4 147 F.24 I welcome new experiences even if they are frightening 2.82 2.85 2.77 2.84 2.63 1 4 144 F.25 I always keep my feelings under control. 3.31 3.54 2.9 3.37 2.8 1 4 145 F.26 In the heat of an argument, I will often say things that I later regret. 2.54 2.24 3.1 2.47 3.19 1 4 147 F27. When excited, I don’t think about the consequences of my actions 2.35 2.29 2.5 2.38 2.06 1 4 147 G.1 Have you been a victim of any type of crime in the past 12 months? .13 .12 .18 .14 .06 0 1 148 G.3 Have you used the weapon on someone or to threaten someone? 0 0 0 0 0 0 0 4 G.4 Have you ever been associated with a gang? .16 .17 .1 .16 .19 0 1 147 G.6 Do your parents or guardians have an adult criminal history? .17 .18 .11 .17 .14 0 1 143 G.9: 0 = Not True, 1 = Somewhat True, 2 = Certianly True G.9 I try to be nice to other people. I care about their feelings. 1.68 1.66 1.74 1.68 1.69 0 2 148 G.14: 1 = Don’t at all agree, 7 = Agree a lot G.14 There are lots of adults in my neighborhood I could talk to about something 3.48 3.75 3.05 3.41 4.06 1 7 148 H.5 - H.6: 0 = Never, 1 = A Little, 2 = A Lot H.5 Do you sometimes imagine hitting or hurting the other person? 1.03 1.02 1.03 1.04 1 0 2 147 H.6 Do you ever daydream about people getting killed? .63 .6 .64 .63 .63 0 2 147 H.7 - H.12: 1 = Don’t at all agree, 7 = Agree a lot H.7 To what extent do you trust the justice system? 2.97 2.84 3 2.79 4.44 1 7 148 H.8 To what extent do you trust the local government? 2.71 2.54 2.95 2.52 4.31 1 7 148 H.9 To what extent do you trust the police in your neighborhood? 2.4 2.15 2.64 2.21 3.94 1 7 148 H.12 Employers discriminate against people with past criminal behavior? 4.63 4.71 4.85 4.74 3.69 1 7 147 I.2 Are you currently enrolled in school? .14 .1 .08 .11 .38 0 1 148 I.13 Are you currently enrolled in vocational training .39 .44 .26 .4 .31 0 1 147 J.3: 0 = 0 - 30,000, 1 = 30,001 - 40,000 2 = 40,001 - 60,000, 3 = 60,001 - 80,000, 4 = More than 80,000 J.3 What is the total monthly income in your household? 1.06 1.22 .56 1.11 .29 0 4 109 J.5 In the past three months, did your household ever run out of food? .61 .58 .67 .62 .5 0 1 147 K.4 Are you currently experiencing problems due to alcohol or drug use? .01 .02 0 .01 .06 0 1 147 K.5 Have you experienced violence related to alcohol or drug use? .03 .05 0 .04 0 0 1 148 USAID.GOV FINAL PERFORMANCE EVALUATION OF THE LOCAL PARTNER DEVELOPMENT ACTIVITY | 103 Table 2: Youth Survey Descriptive Statistics, Part 2 (Binary Variables - Services) Mean Male Female Over18 Under18 Min Max N C.1 What types of supportive services have you received in the last four years? C.1: None .01 0 0 0 .06 0 1 148 C.1: Food and sanitation supply vouchers .18 .16 .23 .19 .06 0 1 148 C.1: Book vouchers, backpacks, and/or school supplies .14 .1 .18 .14 .19 0 1 148 C.1: Therapy/counseling (including Cognitive Behavioral Therapy) .4 .38 .51 .39 .5 0 1 148 C.1: Substance mis-use counseling .16 .19 .08 .16 .13 0 1 148 C.1: Mentorship .45 .56 .31 .46 .38 0 1 148 C.1: Life skills training .66 .72 .62 .67 .56 0 1 148 C.1: Psychosocial support for GBV survivors .05 .06 0 .04 .13 0 1 148 C.1: School reintegration support .2 .19 .23 .18 .38 0 1 148 C.1: Job skills training / vocational training .61 .68 .56 .62 .56 0 1 148 C.1: Apprenticeship placement .25 .25 .28 .25 .25 0 1 148 C.1: Microenterprise support .11 .11 .1 .12 0 0 1 148 C.1: Youth risk screening .13 .12 .18 .13 .13 0 1 148 C.1: Literacy/numeracy training .16 .18 .13 .15 .25 0 1 148 C.1: Case management .51 .51 .62 .5 .56 0 1 148 C.2 What types of supportive services are you currently receiving? C.2: None .18 .18 .21 .19 .07 0 1 147 C.2: Food and sanitation supply vouchers .05 .06 .03 .05 .07 0 1 147 C.2: Book vouchers, backpacks, and/or school supplies .03 .01 0 .02 .2 0 1 147 C.2: Therapy/counseling (including Cognitive Behavioral Therapy) .2 .16 .31 .17 .47 0 1 147 C.2: Substance mis-use counseling .05 .06 .03 .05 .07 0 1 147 C.2: Mentorship .26 .32 .15 .26 .27 0 1 147 C.2: Life skills training .3 .31 .26 .29 .4 0 1 147 C.2: Psychosocial support for GBV survivors .01 .01 0 .01 0 0 1 147 C.2: School reintegration support .07 .06 .08 .07 .07 0 1 147 C.2: Job skills training / vocational training .38 .41 .33 .37 .47 0 1 147 C.2: Apprenticeship placement .17 .18 .18 .16 .27 0 1 147 C.2: Microenterprise support .03 .03 .05 .04 0 0 1 147 C.2: Youth risk screening .04 .05 .03 .04 .07 0 1 147 C.2: Literacy/numeracy training .05 .07 0 .05 .13 0 1 147 C.2: Case management .38 .35 .51 .36 .6 0 1 147 What supportive services have you found most helpful? C.4: None .01 .01 .03 .02 0 0 1 147 C.4: Food and sanitation supply vouchers .1 .08 .15 .11 .07 0 1 147 C.4: Book vouchers, backpacks, and/or school supplies .07 .03 .08 .05 .2 0 1 147 C.4: Therapy/counseling (including Cognitive Behavioral Therapy) .29 .23 .46 .27 .47 0 1 147 C.4: Substance mis-use counseling .1 .12 .05 .1 .07 0 1 147 C.4: Mentorship .35 .42 .23 .36 .27 0 1 147 C.4: Life skills training .49 .54 .44 .51 .33 0 1 147 C.4: Psychosocial support for GBV survivors .01 .02 0 .02 0 0 1 147 C.4: School reintegration support .12 .12 .13 .12 .13 0 1 147 C.4: Job skills training / vocational training .48 .54 .38 .48 .47 0 1 147 C.4: Apprenticeship placement .14 .15 .13 .13 .2 0 1 147 C.4: Microenterprise support .05 .05 .08 .06 0 0 1 147 C.4: Youth risk screening .07 .07 .08 .06 .13 0 1 147 C.4: Literacy/numeracy training .05 .05 .08 .05 .13 0 1 147 C.4: Case management .33 .31 .44 .33 .33 0 1 147 C.5 Are there any services that you did not find helpful? C.5: None .95 .96 .9 .95 .93 0 1 147 C.5: Food and sanitation supply vouchers 0 0 0 0 0 0 0 147 C.5: Book vouchers, backpacks, and/or school supplies 0 0 0 0 0 0 0 147 C.5: Therapy/counseling (including Cognitive Behavioral Therapy) .01 .01 0 0 .07 0 1 147 C.5: Substance mis-use counseling 0 0 0 0 0 0 0 147 C.5: Mentorship .01 .01 0 .01 0 0 1 147 C.5: Life skills training .02 .02 .03 .02 0 0 1 147 C.5: Psychosocial support for GBV survivors 0 0 0 0 0 0 0 147 C.5: School reintegration support .01 0 .03 .01 0 0 1 147 C.5: Job skills training / vocational training .02 .02 .03 .02 0 0 1 147 C.5: Apprenticeship placement 0 0 0 0 0 0 0 147 C.5: Microenterprise support 0 0 0 0 0 0 0 147 C.5: Youth risk screening .01 .01 0 .01 0 0 1 147 C.5: Literacy/numeracy training .01 .01 0 .01 0 0 1 147 C.5: Case management 0 0 0 0 0 0 0 147 C.7 Did you experience any challenges in accessing any of these services? .11 .14 .05 .12 0 0 1 147 C.9 Did you experience any benefits from your participation in this program? .93 .91 .95 .92 1 0 1 147 C.10 What are the benefits? 0 0 0 0 0 0 0 0 C.11 Please rate your level of satisfaction with the services you received 2.01 2.16 1.74 2.07 1.53 1 5 147 C.13 How would you rate your satisfaction with your case manager/mentor? 2.01 2.15 1.66 2.09 1.42 1 5 104 104 | FINAL PERFORMANCE EVALUATION OF THE LOCAL PARTNER DEVELOPMENT ACTIVITY (LPD) USAID.GOV Table 3: Youth Survey Descriptive Statistics, Part 3 (Categorical Variables) Total Male Female Over18 Under18 A1: Parish A1: Kingston / St. Andrew 60 35 14 53 7 A1: St. James 57 37 20 49 8 A1: St. Catherine 32 27 5 31 1 A4: Respondent Type A4: Family member of beneficiary youth 1 0 0 1 0 A4: Youth beneficiary (under 18) 22 7 5 6 16 A4: Youth beneficiary (18 or older) 126 92 34 126 0 Race B10: Black/ Afro Jamaican 138 92 37 122 16 B10: Mixed 10 7 2 10 0 B11: Single 122 87 33 110 12 B11: Married 2 0 1 2 0 B11: Common law marriage (Living together) 19 12 4 19 0 B11: Prefer not to respond 1 0 1 1 0 I.6 Organize a group of students in order to achieve changes at school I6: No 7 5 0 6 1 I6: Yes 10 5 2 7 3 I6: Maybe 3 0 1 1 2 I.7 Take school problems or concerns to my teacher or principal. I7: No 8 3 1 5 3 I7: Yes 10 5 2 8 2 I7: Maybe 2 2 0 1 1 I.8 Teach things to other students in my school I8: No 5 2 1 2 3 I8: Yes 15 8 2 12 3 I.9 Help keep the classroom and school clean. I9: No 2 2 0 1 1 I9: Yes 17 7 3 12 5 I9: Maybe 1 1 0 1 0 I.10 Serve on the student government at my school. I10: No 12 8 1 10 2 I10: Yes 7 2 2 4 3 I10: Maybe 1 0 0 0 1 I.11 Encourage my parent(s) to come to PTA meetings. I11: No 7 4 1 7 0 I11: Yes 12 5 2 6 6 I11: Don’t know 1 1 0 1 0 I.12 Was your school education affected due to the COVID-19 pandemic? I12: No, because I did not attend school for another reason 61 43 16 56 5 I12: No, my classes continued normally 11 10 1 11 0 I12: Yes, I went to virtual or remote/online classes 28 15 7 20 8 I12: Yes, I switched to a combination of virtual/online and in-person classes (m 13 9 3 10 3 I12: Yes, I cut all ties with the school 2 2 0 2 0 I12: Don’t know 21 12 8 21 0 I12: Prefer not to respond 12 8 4 12 0 E.1 Have you ever been diagnosed with a mental illness? E1: No previous or current diagnoses 142 95 38 127 15 E1: Previous diagnoses 4 3 0 3 1 E1: Current diagnoses 2 1 1 2 0 G.2 How many days in the last 30 days did you carry a weapon? G2: 0 days 118 80 29 104 14 G2: 1 days 4 4 0 4 0 G2: 2 days 1 1 0 1 0 G2: 4 days 3 2 1 3 0 G2: 7 days 1 0 1 1 0 G2: 8 days 1 0 1 1 0 G2: 14 days 1 1 0 1 0 G2: 20 days 2 2 0 2 0 G2: 21 days 1 1 0 1 0 G2: 25 days 1 1 0 1 0 G2: 30 days 15 7 7 13 2 G.5 What proportion of your close friends have a history with the criminal justice system? G5: All 72 42 25 63 9 G5: Most 33 24 5 30 3 G5: Some 25 20 4 23 2 G5: Few 8 6 2 8 0 G5: None 2 2 0 2 0 G5: Don’t know 8 5 3 6 2 G.7 How would you describe the discipline, supervision, and support that you receive? G7: Not applicable (no discipline/supervision/support received from parent(s)/guardian 10 7 2 10 0 G7: Adequate 26 19 6 24 2 G7: Sometimes inconsistent 42 25 14 36 6 G7: Overly strict 51 32 15 46 5 G7: Overly permissive 18 15 2 15 3 G7: Don’t know 1 1 0 1 0 USAID.GOV FINAL PERFORMANCE EVALUATION OF THE LOCAL PARTNER DEVELOPMENT ACTIVITY | 105 Total Male Female Over18 Under18 K.5 Have you experienced violence related to alcohol or drug use? K5: No violence 75 53 19 69 6 K5: Violence/discord is occasional 52 34 13 44 8 K5: Violence/discord is severe and/or regular 21 12 7 19 2 I.3 About how often did you attend school in the last month? I3: Rarely 1 0 1 1 0 I3: Absent more often than present 2 2 0 2 0 I3: Present more often than absent 7 3 2 5 2 I3: Almost all the time 3 3 0 3 0 I3: All the time 7 2 0 3 4 I.4 Thinking back over the past year in school, how often did you try to do your schoolwork? I4: Some of the time 1 1 0 1 0 I4: Some of the time 11 5 3 8 3 I4: All of the time 8 4 0 5 3 I.5 In the past year, how many of your best friends have tried to do well in school I5: None 18 11 5 18 0 I5: Few 25 18 5 20 5 I5: Some 28 21 6 24 4 I5: Most 27 19 6 25 2 I5: All 30 19 9 28 2 I5: Don’t know 18 10 8 15 3 I5: Prefer not to respond 2 1 0 2 0 I.14 About how often did you attend training in the last month? I14: Never 1 1 0 1 0 I14: Absent more often than present 4 3 1 4 0 I14: Present more often than absent 20 14 5 19 1 I14: Almost all the time 10 8 2 9 1 I14: All the time 23 18 2 20 3 J.1 Are you currently employed? J1: No 69 47 15 58 11 J1: Yes, part time 30 21 7 28 2 J1: Yes, full time 27 17 9 25 2 J1: Yes, self-employed 22 14 8 21 1 J.2 If yes, about how often did you attend work in the last month? J2: Rarely 2 2 0 2 0 J2: Absent more often than present 3 3 0 3 0 J2: Present more often than absent 8 5 3 7 1 J2: Almost all the time 4 2 1 4 0 J2: All the time 12 8 3 11 1 J2: Don’t know 1 1 0 1 0 J.4 Do you think that your current economic situation is better, the same or worse? J4: Worse 28 19 7 27 1 J4: Same 31 22 7 27 4 J4: Better 83 54 25 74 9 J4: Don’t know 6 4 0 4 2 K.1 How often in the past year have you used an illegal drug or used a prescription for non-medical reasons? K1: Never 135 89 36 119 16 K1: Less than monthly 3 1 2 3 0 K1: Monthly 4 4 0 4 0 K1: Weekly 2 1 1 2 0 K1: Daily or almost daily 4 4 0 4 0 K.2 How often in the past year have you used marijuana? K2: Never 69 36 25 58 11 K2: Less than monthly 12 9 3 11 1 K2: Monthly 15 9 5 13 2 K2: Weekly 5 3 2 4 1 K2: Daily or almost daily 46 41 4 45 1 K2: Don’t know 1 1 0 1 0 K.3 In the past year, how often have you had six or more alcoholic drinks/beverages? K3: Never 58 36 16 49 9 K3: Less than monthly 31 20 7 27 4 K3: Monthly 20 12 8 20 0 K3: Weekly 25 21 4 23 2 K3: Daily or almost daily 13 9 4 12 1 K3: Don’t know 1 1 0 1 0 H.10 A man has a right to discipline his partner with physical violence. H10: Strongly disagree 96 64 25 87 9 H10: Disagree 35 22 10 29 6 H10: Neither agree nor disagree 7 5 2 7 0 H10: Agree 4 4 0 4 0 H10: Strongly agree 5 3 2 4 1 H10: Prefer not to respond 1 1 0 1 0 H.11 Physical violence between members of a couple is a private matter H11: Strongly disagree 28 14 11 27 1 H11: Disagree 24 12 10 21 3 H11: Neither agree nor disagree 14 9 5 14 0 H11: Agree 47 34 9 38 9 H11: Strongly agree 34 29 4 31 3 H11: Don’t know 1 1 0 1 0 106 | FINAL PERFORMANCE EVALUATION OF THE LOCAL PARTNER DEVELOPMENT ACTIVITY (LPD) USAID.GOV ANNEX F: SIGNED DISCLOSURE OF CONFLICT OF INTEREST FORMS DISCLOSURE OF ANY CONFLICTS OF INTEREST [The Evaluation Policy requires that evaluation reports include a signed statement by each evaluation team member regarding any conflicts of interest. A suggested format is provided below.] Name Mario Martinez Title Mr. Organization LINC LLC Evaluation Position? Evaluation Team Leader Evaluation Award Number (contract or other instrument) 7200AA20D00017/72053222F00002 USAID Project(s) Evaluated (Include project name(s), implementer name(s) and award number(s), if applicable) Local Partner Development Activity (LPD), FHI360, Cooperative Agreement # AID-532- LA-17-00001 I have real or potential conflicts of interest to disclose. No If yes answered above, I disclose the following facts: Real or potential conflicts of interest may include, but are not limited to: 1. Close family member who is an employee of the USAID operating unit managing the project(s) being evaluated or the implementing organization(s) whose project(s) are being evaluated. 2. Financial interest that is direct, or is significant though indirect, in the implementing organization(s) whose projects are being evaluated or in the outcome of the evaluation. 3. Current or previous direct or significant though indirect experience with the project(s) being evaluated, including involvement in the project design or previous iterations of the project. 4. Current or previous work experience or seeking employment with the USAID operating unit managing the evaluation or the implementing organization(s) whose project(s) are being evaluated. 5. Current or previous work experience with an organization that may be seen as an industry competitor with the implementing organization(s) whose project(s) are being evaluated. 6. Preconceived ideas toward individuals, groups, organizations, or objectives of the particular projects and organizations being evaluated that could bias the evaluation. I certify (1) that I have completed this disclosure form fully and to the best of my ability and (2) that I will update this disclosure form promptly if relevant circumstances change. If I gain access to proprietary information of other companies, then I agree to protect their information from unauthorized use or disclosure for as long as it remains proprietary and refrain from using the information for any purpose other than that for which it was furnished. Signature Date February 14, 2023 USAID.GOV FINAL PERFORMANCE EVALUATION OF THE LOCAL PARTNER DEVELOPMENT ACTIVITY | 107 108 | FINAL PERFORMANCE EVALUATION OF THE LOCAL PARTNER DEVELOPMENT ACTIVITY (LPD) USAID.GOV USAID.GOV FINAL PERFORMANCE EVALUATION OF THE LOCAL PARTNER DEVELOPMENT ACTIVITY | 109