August 2016 This publication was produced at the request of the United States Agency for International Development. It was prepared independently by Social Impact. Assessment of the Caribbean Basin Security Initiative Final Report ASSESSMENT OF THE CARIBBEAN BASIN SECURITY INITIATIVE August 22, 2016 TO#AID-OOA-TO-15-00050; IDIQ#AID-517-I-12-00001 DISCLAIMER The authors’ views expressed in this publication do not necessarily reflect the views of the United States Agency for International Development or the United States Government. iii CONTENTS Contents ....................................................................................................................................................................... iii Acronyms ...................................................................................................................................................................... v Executive Summary .................................................................................................................................................... vii CBSI Background .................................................................................................................................................. vii Assessment Purpose and Questions ...................................................................................................................... viii Assessment Design, Methods and Limitations .........................................................................................................ix Key Conclusions and Recommendations .................................................................................................................ix CBSI Background ........................................................................................................................................................17 CBSI Overview .......................................................................................................................................................17 USAID CBSI Objectives ........................................................................................................................................... 2 CBSI Activities ......................................................................................................................................................... 5 Assessment Purpose and Questions ............................................................................................................................... 8 Assessment Purpose .................................................................................................................................................. 8 Assessment Questions ............................................................................................................................................... 8 Assessment Design, Methods & Limitations ................................................................................................................. 9 Assessment Design .................................................................................................................................................... 9 Assessment Methodology ......................................................................................................................................... 9 Data collection locations .....................................................................................................................................10 Data collection and analysis methods .................................................................................................................10 Utilization Focused Data Analysis and Reporting ..............................................................................................12 Team Composition and Roles ................................................................................................................................. 13 Limitations ..............................................................................................................................................................13 Scope of Enquiry ................................................................................................................................................13 Data Availability .................................................................................................................................................13 Response Bias .....................................................................................................................................................14 Selection Bias .....................................................................................................................................................14 Definition of Best Practices ................................................................................................................................14 Findings, Conclusions & Recommendations ...............................................................................................................15 Assessment Question 1: Best Practices ...................................................................................................................15 Introduction .........................................................................................................................................................15 Assessment Question 1 Findings ........................................................................................................................16 Assessment Question 1 Conclusions ...................................................................................................................27 Assessment Question 1 Recommendations .........................................................................................................32 Assessment Question 2: Challenges and Opportunities ..........................................................................................34 iv Introduction .........................................................................................................................................................34 Assessment Question 2 Findings ........................................................................................................................34 Assessment Question 2 Conclusions ...................................................................................................................43 Assessment Question 2 Recommendations .........................................................................................................46 Assessment Question 3: Complementarity and Coordination .................................................................................47 Introduction .........................................................................................................................................................47 Assessment Question 3 Findings ........................................................................................................................48 Assessment Question 3 Conclusions ...................................................................................................................53 Assessment Question 3 Recommendations .........................................................................................................56 Annexes .......................................................................................................................................................................58 Annex I: Statement of Work ...................................................................................................................................59 Annex II: Definition of CBSI Assessment Questions .............................................................................................63 Annex III: CBSI Activity Descriptions ...................................................................................................................65 Annex IV: Data Collection Instruments ..................................................................................................................72 Annex V: Documents Reviewed .............................................................................................................................82 Annex VI: Matrix for Analysis of Best Practices ....................................................................................................91 Annex VII: Disclosure of any Conflicts of Interest ................................................................................................. 93 v ACRONYMS ADR alternative dispute resolution ANP A New Path CARICOM Caribbean Community CBO community-based organization CBP community-based policing CBSI Caribbean Basin Security Initiative CCJ Combating Corruption in Jamaica CDA Child Development Agency CJH Community Justice Houses CJSSP Criminal Justice System Strengthened Project COMET Community Empowerment and Transformation Project COMET II Community Empowerment and Transformation Project - Phase II CSO Civil Society Organization CRC Community Resource Center CXC Caribbean Examinations Council CYEP Caribbean Youth Empowerment Program DCS Department of Correctional Services DNP Dominican National Police DoS Department of State DR Dominican Republic EQUI Evaluation Quality Use and Impact ESC Eastern and Southern Caribbean ESCYA Eastern and Southern Caribbean Youth Assessment FGD focus group discussion HIV/AIDS human immunodeficiency virus/acquired immune deficiency syndrome ICITAP International Criminal Investigative Training Assistance Program INL Bureau for International Narcotics and Law Enforcement Affairs IP Implementing Partner JA Junior Achievement JCF Jamaica Constabulary Force vi JJRP Juvenile Justice Reform Project KII key informant interview LGBTI Lesbian, Gay, Bisexual, Transgender and Intersex MoE Ministry of Education NGO non-governmental organization NIA National Integrity Action ODI Overseas Development Institute OECS Organization of Eastern Caribbean States PADF Pan American Development Foundation PC Participación Ciudadana POST Police Officer Standardized Trainings RAPP Resistance and Prevention Program SCI Second Chance Institution SKYE Skills and Knowledge for Youth Employment SI Social Impact TAJ Tax Administration Jamaica TWG Technical Working Group UNDP United Nations Development Programme U.S. United States UNICEF United Nations International Children's Emergency Fund USAID United States Agency for International Development USAID/LAC United States Agency for International Development Latin America and Caribbean USG United States Government YLS/CMI Youth Level of Service/Case Management Inventory vii EXECUTIVE SUMMARY CBSI Background The Caribbean Basin Security Initiative (CBSI) is a regional citizen security partnership that brings together member states of the Caribbean Community (CARICOM)1 and the Dominican Republic to collaborate with the United States (U.S.). At the core of CBSI is the recognition that security is critical to economic and social growth. The following three strategic priorities were identified to address the security threats facing the Caribbean region: 1. Substantially Reduce Illicit Trafficking: through programs ranging from counter-narcotics to reducing the flow of illegal arms/light weapons. 2. Increase Public Safety and Security: through programs ranging from reducing crime and violence to improving border security. 3. Promote Social Justice: through programs designed to promote justice sector reform, combat government corruption, and assist vulnerable populations at risk of recruitment into criminal organizations. The United States Agency for International Development (USAID) is the lead U.S. Government agency of the social justice pillar and has committed over $127 million under CBSI for these efforts in the targeted countries. Each of the USAID Missions in the region other than Haiti (Dominican Republic, or the DR; Eastern and Southern Caribbean, or the ESC; and Jamaica) has supported a unique mix of activities to meet the needs of their respective country or countries and support the overarching priorities of CBSI.2 In the DR, programming has focused on the justice system and supporting at-risk youth to reduce delinquent behavior and encourage them to become more productive members of society. Starting with the 2015–2019 strategy, the ESC Mission is investing CBSI funds in community￾based crime and violence prevention, juvenile justice systems, and an improved evidence base on crime and violence. USAID has a very diverse CBSI portfolio in Jamaica including at-risk youth, basic education, workforce development, community safety and security, and anti-corruption. USAID/Dominican Republic CBSI programming in the Dominican Republic has focused on the justice system and on supporting at￾risk youth to reduce delinquent behavior and encourage them to become more productive members of society. To accomplish its current strategic goal of “Improved Citizen Security to Promote Economic Growth,” this Mission is supporting three justice-related activities that together aim to enhance grassroots access to justice, improve the functioning of the criminal prosecution system, and support police force reform. In relation to at-risk youth, the Mission is implementing an ambitious five-year activity that combines education, job placement, and social and economic services. Most Mission programming (CBSI and otherwise) targets the population of the so-called Duarte Corridor, where the 1 Antigua and Barbuda, The Bahamas, Barbados, Dominica, Grenada, Guyana, Jamaica, St. Kitts and Nevis, St. Lucia, St. Vincent and the Grenadines, Suriname, and Trinidad and Tobago. Note: The Bahamas is not targeted by USAID assistance under CBSI. 2 USAID/Haiti is not a part of CBSI. At the point of CBSI’s creation, Haiti had recently experienced a major earthquake and the Mission was dealing with a significant number of larger development challenges. viii majority of Dominicans live and the major cities are located. Funding for CBSI activities from 2010 through 2015 has totaled approximately $31 million. USAID/ESC USAID/ESC is a regional Mission that currently serves 10 countries: Antigua and Barbuda, Barbados, Dominica, Grenada, Guyana, St. Lucia, St. Kitts and Nevis, St. Vincent and the Grenadines, Suriname, and Trinidad and Tobago. (At the beginning of CBSI, the Mission was referred to as USAID/Barbados and the Eastern Caribbean and did not yet include Guyana, which had its own Mission, Suriname, or Trinidad and Tobago.) USAID/ESC’s 20110–14 strategy focused on youth and workforce development and strengthening of the juvenile justice sector. The Mission moved to adapt programming that had addressed broader issues of youth workforce development to focus more on at-risk youth. CBSI commissioned a Juvenile Justice Assessment in 2011 targeting 9 countries in the region—Antigua and Barbuda, Dominica, Grenada, Guyana, St. Lucia, St. Kitts and Nevis, St. Vincent and the Grenadines, Trinidad and Tobago, and Jamaica—to inform the development of this new program area. In 2012, Suriname, Trinidad and Tobago, and Guyana were added to the Mission’s coverage area and programming was expanded, starting in Guyana. When the Mission designed its new strategy in 2014, its name was changed to reflect this larger coverage area. At the same time, the Mission drew on lessons learned to develop a strategy that includes a public health approach to youth crime and violence reduction, drawing on evolving best practices in this newly emerging field for USAID. This Mission is now focused on investing in activities that support community-based crime and violence prevention, improve juvenile justice systems, and establish an improved evidence base on crime and violence. Funding from 2010 through 2015 has totaled approximately $54 million. USAID/Jamaica USAID/Jamaica began its citizen security programming nearly a decade before the inception of CBSI in about 2000, with activities designed to reform the Jamaican Constabulary Force and improve community/police trust. They later expanded these activities to include a more comprehensive approach to community crime prevention and institutional capacity building of the law enforcement and justice sectors. Under the overarching strategic goal of “Resilience and Social Cohesion of Targeted Jamaican Communities Improved,” USAID/Jamaica has implemented a very diverse CBSI portfolio across the country that focuses on five areas: at-risk youth, basic education, workforce development, community safety and security, and anti-corruption. The first three of these thematic areas directly aim to reduce the vulnerability of Jamaican youth and their involvement as perpetrators or victims of crime and violence. Funding from 2010 through 2015 has totaled approximately $41 million. Assessment Purpose and Questions USAID contracted Social Impact (SI) to conduct this assessment of CBSI activities in order to guide future programming, identify new areas of research and/or implementation for USAID, and locate programmatic and/or geographic gaps in CBSI programs to be filled by USAID and other stakeholders. USAID outlined three specific assessment questions for the assessment team: 1. To what extent do USAID’s CBSI programs reflect the most current understanding of best practices in citizen security, particularly in the Caribbean context? 2. Have there been any unanticipated challenges or opportunities (internal or external) that have affected implementation of CBSI programs? 3. To what degree are USAID’s CBSI programs complementary with other U.S. Government (USG) CBSI programs, to include Department of State (DoS), Department of Defense, and other agencies? Are there instances where these programs are overlapping or working at cross-purposes? ix Assessment Design, Methods, and Limitations This assessment includes 12 USAID/CBSI countries, of which the team focused on five: Jamaica, St. Lucia, St. Vincent and the Grenadines, Suriname, and the DR. Data were also gathered in Barbados from a number of regional stakeholders and the USAID/ESC Mission. From the 23 CBSI activities identified, SI and USAID selected nine “priority activities” for closer examination, while the other 14 were analyzed predominantly through document review. SI has used a primarily qualitative approach, supplementing with quantitative methods where available, including: (1) systematic document review; (2) qualitative data collection through key informant interviews (KIIs) and focus group discussions (FGDs); and (3) analysis of quantitative monitoring data. Information was triangulated to support accuracy of findings. The extensive document review by the assessment team included USAID strategic planning documents; CBSI activity contracts and work plans, monitoring and evaluation plans, progress reports, and evaluation reports; data from government ministries; reports from non-governmental organizations; and research reports produced by USAID and other organizations, such as UNDP, the Overseas Development Institute, and InterPeace. The team conducted 145 KIIs (in person and remotely) with USAID, other U.S. Government agencies, activity implementers and partners, government counterparts, sectoral experts, and other stakeholders in the region. In addition, 15 FGDs were held with beneficiaries, community members, and local partners of the activities. Questions for informants were based on pre-tested interview guides and included components that probed the extent to which women and other marginalized groups have been taken into account in CBSI programming. Limitations on the assessment methodology included response bias, especially in relation to complementarity issues; selection bias, due to partial reliance on USAID and implementers to identify informants; and shortage of reliable evidence of best practices in the region. Furthermore, the compressed timeframe for this assessment meant that not all documents were received and reviewed prior to fieldwork. Key Conclusions and Recommendations ASSESSMENT QUESTION 1 - BEST PRACTICES The assessment defined best practices as “methods and activity models that have been demonstrated and clearly documented as effective in improving citizen security” and focused on practices in the Caribbean region documented since 2010 in the four objective areas of USAID CBSI programming.3 Each activity was examined for consistency with previous evaluations or assessments specifically related to the activity, with USAID or other USG guidelines in the subject area, and with research and analysis by other stakeholders. While assessed activities were not expected to take into account best practices that were developed following activity design or implementation, this assessment question details any incongruence to ensure future programming is based on the most current evidence regarding good practice. Detailed findings in relation to the nine priority activities for this assessment have led to the following key conclusions and recommendations, while additional information on non-priority activities and specific best practices adopted or considered can be found in the body of the report. USAID/Dominican Republic 3 The full definition of the assessment questions, as agreed by SI and USAID, is found in Annex II. x The Alerta Joven activity shows a strong foundation of evidence for its design, has a rigorous monitoring system, and is seeking to build on the data available on youth at risk. The Community Justice Houses (CJH), while not themselves firmly based on a preexisting model or practice, constitute a very promising practice that merits further study. The two rule-of-law activities4 that started more recently have also demonstrated considerable awareness of and willingness to learn from models already proven in other contexts. Experience from the English-speaking Caribbean seems less relevant in the DR (and vice versa), primarily due to language barriers but also owing to differing legal systems. USAID/ESC The concept of the six-country Juvenile Justice Reform Project (JJRP) was based on sound research, and implementation has been fortified by interaction with respected stakeholders such as the United Nations International Children's Emergency Fund (UNICEF).5 JJRP was largely based on findings of the 2011 Juvenile Justice Assessment, which was used to inform the design of the activity.6 While the need for the nine-country Strengthening Second Chance Education activity was identified through sound research, neither document review nor interviews identified either a comparable model in the region or any specific guiding principles for this specific type of intervention. The detailed design of the Kari Yu! Youth Development and Juvenile Justice Project in Suriname was based on USAID experiences under the SKYE activity in Guyana and incorporated learning that stemmed from early assessments funded under the Kari Yu! contract as well as promising practices from the A Ganar activity.7 With a shift towards a public health model (see Box 1) and more holistic view of citizen security in the 2015–2019 strategy, USAID/ESC incorporates both short- and longer-term programming and takes into account the three levels of crime prevention, which is consistent with current thinking in the sector.8 The Mission is continuing to strengthen the evidence base with a regional study on at-risk youth and the design of a companion impact evaluation to be implemented alongside the new juvenile justice and community focused programming. USAID/Jamaica Many elements of CBSI activities in Jamaica are consistent with recommendations from published guidelines, assessments, and evaluations. Interviews and document review did not reveal that best practices had been systematically considered in the design process; rather they indicated a reliance on learning from previous activities. Community Empowerment and Transformation Project - Phase II (COMET II) does reflect components of best practices in the region, especially in the areas of community-based policing and working with vulnerable groups. In accordance with the evaluation of COMET I, the second phase has made a concerted effort to engage communities. While the Combating Corruption in Jamaica activity (CCJ) has succeeded in bringing high-level attention and awareness to corruption issues among stakeholders in the government and public sector, more 4 Criminal Justice System Strengthening Project and Police Reform Project. 5 It should be noted that, at the time that this activity was designed, Suriname, Trinidad and Tobago, and Guyana were not a part of this Mission’s focus, and therefore not included. 6 USAID/Barbados, CBSI Juvenile Justice Assessment, Final Report, June 2011, p. 45. 7 USAID, Cooperative Agreement No. AID-538-A-13-00001, p. 29. 8 It should be noted that much of USAID/ESC’s programming that was developed prior to the start of CBSI was adapted to try and meet its mandate. Given this, it is not expected that this early programming would have taken into account best practices in citizen security at the time of its design. xi needs to be done to engage the broader public. The recent shift by the implementer (National Integrity Action, or NIA) to engage with youth is aligned with the findings of the recent final evaluation of CCJ. The ANP (A New Path) activity has been strategically designed to meet the rehabilitation and reintegration needs of female juveniles detained at South Camp, and to a lesser extent, male juveniles housed in the Metcalfe facility. It has based its work on a thorough assessment and inputs from a multi￾stakeholder steering committee, both promising indicators of its ongoing reliance on evidence and best practices. Empowering Jamaica’s Youth (Junior Achievement, or JA) is largely based on methods developed by JA Worldwide over the years in various countries, which have proven effective in Jamaica. They have been effective in securing government and private-sector support and in reaching diverse vulnerable groups, and the intentional capacity building and support of JA Jamaica ensured transition to local ownership at the end of the activity. Cross-Cutting Conclusions Although the evidence base is expanding (in part thanks to CBSI), rigorous documentation and independent verification of effectiveness are still scarce for citizen security programming in the Caribbean. Based on individual context within their country or countries, the USAID Missions and CBSI implementers have shown a highly varied approach to the use of successful international/external models or practices. Overall, the USAID staff and implementers interviewed by the team referred to very few such models, and CBSI activity design documents rarely made reference to external evidence or guidelines. In most cases, the representatives of USAID staff and implementers who were interviewed for the assessment referred only to past programming in the same country/region when asked to identify models or best practices related to citizen security and the specific sector of their work. Similarly, activity design documents rarely referenced any external evidence or guidelines. USAID itself has published guidance and evaluations on various subjects relevant to some CBSI activities (as noted in the individual sections above and the matrix in Annex VI), but documents of that nature were rarely mentioned by informants or activity documents. Therefore, while at least some correlations described above between CBSI activities and best practice sources were presumably due to the designers/implementers being aware of successful models, it is difficult to reach that conclusion with any certainty. However, the assessment found no evidence of any major discrepancies or contradictions between CBSI activity design and implementation and the identified best practices. Informants emphasized repeatedly that even positive experiences from neighboring nations had to be carefully studied and adapted to the local context. For this reason, it is important to invest in timely evaluations and other research, both linked to specific activities and cross-cutting studies, to continue building the knowledge base. Some USAID Missions and specific activities (such as JJRP, Alerta Joven, COMET II, A New Path, and Kari Yu!) have clearly recognized this need and commissioned research. Sharing of best and promising practices among the CBSI-supported Missions and activities has been fairly limited due to a number of obstacles, notably including geographical divisions that constrain in-person contacts and language barriers (related to the DR and Suriname). However, one of the greatest challenges noted by key stakeholders has been significant differences in subject matter and target groups among the activities, which often have very little in common. Each of the three USAID Missions operate in different contexts, and partner with host-countries who have different needs or priorities. A regional USAID CBSI Technical Working Group was created in 2011 to facilitate exchange of information, sharing of best practices, and development of common approaches to crime prevention and new partnerships between the U.S. and the Caribbean, with a particular focus on at-risk youth. The annual TWG meetings are attended by Caribbean governments, USAID and DoS CBSI interlocutors, thematic experts and implementing partners, and are a key opportunity for information sharing. There is some question about the extent to which participants are using and sharing that information, since the TWG meetings and resources were not mentioned by any in-country informants, although the assessment xii focus did not permit in-depth enquiry into this aspect of CBSI. It was noted by one USAID staff member that ensuring participation of all participating Caribbean governments has been difficult, as many of the smaller island states have put big restrictions on foreign travel as they grapple with the continued economic recession. It was noted by the same informant that moving forward, USAID/ESC has built participation in the CBSI working group into the learning component of its new 2015-19 strategy and will complement it with related learning events aimed at fostering more active sharing of best practices. The ESC Mission is moving towards the “public health approach” to crime prevention.9 This categorization of primary-, secondary-, and tertiary-level approaches to address risk and protective factors of crime and violence is highly relevant to analysis of CBSI best practices. All three levels are important to genuine improvements in citizen security and investments in programming at each level will have different types of results. This conceptual framework can therefore help to guide difficult choices on priorities for CBSI intervention, especially in a context of limited resources. USAID staff and implementers were very conscious of the importance of gender, including but not limited to achieving relative balance of beneficiaries and participants. The assessment also studied the engagement of CBSI activities with specific population groups who are of special interest to USAID for human rights reasons. These include LGBTI individuals, ethnic minorities, and the disabled. Despite some notable efforts by certain activities, it did not appear that specific methods or strategies were used by most implementers and their local partners to reach these individuals, although there were also no signs of active discrimination. Key Recommendations The Mission-specific recommendations of the assessment are primarily aimed at broadening and deepening the knowledge base about citizen security problems and solutions via ongoing or additional research. Cross-cutting recommendations for USAID include:  Conduct independent evaluations of all activities valued at a minimum of $2 million or lasting a minimum of 3 years and sectoral or more specific assessments to inform activity design and/or implementation;  Actively support the exchange of citizen security–related research to help make it more accessible to Mission staff and implementers, for example, organizing periodic teleconferences to discuss new research findings relevant to the region. These should revolve around specific thematic areas on which Mission staff are focused;  Consider the feasibility of information- and expertise-sharing encounters to complement the regional Technical Working Group meetings and ways to overcome language barriers to such exchanges;  Be more intentional and strategic in decision making on crime and violence prevention, considering the risk and protective factors of each context, and the three levels of interventions defined by the public health approach;  Improve outreach to particularly vulnerable or marginalized segments of the population. 9 The approach is described in some detail in Box 1 in the body of the report. xiii ASSESSMENT QUESTION 2 – UNEXPECTED CHALLENGES AND OPPORTUNITIES The assessment studied the key unexpected challenges and opportunities that arose for the USAID￾supported CBSI activities as well as the major effects of such challenges and opportunities on programming. Attention was paid to both internal and external challenges and opportunities. USAID/Dominican Republic A major challenge for Alerta Joven was posed by the large number of sub-grants to local civil society implementing partners; this mechanism takes time, effort, and varying types of support to ensure that activity results are achieved. Slow processing of sub-grants delayed the startup of core activities, and further delays flowed from a significant revision of the scope of the activity to focus more on workforce development. The weak institutional status and political influence of the Ministry of Youth might limit the ability of Alerta Joven to reach all of its goals at the policy level. In spite of considerable uncertainty regarding the future financial sustainability of the CJH, there is cause for optimism based on the positive results found in the recent evaluation and the substantial contribution of governments at local and national levels. USAID/ESC The Juvenile Justice Reform Project experienced unexpected challenges with procurement processes, which arose at both member state and Implementing Partner (IP) levels. The resulting delays affected several participating countries, and was cited in both program documentation and KIIs with IP staff and government stakeholders. The slow pace of attitude change within some governmental ministries and departments is also problematic in terms of pushing through reforms, and the global recession limited available resources for justice-related services by the participating governments. In the Strengthening Second Chance Education Program, the most significant challenges arose from differences between the regional activity’s design and the expectations of local partners. The goal of the activity was “to build a sustainable infrastructure that will allow at-risk groups of all interests and abilities to access programs that may lead to the development of valuable life skills, training/retraining for the world of work, portable certification, and a continuing education platform for future development,” which was to be done by expanding the range of certified organizations that support youth training. The activity had three broad objectives: (1) building strategic alliances/learning partnerships with public, private, and non-profit organizations; (2) taking steps to build a quality management culture and environment; and (3) expanding access to technology in education to facilitate e-learning. The program fell short of its objectives as targeted institutions did not have the internal resources to address the shortcomings identified, and the project was designed only to provide technical assistance and did not include the financial resources to fill the gap. As a result, the number of institutions that benefited was reduced significantly, and the project scaled back. The Kari Yu! activity experienced unexpected challenges in its first two years of implementation due to unrealistically high targets in their initial planning, negative perceptions within the private sector that hindered job placement efforts, and difficulty recruiting young males and youth from more rural parts of Suriname. The activity responded by expanding its outreach to the private sector, revising its indicator targets jointly with USAID, and focusing efforts on areas where youth demand for services was highest. USAID/Jamaica COMET II had to invest additional resources and time to build the capacity of CSOs; this unexpected need led to some delays but also to a better informed and equipped civil society with skills to benefit from COMET II and play a more active role in their communities. xiv While not an unanticipated challenge, the sustainability of NIA as the implementing organization of CCJ was noted by a variety of informants. However, its recent recognition as the national chapter of Transparency International should boost the organization’s credibility and sustainability. The ANP activity has limited the effect of unexpected challenges by various corrective actions, for example, by building the capacity of NGOs and using graduate students to bolster human resources. While Empowering Jamaica’s Youth/JA was relatively unaffected by unexpected issues, sustainability of activities is not guaranteed and may limit its ability to maintain the current reach of its work. Cross-Cutting Conclusions USAID CBSI activities faced a wide array of unexpected challenges, while opportunities were mentioned less by informants and documents. A common area of concern is that the tangible government investment needed to complement inputs from USAID and ensure sustainability has been lacking in many situations. This was especially evident in the small island and coastal nations covered by the USAID/ESC, which have fewer resources available. This was due in part to global economic conditions, although political factors are also at work. Delays in anticipated legislative changes in favor of police and juvenile justice reform have stymied core activity activities in several countries and, generally, political will in these areas is variable. Moreover, activities in USAID/ESC and USAID/Dominican Republic are facing uncertainty in relation to the ongoing level of CBSI funding levels, which could force a scaling back of recently launched initiatives. On the positive side, there is evidence that private-sector interest in supporting citizen security is increasing in some countries, including the Dominican Republic, Jamaica, and Suriname, and there are growing opportunities to create win-win situations that enhance the operating environment for businesses while improving the lives of ordinary citizens. Key Recommendations The Mission-specific recommendations for mitigating challenges and taking advantage of opportunities are as diverse as the programming across the region. They emphasize the importance of anticipating the cost and time required to work effectively with host-governments and local CSOs, analyzing the effort required to build ownership and sustainability of successful initiatives, assessing needs of beneficiaries on a timely basis, and engaging a wide range of stakeholders (depending on the specifics of each activity). Cross-cutting recommendations include:  USAID should attempt to maintain stable CBSI funding in line with projections for each Mission and allow time for Missions to design well-researched and complementary CBSI interventions that maintain a focus on citizen security;  USAID Missions should be as open as possible with implementers about possible changes in funding flows and collaborate with them to mitigate the potential effects of future budget reductions. ASSESSMENT QUESTION 3 – COMPLEMENTARITY AND COORDINATION This section aims to highlight actual and potential complementarity among USAID-supported CBSI activities as well as between USAID activities and programming supported by other USG agencies, identify any overlaps, and comment on coordination efforts. Consultations with USAID/LAC (Latin American and Caribbean) about other USG agencies led the team to focus on the Department of State, in particular its Bureau for International Narcotics and Law Enforcement Affairs (INL). USAID/Dominican Republic The Alerta Joven activity generally complements other CBSI activities, and there are no concerns with overlaps. However, there are potential synergies that have not yet been explored and exploited, for example, with the CJH in relation to peer mediation. The CJH activity has no overlap or conflict of objectives with the other activities but has even greater potential for synergies given the thematic focus xv on justice issues that it shares with the new Criminal Justice System Strengthened Project (CJSSP) and Police Reform Project. There has been weak information sharing and coordination between USAID/DR and implementers of CJSSP and Police Reform Project on the one hand and INL on the other. In recent years, the activities planned by each agency have become more closely related, and there is definite cause for concern that USAID activities could overlap with INL’s work in law enforcement. Irregular communications and coordination have created a high risk of confusion and miscommunication in relation to national counterparts and have made it difficult for the USG to capitalize on investments by each agency. Achieving complementarity may be challenging, as territoriality and competition for funds could undermine the will to share information and seek compromise. USAID/ESC Coordination and complementarity among USAID CBSI activities in the ESC region varied depending on the activity, country, and individual – with some being aware of other interventions, others collaborating, and still others who were wholly unaware of other CBSI programming In many cases, implementers were not aware of other activities in their own countries and the region; in the case of Second Chance and JJRP, only some local counterparts were aware of both interventions. USAID/ESC maintains communication with INL so they are aware of each other’s activities, but they do not routinely collaborate or coordinate in a significant way as they work in two distinct areas. Interviews with informants attributed this to three reasons: the different focus areas of the INL and USAID programming, the varied location of agency personnel, and the difficulties of travel within the region. The overarching programming comes from INL’s regional offices and DC, with INL focusing on counter￾narcotics, law enforcement professionalism, rule of law, and broader-based justice programming; USAID/ESC’s recently closed strategy (2010-2014) focused primarily on juvenile justice reform and workforce development. The staffs of both USAID/ESC and INL are relatively small and are not all based in the same location. The INL and USAID teams in the region cover similar but not identical geographic areas, many parts of which are remote, difficult to traverse, and have an inadequate regional transportation system. Under the most recent USAID/ESC strategy, the Mission embraced the public health approach towards crime and violence prevention. As this new approach is taken, it is anticipated by USAID staff that the stakeholders involved will expand, which creates the potential for overlap and the need for closer coordination. USAID/Jamaica In general, Jamaica’s CBSI activities do not overlap with each other in their objectives or activities, as all are working in quite distinct areas. Several implementers are working with others to leverage strengths and opportunities, albeit in an ad hoc manner, and few concrete collaborative activities have emerged. There is a reasonable level of information sharing and coordination between USAID and USG agencies on CBSI activities, notably in the form of weekly training updates and monthly working group meetings. However, these exchanges are not yet institutionalized and are partially dependent on individual motivation and relationships. The activities of USAID and INL were not seen as overlapping, but even slight shifts in direction could easily jeopardize complementarity, as the roles are not clearly defined. Cross-Cutting Conclusions The nature of the CBSI (and non-CBSI) programming of both USAID and INL continues to evolve. In recent years, INL appears to have amplified (or is looking to amplify) its work with at-risk youth, anti￾corruption, institutional strengthening of law enforcement and judicial sectors, and related areas. At the same time, USAID continues to work on anti-corruption, juvenile justice, and at-risk youth. While both groups specialize in different areas, as priorities for both agencies evolve in the CBSI countries, it creates the potential for either synergy or overlap and highlights the need for strengthened communications between the two agencies. xvi At the Washington level, there are two mechanisms for in-person interagency exchange of information on CBSI: a large formal quarterly meeting and a more frequent informal grouping, both of which appear to focus on overarching issues related to citizen security and USG investments in that sector. In addition, the annual Technical Working Group (TWG) hosted by USAID provides a forum for cross￾fertilization of ideas among the various USG CBSI partners, implementing partners, and host-country government counterparts. Coordination and information exchange between the agencies about individual CBSI activities and their components appears to be relatively ad hoc and is primarily seen as the mandate of staff in the Embassies and USAID Missions. Given that the designers and drivers of USAID and INL activities may be located in Washington or in disparate posts, effective communications strategies at the local level where activities are implemented are essential for true complementarity and avoidance of overlaps among all USG interventions. At that level, information sharing and coordination have been widely variable, although so far major overlaps have been avoided. In general, the level and type of communication between USAID and INL tend to be influenced by personalities, and information is not shared systematically. With the potential for increased budget constraints on both agencies, the chance for overlaps and even unintended competition is likely to increase. Key Recommendations Recommendations specific to each Mission and individual activities are primarily concerned with defining the scope of each activity and agency and ensuring that staff appointed as CBSI coordinators or focal points have a clearly established role. Highlights of cross-cutting recommendations are:  If they have not already done so, each Mission should organize periodic encounters among USAID CBSI implementers to share updates on activity progress and plans, coordinate and synergize activities, and exchange knowledge.  USAID Missions should consider arranging meetings between the CORs/AORs and implementers of its justice and law enforcement–related activities and INL staff. As appropriate, implementing partners should be included. These should be regular, one-on-one meetings to share information and coordinate as needed.  CBSI coordinators should collaborate with decision-makers from USAID and DoS in both Washington and the Missions in the region to develop a clear allocation of types of programming between USAID and INL that could serve as a guideline for their respective staff in Caribbean posts.  Each U.S. Embassy with CBSI activities should consider establishing a regular (quarterly) multi￾agency CBSI working group to ensure increased information sharing and coordination at the activity level.  USAID in LAC and in Caribbean Missions should engage, to the extent possible, its INL DC and INL post counterparts (and vice versa) in major decision processes related to areas of common interest. xvii CBSI BACKGROUND CBSI Overview Crime and violence experienced by Caribbean nations in recent years threaten to reverse the significant development gains that have been achieved. To help address the region’s insecurity challenges, President Obama announced an initial investment of $45 million for the Caribbean Basic Security Initiative (CBSI) at the Fifth Summit of the Americas in 2009.10 CBSI is a regional citizen security partnership that brings together member states of the Caribbean Community (CARICOM)11 and the Dominican Republic to collaborate with the United States Government (USG). At the core of CBSI is the recognition that security is critical to economic and social growth. In forming CBSI, the U.S. interagency teams met with Caribbean countries to develop a common regional strategy. The following three strategic priorities were identified to address security threats facing the Caribbean region: 1. Substantially Reduce Illicit Trafficking: through programs ranging from counter-narcotics to reducing the flow of illegal arms/light weapons. 2. Increase Public Safety and Security: through programs ranging from reducing crime and violence to improving border security. 3. Promote Social Justice: through programs designed to promote justice sector reform, combat government corruption, and assist vulnerable populations at risk of recruitment into criminal organizations. CBSI policy is coordinated by the Department of State’s Bureau of Western Hemisphere Affairs. USAID, the Bureau of International Narcotics and Law Enforcement Affairs (INL), and the Bureau of Political￾Military Affairs are responsible for implementing CBSI assistance in the region. The USG has committed more than $327 million in funding to CBSI programming since 2010. 10 U.S. Congress, House, Foreign Affairs Committee, Testimony of Julissa Reynoso before the House Foreign Affairs Committee Subcommittee on Western Hemisphere Affairs, U.S. Congress, 2009. Print. 11 Antigua and Barbuda, The Bahamas, Barbados, Dominica, Grenada, Guyana, Jamaica, St. Kitts and Nevis, St. Lucia, St. Vincent and the Grenadines, Suriname, and Trinidad and Tobago. Note: The Bahamas is not targeted by USAID assistance under CBSI. 2 Figure 1. Map of CBSI Participating Countries in Caribbean Basin USAID CBSI Objectives USAID is the lead USG agency under the third pillar of CBSI, focused on social justice, and to date has committed over $127 million for these efforts in the targeted countries. USAID works with Caribbean governments and citizens to address insecurity by focusing on four objectives: 1. Increased Economic Opportunities for At-Risk Youth and Vulnerable Populations. USAID provides workforce development training and remedial education programs to at-risk youth. 2. Improved Community and Law Enforcement Cooperation. USAID community-oriented policing activities work to improve effectiveness and professionalism of police; they also work with communities and police to increase mutual trust and cooperation on crime-prevention initiatives. 3. Improved Juvenile Justice Sector. USAID supports regulatory reform for juvenile justice systems, works with police and judges to divert youth from traditional prison sentences, and works within juvenile detention centers to provide psychosocial support and work skills training for youth in custody. 4. Reduced Corruption in Public and Private Sectors. USAID has supported anti-corruption training for government officials and worked with the Customs Department and the Tax Administration in Jamaica to reduce traffickers’ ability to move money and contraband. Each of the USAID Missions in the region (Dominican Republic, Eastern and Southern Caribbean, and Jamaica) has supported a unique mix of activities that both meet the needs of their respective country or countries and align with overarching strategic priorities of CBSI. The assumption is that through this type of programming USAID will help to address the development challenges posed by the escalating crime and violence in the region. At the start of CBSI, each Mission was designated to serve as a “knowledge hub” for specific objectives based on their programmatic strengths at the time. The rationale was that each Mission would be able to share its best practices and lessons learned with the others to help strengthen overall programming. The focus of the Missions was agreed as follows:  USAID/Jamaica: community-oriented policing and anti-corruption 3  USAID/Eastern and Southern Caribbean: workforce development and juvenile justice  USAID/Dominican Republic: monitoring and evaluation Each year, USAID collaborates with CARICOM to host a Technical Working Group meeting (TWG) that brings together the Missions, stakeholders from across the region, and implementers to discuss the situation in their countries and share information on activities designed to increase social justice. Given that programming on citizen security is a relatively new area for USAID and other actors in the region, these TWGs aim to help spread knowledge on the topic. The overall scope of each Mission’s work under CBSI is outlined below, followed by a list of CBSI activities by region and country. The individual activities are further described in Annex III. USAID/Dominican Republic (DR) CBSI programming has focused on the justice system and supporting at-risk youth to reduce delinquent behavior and encourage them to become more productive members of society. To accomplish its current strategic goal of “Improved Citizen Security to Promote Economic Growth,” this Mission is supporting three justice-related activities that together aim to enhance grassroots access to justice, improve the functioning of the criminal prosecution system, and support police force reform. In relation to at-risk youth, the Mission is implementing an ambitious five-year activity that combines education, job placement, and social and economic services. Most Mission programming (CBSI and otherwise) targets the population of the so-called Duarte Corridor, where the majority of Dominicans live and the major cities are located. Funding for CBSI activities from 2010 through 2015 has totaled approximately $31 million. USAID/Eastern and Southern Caribbean (ESC), USAID/ESC is a regional Mission that currently serves 10 countries: Antigua and Barbuda, Barbados, Dominica, Grenada, Guyana, St. Lucia, St. Kitts and Nevis, St. Vincent and the Grenadines, Suriname, and Trinidad and 12 Office of Juvenile Justice and Delinquency Prevention Best Practices to Address Community Gang Problems: OJJDP’s Comprehensive Gang Model, 2d edition, October 2010, p. 3. 13 Draft USAID Crime and Violence Prevention Field Guide, p. 15. Box 1. The Public Health Approach to Crime and Violence The “public health approach” to crime and violence prevention, also known as the epidemiological approach, views crime and violence as analogous to an infectious disease that should be contained and prevented through interventions targeting risk and protective factors at four levels (see Figure 2). 12,13 While it has long informed USAID’s programming, more recently the agency has formally supported a public health approach to addressing crime and violence. At the heart of the public health approach is the identification of risk and protective factors at the individual, relationship, community, and societal levels to assess the likelihood of crime perpetration or victimization and to inform interventions. Interventions under this approach are usually categorized as follows: 1. Primary level: Addresses socioeconomic and community factors at the population level that affect the incidence of crime and violence. 2. Secondary level: Targets individuals or groups exhibiting factors at risk for crime and violence. 3. Tertiary level: Targets individuals who have already engaged in crime and violence or been victimized, primarily in order to reduce recidivism and promote rehabilitation. According to this approach, it is not merely the presence of risk factors that predict an individual’s involvement in crime and violence. It is the presence of multiple risk factors across different levels and the absence of various protective factors that lead to crime. This approach borrows from the Office of Juvenile Justice and Delinquency Prevention’s (OJJDP) Comprehensive Gang Model, which outlines five components to reducing gang violence: 1) primary prevention, 2) secondary prevention, 3) intervention, 4) suppression, and 5) reentry. 4 Tobago. (At the beginning of CBSI, the Mission was called USAID/Barbados and the Eastern Caribbean and covered only Antigua and Barbuda, Barbados, Dominica, Grenada, St. Lucia, St. Kitts and Nevis, and St. Vincent and the Grenadines.) USAID/ESC’s 2011–15 strategy focused on youth and workforce development and strengthening of the juvenile justice sector, supporting CBSI objectives 1 and 3 described above. The Mission moved to adapt programming that had addressed broader issues of youth workforce development to focus more on at-risk youth. CBSI commissioned a Juvenile Justice Assessment in 2011 targeting 10 countries in the region—Antigua and Barbuda, Dominica, Grenada, Guyana, St. Lucia, St. Kitts and Nevis, St. Vincent and the Grenadines, Trinidad and Tobago, and Jamaica—to inform the development of this new program area. In 2012, Suriname, Trinidad and Tobago, and Guyana were added to the Mission’s coverage area and programming was expanded, starting in Guyana. When the Mission designed its new strategy in 2014, its name was changed to reflect this larger coverage area. At the same time, the Mission drew on lessons learned to develop a strategy that includes a public health approach to youth crime and violence reduction, drawing on evolving best practices in this newly emerging field for USAID. Box 1 describes the public health approach, which is illustrated in Figure 2. The 2015–19 overarching goal is “Safer, more prosperous Caribbean communities,” while the key security-related development objective is to “reduce youth involvement in crime and violence in targeted communities.”14 This Mission is now focused on investing in activities that support community-based crime and violence prevention, improve juvenile justice systems, and establish an improved evidence base on crime and violence. Funding from 2010 through 2015 has totaled approximately $54 million. Figure 2. Intervention Levels Targeting Risk Factors under the Public Health Approach 14 USAID, USAID ESC Regional Development Cooperation Strategy 2015–2019 (Public Version), 2015. Societal Community Relationship Individual Primary Intervention Levels Tertiary Risk Levels Secondary Tertiary 5 USAID/Jamaica Under the overarching strategic goal of “Resilience and Social Cohesion of Targeted Jamaican Communities Improved,” USAID/Jamaica has implemented a very diverse CBSI portfolio across the country that focuses on five areas: at-risk youth, basic education, workforce development, community safety and security, and anti-corruption. The first three of these thematic areas directly aim to reduce the vulnerability of Jamaican youth and their involvement as perpetrators or victims of crime and violence. Funding from 2010 through 2015 has totaled approximately $41 million. CBSI Activities The 23 activities appearing in Table 1 were identified in consultation with USAID/LAC and the Missions in the region as those funded by CBSI since 2010, either in whole or in part. Activities already closed as of March 2016 (time of data collection) are indicated in italics. Table 1: USAID CBSI Activities by Mission Mission Activity Implementation Period Studied Countries Covered Estimated USAID Funding Thematic Focus Prime Implementer USAID/ Dominican Republic Dominican Republic At￾Risk Youth Activity / Alerta Joven July 5, 2012 – July 4, 2017 Dominican Republic $19,972,130 Providing integrated support services for at￾risk youth in vulnerable communities Entrena, SRL Community Justice Houses (funded as a component of Criminal Justice System Strengthened Project since 2015) August 10, 2012 – August 10, 2015 (direct funding); August 31, 2015 – March 1, 2016 (indirect funding) Dominican Republic $343,335 for 2015–16 $1,350,000 for 2012 - 2015 Providing dispute resolution services and improved access to justice (2012– 15); Consolidating CJH and expanding dispute resolution services (2015– 16) Participación Ciudadana, prime from 2012–15, grantee under CJSSP from 2015–16 Criminal Justice System Strengthened Project (CJSSP) June 8, 2015 – June 7, 2020 Dominican Republic $21,534,654 Strengthening the criminal justice system by improving the timeliness and effectiveness of criminal prosecutions Chemonics International Inc. Institutional Capacity and Transparency July 20, 2015 – July 19, 2019 Dominican Republic $2,800,000 Strengthening institutional capacity and International Criminal Investigative 6 Strengthening for Police Reform (Police Reform Project) transparency of the Dominican National Police Training Assistance Program (ICITAP) of the U.S. Department of Justice USAID/ Eastern and Southern Caribbean Juvenile Justice Reform Project October 12, 2011 – March 31, 2016 Antigua and Barbuda, Dominica, Grenada, St. Kitts and Nevis, St. Lucia, St. Vincent and the Grenadines $5,800,000 Strengthening the juvenile justice system and working with at-risk youth Organization of Eastern Caribbean States (OECS) Commission Strengthening Second Chance Education in the Eastern Caribbean September 26, 2012 – March 31, 2016 Antigua and Barbuda, Dominica, Grenada, St. Kitts and Nevis, St. Lucia, St. Vincent and the Grenadines, Guyana, Trinidad and Tobago $2,150,647 Increasing educational opportunities for at-risk youth through strengthening second chance education programs Caribbean Examinations Council (CXC) Kari Yu! Youth Development and Juvenile Justice Project February 7, 2013 – September 30, 2016 Suriname $4,500,000 Strengthening the juvenile justice system and working with at-risk youth Pan-American Development Foundation (PADF) Trinidad and Tobago Juvenile Court Project April 23, 2014 – April 22, 2017 Trinidad and Tobago $4,545,765 Strengthening the juvenile courts system United Nations Development Programme (UNDP) Skills and Knowledge for Youth Empowerment (SKYE) August 8, 2011 – September 30, 2016 Guyana $7,700,000 Workforce development, adapted to focus on at-risk youth Education Development Center, Inc. A Ganar September 27, 2009 – September 26, 2015 Dominica, St. Kitts and Nevis, St. Vincent and $8,899,467 (across all 8 countries) Workforce development, adapted to focus Partners of the Americas 7 the Grenadines, Suriname on at-risk youth Caribbean Youth Empowerment Program October 1, 2008 – December 31, 2013 Antigua and Barbuda, St. Lucia, Grenada Total value of activity was $4,173,613, 60% funded by USAID Workforce development, adapted to focus on at-risk youth International Youth Foundation USAID/ Jamaica Community Empowerment and Transformation II (COMET II) December 2, 2013 – December 1, 2018 Jamaica $12,707,527 Promote community￾based partnerships to prevent crime and violence, rule of law, control corruption, and increase youth engagement Tetra Tech ARD Empowering Jamaica’s Youth (Junior Achievement) January 28, 2010 – September 30, 2014 Jamaica $4,223,169 Increase access to economic and business education opportunities to Jamaican youths Junior Achievement Worldwide “A New Path”: Promoting a Healthy Environment and Productive Alternatives for Juvenile Remandees and Offenders in Jamaica September 15, 2014 – January 13, 2017 Jamaica $1,936,543 Supporting the successful reintegration and rehabilitation of female juveniles, and to a certain extent, male juveniles Organization of American States Combatting Corruption in Jamaica August 1, 2012 – July 31, 2015 Jamaica $2,900,000 Challenging corruption and promoting public demand for anti￾corruption National Integrity Action Limited USAID/MoE Education Partnership for Improved Reading Outcomes September 2, 2013 – August 31, 2015 Jamaica $3,097,133 Improving early grade reading toward 100% literacy rate Ministry of Education of Jamaica Fi Wi Jamaica May 21, 2015 – May 20, 2018 Jamaica $2,400,00 Promoting the rights, inclusiveness, University of Technology, 8 and development of women and girls and members of the LGBTI community Jamaica Community Empowerment and Transformation I (COMET I) March 13, 2006 – October 1, 2011 Jamaica $11,405,720.49 Strengthening good governance and economic competitiveness to reduce crime and violence Management Systems International ASSESSMENT PURPOSE AND QUESTIONS Assessment Purpose USAID has been involved in implementing the CBSI since 2010 and wishes to enhance understanding of its progress to date and build upon that experience moving forward. As such, USAID contracted Social Impact to conduct an assessment of CBSI activities to guide future programming, identify new areas of research and/or CBSI implementation for USAID, and locate programmatic and/or geographic gaps in CBSI programs to be filled by USAID and other stakeholders. The resultant data are intended to help guide the work of USAID posts implementing CBSI activities; the USAID CBSI coordinator in Washington; U.S. Department of State CBSI coordinators in Washington; CBSI activity implementers; Caribbean governments; regional organizations such as the Caribbean Community (CARICOM) and the Organization of Eastern Caribbean States (OECS); and academic institutions. This assessment includes all 12 USAID/CBSI countries, with more in-depth attention to five of those countries. In consultation with USAID stakeholders, the team determined that the countries to be visited for fieldwork and explored in more depth would be Jamaica, St. Lucia, St. Vincent and the Grenadines, Suriname, and the Dominican Republic. The team also gathered data in Barbados from a number of regional stakeholders and the USAID/ESC Mission itself. Assessment Questions Social Impact (SI) worked with USAID to clarify the purpose, users, and decision-making uses of the assessment, which led to a focus on three specific assessment questions, defined in the Statement of Work (attached as Annex I) as follows: 1. To what extent do USAID’s CBSI programs reflect the most current understanding of best practices in citizen security, particularly in the Caribbean context? 2. Have there been any unanticipated challenges or opportunities that have affected implementation of CBSI programs? This could include internal factors such as USAID’s strategic planning processes, procurement approval processes, staffing, etc. It could also include external factors such as host countries’ legal frameworks, political will, diplomatic influence of the U.S. Embassy, changes in political administration, etc. 9 3. To what degree are USAID’s CBSI programs complementary with other USG CBSI programs, to include Department of State (DoS), Department of Defense, and other agencies? Are there instances where these programs are overlapping or working at cross-purposes? Those questions were defined in more detail by the assessment team prior to data collection, based on document review and consultation with USAID. The definitions are contained in Annex II. ASSESSMENT DESIGN, METHODS, AND LIMITATIONS Assessment Design This assessment aimed to leverage both context-specific and cross-portfolio analysis to respond to the assessment questions. Social Impact’s conceptual approach to answer these questions started with CBSI activity monitoring data and was complemented with third-party data sources and information obtained during fieldwork including key informant interviews (KII), observations from site visits, focus group discussions (FGDs), and consultations with relevant stakeholders. Throughout the evaluation, the team designed and conducted data collection and analysis with due respect for the inherent diversity of the countries and populations involved in the distinct activities. The assessment utilizes a snapshot design, with data collection taking place at one point in time; however, the team used longitudinal monitoring and third-party data where available to complement primary data collection. A core tenet of SI’s mission is to focus on the use of data to improve programming and, ultimately, development effectiveness. Aligned with this mission, SI actively engaged USAID staff (in DC and the Caribbean Missions), implementing partners, and other stakeholders throughout the assessment process, using its Evaluation Quality Use and Impact (EQUI™) approach. The EQUI approach has guided discussions on methodological choices and presentation of results to help ensure that the assessment will meet USAID and other stakeholders’ needs. Assessment Methodology SI has used a primarily qualitative approach, supplementing it with quantitative methods where available, including (1) a systematic document review; (2) qualitative data collection through KIIs and FGDs with a wide range of stakeholders; and (3) analysis of quantitative monitoring data, where available. SI developed a rigorous design grounded in participatory and gender-sensitive approaches to answer the assessment questions; the team then triangulated data from different sources to ensure accuracy of findings. The data collection tools were designed to have a high level of technical quality and mitigate threats to data validity, and the team conducted pilot-testing of those instruments in Jamaica prior to full roll-out. Principal data sources and data collection methods for each question are presented in Table 2, updated since the inception report to reflect additional learning from the assessment planning process. Table 2: Evaluation Design Matrix Assessment Question Key Data Sources Data Collection Methods Q1: Best practices USAID staff, activity implementers and partners, government counterparts, sectoral experts, other local/regional stakeholders Desk review and qualitative analysis of activity documents; interviews with key informants; literature review Q2: Unanticipated challenges or opportunities USAID staff, activity implementers and partners, government counterparts, beneficiaries, other Desk review and qualitative analysis of activity documents; interviews with key informants; FGDs with beneficiaries; 10 local/regional stakeholders analysis of activity monitoring data Q3: Complementarity and coordination USAID staff, staff of other USG agencies, activity implementers and partners, government counterparts, other local/regional stakeholders Desk review of activity documents and documents related to other USG-funded activities; interviews with key informants DATA COLLECTION LOCATIONS The team carried out data collection in capital cities of each visited country and met with USAID Mission staff, management staff of implementers, national government officials, and others based in those cities and also in selected communities where CBSI activities are implemented, in order to speak with activity field staff, beneficiaries, and local government officials as relevant. The locations visited in each country are listed in Table 3. Table 3: In-Person Data Collection Locations Country Cities/Towns Barbados Bridgetown Dominican Republic Santo Domingo, Santiago Jamaica Kingston, Montego Bay St. Lucia Castries St. Vincent and the Grenadines Kingstown Suriname Paramaribo In addition, remote interviews (by phone or Skype) were held with informants in other countries not visited by the team, including Antigua and Barbuda, Dominica, Grenada, Guyana, St. Kitts and Nevis, Trinidad and Tobago, and the U.S. (Washington, DC). DATA COLLECTION AND ANALYSIS METHODS Qualitative Data Collection The team used semi-structured KII and FGD guides, which can be found in Annex IV. These guides constituted a primary tool to answer all three assessment questions and were thus designed to specifically elicit information in relation to those questions. KIIs and FGDs were facilitated by team members in English, except in the Dominican Republic, where Spanish was used if preferred by participants, and in Suriname, where an interpreter was used with participants who preferred to use Dutch. Although in general the subject matter of the assessment was not considered particularly sensitive, the team was cautious about asking questions about lesbian, gay, bisexual, transgender, and intersex (LGBTI) – related issues, for example, by not making direct references to LGBTI in certain data collection activities based on the advice of USAID Mission staff and regional team members. a) KIIs: The team conducted interviews with 145 key informants throughout the assessment, primarily on an individual basis but at times with multiple respondents. Male participants totaled 60 while female totaled 85. In-person interviews constituted 85% of the total KIIs, while the 11 remainder were conducted by phone. Informants included USAID staff in Caribbean Missions and Washington; implementers; beneficiaries; DoS staff; partner government officials; law enforcement and judiciary personnel; community leaders; sectoral experts; and other donors/partners working in the security and social justice sectors. The KIIs began with a purposive sample of local and regional stakeholders selected in consultation with USAID. Additional stakeholders were flexibly scheduled based upon issues or recommendations raised in the interviews. A breakdown of informants by Mission, type of informant and appears in Table 4. b) FGDs: The team conducted 15 FGDs with a total of 136 participants, with representation from both males (59) and females (77). FGD participants varied depending on the CBSI activity being assessed, but included at-risk youth and their families, members and staff of community organizations, and local implementer staff. These groups were selected based on a convenience sampling approach. Staff of USAID Missions and implementers were consulted to identify participants, and in some cases they were asked to assist with contact information or outreach. c) Gender and social analysis: Complex social relationships and political economies based on gender, race, and other forms of power distribution can play a very important role in development programs. At all levels of data collection, the SI team assessed these variables closely. Key questions for informants sought to document the extent to which women and other marginalized groups have been taken into account and/or played a role in CBSI programming, as beneficiaries or otherwise. The assessment scope of work identified specific populations for particular attention during data collection and analysis, such as ethnic minorities in Suriname and DR, and the LGBTI community in Jamaica, and specific efforts were made to include representatives of those minorities in KIIs and/or FGDs. The team also paid special attention to youth perspectives on CBSI activities by organizing FGDs with exclusively youth participants (ages ranged from 12 to 35) in Suriname, St. Lucia, St. Vincent and the Grenadines, the DR, and Jamaica. d) Direct Observation: Although direct observation of activities was not originally scheduled as part of data collection due to the nature of the assessment questions, the team took advantage of opportunities as they emerged to conduct site visits to activity partners and direct observation of activity activities in order to supplement the information generated through other methods. In all, the team visited five sites in Jamaica and two in the DR. Table 4: Informant Statistics for KIIs and FGDs Mission Type of Informant Total Eastern and Southern Caribbean USAID and other USG officials 7 Implementer staff 26 Government counterparts 24 Other KII 3 FGD participants 31 Total 91 Dominican Republic USAID and other USG 10 Implementer staff 18 Government counterparts 4 Other KII 1 12 Mission Type of Informant Total FGD participants 55 Total 88 Jamaica USAID and other USG 7 Implementer staff 21 Government counterparts 14 Other KII 3 FGD participants 50 Total 95 Other (US-based and regional informants) USAID and other USG officials 8 Other KII 1 Total 9 Total 283 Document Review/Secondary Research The team conducted a preliminary desk review of secondary materials prior to fieldwork, which included a selection of USAID strategic planning documents (Regional and Country Development and Cooperation Strategies and Performance Management Plans); activity contracts, descriptions, Monitoring and Evaluation Plans, progress reports, and evaluations of USAID CBSI activities; national and regional development plans; country-level data from government ministries; reports from local, regional, and international non-governmental organizations present in the region; and studies and reference documents produced by USAID and other organizations (e.g., CARICOM, academic institutions, and the UN). The team used the preliminary desk review to begin studying the assessment questions prior to in￾country visits and to inform the development of data collection instruments. As documents were received from the Missions over time, and as the team’s own research identified additional documents (particularly on best practices), the scope of the desk review expanded. Ultimately, the team reviewed more than 350 documents in order to evaluate their relevance to the assessment. Those found to be of most relevance were assigned to team members (primarily on the basis of the geographic focus of each sub-team) for more in-depth reading and extraction of key information. See Annex V for a list of all documents reviewed. UTILIZATION-FOCUSED DATA ANALYSIS AND REPORTING The assessment team carried out initial analysis concurrently with data collection, and sub-teams regularly communicated initial findings and avenues for exploration with each other. Several times each week, sub-teams summarized key emerging data according to a structured protocol organized by assessment question and shared those data with other team members. The team also conducted weekly check-ins with the USAID Contracting Officer’s Representative during fieldwork to provide updates on emerging patterns in the data and any challenges that had arisen in collection or analysis. 13 At the end of fieldwork, the assessment team organized and disaggregated all data obtained from different methods and sources, reviewed the data for inference and validity, and triangulated it by method and source. They then organized findings according to the agreed-upon assessment questions, drawing conclusions directly from these findings and developing recommendations based on those findings and conclusions. The team inserted preliminary findings, conclusions, and recommendations into a matrix for each activity, ensuring they aligned with the purposes outlined by USAID to feed into decision-making, to facilitate team discussion, and to ensure alignment among all sub-teams. Team Composition and Roles The assessment team consisted of a team leader, assistant team leader, and mid-level evaluation specialist who were complemented by three local subject-matter experts in addition to local and international support staff. A number of team members had previously conducted assessments and/or evaluations in the region on youth and security initiatives, some of which were CBSI-funded and, as such, were well versed with the CBSI context and programming. The assessment team split into three sub-teams during fieldwork, allowing for greater reach and more efficient data collection throughout the region. Each team consisted of two people (a senior- or mid￾level Monitoring and Evaluation Specialist paired with a local expert), who spent between three days to two weeks in each country. The two-person structure allowed for flexible and efficient deployment of resources, while adhering to international best practices of using paired evaluators to minimize potential moderator or reporting biases. The sub-team members shared roles during data collection, with one person leading the questioning and one person taking notes. The sub-teams followed a protocol whereby the note-taker entered the notes into standardized data capture sheets regularly, with review and validation from the facilitator. Limitations SCOPE OF INQUIRY USAID’s programming under CBSI seeks to fulfill a broad range of objectives through a diverse and shifting set of interventions in 12 distinct countries, as described in the CBSI background section. This element of diversity complicates the aggregate snapshot of CBSI, even taking into account the relatively specific nature of the assessment questions, which were not intended to evaluate progress towards objectives. As a relatively short assessment taking place at one point in time, this assessment is inherently limited in developing a complete picture of all USAID-supported CBSI activities. Therefore, the team agreed with USAID in the early stages of the assessment that a reduced list of nine “priority activities” would be subjected to closer examination, while the other 14 activities would be analyzed primarily through document review. Furthermore, the fieldwork period for the assessment ran from February 29 through March 25, 2016, and this compressed timeline did not allow the assessment team to provide each activity with the level of attention that it would receive under a full evaluation. DATA AVAILABILITY The team encountered some limitations related to data availability, particularly in relation to CBSI￾funded activities that ended prior to fieldwork and those not identified as priority activities for this assessment. The compressed timeframe for this assessment meant that not all documents were received and reviewed prior to fieldwork. This was especially challenging for USAID/Jamaica, where a number of activities had closed before the assessment began, and for USAID/ESC, where the team was unable to visit all countries associated with CBSI. The team mitigated this issue by doing an initial scan and prioritization of the key documents for each sub-team and for the team as a whole to read before fieldwork and continued to review documents during and after primary data collection. 14 RESPONSE BIAS Some inputs by informants may have been affected by response bias. In other words, they may have been motivated to provide responses that would be considered socially desirable or otherwise reflect positively on their work or organization. This was most noticeable in relation to the issue of complementarity between USAID programming and INL activities, where informants from both agencies tended to present ideas closely aligned with their own agency’s interests, sometimes resulting in diametrically opposed viewpoints on the same issue. Efforts were made to verify information with more than one source, but where it was not possible to reconcile conflicting views, that fact is indicated in this report. The team also observed a tendency among some activity implementers to downplay or not mention challenges and weaknesses of their activity, which were given little attention in activity documentation as well as in interview responses. SELECTION BIAS Selection bias is an inherent risk when activity implementers and/or funders help to facilitate contact with their partners and beneficiaries, as the assessment team may only hear from informants who are likely to report positive experiences. Given time constraints and the large number of activities to be assessed, the team relied on this type of facilitation for certain data collection targets but also made efforts to broaden the scope of informants to include other viewpoints. To mitigate bias, the team triangulated responses from various data sources and via multiple data collection methods. For example, the views of implementers in the DR on effectiveness of methodologies were checked with local partners during focus group discussions, and documented information on challenges was compared to interview data on the same subject. Furthermore, the team complemented respondent selection through the use of snowball sampling—asking informants to identify other potential informants—and supplemented the list of informants with additional sources known from team members’ experience in the region. That stated, the limited time in each country did limit the number of respondents and scope of triangulation, as mentioned above. DEFINITION OF BEST PRACTICES When designing the assessment, SI anticipated that best practices in the region related to citizen security would have been well defined and identified by reliable sources. What the team found was that the evidence base on what works and what does not work regarding citizen security in the Caribbean is only partially developed, poorly documented, and very fragmented. This is largely due to the somewhat amorphous nature of programming in the sector, as citizen security is a very general concept that encompasses a vast array of interventions (as demonstrated by the variety of USAID CBSI activities) and the research and guidelines that do exist tend to relate to only one or two specific aspects of citizen security, such as workforce development. In many cases, interviewed implementers and USAID personnel did not point to specific sources of best practices in the subject area of the activity(ies) in which they were involved; instead, they spoke in more general terms about what they perceived as effective in those activities. The assessment team worked to mitigate this challenge by conducting independent research on the key objective areas of USAID CBSI programming, then widening the scope to look for authoritative sources in Latin America when thfey could not be found in the Caribbean. That approach is detailed further in Assessment Question 1 below. 15 FINDINGS, CONCLUSIONS, AND RECOMMENDATIONS Assessment Question 1: Best Practices INTRODUCTION While defining the assessment questions, the assessment team defined best practices as “methods and activity models that have been demonstrated and clearly documented as effective in improving citizen security.” As described above in relation to limitations, the knowledge base on current best practices in citizen security in the region is not yet well developed or documented. The activities included in this assessment took place over a period of time and are not expected to take into account best practice documents that were created or published after that particular activity had been designed and implemented. This assessment aims to identify how both closed and ongoing programming has lined up with an evolving understanding of best practices in a diverse sector and/or set of sectors. In doing so, this report seeks to analyze—to the extent possible, given the scope of research and time limitations— the principal approaches and practices used in the CBSI activities to date. Based on this, the team located and reviewed the following four types of documents, through a combination of desk research and KIIs, in order to form an analytic basis for this assessment question:  Documentation of any independent evaluations of the current CBSI activity or a similar (possibly predecessor) activity in the country/region  Documentation of detailed research that informed the design or implementation of CBSI programming (e.g., a comprehensive situation assessment or needs assessment)  Existence of USAID or other USG-produced guidelines or policies directly relevant to the subject matter of CBSI programming  Other documented analysis of practices and outcomes in citizen security programming since 2010, primarily related to the Caribbean region, but also including highly relevant documents focused on Latin America or more broadly The research focused on practices documented since 2010 in relation to the four key objective areas of USAID CBSI programming: (1) increased economic opportunities for at-risk youth and vulnerable populations, (2) improved community and law enforcement cooperation, (3) improved juvenile justice sector, and (4) reduced corruption in public and private sectors. Based on that process, the team created the two-part matrix in Annex VI, which demonstrates the best practices documents identified as most relevant in each CBSI objective area, including a breakdown of the USAID CBSI objectives into sub-topics related directly to the activities funded to date. These guides and documents reflect the most current thinking on best practice; and help demonstrate the evolution in the field of citizen security programming. As many were developed following the design of CBSI activities, the team would not expect each activity to be held accountable to these practices, but rather should be considered for future programming. The following analysis is primarily aimed at assessing the extent to which each CBSI activity is consistent with the documents listed in that matrix, which are mentioned in the text below as 16 relevant to each activity. However, it should be noted that in some thematic areas there was little evidence of best practices that met the criteria of the assessment. ASSESSMENT QUESTION 1 FINDINGS Dominican Republic The design of the Alerta Joven activity in 2011 was based on a comprehensive assessment,15 which examined not only the needs of vulnerable youth in the DR but also the outcomes of past and current interventions targeting such youth. It looked closely at various methods used to promote education, health, employment, and civic participation/governance, including those supported by USAID, other donors, and the government. The core recommendation was to establish a single youth program to provide a social safety net for vulnerable out‐of‐school youth within targeted areas, and Alerta Joven was the outcome. The activity has promoted teaching and educational re-insertion methods that were positively reviewed by the cross-sectoral assessment. The activity design is closely aligned with United Nations Development Programme’s (UNDP) Caribbean Human Development Report 2012, which focused on at-risk youth in the Caribbean. In particular, Alerta Joven’s high priority on boosting employment and education levels, on gender balance and inclusive targeting (working with a broad range of youth from at risk communities, not just those with highest risk of involvement in crime and violence), and on a holistic approach to address multiple risk factors is consistent with the recommendations of that report.16 Informants considered its “integrated attention” model to be in line with best practice, though the original idea was adapted after the first year to emphasize employment and entrepreneurship. Activity monitoring data and reports show that the implementer and its partners have strived for and achieved an overall balanced participation of male and female youth and considered preferences of both in the choice of vocational training options and other activities. The team found the “public health approach” to be highly relevant to Alerta Joven. Current literature on this approach conceptualizes crime as analogous to an infectious disease and generally categorizes crime￾prevention interventions as operating at three levels. At the primary level the strategies target the general population and seek to address risk and protective factors leading to crime and violence. At the secondary level, they target groups and individuals facing multiple risk factors and who have already been either victimized or engaged in violent behaviors. At the tertiary level, strategies are designed to prevent recidivism and promote rehabilitation measures. The levels were clearly defined in a 2012 report on Central America by Interpeace17 and more recently by a draft USAID guide, as explained in later sections of Assessment Question 1. The Interpeace report made specific recommendations for youth-focused programming at all three levels.18 Alerta Joven is directly in line with several of those strategies, including the three at the primary 15 Education Development Center, Inc. USAID/Dominican Republic Cross-Sectoral At-Risk Youth Assessment, August 2010. 16 United Nations Development Programme, UNDP Caribbean Human Development Report: Human Development and the Shift to Better Citizen Security, 2012. 17 Interpeace, Nine Strategies to Prevent Youth Violence in Central America, 2012. 18 Ibid., p. 11. The levels are similarly defined in the Draft USAID Crime and Violence Prevention Field Guide, 2016, provided to the team during analysis stage by USAID/LAC, on page 21. 17 level: (1) programs and appropriate funding to guarantee fundamental rights linked to reducing youth violence: education, health, and employment, with a focus on violence prevention; (2) promotion of youth development through arts, sports, recreation, and community participation; and (3) improvement and expansion of community-based, preventative approaches to public and citizen security. More generally, the Interpeace report commented that: “Large numbers of youths in the region live in generalized risk due to social exclusion and violation of their social, economic, and cultural rights. Primary prevention strategies are appropriate for all youth facing these challenges…” 19 The Alerta Joven approach is consistent with this guidance. Although it has elements of secondary prevention, most of Alerta Joven’s activities are better described as primary prevention since targeting was broad, largely based on residence in “barrios calientes” (“hot” or dangerous neighborhoods) as compared to specific individual risk factors. As described by the prime implementer: “Alerta Joven identifies and works with those youth in high-risk situations who have not yet taken the wrong path but could easily do so.”20 However, there is one aspect of the activity (added during the second year) that focuses on youth in conflict with the law by collaborating with the Attorney General’s office on enhanced systems and procedures. Generally, while Alerta Joven does aim to reduce crime levels in the long term, the nature of the activity would make it very difficult to attribute changes in perception or incidence of crime to its interventions. Activity records and KIIs reveal that Alerta Joven has made a significant investment in results monitoring and research more generally, including a comprehensive database of more than 85,000 youth participants (based on a 95-question survey). This responds to the need for reliable data on youth development and risk factors to support evidence-based planning. This need was underlined by the UNDP regional report mentioned above and informants in the DR. However, the database is not yet fully functional, and options for its sustainability and accessibility are still being explored, a challenge further explained under Question 2. The Community Justice Houses (CJH) are an example of multi-stakeholder collaboration in the form of a civil society organization (CSO) managing an initiative involving various national and local government agencies in ongoing service delivery and cost sharing, which the assessment team considers unusual. Focused in conflict-prone vulnerable neighborhoods, the eight existing facilities offer a combination of alternative dispute resolution (ADR), facilitation of access to the formal justice sector, and public information services. Participación Ciudadana (PC) launched the model in 2005 without a definitive link to any other model, although the leadership was reportedly aware of other experiences in Latin America. The team has identified neither a closely comparable access-to-justice model in the region nor any specific guiding principles in the region for this type of intervention. The Colombia “Justice Houses” experience was mentioned as relevant by several informants, who indicated that USAID has supported periodic exchanges with the Colombia initiative. Although USAID support to these initiatives in both countries has been significant over the years, the latest report of the activity responsible for recent USAID support to the Colombia houses21 shows that there are fundamental differences—notably the government-run nature of the Colombia houses. The report also 19 Ibid., p. 3. 20 Alerta Joven Project Annual Report 2014–15, p. 11. 21 USAID/Colombia Access to Justice Activity, Third Annual Technical Report, October 2015. The team also reviewed the 2015 Mid-term Evaluation of the Access to Justice Activity but, unfortunately, the evaluation scope did not include the Justice Houses, as USAID had already decided to phase out direct support to the Houses. 18 describes strategies and tools that could be useful in the DR, if they are seen to be effective. For example, Colombia houses are transitioning away from USAID support by developing a model for public-private alliances to support new houses. In addition, the activity has devised a tool to periodically assess the sustainability of 24 houses and a web-based information system. An independent evaluation of the CJH (covering 2012–2015) in the DR has just been completed, and a draft report was shared with the assessment team; it contains both quantitative and qualitative data on the model. Overall, the evaluation concluded that: CJH services are in increasingly high demand from the public; the proportion of cases settled via mediation is impressive (just over 50%); and user satisfaction levels are very high (even among the 60% of users who say that their problem recurred after initial resolution). FGDs and KIIs by this assessment generally corroborated those findings. The principal issue raised by the evaluators, and also by this assessment, was sustainability; that challenge is discussed under Assessment Question 2. The evaluation draft report indicated that beneficiaries of the CJH were almost equally divided among males and females; it found no significant gender-related differences in levels of satisfaction with services. Both the CJSSP and Police Reform Project are still in their startup stages, having been funded in mid-2015. As a result, this assessment did not prioritize these two activities; the best practices analysis was of necessity focused on the design of these interventions, and informants were largely confined to USAID and the implementers. With respect to CJSSP, implementer staff and the first year work plan indicated that activity plans have drawn upon capacity-building guidelines published by USAID, notably the USAID Human and Institutional Capacity Development Handbook,22 as well as documents describing successful interventions in prosecutorial capacity building in Latin America and the Caribbean. The implementer staff demonstrated awareness of several relevant experiences in Mexico and Colombia, which they considered as useful sources of ideas that could be adapted to the DR, but none were specifically referenced in the design documents or reports. The assessment team did not identify any specific documentation of best practices in the region in the area of prosecutorial effectiveness. Regarding the Police Reform Project, informants within USAID, the implementer, and the Dominican National Police (DNP) all described a heavy reliance on experience with and expertise in the Colombia national police force, since the situation in the DR today is similar to that of Colombia 20 years ago. Both were originally based on a “military model”, and Colombia is seen as a successful example of transition to civilian policing practices. The implementer, the International Criminal Investigative Training Assistance Program (ICITAP), is familiar with U.S.-based practices and plans to adapt the Police Officer Standardized Trainings (POST) developed in the U.S. to improve policing quality and consistency. ICITAP also relies on the International Association of Directors of Law Enforcement Standards and Training for guidance in designing training activities. Generally with respect to police reform, informants and document review revealed that community￾based policing (CBP) is not well entrenched in the DR. Although CBP is not a “cure-all” for problems in law enforcement, it has been heavily promoted by USAID and other USG agencies in order to engage citizens in the security of their communities and break down barriers between citizens and police. The commitment of DNP leadership to community-based approaches has been variable, according to interviewees familiar with the force, and a review of the new Strategic Plan (developed by the DNP with 22 USAID Human and Institutional Capacity Development Handbook, 2010. 19 significant support from Institutional Capacity and Transparency for Police Reform) revealed no evidence of any change in that respect. The 2012 UNDP Caribbean Human Development Report states: “Caribbean countries suffer from unreformed police structures. Despite progress in some nations, the predominant policing model is still focused on state security, not citizen security. The transition to citizen security requires institutional reforms within police forces and changes in police work and attitudes so that community-based policing becomes the norm.”23 Furthermore, according to USAID’s own Field Guide on Assistance to Civilian Law Enforcement: “Successful implementation of community-based policing requires an organization-wide commitment to this philosophy and involves a profound transformation from traditional civilian police values, organizations, attitudes, policies and approaches to a new style/culture of law enforcement management and operations.”24 Two informants mentioned the expectation that the proposed legislative reform of the police force, pending since at least 2014, would ultimately foster moves towards entrenchment of CBP in the DR. (Note: the Police Organic Law was passed in July of 2016, shortly after the team completed data collection for this assessment.) Eastern and Southern Caribbean The design of the Juvenile Justice Reform Project (JJRP) was largely based on findings of the 2011 Juvenile Justice Assessment.25 That report emphasized specific areas of concern in nine countries, including those in the Eastern Caribbean, Trinidad and Tobago, Guyana, and Jamaica, along with region￾wide concerns. Some recommendations adopted by JJRP include funding the development of a basic comprehensive case-management system that could be tailored to each country and encouraging the use of mediation and other forms of diversion for juvenile cases. Consistent with the medium- to long-term objectives from the Juvenile Justice Assessment, JJRP funded work with regional governments to eliminate institutional barriers to diversion, shift the focus of how countries treat juvenile offenders to have a more restorative and rehabilitative focus in general, and make improvements in juvenile rehabilitation facilities.26 Aspects such as a greater focus on rehabilitation and the use of diversion programs is consistent with best practices that have come out of other regions, such as Central America.27 JJRP is also congruent with the Eastern and Southern Caribbean Youth Assessment (ESCYA), which highlighted the need to address issues of juvenile justice in the region, as they have a negative impact on youth development—and, by extension, economic growth, social cohesion, and stability.28 JJRP was designed to include both short- and longer-term interventions, which are recognized as a characteristic of successful citizen security programming.29 In addition, JJRP offers training for those who work at some level with youth who are in the juvenile justice system. Such trainings were highlighted as a key need in the region by the 2011 Juvenile Justice Assessment. Based on interviews, both participants and policymakers within host country governments saw the JJRP trainings as extremely useful. Informants particularly commented on the use of experts 23 UNDP Caribbean Human Development Report, UNDP, 2012, p. 114. 24 USAID, Field Guide for USAID Democracy and Governance Officers: Assistance to Civilian Law Enforcement in Developing Countries, 2011, p. 27. 25 USAID/Barbados, CBSI Juvenile Justice Assessment, Final Report, June 2011, p. 45. 26 Ibid., p. 45. 27 Interpeace, Nine Strategies to Prevent Youth Violence in Central America, 2012, p. 16. 28 ESC Youth Assessment, 2013, p. vii. 29 Draft USAID Crime and Violence Prevention Field Guide, p. 9. 20 from the region as a way to ensure training is contextually relevant and to help participants from different countries share resources. One USAID informant noted that the Strengthening Second Chance Education concept emerged from a request from OECS and the Caribbean Examinations Council (CXC), when they realized that there was a need for training and certification for youth who left school with little or no formal qualifications. This rationale is supported by the ESCYA, which noted access to education and training as a challenge in the region.30 Although the activity focused on a critical need, numerous implementation challenges prevented both training organizations and targeted youth from fully participating. The assessment team has identified neither a comparable model in the region nor any specific guiding principles for this specific type of intervention. The original design of Kari Yu! was based on previous activities in the country and region. Key informants from USAID/ESC noted that the Skills and Knowledge for Youth Employment (SKYE) experience in Guyana influenced the development of Kari Yu!. In addition, the Cooperative Agreement for the activity noted that the implementing partner would take into consideration promising practices from programs such as A Ganar.31 In addition, the implementer researched needs during implementation, notably through several assessments in its first phase, which examined at-risk youth, school dropouts, a labor market assessment, and youth in conflict with the law. These studies provided information on the country context, on related interventions in Suriname, and on current stakeholder capacity; and were particularly relevant given the programming’s partnership with host governments. In addition, the assessments provided recommendations for improvements to current programming, which were primarily aimed at the government of Suriname, but some were also adopted by the implementer to guide Kari Yu!. Although the 2011 Juvenile Justice Assessment did not include Suriname, in part due to its status as being ahead of most countries in the region with respect to treatment of youth in conflict with the law, the general recommendations mentioned above in relation to other Caribbean countries may still be considered relevant to the Suriname context. Examples include support for job skills training, mentorship, and employment assistance programs, particularly for older youth, all of which were reflected by Kari Yu!. One element of Kari Yu! that USAID informants considered good practice was the development of partnerships with reputable local organizations and the private sector, which were seen as key to activity results. The implementer (Pan American Development Foundation, or PADF) involved local CSOs with a track record of working with the target population, and the team specifically made efforts to involve the private and public sectors. These methods are consistent with regional good practices in juvenile justice defined by UNICEF, which include the involvement of civil society and the private sector.32 The ESCYA noted that the challenge of reintegrating youth offenders into society requires work with the ex-offender as well as his or her family.33 The involvement of parents to understand and reiterate the lessons being taught to youth is important, and ESCYA recommends training parents and guardians 30 Ibid., p. vi. 31 USAID, Cooperative Agreement No. AID-538-A-13-00001, p. 29. 32 UNICEF, Juvenile Justice Systems: Good Practices in Latin America, 2003, p. 7. 33 ESC Youth Assessment, p. 86. 21 to better support their children. Several interviewees observed that Kari Yu! does not focus on parental involvement and expressed concern that some gains of the activity were lost once youth returned to the communities. The Trinidad and Tobago Juvenile Court Project is working in close collaboration with the judicial authorities to establish a new juvenile court, promote use of rehabilitation strategies, and train judicial personnel on how to deal with juvenile offenders. All of these aspects are in line with the Juvenile Justice Assessment, which recommended capacity building with courts; the use of community service, mediation, and other forms of diversion; specialized training for judges and court personnel; and work with governments to eliminate barriers to diversion. More generally, the transition towards rehabilitative and restorative practices and away from punitive models of juvenile justice is consistent with the Juvenile Justice Assessment. The implementer (UNDP) indicated that the assessment has helped to inform its activities, while it also refers regularly to the UNDP Caribbean Human Development Report for guidance on relevant practices. The activity objectives are aligned with the ESCYA, which found that the country does not have sufficient capacity in the juvenile justice system and does not differentiate between violent offenders, youth held for minor transgressions, and young victims of neglect and abuse. The new juvenile court and other activities will aim to strengthen capacities and systems to treat cases with measures that are more tailored to the situation. Informants mentioned that a youth offender risk assessment had been introduced by the activity: the Youth Level of Service/Case Management Inventory (YLS/CMI) tool is considered a staple of the juvenile justice field in the U.S. According to the Risk Assessment in Juvenile Justice: A Guidebook for Implementation,34 “Many jurisdictions, juvenile justice agencies, and practitioners have adopted risk assessment as a part of their practice… Juvenile court decision-makers often must decide whether youth need certain interventions to reduce the risk of harm to others. Risk assessment can assist with these decisions.” The Guidebook does not recommend any specific tool but mentions YLS/CMI as one of the resources most backed by research. The Skills and Knowledge for Youth Employment (SKYE) activity aimed to achieve its objectives in three main ways: (1) working with the judiciary and government to develop alternatives to detention for youth and rehabilitation services; (2) providing formal training and mentorship for at-risk youth, designed to improve life and employment skills; and (3) training youth in entrepreneurship and providing support to start businesses. These objectives are congruent with priorities highlighted in assessments commissioned by USAID: the ESCYA found that the two most pressing challenges for Guyanese youth related to the lack of economic prospects and inadequate education and training systems,35 and the Juvenile Justice Assessment noted that the priority needs for Guyana included supporting the reform of the juvenile justice system within the country to have a more rehabilitative focus.36 USAID has supported the implementer to incorporate research into SKYE, including assessments in both phases to inform its ongoing work. Phase I included an assessment of the context impacting at-risk youth37, while Phase II included a labor market survey to identify job sectors and potential employers to target. In addition, a midterm evaluation was carried out in 2014 to assess whether or not the activity 34 Gina M. Vincent, Laura S. Guy, and Thomas Grisso, Risk Assessment in Juvenile Justice: A Guidebook for Implementation, November 2012. Available at http://www.modelsforchange.net/publications/346 35 ESC Youth Assessment, pp. 65–66. 36 USAID/Barbados Juvenile Justice Assessment, p. 34. 37 USAID SKYE in Guyana Assessment Report and Program Design, 2011. 22 was on target to meet objectives and make recommendations for the remaining period of implementation. Review of the evaluation report and subsequent activity reports indicates that the implementer made adjustments to respond to those recommendations. The A Ganar youth development concept was originally designed and implemented in 2009 by Partners of the Americas in Brazil, Uruguay, and Ecuador as a broad-based, youth-focused, workforce development approach. The model was considered successful in those countries, which led to the decision to expand to additional countries, including some of those now within the mandate of the USAID/ESC Mission.38 Although the Caribbean programming predated the creation of CBSI, the workforce development focus was adapted to meet the needs of the CBSI mandate. Supporting this idea, the 2011 Juvenile Justice Assessment identified A Ganar as an example of an activity that had a solid foundation in providing individualized care and could be adapted to be effective juvenile justice programming. 39 Research from organizations outside of the USG also highlight how sports can be used in the context of juvenile justice programming. In Nine Strategies to Prevent Youth Violence in Latin America, the authors highlight the use of sports and recreation to promote youth development and decrease violence.40 It should be noted that the model continues to be replicated; there is now programming in Barbados and Trinidad and Tobago that uses the A Ganar methodology with funding from other donors.41,42 The Caribbean Youth Empowerment Program focused on using workforce development programs to reach vulnerable youth. Although it began implementation well before the creation of CBSI, the activity was adapted to have a greater focus on at-risk youth in order to help meet the mandate of CBSI. Although the activity also predated the Juvenile Justice Assessment, its objectives were in line with that report’s recommendations to support job skills training, mentorship, and employment assistance programs, particularly for older youth. The implementer primarily used a methodology that it had developed in other countries known as “Entra21.” The midterm evaluation of A Ganar and the Caribbean Youth Empowerment Program (CYEP) found that both activities had achieved positive results in various areas including life skills, graduation rates, and employment status and did not identify any major areas of concern in relation to their approaches or methods. General ESC Findings. Currently there is a shortage of (1) reliable data on crime and justice and (2) research into the causes and implications of crime. Several informants attributed this to a lack of capacity to accurately collect and track this information while others noted concerns by regional governments that such data may produce negative perceptions and harm tourism if made public. The ESC Mission has strived to increase this body of knowledge as a base for CBSI and other programming, for example, by funding the ESCYA in 2013 to inform its new strategy and current research analyzing 38 A Ganar, Monitoring and Evaluation Plan, p. 1. 39 USAID/Barbados Juvenile Justice Assessment, p. 11. 40 Interpeace, Nine Strategies to Prevent Youth Violence in Central America, p. 12. 41 Currently, A Ganar is part of a larger five-year longitudinal randomized control trial impact evaluation being carried out in Honduras and Guatemala by Social Impact, which should provide further evidence in relation to the effectiveness of activity methods. To date, baseline and mid-line data for each country have been collected. End line research is scheduled for 2016. (Source: Ibid., p. 6.) 42 USAID, The A Ganar Alliance: Using Sport to Impact Youth Employment and Youth Engagement, Final Report, September 2009 – September 2015, p. 13. 23 “troublesome youth” in the region. This new study will help inform USAID programming and serve as a resource for governments and other donors. Since the creation of CBSI, there has been an evolution in international understanding of the factors that promote and prevent youth involvement in crime and violence. Recent analysis supports the idea that for crime prevention programming to be effective, the portfolio should be cross-sectoral.43 The early CBSI activities in the ESC focused predominantly on workforce development, but the Mission now demonstrates an explicit focus on juvenile justice reform and workforce development specifically for at￾risk youth, as exemplified by Kari Yu! and JJRP. In the Regional Development Cooperation Strategy 2015–2016 (Public Version), the Mission noted that it would be taking a “more holistic approach,” focusing not just on workforce development and juvenile justice. The document goes on to note that “USAID must also address the environment in which these youths live, including their families, the broader community, and the educational system that affect the opportunities youth have.” 44 USAID/ESC staff have indicated in interviews their plans to more intentionally incorporate a “public health approach” in upcoming programming.45 This approach calls for identification of risk and protective factors leading to crime and violence and for interventions to target these factors across primary, secondary, and tertiary levels (see Box 1 for more information). The current USAID/ESC portfolio incorporates all three of these levels; Mission staff indicated that interventions have been identified for each country based on local priorities and availability of funds.46 Jamaica In the final evaluation of COMET47, the predecessor to COMET II, the report applauded the activity’s efforts in assisting the Jamaica Constabulary Force (JCF) to operationalize the concept of CBP. In addition, an Overseas Development Institute (ODI) report described COMET’s efforts as “a valuable example of a community policing program which has been a formal state-led process, but has taken place in a security and justice arena that has received significant support from multiple donors.”48 COMET II has continued efforts to support implementation of CBP across the country. Community-police relations remain central to the activity’s activities, including the dedication of a local police office at all Community Resource Centers (CRCs). The intention is to encourage closer community-police ties by sharing premises. 43 Draft USAID Crime and Violence Prevention Field Guide, 2016; UNICEF, Juvenile Justice Systems: Good Practices in Latin America, 2003. 44 USAID, USAID ESC Regional Development Cooperation Strategy 2015–2019 (Public Version), 2015, p. 18. 45 Under this approach, crime and violence are treated as one would an infectious disease, which should be contained and prevented through tailored interventions based on risk and protective factors. (Draft USAID Crime and Violence Prevention Field Guide.) 46 For example, JJRP contains both a focus on changing attitudes towards how societies treat juvenile offenders and reforms to juvenile justice legislation (primary), as well as capacity building for those who work with juveniles in conflict with the law and physical upgrades to detention facilitates (tertiary). Examples of secondary preventions include Strengthening Second Chance Education and Kari Yu!, which have targeted at-risk youth. 47 IBTCI, USAID/Jamaica Community Empowerment and Transformation (COMET) Program Final Evaluation, December 2012. 48 Overseas Development Institute, ODI Securing Communities and Transforming Policing Cultures: A Desk Study of Community Policing in Jamaica, May 2014, p. vi. 24 The COMET evaluation highlighted the need for greater engagement of civil society to increase buy-in and facilitate the adoption of CBP. “COMET’s methods and approaches [had not had] much success in increasing citizen participation in community security or in building sustainable partnerships between police and local communities through working with civil society.” 49 As a result, COMET II has deepened its focus on engaging communities and national stakeholders and responded to capacity gaps of local non￾governmental organizations (NGOs) and community-based organizations (CBOs) by introducing grants application and management trainings. COMET II has built a strong reputation as an inclusive organization, particularly in its work with the LGBTI community and at-risk youth (although one informant alluded to the appearance of prioritizing LGBTI concerns over other populations). While gender was not singled out as a main factor in activity design and implementation, an informant mentioned that COMET II strives to include women in the construction and design of CRCs. The USAID Field Guide for Assistance to Civilian Law Enforcement50 points to the need for organization-wide commitment to a change in culture of law enforcement, values, attitudes, policies, and structure in order to successfully implement CBP. The abovementioned ODI report noted that while CBP has received senior-level support from stakeholders, support within the JCF has been inconsistent, with the force’s Community Safety and Security Branch taking most interest and responsibility in implementing CBP.51 It went on to say that efforts in Jamaica are driven too much by external actors and donor funding cycles, leading to questions about internal buy-in and sustainability. A USAID assessment in 2008 found that Jamaica was covered in a “blanket of corruption” but also indicated real opportunities to lift the blanket due to the presence of a number of anti-corruption champions in key public-sector positions.”52 Combatting Corruption in Jamaica (CCJ), implemented by National Integrity Action (NIA) Limited, was designed in response to that assessment’s findings and recommendations, in particular to leverage the presence of anti-corruption champions throughout the public sector and unite them to move the needle on corruption in the country. USAID’s Practical Guidance on Anticorruption Programming advocates a multipronged, multisector, and whole of government approach to anti-corruption. It outlines five phases to adopt when planning anti-corruption activities: (1) assessment of points of corruption vulnerabilities, (2) definition and prioritization of goals and strategies, (3) selection of entry points for anti-corruption initiatives, (4) practical and appropriate programming, and (5) monitoring and evaluation.53 In general, the CCJ activity was consistent with these guidelines in designing its activities. In its work, CCJ has focused on engaging high-level government and public-sector stakeholders in efforts to tackle corruption issues. However, activity documents and informants suggest that more could have been done to engage with current or potential victims of corruption, in particular with youth. This was underlined in the recent CCJ final evaluation: “One of the more unexpected and repeated findings was how frequently–and from what surprising sources–came the recommendation: If you want to do something meaningful in the future–focus on civic education, reach out to young people in schools, cultural and sports 49 COMET Final Evaluation, p. vii. 50 USAID Field Guide for USAID Democracy and Governance Officers: Assistance to Civilian Law Enforcement in Developing Countries, January 2011. 51 ODI report, 2014. 52 Management Systems International, USAID Corruption Assessment for Jamaica, September 2008, p. 28. 53 Management Systems International, USAID Practitioner’s Guide for Anticorruption Programming, January 2015. 25 associations.”54 As a result, NIA has now started targeting youth in their follow-on activities through associate programs at secondary schools and universities. USAID guidance on anti-corruption programming advocates for the involvement of the justice sector in addressing the problem, particularly in providing protection to citizens to report on wrongdoings.55 CCJ worked to build the capacity of Jamaica’s justice sector to serve this role by providing training on new legislation to the judiciary. Interviewees spoke highly of those trainings, while noting that training judges is the responsibility of the Justice Training Institute.56 Activity documents and interviews indicate strong alignment between the design and implementation of the A New Path (ANP) activity and the 2015 needs assessment57 of South Camp and Metcalfe Juvenile Correctional Centers. That assessment made a number of recommendations to better meet the needs of juveniles in government custody at these centers, including:  Increase focus on training staff to deal with psychological and emotional needs of juveniles  Increase the number of social workers and psychologists  Leverage graduate training programs in social work and clinical or counseling psychology  Involve and train parents and/or guardians to better support reintegration and rehabilitation of juveniles  Identify community programs to connect juveniles to once they leave centers  Develop an electronic case-management system to track and utilize information about wards Specifically, ANP has hired a Lead Social Worker to work with staff and girls to provide social, emotional, and rehabilitation support. However, informants emphasized the difficulties in identifying more qualified social workers to join the activity, as well as challenges in hiring a full-time psychologist for the team. ANP has mitigated this challenge by using graduate students from Social Work and Clinical/Counseling Psychology programs at South Camp as part of their training, which was described by informants as a “win-win situation.” In addition, to support the reintegration of juveniles into their home communities, ANP has started engaging parents and guardians while girls and boys are still in correctional centers. ANP chose to focus the majority of its work in this component of the activity on females, having noted that other actors were providing support to facilities catering to juvenile males. Aside from activity documents, the mention of the involvement of boys in ANP’s activity was starkly missing in interviews with informants, except for one. One of ANP’s objectives is to develop an electronic case-management system to track girls and boys from South Camp and Metcalfe from entry to graduation and beyond. An electronic case-management system has been recognized as a priority by both the regional Juvenile Justice Assessment of 201158 and 54 Dexis Consulting and Management Systems International, USAID/Jamaica Final Report of Combatting Corruption in Jamaica Final Performance Evaluation, October 2015, p. 38. 55 USAID Practitioner’s Guide on Anticorruption Programming. 56 The Institute is not operational due to lack of funding from the Government of Jamaica. 57 Rosemarie Johnson, Needs Assessment of South Camp Juvenile Correctional and Remand Centre and Metcalfe Street Juvenile Centre, April 2015. 58 USAID/Barbados Juvenile Justice Assessment. 26 the 2015 needs assessment in Jamaica to enable better tailoring of services to individual juveniles and facilitate aggregation of data. With over 150 CSOs active in reintegration of juveniles into communities in Jamaica, the latter assessment noted that activities are bound to be disconnected and duplicative in the absence of an integrated national system for managing cases.59 The hopes are that this system will eventually serve the entire juvenile care system of Jamaica. It may also present an opportunity to develop a standard software package that can be tailored to the needs of other countries in the region, according to the Juvenile Justice Assessment. The Child Development Agency (CDA) is working towards a similar system with other donor funding, and informants underlined the need for compatibility of such systems. KIIs and a review of activity documents indicate that Empowering Jamaica’s Youth implemented by Junior Achievement (JA) does adopt elements of best practices, particularly by drawing from the training models and curriculum of JA Worldwide.60 JA’s partnership with the Ministry of Education (MoE) has led to an expansion of services across the country, especially of JA’s Company of Entrepreneurship. Sustained support from the private sector, both strategically as board members and also financially as donors, has also been instrumental in their success. One of the more visible outcomes of this positive public-private partnership is the establishment of JA Biztown in Kingston as a space for students to apply their business, financial, and entrepreneurial knowledge and skills learned in classrooms. JA has made a significant effort towards being inclusive by proactively engaging with a range of populations who are of interest to USAID; it adapted select curricula into Jamaican Sign Language to make them accessible to the deaf and hard of hearing, and it targeted students with disabilities by implementing Career Week, a one-week personal development training program. Top graduates of this training have successfully been placed in apprenticeship programs. JA also partners with ANP and corrections authorities to deliver training to girls and boys in conflict with the law. Recently, JA Worldwide passed responsibility for continued implementation of select JA programs to JA Jamaica. Transition of management of the activities to a local chapter of the organization has been recognized by informants, as well as espoused by the principles of USAID Forward, as key to sustainability. As part of the objectives of USAID/MoE Partnership for Improved Reading Outcomes, a government-to-government initiative, manuals to facilitate the training of teachers and effective implementation of the national literacy program were developed. The activity aimed to facilitate free access to all literacy resources developed under the previous MoE and USAID Basic Education Project. For example, the activity produced a Gender Manual and Monitoring and Evaluation Toolkit and delivered both to the MoE to enable continuation of efforts and ownership by the government to improve literacy outcomes. The Gender Manual aimed in part to close the gender gap in literacy attainment; it has received the full endorsement of the MoE and been adopted for use by the National College for Educational Leadership, which is responsible for training school principals. Fi Wi Jamaica was recently launched to promote LGBTI-inclusivity efforts through the promotion of rights and empowerment and to reduce gender-based violence and mitigate its harmful effects on women and girls and communities. The activity is modeled on earlier experiences from UTech Cares, an 59 Needs Assessment of South Camp, 2015. 60 Empowering Jamaica’s Youth Project Completion Report, November 2015. 27 initiative that started in response to the assault of a university student based on his perceived sexual orientation. ASSESSMENT QUESTION 1 CONCLUSIONS Dominican Republic The holistic design of the Alerta Joven activity by USAID was based on findings and recommendations produced by in-depth independent research, though over time some types of assistance to youth have been emphasized more than others. The activity has capitalized on approaches with a strong track record in the DR, particularly in education, and prioritized the use of reputable existing training institutions rather than creation of new activities. Generally, the activity is consistent with principles defined by international agencies such as UNDP and Interpeace, as well as USAID, for reducing the risk of youth engaging in violence and crime. The activity has made a commendable commitment to not only monitor its own interventions but also to establish a database that could be valuable to USAID and others as they analyze the problems of youth and develop policies, strategies, and interventions. This has the potential to be an important legacy of the activity, although the lack of a clear vision of the future use or users of this resource poses a significant risk to sustainability of this investment. Based on the evidence so far, the assessment considers that the CJH represent a highly promising practice that could be a model for other countries and be further replicated successfully in the DR. The current effort of systematization will be immensely helpful in describing the details of the model and serving as a guide for future Houses. There is still little data on the longer-term effects of CJH interventions on conflict and security in the targeted communities, including what happens to the many disputes that do not “stay resolved” and to users sent towards a court system with limited capacity. CJH staff and Alerta Joven partners working at the grassroots level demonstrated limited awareness of the needs of populations such as disabled and LGBTI individuals and how to optimize assistance to such individuals. This may be partially due to the highly sensitive nature of the LGBTI issue in the DR, though that was not mentioned by informants. With respect to CJSSP and the Police Reform Project, it appears that both of these new activities have identified relevant guidelines and models on which to base their activities, including considerable emphasis on experience in Colombia and the U.S. It remains to be seen to what extent they are able to incorporate and adapt those practices to the country and activity context. Examples and experience from other Caribbean countries seem to be of less relevance to the implementers and their national counterparts in the DR, probably due to language barriers as well as differing legal systems. Although not mentioned by informants, these differences would also hinder access by the English-speaking Caribbean to information about effective practices in the DR (and Latin America more broadly). Eastern and Southern Caribbean The concept of JJRP was based on solid research from earlier USAID assessments and research report. Assessments and evaluations were informative in the design of JJRP and will be continue to be useful for future programming in the countries within USAID/ESC. Although the local context is different, document review suggests that the focus and strategies that USAID/ESC is pursuing are in line with best practices that are being implemented in other regions. Based on a wide array of KIIs, the trainings that used specialists from the region and brought together people from different countries who work directly with at-risk youth produced positive results and were extremely well received. Strengthening Second Chance Education experienced significant implementation challenges related to the differences in the design as envisaged by CXC and the expectations of its partners – which may have been, at least in part, due to the lack of similar interventions or best practices upon which to base the design. While current literature and key informants suggest that the need for 28 certification programs for out-of-school youth within the region remains, this activity has, at least in the eyes of many respondents, been unable to adequately meet that need. The detailed design for Kari Yu! was based on early assessments that were included as part of the contract and learning from earlier USAID programming. The knowledge generated from the assessments was useful in guiding the implementing partner and increasing the body of knowledge about factors that contribute to youth being at risk in Suriname. The Trinidad and Tobago Juvenile Court Project is making a concerted effort to address some shortcomings identified by the 2013 ESCYA—notably supporting the creation of juvenile courts in both Trinidad and Tobago and public education about new laws related to juvenile delinquency and importance of rehabilitative approaches. The activity is using and adapting methodologies and tools that have been proven effective in other contexts. The SKYE activity focused on working with the judiciary and government to develop alternatives to detention for youth and rehabilitation services, incorporating job skills training and mentorship, and assisting youth with obtaining employment, which were all highlighted as needs in the 2011 juvenile justice report. The A Ganar model has been replicated in multiple countries with varying degrees of success. Although it needs to be adapted to local realities, the basic model has proven effective, as confirmed by evaluations of ESC iterations and those implemented in other locations. The CYEP also had reasonably positive outcomes in workforce development, but it was not clear to what extent it relied on best practices in citizen security. USAID/ESC’s shift towards a public health model that incorporates both short- and longer-term programming and takes into account the three levels of interventions is positive. The results from the Troublesome Youth study, which will contain specific information on each of the USAID/ESC countries, crime statistics, and other information obtained from local police forces, will be used to target at-risk youth in future interventions. While there is minimal specific targeting of disabled and LGBTI individuals under USAID/ESC’s current portfolio, there was also no evidence of active discrimination by the activity. Jamaica Overall, findings suggest that many elements of CBSI activites in Jamaica are guided by recommendations from published guidelines, assessments, and evaluations. However, this does not tend to happen systematically or intentionally but more often as a result of learning from experience or slightly revising previous activities. Jamaica is one of the few CBSI countries that was implementing citizen security activities prior to the start of CBSI, which means that the Mission and some implementers have more thematic experience compared to other countries in the region. The assessment data indicates that the key elements of COMET II reflect best practices in the region, especially in the areas of promoting CBP with the police, engaging communities in CBP through consultations, and working with vulnerable groups. In accordance with COMET I’s evaluation, COMET II has made a concerted effort to engage communities in its activities. Recognizing the importance of active community groups to partner in its efforts, COMET II is working to analyze needs and build the capacity of CBOs and community leaders by providing grants application and management training. Informants commended this change of approach. However, continued reliance on external donor funding can threaten sustainability of CBP and COMET II’s results. Exclusive reliance on the Community Safety and Security Branch as the main champion of CBP can limit force- and country-wide internalization of community-based practices and the move away from JCF’s paramilitary approach. NIA’s recent shift to engaging with youth is aligned with the findings of CCJ’s recent evaluation. While the implementer’s success in bringing attention to corruption issues has been due in large part to its ability to engage with high-level stakeholders in the government and public sector, it has not done as 29 much to engage and raise awareness of the broader public. Through CCJ, NIA has effectively engaged the justice sector on anti-corruption issues by serving as trainer on related legislation. However, it is not a long-term solution for a CSO to act as sole trainer of judicial officials. ANP has been strategically designed to address specific limitations in meeting the rehabilitation and reintegration needs of female juveniles at South Camp and male juveniles from Metcalfe facility. Its work is based on a thorough assessment and is guided by a multi-stakeholder steering committee, which are both promising indicators of its ongoing reliance on evidence and best practices. Empowering Jamaica’s Youth (JA) relies heavily on methods developed by JA Worldwide over the years in various countries, which have proven effective in Jamaica. Their achievement is closely tied to strong government and private-sector support and outreach to target diverse vulnerable groups. Furthermore, the intentional capacity building and support of JA Jamaica built into the activity by JA Worldwide ensured transition to local ownership at the end of the activity. Based on the assessment team’s limited inquiry, there is little evidence that the Partnership for Improved Reading Outcomes and Fi Wi Jamaica activities were developed based on best practices, beyond specific elements drawn from prior activity implementation. Promising components include (1) the effort by Partnership for Improved Reading Outcomes to promote continued improvement in literacy outcomes by developing materials and resources for MoE to adopt and (2) the decision by Fi Wi Jamaica to leverage the reputation of other local organizations to successfully implement activities targeting socially marginalized groups. Cross-Cutting Findings and Conclusions Evidence Base Although the evidence base is expanding (in part thanks to CBSI), rigorous documentation and independent verification of effectiveness are still scarce when it comes to citizen security activites in the Caribbean. The need for reliable data on youth development and risk factors in particular was underlined by the UNDP in 2012: “It is perhaps too early in many cases to assess fully the success of various programmes implemented in the Caribbean, especially given the lack of data.”61 The report goes on to refer to “…the limited regional capacity for systematic data collection on youth to support evidence-based planning and programme design.” Models and analysis of interventions in other regions and general guidelines based on global experience can be very useful to inform activity design and implementation; however, informants in the targeted countries emphasized repeatedly that even successful experiences from neighboring nations in the Caribbean had to be carefully studied and usually adapted to the local context. For this reason, it is important to invest in timely evaluations and other research, both linked to specific activities and cross￾cutting studies, to continue building the knowledge base of what works best in each country or sub￾region of the Caribbean. Some of the USAID Missions activities have clearly recognized this need and commissioned research to inform activity strategy, either at the design stage or early in implementation. Out of the 23 CBSI activities included in this assessment, 12 have been completed; three of those had a 61 UNDP Caribbean Human Development Report, 2012, p. 57. 30 final evaluation and two had midterm evaluations. 62 Of the 11 current activities, only SKYE had had a midterm evaluation as of March 2016. The USAID Missions in the region and CBSI implementers have adopted a highly varied approach to the use of international/external models or practices identified as successful. In most cases, the representatives of USAID staff and implementers who were interviewed for the assessment referred only to past programming in the same country/region when asked to identify models or best practices related to citizen security and the specific sector of their work. Similarly, activity design documents rarely referenced any external evidence or guidelines. USAID itself has published guidance and evaluations on various subjects relevant to some CBSI activities (as noted in the individual sections above and the matrix at Annex VI), but documents of that nature were rarely mentioned by informants or activity documents, even for those which were designed following the dissemination of those documents. While other activities were designed prior to the distribution of these practices, neither design documents nor key informants referenced other (prior) models or best practices. Therefore, while at least some correlations described above between CBSI activities and best practice sources were presumably due to the designers/implementers being aware of successful models, it is difficult to reach that conclusion with any certainty. Nevertheless, it is important to recognize that the assessment found no evidence of any major discrepancies or contradictions between CBSI activity design and implementation and the best practices identified by the team. Sharing of Experience Although not directly within the scope of work for the assessment, the team made some observations in relation to the sharing of best and promising practices and other lessons learned among the CBSI￾supported Missions and activities. These emerged from document review and comments by informants. Overall, this kind of information sharing has been fairly limited due to a number of obstacles, notably including geographical divisions that constrain in-person contacts and language barriers (especially related to the DR and Suriname). However, one of the greatest challenges has been posed by the significant differences in subject matter and target groups (among others) among the individual activities supported by each Mission; in many cases, they seemed to have very little in common beyond an overarching theme of citizen security. Inevitably, that has presented difficulties for the identification of practical information and tools that have relevance across multiple activities. One mechanism established to foster information sharing on these topics is the Technical Working Group (TWG) on Preventing Crime by Focusing on At-Risk Youth and Vulnerable Populations hosted by USAID on an annual basis in collaboration with CARICOM. Participants include Caribbean governments, CBSI-supported USG agencies (including USAID and DoS), thematic experts, and implementing partners. Among other diverse objectives described in its Terms of Reference, this TWG aims to facilitate exchange of information, sharing of best practices, and development of common approaches and new partnerships between the U.S. and the Caribbean.63 It should be noted that there are other TWGs led by other U.S. CBSI implementing agencies, but informants at USAID/LAC indicated that the other TWGs operate on a more ad hoc basis and do not have the same level of cross-fertilizations as the 62 The assessment team viewed reports of final evaluations of COMET I and Combating Corruption in Jamaica, while evaluation of the predecessor activity to the current funding of Community Justice Houses was underway at the time of this assessment. A combined midterm evaluation of A Ganar and CYEP was done in 2013. 63 Terms of Reference, CBSI Technical Working Group (TWG) on Preventing Crime by Focusing on At-Risk Youth and Vulnerable Populations, revised 8 October 2014. 31 USAID CBSI TWGs have. Topics for those TWGs have included Anti-Corruption, Maritime and Aerial Domain Awareness, Law Enforcement Information Sharing, and Law Enforcement Cooperation and Capacity Building. The team’s enquiry into this particular aspect of CBSI was limited due to time constraints. However, it was noted that the TWG was not mentioned by any in-country informants during the assessment. This may be, at least in part, due to the difficulties encountered in ensuring consistent TWG participation by the right individuals. While not all informants had participated in TWG events, even TWG attendees did not refer to information from those meetings in response to questions about best practices and models from other countries. Thus, the extent to which TWG participants are internalizing, using, and further sharing the information from the annual meetings is unclear. One informant commented that annual meetings of short duration (three days) were not sufficient to foster meaningful exchange of experience among USAID CBSI stakeholders. The findings of the assessment team are consistent with that opinion. Interpretation support is provided at the annual meetings for those not fluent in English, but the presentations provided to the assessment team were all in English, and it was not clear to what extent they were made available in other languages. This would necessarily limit the options of TWG attendees from the DR and Suriname to use and disseminate information to stakeholders in their home countries. Public Health Approach As mentioned above, USAID/ESC is starting to more intentionally integrate the public health approach into the design and implementation of activities. The model specifies interventions across levels targeting risk and protective factors associated with crime and violence and is highly relevant to analysis of CBSI best practices. In USAID’s 2016 draft field guide on crime and violence prevention, this approach has been highlighted as a best practice: “While there is no universally accepted definition of citizen security, the following features generally characterize successful citizen security programming: …Integrates all three levels of intervention: prevention initiatives combine primary, secondary and tertiary prevention to address the full universe of potential and actual offenders and victims.” 64 In most cases, a single activity would not be suitable for coverage of all three levels, although it is not uncommon for two levels to be included. Looking forward, the assessment team considers this categorization to be of most relevance at the Mission portfolio level, where difficult choices have to be made on priorities for CBSI intervention based on limited resources. Gender, LGBTI, and Disabled Populations The assessment took gender-related issues into account as a cross-cutting topic, and informants were consistently asked about ways that both CBSI activities and the initiative overall had considered gender and the differing needs of males and females in their design and implementation. Though the importance of gender was not an acute focus of the assessment, the team found that USAID staff and implementers were very conscious of it, achieving a relative balance of beneficiaries and participants. All implementers appear to be tracking the gender identity of beneficiaries in their reporting, and several mentioned specific efforts to engage appropriately based on gender or adjust activities to suit males or females, while also recognizing that for certain types of activities (e.g., targeting incarcerated youth), equality in targeting is neither possible nor desirable. With respect to groups of interest to USAID/LAC for this assessment, such as LGBTI individuals, ethnic minorities, and the disabled, there were some notable specific efforts to engage and support these 64 Draft USAID Crime and Prevention Field Guide, p. 9. 32 persons and protect their rights. Examples include JA’s work with the disabled in Jamaica, Alerta Joven’s targeting of Haitian-descent immigrants in need of legal documentation in the DR, and Fi Wi Jamaica’s work to defend the rights of LGBTI people as a primary target group. In addition, Kari Yu! in Suriname has incorporated elements that aim to increase tolerance towards these priority groups. However, those were the exception rather than the rule for the assessed activities. While there has been minimal specific targeting (outreach or tailored approaches) of persons with disabilities and LGBTI individuals under the current portfolio, there were also no signs of active discrimination. The most common response to the team’s questioning on the subject was “Our doors are open to all,” and even after the team probed further, local partners in particular seemed unaware of the potential benefits of more proactive approaches to engaging members of marginalized population groups. Several implementers and USAID staff in the ESC emphasized the extreme sensitivity of LGBTI rights in the region, and some expressed concern that singling these groups out could do more harm than good. During data collection, key USG officials in the DR were being criticized in the media for having supported certain activities of LGBTI rights groups, including through USAID (non-CBSI) programming. On the other hand, senior USAID personnel in Jamaica opined that the efforts of USAID and others over the past five years, with at least tacit support from key government figures, had succeeded in “moving the needle” of public opinion towards greater acceptance of sexual and gender minorities. ASSESSMENT QUESTION 1 RECOMMENDATIONS Dominican Republic 1. USAID should support a rigorous independent evaluation of the Alerta Joven activity prior to its completion to inform design of any follow-on activities. That evaluation should try to isolate the effects of different combinations of activities on participating youth and families, including through use of the extensive database created by the activity, and include some analysis of the most closely related activities funded by other donors. This would provide additional evidence on effectiveness of the various preexisting models and practices incorporated by the activity, as well as models being used by other organizations, thereby supplementing and updating the findings of the major assessment that formed the basis for the activity’s design. 2. USAID and Entrena should devise a strategy to ensure appropriate accessibility of the Alerta Joven database after the activity ends, based on careful analysis of the primary uses and users of the data and consultation with the DR government. Once refined, this experience could help to inform a youth development index or similar tool at the national or regional level. 3. USAID should perform an in-depth analysis of the rich data and evidence generated by the recent evaluation of the CJH as a potential emerging best practice and invest in additional research to analyze the longer-term effects of the CJH on individuals and surrounding communities. Highlights of the evaluation should be shared with CBSI (and other USG-supported) implementers in the DR and beyond in English and Spanish, and USAID should support the ongoing exchange of lessons learned with the CJH, especially on sustainability and information management systems. Eastern and Southern Caribbean 4. Activities like Strengthening Second Chance Education that address the need for education and training for at-risk youth are still needed in the Caribbean region. To inform any future activities of this nature, USAID and implementers should first conduct a thorough needs assessment of participants and potential partner institutions to make sure that they are fully able to participate and achieve expected results with the support that can be provided by USAID. Similarly, while recognizing that the intervention is not currently a priority for future Mission programming, should future interventions work to address this issue, the team recommends delving further into Strengthening Second Chance Education to shed more light on lessons learned. 33 Jamaica 5. The implementer of COMET II should continue engaging communities using its tailored approach to local needs but with a longer time frame for startup of community engagement to account for capacity-building needs of CSOs and community members. 6. To support greater ownership in and commitment to CBP, USAID and the COMET II implementer should seek broader agency-wide support within JCF, including through collaboration with other international actors, rather than rely solely on one unit (the Community Safety and Security Branch). 7. NIA (potentially with renewed USAID support) should enhance efforts to engage community members at all levels in anti-corruption initiatives, which is in line with the recent CCJ evaluation. In follow-on activities, NIA should consider a train-the-trainer approach with the justice sector to facilitate transition of training responsibilities back to the government. Cross-Cutting Recommendations 8. USAID (LAC and regional Missions) should continue to develop the regional evidence base by conducting independent evaluations of all activities valued at a minimum of $2 million or lasting a minimum of 3 years and supporting sectoral or more specific assessments to inform activity design and/or implementation. More specifically:  Consider conducting country-level crime and violence prevention assessments, if sufficient data is not available to inform decision making on citizen security interventions. (Note: this poses specific challenges for the ESC as they cover 10 countries with limited staff)  Take into account in research design and timing that the effects of crime and violence prevention programs are often only realized several years later, and attribution of changes in actual and perceived levels of crime can be extremely difficult, even if controls are used in the study methods.  Build evaluation into the design of new programming to ensure timely availability of information for design of future interventions. 9. Actively supporting the exchange of the most relevant citizen security–related guidelines and research, starting with the best practice documents included in Annex V to make it more accessible to Mission staff and activity implementers in the region and taking into account that human resource limitations may constrain their ability to identify and/or extract the most relevant information. In addition, some staff may have had very limited exposure to programming outside that particular context. It may be helpful to prepare and circulate excerpts or summaries of certain key documents to targeted recipients and/or organize briefings by relevant technical staff of USAID as new research comes to light or is published. 10. USAID and CARICOM should consider the feasibility of information- and expertise-sharing encounters to complement the Technical Working Group annual meetings, including one-on-one interactions and facilitated exchanges (in-person and remote) between closely related activities. This would allow for implementers and/or responsible USAID staff to discuss their activities in more detail and identify concrete lessons and successful practices. (Note: this poses specific challenges for the ESC as they cover 10 countries with limited staff). 11. To further promote practical and ongoing exchange of experience and ideas, USAID and CARICOM should examine cost-effective methods to overcome the language barriers to sharing lessons and models, which affect Suriname and the DR in particular. These could include translation of excerpts or summaries of key documents to Dutch and Spanish, based on relevance to programming in the target country. This would enable valuable information to reach stakeholders 34 beyond just the immediate participants of TWG and facilitate use of information received at TWG meetings. Although the DR is not a CARICOM member, thus presents some organizational challenges, it would none-the-less be beneficial to USAID and CARICOM to explore these options. 12. Although USAID CBSI activities as a whole have to some extent been active at all three levels of crime and violence prevention outlined under the public health approach, USAID should strive to be more intentional and strategic in its decision making on this front, especially in light of potentially declining resources for the region. A balance needs to be struck that also considers needs for institutional strengthening in the security sector. USAID needs to prioritize carefully as it designs new activities and chooses target populations, including by considering four key questions:  What are the primary risk and protective factors relevant to the local context?  What kind of prevention is most appropriate to the local context: primary, secondary, and/or tertiary?  What is within the manageable interest of the program?  Are there other host country assets and/or donor focus on the area? 65 13. USAID should work with implementers to analyze and strengthen the access of LGBTI, persons with disabilities, ethnic minorities and other marginalized segments of the population as appropriated to CBSI-supported services, including through the exploration of “affirmative action” or outreach activities catering to needs of specific groups. This could include use of social media and information sharing via formal and informal civil society organizations with strong connections to the targeted population. Considerable care is needed in relation to direct targeting of LGBTI populations and groups, since discrimination levels remain severe and U.S. government support for that community is a highly sensitive topic in much of the Caribbean basin. Sharing of experiences with this type of intervention among the Missions and Embassies in the region could help to inform future actions and mitigate any negative repercussions. Assessment Question 2: Challenges and Opportunities INTRODUCTION In this section, the assessment describes the key unexpected challenges and opportunities that arose for the USAID-supported CBSI activities and the major effects of those challenges and opportunities. The team sought to identify and understand any major events or circumstances that either impeded or facilitated the implementation of the activities and/or the achievement of expected results. In accordance with the assessment question, attention was paid to both internal and external challenges and opportunities, identified through document review, KIIs, and FGDs. ASSESSMENT QUESTION 2 FINDINGS Dominican Republic With respect to Alerta Joven, one key unexpected challenge arose in relation to the activity’s strategy to deliver services to youth through civil society organizations (CSOs) in different parts of the country. 65 Based in part on Draft USAID Crime and Violence Prevention Guide, p. 45. 35 In order to cover the targeted area, Entrena as prime implementer awarded sub-grants to 24 Dominican CSOs of varying sizes and levels of experience. The sheer number of partners, most selected at the same time, put a significant burden on staff at both USAID/DR and the implementer (based on interviews with both), given the necessary processes of negotiating agreements and budgets as well as the vetting requirements of USAID and the USG more generally. This led to delayed startup of core activities, which was exacerbated by the low capacity of some CSO partners. Due to time constraints, the team was unable to further investigate cause of this. Further, implementer staff and reports showed that most partners needed considerable support to carry out required data collection and reporting tasks due to lower-than-expected abilities in this type of rigorous monitoring. A significant revision of the activity to increase the emphasis on workforce development contributed to additional delays during its first year. In addition, placing at-risk children and youth in jobs and schools (depending on age and needs) was more difficult for Alerta Joven partners than expected, according to FGDs. Obstacles to employment of beneficiaries included the very low salaries offered for many jobs open to youth and the limited commercial activity in some target areas; in terms of school (re)insertion, the partners cited the lack of space in schools and some reluctance to accept the targeted youth. The activity struggled to meet its targets in these areas,66 although Year 3 (2014–15) was better due to changes in approaches, including the engagement of labor insertion specialists. Although anticipated to some extent, Alerta Joven has been hampered by the constrained human resources, limited mandate, and relatively weak political influence of the Ministry of Youth, the key governmental counterpart of the activity. The implementer and external stakeholders indicated that the Ministry has limited influence within the national government and that political will generally in relation to at-risk youth is not strong. As a result, one key activity objective—improvement of policy on youth protection—has been largely stymied on the national level. The Ministry of Youth has not been able to advance the National Youth Plan/Strategy after considerable investment by the activity. Alerta Joven and partners have instead focused their attention on municipal-level policies and implementation of legal frameworks at that level, including setting up youth councils and raising awareness of relevant laws among youth themselves. Also, in relation to Alerta Joven’s work on policy issues, a dramatic change in national policy on documentation of migrants occurred in 2013 by way of a constitutional court decision.67 This presented unexpected difficulties, as indicated by Alerta Joven partners working to assist undocumented youth in vulnerable situations to regularize their legal status. The 2014 National Plan for Documentation of Immigrants mitigated the situation considerably but with a limited window of time for regularization of migration status. The decision by the constitutional court made it temporarily harder for the activity to help ethnically Haitian youth to secure such documentation, while the National Plan had the effect of 66 For example, the activity’s annual report 2014–15 showed that three-year cumulative totals for the indicator “Number of learners enrolled in primary schools and/or equivalent non-school-based setting with USG support” hovered around 73% for both male and females. Cumulative totals for “Number of persons receiving new employment or better employment (including better self-employment) as a result of participation in USG-funded workforce development programs” were also well below targets (68% female and 77% male), but those showed dramatic improvement in Year 3. 67 For more information, see the DR government website at http://presidencia.gob.do/noticias/plan-de￾regularizacion-inicio-hoy-gobierno-confia-en-exito-del-proceso and news article at http://elnacional.com.do/plan￾de-regularizacion/ (both in Spanish). 36 increasing demand for assistance, according to FGDs with CSO partners. Many cases are still pending as a result. CSO partners also indicated that in order to secure documentation for a child or youth, it was often necessary to first sort out documentation problems affecting their families, which was more costly and time-consuming than expected. An opportunity that was seized by Alerta Joven related to collaborative work with the Attorney General’s office on reviewing and revamping procedures for handling youth in conflict with the law. To inform this process, the activity has provided funds and technical assistance for several studies that were highly valued by the Attorney General’s office. For example, youth in detention centers and in conflict with the law were surveyed to identify various risk factors that may have contributed to their delinquent behavior. These studies and other technical assistance funded by Alerta Joven are supporting the development of a new “model for intervention” to be used across the country to guide efforts at rehabilitation of these extremely vulnerable youth. This supplementary intervention has been especially significant as it provides a window for Alerta Joven to work at the secondary level of crime prevention on issues affecting some of the most at-risk youth in the country. For the Community Justice Houses, sustainability is the key challenge; there was consensus among informants within and outside the activity that their services are valuable, and they need to continue (and expand) without ongoing donor support while preserving the positive collaboration among government agencies. This issue was not unexpected by USAID or PC in relation to the current period of funding; however, the costs of the planned new houses are considerably higher than expected (and budgeted) according to USAID (one informant mentioned 30% higher). Plans to open seven new houses with USAID support are currently being re-evaluated in light of this information. The primary strategy mentioned by informants for ensuring the survival and expansion of the CJH and their services is premised on full coverage of costs by the government. This is consistent with PC’s report on CJH implementation from 2012 to 2015, which stated that 65% of costs were being covered by DR government sources (from a base of 55% in 2012), as compared to a target of 100%. USAID is supporting efforts to draft and advocate for legislation for the CJH in the hopes that it will guarantee stable financial support from the government. Sustainability models from other countries are being considered, according to PC, but it is not easy to identify similar interventions that have successfully been weaned from donor support. Colombia’s Justice Houses are still in a period of transition from USAID support, which includes developing a model for Public-Private Alliances to support new houses; the results of that process remain to be seen.68 Notwithstanding the abovementioned challenges, the most important opportunity for the CJH also relates to sustainability. It was found that the private sector is showing some increased interest in supporting the houses; for example, USAID and PC commented that the newly inaugurated CJH in Moca has attracted considerable support from the business community. Each CJH is currently examining possibilities for corporate donors in its locality. Mediation of commercial and labor disputes is a service that the CJH already offer to local businesses, and one expert remarked that this could present an avenue for “selling” the wider private sector on the value of the CJH, whether on a “fee-for-service” basis or otherwise. 68 USAID/Colombia Access to Justice Activity, Third Annual Technical Report, October 2015, p. vi. The same report mentions that a tool has been developed in Colombia to periodically assess the sustainability of their houses (the “JH Development and Sustainability Index”). 37 For CJSSP and the Police Reform Project, still in their early stages, the unexpected challenges and opportunities have been few. CJSSP has experienced some unanticipated changes to the original scope of the activity. Informants and the Request for Proposals indicate that the judicial branch was not going to be included at all in this initiative, although it was a major recipient of past USAID support. However, some new activities that target the judiciary have been added to CJSSP at USAID’s request. On the other hand, CJSSP had planned certain activities targeting the DNP and considered them as one of their key national counterparts. As described in more detail in Assessment Question 3, difficulties have arisen in defining the roles of CJSSP, the Police Reform Project, and INL with respect to the police, which has led to a recent temporary suspension of CJSSP’s work with the DNP. The Police Reform Project was able to rapidly achieve one of its key early objectives, by facilitating the adoption of a comprehensive new strategic plan by the DNP. This represents a significant opportunity for this activity and other USG-supported actors to intervene in a targeted and coordinated manner to address key challenges for the police force. The plan has reportedly been well received by the DNP leadership as well as other stakeholders and has thus laid a solid foundation of credibility for the Police Reform Project and its key personnel. More generally, both the Police Reform Project and CJSSP (and by extension the CJH) are facing considerable uncertainty in terms of their funding streams due to the overall unpredictability of CBSI financial allocations to the DR and the expectation among some informants that the DR funding level would decline. This is especially difficult for new activities as they attempt to form collaborative arrangements with other stakeholders, get some activities underway quickly, and make concrete plans for next steps, while being unsure if they will ultimately be able to carry out all planned activities. As of March 2016, the allocation for CBSI USAID programming in the DR for the 2016 financial year had not yet been decided. Another issue affecting both of these activities is the delayed enactment of police reform legislation. Much of the planned intervention of the Police Reform Project is contingent upon the passage of those reforms—which were promised by the current president, Danilo Medina, when he was elected in 2012—but the legislation has yet to become a reality.69 National elections were (at the time of data collection) scheduled for May 2016, and informants agreed that there was little chance of the legislation being passed until elections were over. Eastern and Southern Caribbean The Juvenile Justice Reform Project experienced unexpected challenges with procurement processes in its early stages, which arose at both member state and IP levels. The resulting delays affected several participating countries, and in the cases of Grenada and St. Kitts, KIIs noted that some items were received more than one year after procurement had begun. These challenges were highlighted by a number of informants from various member states and from the implementer, as well as in the Annual Report from 2015.70 It was found that many of the delays were due to OECS not having enough capacity to fully implement procurement at both the regional and country levels. This challenge was recognized by OECS, which made efforts to ameliorate the problem by bringing in a procurement 69 Prior to publication of this report, the new organic law of the national police in the DR was passed (July 15, 2016) as “Ley 590-16”. See for example: http://www.listindiario.com/la-republica/2016/07/16/427211/el-presidente-medina-promulga￾ley-organica-de-la-policia 70 JJRP Annual Progress Report, October 2014 – September 2015, pp. 8, 22–27. 38 officer for a six-month time period. This had a positive impact, but once the officer left the challenge returned, despite the fact that a longer-term officer came on board. Challenges at the member state level also contributed to the procurement challenges. This was particularly true in the case of the Small Grants Facility, which was attributed to a lack of absorptive capacity at the member state level.71 A third significant factor was the difficulty of sourcing supplies in the region; in a number of cases materials were not available locally and obtaining them internationally added time to the process. The systemic problem of too few staff in the country governments and organizations attempting to work with too few resources was a significant problem for JJRP. Although this did not come as a surprise for USAID and implementers, it was emphasized in most interviews related to this activity. Many implementer staff felt they were unable to really serve youth in the way they should, and while the support they received was important, in many cases they were not able to properly implement it due to resource constraints. For example, in both St. Lucia and St. Vincent informants pointed out that risk assessments of youth were helpful, but they were “unable to treat the problems the assessments found” because they did not have the budget or staff to do so. JJRP also faced difficulties in changing the attitudes of people from a punitive approach to juvenile offenders towards a more restorative and rehabilitative orientation. Those who worked on JJRP from the member states expected resistance from some parents and community members but were surprised by the level of opposition among the staff in the justice and social work sectors. This was most pronounced in St. Vincent and the Grenadines, where a group of informants discussed how some staff had been more receptive than others to the shift towards a more rehabilitative system. The discussion highlighted that this slow pace of attitude change, combined with staffing shortages, has made it more difficult to ensure that reforms related to treatment of youth offenders are systematically implemented across the board. In St. Lucia, an unexpected opportunity emerged from JJRP for the Boys’ Training Center. One informant noted that the improved recognition of the Center due to JJRP resulted in additional donations of equipment and some funding for a music program. It was not apparent whether similar opportunities arose for partners in other countries. Although the assessment team was not able to provide an in-depth evaluation of this activity, document review and interviews with USAID, CXC, and staff from the selected institutions who participated in the activity highlighted a number of challenges that affected the Strengthening Second Chance Education Program. The stated goal of the activity was “to build a sustainable infrastructure that will allow at-risk groups of all interests and abilities to access programs that may lead to the development of valuable life skills, training/retraining for the world of work, portable certification, and a continuing education platform for future development,” which was to be done by expanding the range of certified organizations that support the training of youth.72 The activity had three broad objectives: (1) building strategic alliances/learning partnerships with public, private, and non-profit organizations; (2) taking steps to build a quality management culture and environment; and (3) expanding access to technology in education to facilitate e-learning. 71 Ibid., 26–27. 72 Strengthening Second Chance Education in the Eastern Caribbean, Cooperative Agreement no. AID-538-A-12- 00005, p. 14. 39 The first challenge that the activity faced was related to the resources that CXC as implementer was able to provide to institutions selected to work with the activity. Initial assessments on potential Second Chance Institutions (SCIs) were done, but CXC did not do enough research into the types of support the SCIs would need to implement activities in each country. Originally, CXC identified 30 organizations for collaboration in the activity, but in the end, less than a third were able to effectively do so. A second unanticipated challenge was that, according to CXC personnel and SCIs, the design of the activity did not take into account the students’ needs for support with soft skills, which are often a major factor in the failure to complete schooling through the traditional education system. Finally, interviews with SCIs and participants found that the activity also did not anticipate the needs of students for material support to attend classes, such as funds for transportation, food, and child care. While providing this type of support was not the intention of the activity, the breadth and depth of these issues varied by country and SCI and significantly impacted the execution of the program. In some cases, opportunities arose as partners looked for ways to mitigate this challenge: in St. Vincent and the Grenadines, some students needed childcare, so they made an arrangement with local childcare centers. While the students were in class, their children attended the centers; in exchange, the students volunteered at the centers at other times of the day. In addition, there were communication challenges between SCIs and CXC, which affected at least two countries. One example was the significant confusion among partner staff in St. Vincent over the discontinuation of the activity. The lack of clarity resulted in some students being enrolled who needed the funds in order to complete the exams. Once it was determined that those funds would not be available, some had to drop out. Finally, some SCIs struggled to meet the infrastructure requirements needed to work with CXC on vocational training. In both documentation and interviews with SCIs, it emerged that while it was helpful for CXC to identify needs for facility and equipment upgrades, the analysis was of limited utility in the absence of funding for those upgrades. Based on document review and interviews with USAID/ESC staff, the final obligated amount ($2,150,647) for Strengthening Second Chance Education was less than was originally anticipated under the contract ($4,125,714), which was due to the slow rate of implementation caused by the above challenges. As the activity underperformed, the budget and scope were reduced accordingly. One of the early challenges for Kari Yu! was a number of unrealistically high targets outlined in the original results framework by PADF. Examples included the number of internship placements, number of youth employed, and number of youth transitioning to further education and training programs. As a result, initial results were far short of targets. A revision in March 2015 reduced targets and allowed the activity to work towards more reasonable goals. Another challenge for Kari Yu! was that private-sector groups had a negative view of the youth whom the activity was trying to place in jobs, which resulted in limited opportunities for job placements for the participants. These challenges were outlined in activity documents and also mentioned by implementers and have necessitated significant efforts to sensitize the business community about the objectives and successes of the activity. Recruiting a variety of youth, in particular males and youth from the hinterland, proved more difficult to achieve than anticipated. It was reported that some males were more interested in immediate gains from informal economic activities rather than uncertain benefits from formal employment at a later date. Kari Yu! also experienced delays in finding an appropriate firm to 40 conduct initial assessments and in negotiating contracts with CSOs receiving grants; activity documents indicate the combined effect was a six-month delay.73 For the Trinidad and Tobago Juvenile Court Project, some informants expressed the need for stronger leadership to steer reform of juvenile justice policy in the country. Informants in the sector hoped that the new Children’s Authority established by the government would take the lead, which is occurring - but they noted various challenges facing that entity, including a very broad mandate, shortage of qualified staff, and budget reductions – all of which hinder its ability to truly champion the issue. One of the unanticipated opportunities mentioned by the implementer was the piloting of the peer resolution system, a methodology that is available for schools. Originally intended for the new Youth Courts, the expansion to schools has been an unexpected benefit. Review of the documentation for SKYE in Guyana and interviews highlighted a few challenges that appear to be unanticipated. Many youth who have been in conflict with the law have lower levels of literacy than the implementer initially expected, so they revised the activity to add a literacy strengthening component. According to a key stakeholder, this assisted some participants, but there were still others who were below even that level. In addition, job placement for youth proved to be a major hurdle, although the available data do not indicate if this was unexpected. These concerns were expressed in several quarterly reports, which showed that while placement targets were exceeded in 2013, levels dropped in 2014. As of late 2015, this remained a problem, although the implementer continued to engage public- and private-sector representatives to find opportunities for youth. A major component of A Ganar was the use of preexisting local vocational training providers to help provide youth with needed skills. According to the final activity report, these institutions were not as prolific in St. Vincent and the Grenadines, St. Kitts and Nevis, Dominica, and Suriname as in the countries where the activity model was developed.74 The report noted that once this was recognized, USAID and the implementers made a course correction to put more emphasis on internships and on￾the-job training, but this resulted in delays in the vocational skills portion of the activity. In the Caribbean Youth Empowerment Program, entrepreneurship was one component of the activity. It was noted by informants and the final activity report that the population in some countries had a rather negative perception of entrepreneurship. In many communities, it is perceived that people start their own businesses because they are unable to obtain or keep a job. This was especially true in Grenada and St. Lucia, where families discouraged youth from following this career path.75 This resulted in lower rates of participation in the entrepreneurship component, where the implementers found that at-risk youth required customized support and more attention than adults or youth who do not suffer from poverty or low educational attainment. Declining CBSI funds both overall and specifically for USAID are a distinct concern for USAID/ESC. Several regional informants from USAID and DoS mentioned this issue, which some viewed as a function of USG resources being shifted to other countries or regions perceived to be a higher priority by some decision makers. In addition to these external pressures, USAID/ESC has experienced some unique 73 Kari Yu! Revised Results Framework, March 2015. 74 Partners of the Americas, The A Ganar Alliance: Using Sport to Impact Youth Employment and Youth Engagement, Final Report, November 2015, p. 18. 75 International Youth Foundation, CYEP Final Report, February 2014. 41 challenges that resulted in slower rates of spending than expected. Internal pressures noted by USAID informants included significant staff turnover and shortages. One USAID staffer noted that the significant restructuring of the Mission fed into delays in the design of the strategic plan for the Mission, which they saw as a factor in losing some anticipated CBSI funding. All of these factors have impacted the way USAID/ESC has been able to implement CBSI in the countries under its purview. Jamaica The paradox of COMET II is that while its activities aim to empower communities to chart their own transformation, the low capacity levels of community members and CBOs have been an obstacle to their participation in the activity. Interviewees mentioned that many CBOs were not registered and thus could not apply for USAID grants. Community members in FGDs spoke of how they did not have the knowledge to apply for COMET II grants nor manage the small activities meant to empower them. In response, the implementer organized training sessions on grant application and management and supported the formal registration of many organizations. To better understand needs, COMET II recently commissioned a capacity assessment of CBOs to investigate further, identify gaps, and develop a strategy for future work with CBOs. COMET II met with significant challenges in securing land for CRCs, which far exceeded the implementers’ expectations. For example, the activity took almost one year to identify a suitable location for Stoney Hill’s CRC. However, right before signing the lease, the implementer and community realized that the land’s association with one political party would risk the appearance of political bias. This challenge delayed implementation of this important component; according to informants and activity documents, COMET II was supposed to have built nine CRCs by the time of the assessment but only three have so far been established. In addition, informants have shared concerns about the capacity for communities to govern and sustain activities once the activity ends. In the words of one informant, “When something is owned by all, it’s owned by none.” The concern is on balancing community ownership and inclusivity with the need for clear roles and responsibilities, oversight, and accountability of activities. COMET II faced some difficulties in identifying an appropriate Chief of Party; according to USAID staff, two international candidates were tried but neither proved satisfactory to lead this complex activity. Ultimately, a senior Jamaican staffer of the activity was appointed, and informants from USAID, government and beyond agreed that he had earned wide respect and became the driving force of COMET II. So, in essence a major challenge was transformed into an opportunity for strong local leadership. COMET II’s success in influencing community dynamics and advocating for the rights of vulnerable groups, particularly LGBTI issues, can be partially attributed to him. As one interviewee summed it up, “[The COP] is outstanding: he’s a Jamaican and rose within the ranks of COMET.” Combatting Corruption in Jamaica was implemented by National Integrity Action Limited (NIA), founded and led by a well-known public figure in the country. Dr. Trevor Munroe has effectively brought visibility to the issue of corruption and successfully rallied the support of many senior stakeholders in the government and public sector around the issue. Over time, Dr. Munroe’s personality and actions have become so integral to NIA that many interviewees said it was difficult to dissociate the organization from the founder. This poses a challenge for sustainability of the organization, especially given the end to USAID funding in early 2016 and the looming end of other core funding. Several informants, including government and activity stakeholders, have expressed concern that NIA is not as inclusive as it should be, which could make it more difficult to expand the funding base beyond traditional donors. NIA was accredited in early 2015 as Transparency International’s national chapter. This serves as recognition for all the work it has done to move the needle on corruption in the country, including through the CCJ activity. The formal affiliation with Transparency International, while coming late in the 42 CCJ activity, should provide more opportunities for NIA in terms of access to funding, expertise to guide actions, and sustainability of the organization. The corrections authorities of Jamaica have been sending girls deemed “uncontrollable” to juvenile correctional centers, despite being mainly cases of disciplinary issues. Informants noted that recent developments have led to fewer and younger girls being sent to South Camp. ANP designed its educational, vocational, and psychological interventions to target older girls or young female adults, and the younger cohort of girls means that these interventions are not always age-appropriate, for example, entrepreneurial training. As a result, the implementer has struggled to meet the targets initially set for the number of girls to be served and has had to adjust its approaches. Moreover, they did not anticipate such a high proportion of remandees, who remain at South Camp for unpredictable lengths of time, ranging from one day to several months. This poses logistical challenges for the timing and planning of educational or vocational activities for remandees and also limits the benefits for those girls from ANP’s interventions. ANP is designed to work through local CSOs in delivering many services to the girls. However, staff of the implementer reported that they did not anticipate the limited capacity levels of local NGOs, many of which have not worked previously with youth in detention and thus lack the skills and experience to effectively manage this population. The activity has addressed this by partnering with the Council of Voluntary Social Services to build the capacity of civil society groups. ANP has also struggled to hire social workers due to a dearth of this particular skillset in the labor market. Having only one social worker on board (at the time of assessment) has hampered plans to increase the engagement of parents and guardians of girls to facilitate reintegration and rehabilitation and avoid recidivism after leaving South Camp. No significant unexpected challenges or opportunities affected the Empowering Jamaica’s Youth (JA) activity during the CBSI funding period. One important ongoing challenge mentioned by informants was JA’s reliance on private-sector support, not only to set the strategic direction of the organization as board members but also to fund activities to a significant extent. The challenge in maintaining a steady flow of private-sector support from year to year affects consistency in the reach of beneficiaries and schools. On the other hand, the transition from JA Worldwide to JA Jamaica to manage the next iteration of the activity presents an opportunity for local ownership, increased support, and enhanced sustainability. An unexpected challenge arose in the USAID/MoE Partnership for Improved Reading Outcomes activity when CBSI funds were used to provide additional coaching and resources to lower￾performing schools. A USAID informant mentioned that the implementers were surprised when these schools did not record improved reading outcomes compared to other schools supported by the activity, which raised questions on the impact of that additional support. No unexpected opportunities were mentioned in activity documents, which were the primary source of information on this activity. Fi Wi Jamaica is being implemented by the University of Technology of Jamaica (UTech) and includes activities to protect the rights of LGBTI individuals and other socially marginalized groups, such as victims of sexual and gender-based violence and victims of trafficking in persons. The activity is just getting off the ground but has faced unexpected difficulties working through the university’s Procurement Office to obtain goods and services; some informants alluded to the possibility of certain employees in that office impeding the process due to personal views of the activity. This has delayed the hiring of consultants to help with certain activities. Across the activities, several informants mentioned the effects that the recent change in the ruling party in national government may have on CBSI activitiess going forward. While CBSI programming had steady backing from the previous government, there is some concern about continued support by the different 43 government agencies. This is particularly pertinent for activitiess such as Fi Wi Jamaica that focus on controversial topics. ASSESSMENT QUESTION 2 CONCLUSIONS Dominican Republic With respect to Alerta Joven, one lesson learned is that sub-grants to local partners, especially CSOs, are a mechanism that takes time, effort, and varying types of support to ensure that activity results are achieved, both in terms of quality and quantity. It is likely that Entrena (and maybe some USAID/DR staff) did not realize the complexity of this undertaking, especially in the context of USAID rules and regulations, when they planned this activity. Careful assessment of the CSO universe in targeted areas and of the time required for all approvals of sub-grants might have helped to alleviate the situation in terms of realistic scheduling and anticipating needs for CSO support. However, it is difficult to see how the activity could have achieved its results without the substantial engagement of CSO partners with local knowledge of the targeted areas. Generally, Alerta Joven has set ambitious targets, some of which have proved to be problematic. Factors beyond the control of the activity were largely responsible, although some targets were probably set with insufficient knowledge of what would be realistic. The implementers have worked closely with CSO partners to set targets for indicators and responded appropriately to shortfalls, making key changes in areas such as labor insertion. The weak institutional status and political influence of the Ministry of Youth might have a negative impact on the chances of Alerta Joven reaching all of its goals at the policy level; it is even possible that the Ministry will be eliminated and integrated with other Ministries. However, the activity strategy of working with local governments and others to achieve change at the ground level and support policy implementation and awareness appears to be both suitable and effective. In addition, Alerta Joven has capitalized on an opportunity to support the enhancement of procedures for dealing with youth in conflict with the law. While the process is still underway, the results could have far-reaching effects on the well-being and future of youth who come in contact with the judicial system. In spite of considerable uncertainty regarding the future of the CJH and challenges to expansion in particular, there is cause for optimism based on the positive results described under Assessment Question 1 and the substantial contribution of governments at the local and national levels. The recent evaluation endorses the full incorporation of the CJH into government as the only route to sustainability, and 100% government funding appears to be favored by the implementer. However, there are other avenues that could be explored (such as charging fees for certain services), considering international access-to-justice models and Colombia’s closely related experience. With respect to CJSSP, there is cause for concern that uncertainty in its scope of work and available funding could affect the implementer’s ability to plan effectively and meet its objectives. The changes of focus in the first nine months (both adding and subtracting areas of work) have created some internal and external confusion, which could affect credibility of the activity if communications are not managed very carefully. For the Police Reform Project, the new DNP strategic plan is an important step forward for enhancement of the institutional capacity of the force, especially as the plan is accompanied by implementation details and budgetary allocations. However, ensuring buy-in amongst the leading cadres of the DNP will be essential to achieving the goals that have been set. Eastern and Southern Caribbean The Juvenile Justice Reform Project procurement process produced significant challenges for the partner countries, indicating a need to streamline procedures and ensure adequate staffing in any follow￾on programming to improve the pace of implementation. The unexpected slower pace of attitude 44 change within some government ministries and departments may well have been a factor that hampered the consistent implementation by government staff of certain elements of reforms. In spite of efforts by USAID, OECS, and individual partners in the countries, some challenges are heavily dependent on external factors. The resource constraints on service providers will continue until the economies of these countries recover from the global recession and/or those in power place higher priority on juvenile justice reform. Shifts in political power dynamics and related difficulties in adopting and implementing legislation are affected by each country’s evolving situation, and thus the pace of reform is unpredictable. In the Strengthening Second Chance Education Program, the most significant challenges arose from flaws in its design. There is evidence of insufficient initial research into the needs and resources of the SCI organizations and the youth that they serve. The lack of attention to soft skills and consideration of other needs of students directly contributed to lower participation rates. While CXC did research during the first phase on the needs and capabilities of the SCI organizations, the changes needed were found to be more extensive than were possible to correct within the scope of the activity. However, these critical design issues did encourage SCIs to think creatively to solve some of the resource shortages. In addition to the design issues, there was also not enough clarity between CXC and SCIs about the changes in funding during the second year, which had a negative impact on some students, who lacked the required resources to contribute to course and examination fees. The Kari Yu! activity has experienced a number of challenges, including unrealistically high targets, negative perceptions within the private sector, and difficulty in recruiting some youth. Activity staff have worked diligently to resolve these issues and were able to resolve those associated with the results framework. Those challenges related to the engagement of the private sector and the more limited involvement of young males and youth from the hinterlands remain issues. Finding a firm with the requisite experience to conduct the initial assessment was a challenge that cost the activity a little time, but the research was ultimately considered valuable by the implementers. In addition, CSOs are critical to the success of the activity, and the work done to shore up their capacity was relevant to the overall objectives; therefore, it was an important hurdle to overcome. For the Trinidad and Tobago Juvenile Court Project, the creation of the Children’s Authority was a positive step toward establishing a lead government agency for juvenile justice, but properly defining its role and priorities and adequately resourcing it will be crucial to its effectiveness. The Juvenile Court Project is still in its early stages and is moving slowly, but it has shown promising signs of being able to seize unexpected opportunities. In order to account for lower levels of literacy than expected, SKYE added an education component, which indicates the implementer’s responsiveness to identified challenges. Finding placements for youth has been a problem throughout the activity cycle. The implementer attempted to analyze this issue through a labor market survey and appears to have made significant efforts to place youth in positions with some success, increasing placements from 2014 to 2015. For A Ganar, the implementers experienced a challenge with the lack of preexisting vocational training institutions in the USAID/ESC countries. Based on the Final Report, they shifted the focus to more internships and on-the-job training, which enabled them to improve their targets in this area. Plans for upcoming programming in USAID/ESC are robust and, according to a USAID informant, are based on a pipeline of about 30 million USD over the next three years. If the funding that USAID/ESC expects to see is reduced, it could significantly impact the programs they plan to implement. Jamaica COMET II had to invest additional resources and time to build the capacity of CSOs, which delayed activity implementation. However, this also led to the unintended outcome of a better informed and 45 equipped civil society with skills to benefit from COMET II’s activities and generally play a more active role in their communities. While having a sound strategy to empower communities to lead change processes is critical, that is difficult to achieve if community groups are not solidly established and active, with strong leadership willing to take the lead—inevitably, this will not be the case in every targeted community. The nature of COMET II’s activities and political context of the country means that allocation of land, and even other apparently innocuous activities, can always generate political controversy. It is crucial for activities to be staffed by people who are well informed of these risks and are diplomatic and savvy about navigating controversial areas. Sustainability of NIA, and inevitably, any future follow-on activity to CCJ, remains uncertain given the close link between the organization and its leader, although their recent recognition as TI’s national chapter, a major achievement for an organization founded officially only four years before, should be a boost for the organization’s credibility (especially among international stakeholders) and sustainability. At the same time, the perception of exclusivity of NIA can affect long-term efforts to diversify its support base, especially at the grassroots level. ANP’s objective of supporting rehabilitation and reintegration of juveniles is critically needed in the country, and while progress is being made towards that objective, the activity has met a series of unexpected challenges beyond the control of the implementers. The implementer has taken corrective action to address these challenges with the support of USAID by investing in building capacity of local NGOs and utilizing graduate students from social work and/or clinical psychology programs to support ANP’s social worker. Both strategies have generated unintended positive outcomes for the participants, which should last beyond the duration of this short activity. Empowering Jamaica’s Youth was relatively unaffected by unexpected issues, and JA appears to have a solid base of supporters in government, schools, and the private sector. However, sustainability is not guaranteed and its scope of intervention is still dependent on factors beyond its control. Under the MoE Partnership for Improved Reading Outcomes, other factors could have contributed to the lower-than-expected improvements in reading outcomes among CBSI-funded schools (for example, starting at a lower reading level on average than other schools). Additional data is needed to understand the problem before it can be addressed. Fi Wi Jamaica tackles very sensitive issues within Jamaica, which increases the likelihood of meeting some resistance; however, this was not expected from within the implementer itself. USAID/Jamaica secured assistance from the MoE, which provides oversight of the university, to resolve the apparent resistance from UTech’s Procurement Office. While the problem has been addressed, it did delay implementation. This issue raises the broader question of the potential impact of the change in the ruling government in Jamaica on ongoing support for this activity. Across the country, politically related challenges are difficult to predict; there will always be the potential for politics to affect activities. Activity implementers should be mindful of the potential for political sensitivities to affect implementation and incorporate these factors into work plans. Cross-Cutting Findings and Conclusions USAID CBSI activities across the region faced a wide array of unexpected challenges and, to a certain extent, opportunities. The activities aimed to respond with fairly limited resources to huge demands and needs in relation to at-risk youth, local communities, and government agencies in a context where most host country governments were facing worsening financial constraints due to the global economic downturn, among other factors. This has meant that the tangible government investment (in human resources, infrastructure, etc.) needed to complement the inputs from USAID and ensure their sustainability has been lacking in many situations. 46 Moreover, a number of CBSI activity implementers and partners are facing uncertainty in relation to the ongoing level of CBSI funding while conversely experiencing pressure to spend allocated CBSI funds to demonstrate funding relevance and utility. This is especially the case for the USAID Missions in the DR and ESC. The uncertainty can be a disincentive for USAID and implementers to develop strategic multi￾year plans and conduct in-depth research to inform activity design. Many of the activities rely heavily in some way on CSOs, particularly those with deep roots and knowledge of local communities. Working through locally based partners and grantees from civil society or government is often time-consuming; in particular, capacity in systematic monitoring of outputs and outcomes is often less than optimal for fully capturing activity progress and identifying issues on a timely basis. Nevertheless, partnership with local stakeholders is important for sustainability of results and linkages with vulnerable populations, among other benefits, and is in line with USAID Forward principles. Activity implementation and results have also been affected by political dynamics in individual countries and the region. Delays in anticipated legislative changes in favor of police and juvenile justice reform have delayed certain core activities and continue to do so. Political sensitivities associated with the location of some activities have added another layer of complexity. On the positive side, there is evidence that private-sector interest in supporting certain areas of citizen security programming is increasing, and there are growing opportunities to create win-win situations that enhance the operating environment for business. Examples of this include the Community Justice Houses in the Dominican Republic, Junior Achievement in Jamaica, and Kari Yu! in Suriname. ASSESSMENT QUESTION 2 RECOMMENDATIONS Dominican Republic 1. USAID and implementers should recognize that, in most cases, working with CSOs to carry out and monitor significant activity components will require investments in capacity building and often involve longer time frames to reach objectives and should make allowances for this in plans and budgets. USAID and implementers should ensure that timelines take into account the necessary steps for sub-grant solicitation, negotiation, and approvals, and seek to expedite those processes as much as possible. One option for the future is a phased approach that spreads out CSO grants over time as compared to a single wave of grants, which would ease pressure on USAID and implementer staff. 2. USAID and its implementers should continue to explore and analyze options for sustainability of the CJH, including consideration of the Colombia experience as it evolves. Income-generation strategies such as fee-based mediation for the private sector should be studied, including regional experience (e.g., the Dispute Resolution Foundation in Jamaica) and access-to-justice models in which some users contribute towards costs. 3. For CJSSP, USAID should try to limit further adjustments of activity scope to allow the activity to set a stable course and effectively manage the expectations of national stakeholders. 4. With respect to the Police Reform Project, USAID and ICITAP should work with the DNP to ensure that other USG and international actors are well informed about the strategic plan and ensure that this plan is respected and actively supported by all USG actors. This will serve to boost credibility of the plan, foster timely implementation, and set a positive example of harmonized intervention. 47 Eastern and Southern Caribbean 5. For new activities involving significant procurement of goods or services, USAID/ESC should work with implementers to ensure that they have a clear and efficient procurement process in place, with sufficient staff to support that process. 6. For future activities along the lines of the Strengthening Second Chance Education Program, USAID and implementing partners need to carefully assess the situation in each country during design and consider needs for soft skills, financial support for students, and physical upgrades to facilities. 7. Private-sector support is critical to activities like Kari Yu! and SKYE with youth employment at their core; implementers should continue to make efforts to engage representatives from private-sector bodies from the earliest stages to ensure buy-in and identify potential champions. 8. Identifying additional funders and partners in the design phase may help to insulate projects from the worst effects of funding cuts. USAID should encourage implementers to co-fund projects, work to strengthen collaboration with agencies with related interests such as UNICEF, and liaise with other USG agencies to see if synergies between activities could help to meet some costs. If identification of additional funders is problematic, a wide-ranging stakeholder/donor mapping exercise might help identify opportunities at a broad scale, which could then feed into specific design decisions and outreach. Jamaica 9. USAID/Jamaica and implementers of community-based policing and community development activities (like COMET II) should continue engaging with CSOs and community members during design and implementation phases, including the integration of capacity-building needs of these groups and planning for a longer timeframe to account for these engagements 10. USAID/Jamaica should prioritize development and implementation of financial and results sustainability plans in future anti-corruption activities by NIA or other implementers. More specifically, NIA should reassess its funding strategy to diversify beyond international donors to include grassroots supporters and the private sector and make a concerted effort to increase demarcation between the leader and the organization. Cross-Cutting Recommendations 11. USAID should attempt to maintain stable CBSI funding in accordance with projections for each Mission, allowing reasonable time for Mission teams to develop well-researched and complementary interventions. It is important to maintain the focus of CBSI on citizen security, in spite of financial pressures on Missions that are receiving fewer resources from other USAID funding streams. 12. Given the uncertainty of future CBSI funding streams, USAID Missions should be as open as possible with implementers about possible changes in funding. When making/reviewing work plans and other key documents, USAID and implementers should consider the potential effects of budget reductions, for example, identifying activities that could be dropped or scaled back if needed, highlighting those likely to be resource-intensive without major impact, and flagging those that could be damaging if the full activity plan cannot be carried out for full duration. Assessment Question 3: Complementarity and Coordination INTRODUCTION In this section, the assessment aims to highlight actual and potential complementarity among USAID￾supported CBSI activities, as well as between USAID activities and activities supported by other USG agencies, identify overlaps if they exist, and comment on coordination efforts where relevant. Given the 48 nature of each agency’s mandate, SI agreed with USAID/LAC that the focus in terms of other USG agencies should be on the Department of State, in particular the INL Bureau. Because of the sometimes closely related areas of intervention of USAID and INL, the risks and opportunities for complementarity, overlap, and coordination are significant. Therefore, while the team remained alert to the possibilities of common ground with other agencies such as the Department of Defense and phrased its questions to informants to encompass other USG entities, the assessment did not specifically investigate the activities of those other entities. ASSESSMENT QUESTION 3 FINDINGS Dominican Republic The design and implementation of the Alerta Joven activity are generally consistent with other USG￾funded activities in DR, and no concerns with respect to overlapping or conflicts emerged from informants or document review. There was no evidence that the activity was particularly supportive of the goals of the other three CBSI activities, which are largely justice- and law enforcement–related, or vice versa. Examination of activity documents and in-country data collection indicated that although differing in focus, there were areas of potential synergy with other activities that had not yet been explored. For example, some Alerta Joven partners support peer mediation processes among youth, and this could potentially be supported by the experienced mediators of CJH in areas of geographic overlap. In relation to the CJH, there was no evidence of overlaps or conflict with other USAID or USG activities, and their objectives are seen by informants as consistent with other CBSI activities—especially the relatively recent CJSSP and Police Reform activities of USAID, which are focused on the “supply side” of justice, while the CJH is firmly targeted on the “demand side.” Review of activity documents for CJSSP (in which CJH has been subsumed since 2015) indicates that the CJH could benefit from certain aspects of the CJSSP; for example, streamlining the management of cases by the Attorney General’s office should have a positive effect on cases handled by prosecutors working with the CJH. In addition, improved relations and cooperation between the Attorney General’s office and the DNP, a shared objective of the CJSSP and Police Reform Project, could be relevant to the provision by the CJH of streamlined assistance in criminal cases. Conversely, the long practical experience of the CJH staff and partners in helping resolve cases of violent conflict (albeit a small proportion of their caseload) through cooperation with prosecutors and police could be a valuable resource for both CJSSP and the Police Reform Project, as they strive to identify the best ways of strengthening both institutions and their ability to serve the Dominican population. Informants indicated that discussions had begun between Participación Ciudadana and ICITAP (implementer of the Police Reform Project) in relation to specialized training of police officers, aimed at strengthening their links with the CJH. Although the CJH activity is now technically part of CJSSP (as a grant under contract), the team observed by triangulating several sources that the working relationship between PC and Chemonics (prime implementer of CJSSP) is not yet firmly established, and in fact there may be competing ideas about the planned expansion of the CJH operation. CJSSP was primarily designed to support the improvement of criminal prosecutions via work with the Attorney General’s office. However, the plans (as described in the Request for Proposals) also included some support to the police force—notably in relation to the protection of vulnerable population groups and cooperation with the prosecution service, but also in efforts to enhance oversight and ethics in law 49 enforcement and criminal prosecutions. Informants reported difficulties in achieving a shared understanding between CJSSP and the Police Reform Project about allocation of roles, because the latter is aiming for some similar objectives. As indicated in the CJSSP report on its first six months of operation: “In late 2015, demarcation of assistance areas to be provided from respective USG funded entities was still being defined and discussions on the CJSSP role still in progress.” 76 The situation is exacerbated by the fact that most work by both activities with the police is contingent on the adoption of legislation to reform the police force, which remains pending in spite of high expectations that it would be passed in 2015. On the positive side, informants indicated that there is strong will within both activites and within USAID to ensure that complementarity is achieved and coordination is optimized. With respect to other USG agencies, there is evidence of weak information sharing and coordination between USAID staff and implementers of, on one hand, CJSSP and Police Reform Project and, on the other, INL of the Department of State. Informants both in the Mission and in Washington reported that until recent years, the types of support provided by USAID and INL to law enforcement agencies were quite distinct, thus there was little need for close coordination. However, in the case of the DR (and according to some informants, in other locations as well), the activities planned by each agency have become more closely related, especially since adoption by USAID of the 2014–18 Country Development Cooperation Strategy, with its explicit emphasis on citizen security. Complementarity was especially problematic between the Police Reform Project and INL, as the latter agency has shown interest in working on institutional strengthening of the DNP, according to USAID and DNP interviewees. Although the assessment team did not receive a copy of the INL work plan, there is definite cause for concern that USAID activities could overlap with INL’s work in law enforcement. One informant highlighted the need for investments by INL in equipment, software, and training to be taken into account as USAID invests in the internal systems of the DNP. Dominican stakeholders indicated that they have already noticed the lack of a clear division of labor and appropriate coordination between the work of USAID implementers and INL. While efforts have been made by various parties in the Mission, informants among USAID and Department of State staff consider that there is currently no effective forum or process for these agencies to share detailed plans and ensure that they are complementary. Most see the Embassy’s Law Enforcement Working Group as unsuitable in this respect, due to constraints on individuals who can participate and the so-called “drugs, guns, and thugs” orientation of the group, which bears little relation to USAID programming on citizen security. Synthesis of data from informants and documents indicates that coordination and complementarity among USAID/DR activities had previously been smooth, but the introduction of two complex new activities (CJSSP and Police Reform) in mid-2015 coincided with the departure of the CBSI coordinator, a role played by the Youth, Education, and Security Office Director. Since that time, staffing turnover and gaps have affected the coordination role, and the lack of a scope of work to define that role may also have contributed to difficulties. Both INL and USAID staff reported that efforts to obtain and share information between the two agencies had not been fruitful, and some suggested that the Embassy leadership might have to play a stronger role in coordinating efforts between INL and USAID. Several informants commented that relations between USAID and INL were highly dependent on personal relationships, which meant that the will to coordinate was highly variable over time. It is worth commenting that the difficulties facing 76 Chemonics for USAID, CJSSP Quarterly/Startup Report, June 6, 2015 – December 31, 2015, p. 24. 50 the DR with respect to coordination and complementarity are by no means unique. According to a field guide on law enforcement programming by USAID Missions77: “Despite its size and reach, U.S. law￾enforcement assistance can be a source of controversy within the U.S. Government. Responsibility for the delivery of assistance is fragmented among a host of departments and agencies, notably State, Justice, Defense, Treasury, Homeland Security, and USAID. There is no consolidated assistance budget or sharing of lessons learned.” The guide goes on to say: “With this number of USG agencies involved in overseas civilian police assistance, it is not surprising that different approaches and philosophies are brought to bear.” 78 Eastern and Southern Caribbean Document review and interviews with informants indicated neither overlap nor significant complementarity between the Juvenile Justice Reform Project and other USAID CBSI activities. One area in particular that the team explored was the interaction between JJRP and the Strengthening Second Chance Education Program, which as regional activities aimed to assist at-risk youth in many of the same countries. While a few key JJRP informants were aware of both activities, most participating organizations and government counterparts were unaware of Strengthening Second Chance Education, or were unable to meet the CXC requirements, so there were few opportunities for complementarity. One exception was noted in St. Lucia, where the Boys Training Center (BTC) received funds from JJRP to make upgrades to its technical workshop in the form of equipment. Those upgrades enabled the BTC to better take advantage of the opportunities for certification under the Strengthening Second Chance Education Program. In terms of complementarity between JJRP and other USG agencies, informants agreed that INL has not implemented activities related to JJRP’s work in the same countries; thus there is no overlap and little opportunity for complementarity. The only common ground mentioned is the justice advisor that INL is co-funding with the British government; she works with all six states covered by the INL post in Barbados on revising legislation and justice systems but has not focused on juvenile justice. One USG informant observed that the work of JJRP and INL is related, so there was room for more coordination to avoid duplication in future and potentially foster collaboration—for example, INL’s work with the rehabilitation of prison populations, which contain large percentages of youth. Based on interviews, the assessment found that most CBSI implementers were not aware of other USAID CBSI activities in their countries. This was most apparent in the case of JJRP and the Strengthening Second Chance Education Program, which had the most geographic overlap, even though their objectives were very distinct. One implementer noted that they were not made aware by USAID of other relevant activities; they found out about them through other channels. When asked about complementarity and coordination, interviewees repeatedly noted that UNICEF complements the work of USAID/ESC. UNICEF provides both technical and financial support in the region (particularly the OECS countries) in the area of juvenile justice, including legislative reforms and interventions targeting youth at risk or in conflict with the law. JJRP has received UNICEF support for additional attendees to participate in trainings and for a JJRP coordinator in St. Lucia for a period of time. According to interview data, although UNICEF is on the regional advisory group of JJRP, additional discussions between the two groups during the development of upcoming programming would have 77 USAID, Field Guide for USAID Democracy and Governance Officers: Assistance to Civilian Law Enforcement in Developing Countries, 2011, p. 9. 78 Ibid., p. 23. 51 been welcomed and would have helped UNICEF be more deliberate in its own planning and resulted in an even more coordinated approach. An example of this was cited from St. Lucia and Grenada where UNICEF had been developing a non-custodial diversion program. UNICEF would have liked to work with USAID and the OECS on the follow-on to JJRP to help it incorporate the learning from the earlier UNICEF programs. In Suriname, there are currently two significant USG-funded activities that focus on youth and juvenile justice. USAID’s Kari Yu! works specifically on policy-level reforms related to juvenile justice and the provision of employment, training, and mentorship specifically for vulnerable youth. The INL-funded Resistance and Prevention Program (RAPP) works more broadly on improving interactions among communities, police, and youth. RAPP started operations in Suriname much later than Kari Yu! and was recently closed; the two ran concurrently for only seven to eight months. Based on interviews with the implementer for both activities (PADF), INL staff, and activity documentation, the objectives were complementary and the only overlap was the targeting of some of the same youth. Multiple informants observed that the activities strove to coordinate and even collaborate when possible. For example, when PADF discussed violence in the Kari Yu! basic life skills training component, representatives from RAPP led dialogues with the participating youth. According to staff, this coordination was largely due to the fact that PADF implemented both activities. The Trinidad and Tobago Juvenile Court Project is the only USAID CBSI activity in the country and thus there is little scope for coordination or complementarity with other USAID CBSI programming outside of meetings such as the regional TWG. The implementers indicated very little linkage with Kari Yu! but ongoing occasional contacts with JJRP, which has some similar objectives. Linkages with other activities in the region were primarily facilitated by the National Center for State Courts (the primary technical partner). With respect to other USG agencies, the only other significant activity in the country was RAPP, mentioned above. RAPP has a rather different mandate than the Juvenile Court Project, and the team found no cause for concern about overlaps. From the perspective of coordination, while the implementers of RAPP were aware of the Juvenile Court Project, they did not interact in any meaningful way, and it does not appear that collaboration was explored. The assessment team also examined other CBSI activities that worked primarily on youth engagement and employment, namely A Ganar and CYEP. The team also reviewed SKYE, which includes a juvenile justice reform component. In the case of all three there is no apparent overlap or complementarity with other USG CBSI activities. Given that the team performed only document review on these activities (two of them closed), it was not possible to fully explore ties with other CBSI activities; those links did not appear from the documentation. With respect to other USG agencies funded by CBSI, informants in both INL and USAID indicated awareness of each other’s activities but believed that their focus on very different areas made the risk of overlap unlikely. One informant described the interagency relationship as “high fences make good neighbors,”—in other words, limited interaction has allowed for cordial relations. The lack of overlap and limited interaction was corroborated by conversations with USAID primary and secondary implementers and government counterparts. There was little evidence of coordination between USAID/ESC and the INL officers based at posts around the Caribbean. Some informants thought that more communication would be beneficial, but there was no significant problem at present. Informants underlined that USAID/ESC faces some unique challenges compared to other CBSI Missions. The staff of USAID/ESC is very small and not all based in the same location, adding an additional layer of complexity to intra-staff communications. In order to conduct the business of the Mission, staff are required to travel among 10 countries that do not all have direct flights, which adds significant time to any action requiring travel. Finally, they face unique challenges of operating with countries that share similar histories, cultures, and regional structures but that are also sovereign states. The INL and USAID teams in the region cover similar but not identical 52 geographic areas, many parts of which are remote, difficult to traverse, and have an inadequate regional transportation system. Jamaica The team’s analysis of the objectives and activities of the six current CBSI activities demonstrated that the activities are not overlapping and, to a certain extent, can be said to be complementary of each other in terms of the broader CBSI objectives. More specifically, COMET II’s emphasis on community￾law enforcement relations complements the efforts of A New Path on addressing juvenile justice reforms, Combatting Corruption in Jamaica on reducing corruption, and Empowering Jamaica’s Youth (JA) on increasing economic opportunities for at-risk youth and vulnerable groups. Interviews and activity documents revealed the presence of several other government and donor-funded agencies aside from COMET II that work on community development in Jamaica.79 While these activities have slightly different objectives, they strive toward the broader goal of empowering communities for peace and development. The COMET II team has made efforts to avoid geographic overlaps and supports an ongoing effort to develop an overarching Monitoring and Evaluation framework. Although external complementarity was not part of this assessment, several informants in government and the international community commented that there was room for more synergies among these initiatives. The team’s review of activity objectives demonstrated that CCJ did not appear to have overlapped with the other USAID CBSI activities, and in fact there are several examples of CCJ attempting to coordinate and leverage activities with COMET II. For example, CCJ took advantage of COMET II’s presence in communities to promote awareness and increase support against corruption among community members. In addition, CCJ delivered training to police officers and provided anti-corruption materials to participants of COMET II’s community journalism activity. A New Path has a technical steering committee led by the Department of Correctional Services (DCS) that brings together stakeholders from several ministries and international agencies to provide technical oversight and coordinate activities in the juvenile justice sector. This has served ANP well as a source of technical guidance as well as a coordination mechanism, although there were delays in activity startup as a result of the need to consider different stakeholders’ input. ANP has reached out to COMET II to access its community activities for rehabilitation of girls coming out of South Camp; however, informants reported that little progress has been achieved because COMET II’s activities are not always suitable for the purposes and target group of ANP. Interviews revealed that several Jamaican entities, such as the government’s Child Development Agency and the donor-funded Citizen Security and Justice Project, are working on development of an electronic case-management system, with the CDA being the furthest along in developing a system. While this activity is in ANP’s work plan, it was not yet clear how the implementer was planning to coordinate with 79 The Planning Institute of Jamaica leads the government’s efforts in community transformation based on its Community Renewal Program strategy; the Social Development Commission focuses on empowering communities through economic, governance, and social initiatives; and the Citizen Security and Justice Project implements the Ministry of National Security’s Crime Prevention and Security Strategy to reduce crime and violence in communities. The Citizen Security and Justice Project is funded by the Inter-American Development Bank, Global Affairs Canada (formerly known as Department of Foreign Affairs, Trade and Development), and UK Department for International Development. 53 other agencies to support the government’s goal of an integrated system for use by all stakeholders in the juvenile justice system. Similar to the other activities, Empowering Jamaica’s Youth (JA) does not appear to be overlapping with the other CBSI activities in the country. There have been some successful efforts at complementarity with other activities, for example, by coordinating with ANP to tailor and deliver its curriculum to the girls at South Camp. Findings also indicate that there was neither overlap nor particular complementarity of USAID/MoE Partnership for Improved Reading Outcomes and Fi Wi Jamaica with the other USAID CBSI activities. Document review and interviews indicated that USAID and other USG CBSI activities in Jamaica do not directly overlap with each other. USAID’s COMET II activity has most in common with INL, given their closely related objectives of building the capacity of the national police. Informants in the Mission reported a modest level of information sharing and coordination among USAID and other USG agencies in Jamaica on CBSI activities. INL circulates a weekly list of upcoming law enforcement training events organized by the different USG agencies in the country, including those funded by CBSI, a copy of which was shared with the team. INL also takes the lead on putting together a monthly report on all CBSI activities that is shared with the interagency group. At a broader level, coordination among USAID and other USG agencies occurs through a Law Enforcement Working Group that meets monthly to discuss activities. However, some informants expressed concern about the lack of a clear division of labor between USAID and INL. While it was generally perceived that INL emphasizes provision of physical materials and related technical training, and USAID focuses on longer-term capacity development, especially to promote CBP methods, this de facto allocation of roles is not documented anywhere. Furthermore, interviewees commented that information sharing and coordination is mainly dependent on individuals and personalities at the Embassy. USAID is not always invited to the Law Enforcement Working Group’s meetings, and as reported by USAID/Jamaica as well as informants in other posts, locally employed staff are not able to participate due to the sensitive subjects that may arise. One USAID staff member lamented that “We’re not always informed when they [U.S. forces] are planning military operations in an area where USAID is working,” and observed that even if activities are not overlapping or closely related, the lack of information sharing can be detrimental to the work of other USG agencies. ASSESSMENT QUESTION 3 CONCLUSIONS Dominican Republic The Alerta Joven activity generally complements other CBSI activities, and there are no concerns with overlaps. However, there are potential synergies that have not yet been explored and exploited, including with the CJH in relation to peer mediation and with CJSSP in relation to youth in conflict with the law. The CJH activity is also consistent with other CBSI activities but with even greater potential for synergies given the thematic focus on justice issues that it shares with CJSSP and the Police Reform Project. Improved performance and case handling by prosecutors and police can only be a positive for the communities served by the CJH, and thus for the CJH activity itself. In general, the CBSI programming in the DR reflects a common theme of promoting the rule of law and peaceful dispute resolution, and thus appears to be internally complementary at the Mission level. Achieving complementarity between the law enforcement–related activities of USAID activities and INL may be more challenging, as territoriality and competition for funds are factors that can undermine the will to proactively share information and seek compromise. Irregular communications and coordination have created a high risk of confusion and miscommunication in relation to national counterparts and have made it difficult for the USG to capitalize on investments by each agency and thus optimize use of limited resources. Informants consistently stated that there was “so much to be done” to enhance law 54 enforcement in the DR that there should be no problem for each agency and activity to identify suitable high priority areas of intervention. The assessment team agrees with this conclusion. Eastern and Southern Caribbean Coordination and complementarity among USAID CBSI activities varied. The geographic and thematic diversity of programming in the ESC were factors that limited the possibilities—as well as the potential benefits—of interaction among activity implementers and other stakeholders. In some cases, implementers reported that they were not aware of other activities within their own countries and the region. In the case of Strengthening Second Chance Education and JJRP, some country-based counterparts were aware of both activities while others were not. The level of collaboration and coordination varied depending on the activity, country, and individual – with some being aware of other interventions, others collaborating, and still others who were wholly unaware of other CBSI programming. There is not enough information available to know what the impact of increased coordination and collaboration between the two activities could have produced. USAID/ESC’s new strategy focuses solidly on reforming and strengthening juvenile justice legal frameworks and institutions, providing social support to at-risk youth to reduce the likelihood that they will come into conflict with the law, and developing a body of data intended to provide countries with information to make decisions about their juvenile justice work and inform future USAID programming. USAID/ESC maintains communication with INL so they are aware of each other’s activities, but they do not routinely collaborate or coordinate in a significant way. This is in part due to the varied location of agency personnel and the difficulties of travel within the region. The one exception is the interactions between the USAID’s Kari Yu! and INL’s RAPP. Implemented by the same organization, they not only coordinated but actively sought opportunities for collaboration and synergy. Examining the overall picture, there is currently no significant overlap between USAID CBSI activity in the ESC and that of other USG CBSI implementing agencies. Coordination and collaboration with UNICEF has been beneficial to JJRP thus far and has the potential to bolster both new USAID programming and new UNICEF programming. There were some accounts that the relationship could be enhanced even more in order to improve both entities’ programming. Jamaica The Jamaica Mission had been supporting activities addressing problems related to citizen security prior to the initiation of CBSI. While not initially designed to directly address CBSI objectives, given their close alignment, many of these activities were adapted midstream to fit with CBSI. In general, the team found that Jamaica’s CBSI activities do not overlap with each other in their objectives or activities, as all are working in quite distinct areas. Several implementers are working with others to leverage strengths and opportunities; however, this is occurring in an ad hoc manner, and few concrete collaborative activities have emerged. Some potential synergies between activities have not yet been developed and, , in the absence of strong coordination mechanisms, there is a definite risk of overlaps between COMET II and other agencies working in communities to foster participation in improved citizen security. In general, there is a reasonable level of operational information sharing and coordination between USAID and USG agencies on CBSI activities in the form of weekly training updates and monthly working group meetings. However, this is partially reliant on individuals’ motivation and relations with others, and the transition of personnel in and out of the Embassy poses a risk to the sustainability of current levels of coordination. On the surface, the division of labor between USAID and INL appears to be complementary, but shifts in direction by either agency (as observed in the other CBSI Missions) could easily jeopardize that complementarity, as the roles are not clearly defined. 55 Cross-Cutting Findings and Conclusions The nature of the CBSI (and non-CBSI) programming of both USAID and INL continues to evolve. In recent years, both agencies appear to have ramped up (or are planning to ramp up) their work on at￾risk youth, anti-corruption, institutional strengthening of law enforcement and judicial sectors, and related areas. Based on conversations with INL staff in DC, it is possible that in the future there could be areas where programming between the two groups may be able to either complement each other or overlap with one another. Examples include Jamaica, where INL is planning anti-corruption work that has the potential to either complement or overlap with any USAID follow-on of the CCJ activity. Another example of this is that in the region covered by USAID/ESC, INL is considering an expansion of RAPP program to other countries. Given how RAPP and Kari Yu! were able to complement each other in Suriname, this could result in a similar relationship in other countries. There are also signs of increased interest in INL to do more justice sector work that could either complement or overlap with any follow-on to JJRP. In the Dominican Republic, USAID’s shift towards a more explicit emphasis on citizen security has led to new activities that are more closely related to ongoing law enforcement support activities of INL. Given this situation, one informant noted that “there is always going to be overlap. It’s a question of minimizing, not eliminating it. This is the price to pay for multiple agencies being involved.” As priorities evolve in these countries, it creates the potential for either synergy or overlap and highlights the need for strengthened communications between the two agencies. Based on conversations with informants, the team identified that there are two main levels of communication: that at the Washington level and that at the Mission/Embassy level. At the Washington level, coordination tends to focus on larger policy issues rather than specific activities. Currently there are two main avenues for in-person information sharing and coordination between USAID and DoS: large CBSI interagency coordination meetings and a smaller group of CBSI funding coordinators. The interagency coordination meetings occur once a quarter, where policy issues and major activities are discussed. The number of attendees varies, but there is generally representation from the DoS, Department of Justice, USAID, and Department of Defense. In general, this meeting does not address overall strategic planning or specific activity planning; there is often a specific theme selected and those interested will attend. The bi-weekly CBSI coordinators group meets at the DoS offices and focuses on upcoming CBSI activities, public diplomacy updates, travel plans, and each coordinator’s agency updates. The attendees include WHA/CAR, INL, Pol-Mil, USAID, PPC, and the WHA Security Advisor. Otherwise, coordination by USAID and INL related to individual CBSI activities is on an ad hoc basis: one notable example was the participation of the USAID CBSI Coordinator, Debra Banks, on a review panel for INL’s RAPP activity; this was seen as an exception to the more common practice of keeping within agency silos. The findings suggest that this type of early interaction can set the tone for how activities funded by different agencies will relate to each other and foster a framework for increasing complementarity. At the Mission/Embassy level, information sharing and coordination have been widely variable, although so far it appears that major overlaps have been avoided. In general, the level of communication between staff of the two agencies tends to be based on personalities, and information sharing does not occur systematically. In Jamaica and the DR, the Law Enforcement Working Group is sometimes used to discuss CBSI issues, although not consistently. In Barbados, USAID/ESC does not participate in meetings of that Group. On the other hand, in Guyana a specific working group has been established in the Embassy to bring together all agencies with CBSI activities in that country. Every two weeks, representatives share updates on the work they are doing and planning, and it was reported that activities were complementing one another as a result. While some other Missions have considered a similar forum, none has been established to date. Since the Missions and Embassies are the designers and 56 drivers of most USAID and INL activities, effective communications strategies at the local level are essential for assuring true complementarity and avoiding overlaps among all USG interventions. Informants in both USAID and DoS expressed concerns that overall funding for CBSI would continue to decline in coming years, primarily owing to emerging priorities in other regions seen as having more serious security problems or being more strategically significant for U.S. interests. With the potential for increased budget constraints on both DoS and USAID, several informants believed that the chance for overlaps and even unintended competition would also increase. According to some informants and the documentation review, USAID and INL have different operational styles, with the former inclined to work on the basis of detailed multi-year plans and produce lengthy documents to describe its strategies and activities, which take considerable time to prepare and are not always easily digestible by external readers. On the other hand, informants described INL as more likely to base activities on more flexible and less wordy work plans lasting one year (or less), and to support “one-off” or short-term interventions. This has been one factor in the “disconnect” observed by multiple informants, one of whom suggested that USAID communications directed at INL and other USG agencies would be much more effective if adapted to that specific audience, including short summaries of lengthy documents. In addition, coordination is complicated by the fact that most USAID activity planning and management takes place at the Mission level, while for INL, those processes tend to be more centralized in Washington. ASSESSMENT QUESTION 3 RECOMMENDATIONS Dominican Republic 1. USAID/DR should clearly and concisely document the exact scope of activity (including any specific limitations) for each CBSI activity dealing with law enforcement authorities in any way, including any adjustments over time or contingency plans and ensure that information is shared with other CBSI implementers, USG agencies, relevant national counterparts, and international actors in the sector. 2. USAID/DR should clearly define responsibility for CBSI coordination within its team, including development of a detailed scope of work for the person responsible, suggested guidelines for information sharing among activities and other mechanisms, and a system for liaison between that person and the other USG agencies involved in citizen security interventions. Eastern and Southern Caribbean 3. USAID/ESC and INL should review current programming across the region to identify any existing or pending areas of potential synergy, complementarity, or overlap. One current area of opportunity is the work of the INL Justice Advisor; USAID should keep informed of the INL Justice Advisor’s work plan, and coordinate with INL to capitalize on synergies. Another is related to INL’s planned expansion of RAPP (and other activities) and the rollout of new USAID activities. 4. USAID/ESC should seek to enhance coordination with UNICEF in areas related to child protection, at-risk youth, and juvenile justice to identify priority needs and leverage resources. Jamaica 5. To facilitate an integrated rollout of the country’s electronic case-management system, USAID and ANP should identify ways to integrate its plans with the CDA, which has been making significant progress on the development of such a system. USAID should also consider an assessment of how this system can be useful to other CBSI activities and countries. Cross-Cutting Recommendations 6. If they have not already done so, each Mission in the region should organize periodic (at least quarterly) encounters among CBSI implementers in their respective coverage areas to share updates 57 on activity progress and plans, coordinate and synergize activities as appropriate, and exchange knowledge and lessons learned. These encounters should include activities working in thematic areas that are not directly related, especially where there is current or potential geographic overlap. Multilateral meetings could be supplemented by bilateral exchanges for more closely related activities. Depending on the situation, these could be in-person or remote meetings. 7. In cases where CBSI interventions have closely related objectives and/or target groups, Each Mission should consider arranging meetings between the CORs/AOR of its justice and law enforcement– related activities and INL staff. As appropriate, implementing partners should be included. These should be regular, one-on-one meetings to share information and coordinate as needed. As needed, the Embassy should facilitate these exchanges, which should occur at least quarterly. 8. To optimize complementarity in the longer term, each Mission in the region should consider creation of a multi-agency working group on rule of law or citizen security, which could be tasked with devising an overall national strategy for USG support to civilian police and other law enforcement bodies. 9. CBSI coordinators should collaborate with decision makers from USAID and DoS in both Washington and the Missions in the region to develop a clear allocation of types of programming between USAID and INL that could serve as a guideline for their respective staff in Caribbean posts as they develop strategies and design/adjust activities. This allocation should be the basis for division of responsibilities in each post, although exceptions could be agreed upon in order to respond to specific situations. 10. While meetings at the Washington level are useful for tackling overarching issues, increased coordination at the Mission/Embassy level is needed to address items at the activity level. Each Embassy with CBSI activities should consider establishing a quarterly working group based on the Guyana experience and examine other potentially useful tools for information sharing such as the weekly training list circulated by INL in Jamaica. 11. USAID in LAC and in Caribbean Missions should systematically engage its INL counterparts (and vice versa) in strategic planning, activity design, and other major decision processes related to areas of common interest, especially law enforcement. Planning documents should be openly and regularly shared at both the Washington and Mission levels in drafting stages and targeted summaries with essential information used to supplement the sharing of complex documents. 58 ANNEXES 59 ANNEX I: STATEMENT OF WORK80 II. ASSESSMENT RATIONALE A. Assessment Purpose The U.S. Agency for International Development (USAID) is in the fifth year of implementation of the Caribbean Basin Security Initiative (CBSI) and wishes to do an assessment of progress to date. The purpose of this assessment is to review both completed and ongoing CBSI programs across the 12 countries where USAID is implementing programs in order to assess: (1) the progress of each USAID Mission’s implementation of CBSI programs; and (2) whether USAID’s CBSI programs are using technically sound programmatic approaches. Audience  CBSI implementers  Caribbean governments  Regional organizations (Caribbean Community (CARICOM) and the Organization of Eastern Caribbean States (OECS))  Academic institutions (e.g., University of the West Indies, University of Arizona)  USAID CBSI coordinator in Washington  U.S. Department of State CBSI coordinators in Washington  Caribbean Missions and posts implementing CBSI programs in the region Intended uses  Guiding future USAID CBSI programming in the region  Identifying new areas of research and/or implementation for USAID CBSI programs  Locating programmatic and/or geographic gaps in CBSI programs to be filled by USAID, other U.S. Government (USG) actors, other donors, and/or partner country governments B. Assessment Questions  To what extent do USAID’s CBSI programs reflect the most current understanding of best practices in citizen security, particularly in the Caribbean context?  Have there been any unanticipated challenges or opportunities that have affected implementation of CBSI programs? This could include internal factors such as: USAID’s strategic planning processes, procurement approval processes, staffing, etc. It could also include external 80 While there was no Statement of Work as such, these sections outlining the work requested of Social Impact at the award stage are adapted from pages 9–13 of the original contract. 60 factors such as host countries’ legal frameworks, political will, diplomatic influence of the U.S. Embassy, changes in political administration, etc.  To what degree are USAID’s CBSI programs complementary with other USG CBSI programs, to include State Department, Department of Defense, and other agencies? Are there instances where these programs are overlapping or working at cross-purposes? III. BACKGROUND A. Identifying Information of the Initiative to be Evaluated CBSI is one pillar of a U.S. security strategy focused on citizen security throughout the hemisphere. CBSI brings members of CARICOM and the Dominican Republic together to jointly collaborate on regional security with the United States as a partner. The United States has committed more than $326 million in funding to the CBSI since 2010. The United States and Caribbean countries have identified three strategic priorities to address the security threats facing the Caribbean:  Substantially Reduce Illicit Trafficking: through programs ranging from counternarcotics to reducing the flow of illegal arms/light weapons.  Increase Public Safety and Security: through programs ranging from reducing crime and violence to improving border security.  Promote Social Justice: through programs designed to promote justice sector reform, combat government corruption, and assist vulnerable populations at risk of recruitment into criminal organizations. USAID is the lead USG agency under the pillar of Social Justice; through this initiative USAID works with the partner governments and other stakeholders to provide social and economic opportunities for at￾risk youth and their communities; improve juvenile justice systems’ focus on rehabilitation; enhance cooperation between communities and law enforcement; and reduce corruption. To date, USAID is implementing activities valued at $102 million for these efforts in Antigua and Barbuda; Barbados; Dominica; the Dominican Republic; Grenada; Guyana; Jamaica; St. Lucia; St. Kitts and Nevis; St. Vincent and the Grenadines; Suriname; and Trinidad and Tobago. Interventions are tailored to each individual country, but all focus on two primary goals: a. Improving justice sector and social services institutions so that they can better prevent and mitigate the effects of insecurity; and b. Providing improved social and economic opportunities for at-risk youth and their communities. B. Approach and Intended Results USAID works with Caribbean governments and citizens to address insecurity by focusing on four objectives: 1. Increased Economic Opportunities for At-Risk Youth and Vulnerable Populations. USAID provides workforce development training and remedial education programs to at-risk youth. Since 2011, almost 92,000 young people have participated in USAID’s education and workforce development activities through CBSI. 2. Improved Community and Law Enforcement Cooperation. USAID community-oriented policing activities work to improve effectiveness and professionalism of police; they also work with communities and police to increase mutual trust and cooperation on crime prevention initiatives. 61 3. Improved Juvenile Justice Sector. USAID supports regulatory reform for juvenile justice systems, works with police and judges to divert youth from traditional prison sentences, and works within juvenile detention centers to provide psychosocial support and work skills training for youth in custody. 4. Reduced Corruption in Public and Private Sectors. USAID has supported anti-corruption training for more than 1,600 government officials through CBSI, working with the Customs Department and the Tax Administration Jamaica (TAJ) to reduce traffickers’ ability to move money and contraband. TAJ has recovered more than $100 million of funds through a USAID program that cost less than $2 million. C. Existing Data A variety of documents and datasets will be made available to the assessment team upon award, including but not limited to the Regional/Country Development Cooperation Strategy for each USAID Caribbean Mission (USAID/Eastern and Southern Caribbean (USAID/ESC), USAID/Dominican Republic, and USAID/Jamaica); program descriptions for each program; and any completed evaluations of CBSI programs. IV. ASSESSMENT DESIGN AND METHODOLOGY A. Assessment Designs The Contractor must answer the three specific assessment questions presented above in the performance assessment. The Contractor’s conceptual approach to answer these questions will start with CBSI activity monitoring data and will be complemented with third-party data sources and information obtained during fieldwork including key informant interviews, perception surveys, observations from site visits, focus groups, consultations with relevant stakeholders, and other data￾gathering methods as proposed by the assessment team in its inception report. The Contractor’s independent external consultants must work in conjunction with other team members to plan and implement the proposed assessment. USAID staff from Washington and from USAID’s Caribbean Missions will need to be involved with design, planning, and logistics, but the assessment team must provide significant and overall leadership and direction, as well as full responsibility for the assessment duties and deliverables. B. Data Collection and Analysis Methods The assessment team will start by reviewing the activity monitoring information collected by USAID CBSI programs and identify information gaps to guide additional data collection required as part of the assessment. Data requirements, collection methods, and required analyses will be explicitly defined in the inception report. Once existing information has been thoroughly reviewed and gaps in monitoring information for purposes of the assessment are identified, the assessment team will complete the inception report, accompanied by narrative describing its content and the rationale behind the intended approach. As with all deliverables, the first drafts may receive comments and requests for improvement before approval is provided. The inception report will be prepared and discussed and a final version approved by the Task Order Contracting Officer’s Representative before fieldwork will be approved. Interviews with activity staff cannot be the sole source of information collected by the assessment team to complement monitoring data and answer the assessment questions. Documentary evidence and discussions with community and other stakeholders will be important, among other sources the assessment team may propose. Consistent with ADS 203.3.1.6 guidance on assessment methodologies, a combination of quantitative and qualitative methods in data collection and analysis should be employed by the assessment team in the process. A triangulation method should be applied by the assessment team to increase the level of validity in data collection and processing of results. 62 USAID encourages the assessment team to use a multiple-perspective view that triangulates data sources. Even if an assessment question can be answered using only one method, often it is preferable to combine multiple methods to answer the same question in order to gain a more complete understanding of the issue and more confidence in the findings. By approaching the same question from more than one perspective or by using more than one technique, evaluators can then compare and contrast the results from these different methods. If the findings from the different methods are similar, or reinforce one another, then users can have greater confidence in the findings than if they are based on only one method. If, on the other hand, the findings from different sources do not coincide, that often signals the need for deeper probing and investigation before reliable conclusions can be determined. The assessment team must ensure that its methods are gender-sensitive, capturing quantitative and qualitative data on how USAID’s CBSI programs address issues specific to women and girls; boys and men; and lesbian, gay, bisexual, transgender/gender non-conforming, and intersex (LGBTI) persons. Site Visits and Data Collection USAID supports CBSI efforts across 12 countries, but the team is only expected to visit some of these. The number of sites to be visited, along with the process for their selection, needs to be stated explicitly in the inception report and determined in consultation with USAID. It will likely include Jamaica and the Dominican Republic, as well as a representative selection of the 10 countries covered by USAID/ESC. Visits will be conducted both in capitals in order to meet with government officials (from national governments and regional organizations, i.e., CARICOM and the OECS, as appropriate) and in communities where CBSI programs are implemented in order to speak with program staff and beneficiaries. 63 ANNEX II: DEFINITION OF CBSI ASSESSMENT QUESTIONS Question Interpretation 1. To what extent do USAID’s CBSI programs reflect the most current understanding of best practices in citizen security, particularly in the Caribbean context? Current understanding of best practices in citizen security = methods and program models that have been demonstrated and clearly documented as effective in improving citizen security (as defined below). Assessment will focus on practices developed in the Caribbean region (via literature review and primary research) but also conduct limited desk research into practices from outside the region in the main programmatic sectors addressed by CBSI. Emphasis will be on practices documented since 2010. Assessment will examine the extent to which both initial design and subsequent implementation of programs have been consistent with the identified best practices. This will include consideration of how groups such as women, persons with disabilities, and LGBT individuals have been affected by or engaged in programming. Citizen security = activities or initiatives aimed at tackling the issues highlighted in the four objectives of USAID’s CBSI social justice programming: 1. Increased economic opportunities for at-risk youth and specific populations listed above. 2. Improved community and law enforcement cooperation. 3. Improved juvenile justice sector. 4. Reduced corruption in public and private sectors. 2. Have there been any unanticipated challenges or opportunities that have affected implementation of CBSI programs? This could include internal factors such as USAID’s strategic planning processes, procurement approval processes, staffing, etc. It could also include external factors such as host countries’ legal frameworks, political will, diplomatic influence of the U.S. Embassy, changes in political administration, etc. Unanticipated = not taken into account in the design by USAID and/or the implementer(s), which will largely be analyzed by review of activity design, award, and initial planning documentation to see if actual challenges and opportunities (that are found to emerge during implementation) were considered or expected at that time. Assessment will outline the major effects that such unanticipated challenges and opportunities have had on USAID CBSI programming, on activity-specific level, and more broadly, as applicable. Challenges = events or circumstances that have impeded the implementation of USAID CBSI programs and/or the achievement of expected results. Opportunities = events or circumstances that have promoted or facilitated the implementation of USAID CBSI programs and/or the achievement of expected results. External includes factors related to security conditions, political will and other political developments, legal framework, relations among USG, implementers and host government, actions of other donors or organizations working in related sectors, etc. Internal includes factors related to staffing of USAID or activity 64 Question Interpretation implementers, USAID planning, reporting and approval processes, available funding (including reductions or increases in resources), communications and coordination among activities, etc. 3. To what degree are USAID’s CBSI programs complementary with other USG CBSI programs, to include DoS, Department of Defense, and other agencies? Are there instances where these programs are overlapping or working at cross-purposes? Complementary = a) design of USAID CBSI programs is not conflicting or overlapping with those other USG programs and is supportive of their goals b) implementation of USAID CBSI programs is not conflicting or overlapping with those other USG programs and is supportive of their goals c) responsible staff in USAID and its implementers are coordinating with their counterparts in other related programs, as appropriate The assessment will examine these issues and identify any key areas of activity in which programming is not complementary, while also highlighting examples of complementarity in design or implementation. 65 ANNEX III: CBSI ACTIVITY DESCRIPTIONS USAID/DR At-Risk Youth Initiative (Alerta Joven) The Alerta Joven activity is a five-year at-risk youth program that combines academic and vocational education, job placement, and HIV/AIDS counseling and testing services in a holistic effort to improve social and economic opportunities and outcomes for this vulnerable population. The activity began in July 2012 and is slated to be completed in July of 2017. In order to provide relevant training, the activity conducted a labor market survey to identify labor needs in the targeted communities and in the overall economy. This program began by identifying what jobs are available or are likely to be created in the private and public sectors and then worked backwards through the “supply chain” to design vocational education programs that satisfy market demand for specific labor shortages.81 Based on the implementation data that have been received for the first few years of the program, Alerta Joven has made strides in reaching large numbers of participants through coordinated programming. The program reports that, as of June 30, 2015, it has reached 74,474 youth through 26 partner organizations. Community Justice Houses (CJH) Community Justice Houses offer residents an open and safe space to access critical services in dispute mediation, along with support and related services for victims of family and gender-based violence, linked to the criminal justice system where necessary. Community Justice Houses operate primarily in high-crime, under-served municipalities in cities such as Santo Domingo, Santiago, La Vega, and San Francisco de Macoris. Currently, there are eight community justice houses run by the Dominican organization Participación Ciudadana, all of them started with USAID/Dominican Republic funds.82 Previous funding was direct to that organization, although since mid-2015 it is a sub-grantee of Chemonics under the larger CJSSP activity described below. The Community Justice Houses reach over 20,000 people annually. Importantly, 75% of the total operating costs of each Community Justice House are covered by non-U.S. government sources (CBSI contract). However, USAID is aiming to establish new houses that will provide 125,000 people in marginalized urban neighborhoods with access to justice services, including ADR and conflict mediation. Criminal Justice Systems Strengthening Project (CJSSP) The program principally targets key directorates of the Office of the Attorney General (including Women and Gender, Human Rights, and Complex Crimes). The activity, led by Chemonics, seeks to build the government’s capacity to conduct prosecutions by improving police/prosecutor coordination, strengthening case management, and expanding prosecutor availability in high-volume crime districts such as the Santo Domingo province. Begun in mid-2015, the program is scheduled to run until 2020. 81 USAID/Dominican Republic, Dominican Republic Country Development Cooperation Strategy, 2014. 82 Ibid. 66 Institutional Capacity and Transparency Strengthening for Police Reform (Police Reform Project) This four-year activity launched in mid-2015 provides training and technical assistance to strengthen police transparency and management, with an emphasis on leadership, planning, inter-agency coordination, and community relations. The training is also offered on the management of family and gender-based violence, and protecting other vulnerable populations. This programming is implemented by the International Criminal Investigative Training Assistance Program (ICITAP) under the Department of Justice. The activity is composed of four components: 1) Development of the Dominican National Police’s (DNP) Management Capacity, Minimum Standards, and Organizational Capacity Building to Strengthen Internal and External Oversight Practices 2) Modernization of the DNP’s Human Resources Department and Support Sustainability of Training Capacity in the DNP Training Academies 3) Improvement of Police, Investigator, and National Prosecutor Communication and Coordination 4) Strengthening of Community Relations Practices, Civil Society Coordination, and Policing Services for Vulnerable Populations USAID/ESC Juvenile Justice Reform Project (JJRP) Implemented by the OECS Commission, JJRP has a multi-country focus and includes Antigua and Barbuda, Dominica, Grenada, St. Kitts and Nevis, St. Lucia, and St. Vincent and the Grenadines. JJRP is a four-year, $5.8 million initiative that is slated to conclude on June 30, 2016.83 This activity seeks to address the problem of limited formal sentencing options available for juveniles who come into conflict with the law, with non-custodial sentences and court avoidance procedures being uncommon. In these countries, there is a lack of investment in alternative sentencing and restorative justice models. JJRP seeks to ameliorate this issue by strengthening the juvenile justice systems in the aforementioned countries. The intervention encompasses a group of juvenile justice programs at both the national and sub-regional levels. The activity has four main components: improved legal and regulatory frameworks for the juvenile justice system (Component 1); capacity building for effective administration of juvenile justice (Component 2); modernization of diversion, detention, and rehabilitative processes in the juvenile justice system (Component 3); and improving linkages with civil society and other supporting structures 83 USAID, Juvenile Justice Reform for the OECS Summary Sheet, (accessed on January 5, 2016). 67 to strengthen the system (Component 4).84 The main activities under each of these objective areas are as follows: Component 1: Working with participating countries on the development and adoption of juvenile justice policy statements that encourage harmonization across national policies and on legislation that ensures that their laws are in line with international conventions, such as the Convention on the Rights of the Child, that have been signed and ratified by the Member States. The program provides technical assistance through short-term consultancies to achieve these goals.85 Component 2: Training members of the judiciary, social workers, probation officers, and others to ensure full understanding of the laws and expected practices required to support an effective juvenile justice system. The program also helps countries to develop procedures that guide the operations of key aspects of the juvenile justice system. Additional activities include strengthening statistical and research capabilities to support databases that facilitate effective decision making; building capacity in monitoring and evaluation of juvenile justice systems; testing mental health screening tools; and introducing case￾management software that improves the level of care and connects the various agencies.86 Component 3: Funding the rehabilitation of youth detention facilities in St. Kitts, Antigua, Dominica, St. Lucia, Grenada, and St. Vincent. This component provides the equipment necessary to ensure the full functionality of juvenile facilities, including those managed by NGOs.87 Component 4: Developing a Functional Family Framework in each country. Under this objective the program also includes media training and the development of public awareness and education strategies to support increased understanding by the populace of the rights of children and to support greater advocacy for a reformed juvenile justice system.88 Strengthening Second Chance Education in the Eastern Caribbean This program sought to build a sustainable infrastructure that would allow at-risk groups of all interests and abilities to access programs leading to development of valuable life skills, training/retraining for the world of work, the attainment of certification, and a continuing education platform for future outcomes. This program was implemented by Caribbean Examinations Council (CXC) in nine countries in the Caribbean. The program had three broad objectives:  Objective 1: Building strategic alliances/learning partnerships with public, private, and nonprofit organizations 84 Ibid. 85 Organization of Eastern Caribbean States, OECS Juvenile Justice Reform Project Extension Workplan (October 2014 – October 2015), United States Agency for International Development, (accessed on January 5, 2016). 86 Ibid. 87 Ibid. 88 Ibid. 68  Objective 2: Taking steps to build a quality management culture and environment  Objective 3: Expanding access to technology in education to facilitate e-learning89 Kari Yu! Youth Development and Juvenile Justice Project The Kari-Yu! initiative, implemented by the Pan American Development Foundation (PADF), has two interrelated goals: (1) to expand opportunities for at-risk youth ages 15 to 24; and (2) strengthen the juvenile justice system in Suriname. PADF implemented the program in two phases, the first being an assessment period and the second, implementation. Program activities include training, mentorship, and internship opportunities for at-risk youth. In addition, the activity promotes capacity building for youth￾oriented public-sector ministries, private-sector agencies, and nongovernmental organizations (NGOs). Kari Yu! is scheduled to end September 30, 2016. As of the end of FY2015, the program had made some significant progress against its indicators for both objectives. Examples include the number of juvenile justice reforms introduced, the number of youth accessing and maintaining internship opportunities, and the number of youth participating in vocational training and education opportunities.90 A Ganar A Ganar leveraged a sports-based curriculum to address youth employment and engagement challenges through a 7–9 month integrated job training program that includes vocational training, life skills training, and other complementary activities. This sports-based focus sought to impact youth between the ages of 16–24 years or those considered at risk as a result of dropping out of school or not achieving the requisite school-leaving certification by providing training opportunities to enable them to become employable and contribute positively in the region. The program operated in four countries under the purview of USAID/ESC: Dominica, St. Kitts and Nevis, St. Vincent and the Grenadines, and Suriname. The program began in 2009 and closed out in September 2015. A Ganar was implemented through four major phases:  Phase I: Employability skills enhancement. Youths are taught teamwork, communication, discipline, respect, a focus on results, and continual self-improvement through a sports-based lens.  Phase II: Vocational skills training. Youths are trained on market-relevant technical, entrepreneurial, and/or vocational skills so that they are well positioned to enter the labor force.  Phase III: Internship. Youths are encouraged to apply the knowledge and skills gained through the earlier phases in a professional environment. 89 USAID, Strengthening Second Chance Education in the Eastern Caribbean, AID-538-A-12-00005, 26 Sept. 2012. Contract (accessed January 11, 2016). 90 Pan-American Development Foundation, Kari Yu! Annual Report, October 1, 2014 – September 30, 2015. PADF, 2015. 69  Phase IV: Follow-on support. Youths are supported in their efforts to find jobs, start a business, or further their education. Address Crime Prevention by Enhancing Skills and Increasing Economic Opportunities for At-Risk Youths (SKYE) The primary goal of SKYE is to reduce crime and violence in Guyana through strengthened economic participation and civic engagement of at-risk youths. The program achieves its objective in three ways: (1) working with the judiciary, government, and ministries to develop alternatives to detention for youth, as well as developing rehabilitation services for juvenile offenders; (2) providing formal training and mentorship program for at-risk youths, designed to improve life and employment skills; and (3) training youths in entrepreneurship and providing business support for youths to start their own businesses. As a part of the activity, the IP made efforts to include both the public and private sectors in these activities. Caribbean Youth Empowerment Program (CYEP) The objective of this six-year program was to offer vulnerable youth ages 17 to 25 training in life and technical skills, with the goal of increasing their participation in the job market.91 The activity implementers also included a focus on developing private and public partnerships to increase community support for the work. Within the program there were three main tracks the youth could follow: vocational, entrepreneurship, and career guidance.92 The program was implemented through 13 individual activities. The program ran in Antigua and Barbuda, Grenada, St. Lucia, and Jamaica, the first three of which fell under the umbrella of USAID/ESC.93 Overall, the program exceeded its target rate for participation in the program, with 75% of those who participated in one of the tracks graduating. Follow-up studies conducted six to nine months after graduation showed that on average 49% of youth were either employed or studying. The final activity report noted two distinct challenges across countries, including the global recession (an unexpected challenge), and highly competitive job markets. There was a midterm performance evaluation that covered both CYEP and A Ganar, which observed that the most notable perceived outcome was the positive effect of both programs on self-confidence, self-awareness, and outlook on life.94 Trinidad and Tobago Juvenile Court Project This program provides support to ensure the successful implementation of the package of children’s legislation enacted in Trinidad and Tobago, including by ensuring the judiciary’s ability to use the disposition powers provided and to encouraging the implementation of supportive mechanisms. It also 91 USAID, Caribbean Youth Empowerment Program, Final Report, The International Youth Foundation, USAID, 2014. 92 Ibid. 93 Ibid. 94 Balestino, Ramon, Mark Bardini, and Lindsay North, A Ganar and Caribbean Youth Empowerment Program: A Mid￾term Performance Evaluation, Dexis Consulting Group and Management Systems International, 2013. 70 seeks to improve critical aspects of the juvenile justice system through systemic reforms and related capacity building. The program has funding to implement activities from 2014 to 2017. Core activities planned are:  Establishment of two pilot juvenile courts in Trinidad and one in Tobago that meet standards for adjudicating juvenile cases.  Building of institutional capacity of juvenile courts to adjudicate cases in an efficient and effective manner in accordance with new laws and principles of restorative justice.  Building of institutional capacity of organizations providing services to children in conflict with the law through the creation of effective referral and coordination mechanisms, training on new laws and procedures, improved programs, and improved ability to fulfill the reporting and monitoring requirements of the juvenile courts.  Establishment of youth/peer courts based on principles of restorative justice to facilitate the adjudication of minor offenses by children and youth.  Public awareness campaigns to educate and sensitize the public to the reform of juvenile justice system and the implementation of the Children’s Act. USAID/JAMAICA Combatting Corruption in Jamaica The Combatting Corruption in Jamaica (CCJ) program seeks to change cultural attitudes toward corruption using a three-tiered approach of raising public awareness of corruption; increasing public demand for a cleaner government; and improving institutional, legal, judicial, and media infrastructure to support anti-corruption activities. The program aims to build capacity through training and workshops of key judicial, law enforcement, and legal officials, among other strategies. Community Empowerment and Transformation II (COMET II) COMET II seeks to build safer communities through strengthening community and civil society organizations, increasing accountability, supporting at-risk youth organizations, and improving community policing practices. COMET II is an extension of COMET, which was implemented between 2006 and 2012, and began with a focus on community-based policing in three communities—Central Village, Flanker, and Grant’s Pen. The program has since evolved to focus on providing institutional support to the Jamaica Constabulary Force (JCF), particularly as it relates to the national roll-out of community-based policing and implementing the recommendations of the JCF Strategic Review. COMET II is also expanding to integrate global climate change adaptation issues into select activities, particularly by helping urban poor communities to be better prepared for extreme weather changes. Empowering Jamaica's Youth (Junior Achievement) The Junior Achievement Program (JA) is known internationally as an evidence-based model for positive youth development. This Jamaica program seeks to establish a viable, sustainable, and functioning JA organization as a local NGO capable of offering quality educational programs and mentors to introduce children and youth to fundamental concepts in business, economics, and individual accountability. JA Jamaica targets youth aged 5 to 25 with a special emphasis on at-risk youth in the Government of Jamaica’s priority Community Renewal Program Communities. 71 Since inception, the program has trained more than 40,000 youth in approximately 200 schools and local community-based organizations in basic financial literacy, workforce readiness, and entrepreneurship. The JA programs have also been adapted and taught in all schools for the Deaf and Hard of Hearing across Jamaica, and some out-of-school deaf youths have successfully attained full-time employment after undergoing a residential JA Success Skills program. A New Path: Promoting a Healthy Environment and Productive Alternatives for Juvenile Remandees and Offenders in Jamaica A New Path is a comprehensive initiative to address the emotional, social, educational, and economic challenges faced by Jamaican youth who have been in the justice system. The program spans 28 months and seeks to benefit approximately 500 female youths housed at South Camp and their families. The program offers the young women at the facility marketable technical skills, life skills, and individualized psychosocial attention to assist in their successful reintegration into society. The program also offers those released from the facilities an opportunity to access educational, vocational, and internship/employment opportunities through community-based support and monitoring for six to twelve months following release. A major effort is being made by the program and national stakeholders to develop a comprehensive case-management system that would facilitate information sharing and tracking of services provided to youth involved with the social protection system to ensure a more structured and successful integration into society. USAID/MoE Partnership for Improved Reading Outcomes Targeting areas facing risk factors such as crime, poverty, and unemployment, the Partnership for Improved Reading Outcomes is a government-to-government effort aimed at improving reading for students in grades 1 to 3 from 450 poorly-performing primary and all-age schools. The program seeks to enhance the knowledge and capacity of teachers, school principals, education officers, and parents to further support children’s reading performance. This program supports 54 schools in Regions 1, 4, and 6, reaching approximately 11,000 students and 200 teachers. The program has reportedly been successful in improving primary education in schools in the regions with the highest crime rates, which are also part of the Government of Jamaica’s Community Renewal Program. Fi Wi Jamaica Fi Wi Jamaica seeks to empower members of socially excluded and vulnerable groups through social, cultural, and economic interventions. The program aims to promote the protection and respect of human rights and dignity of all, regardless of gender and sexual orientation, by raising the level of dialogue regarding inclusivity, diversity, and fairness for the lesbian, gay, bisexual, transgender, and intersex (LGBTI) community, as well as support key partners in delivering services to women and girls who are victims or potential victims of gender-based violence and trafficking. The program will capitalize on existing Government of Jamaica and USAID/Jamaica efforts to advance the freedom, dignity, and development of women and girls and members of the LGBTI community. Program implementation began in 2015 and will continue to 2018. The program is still in early stages of implementation and has yet to produce results. 72 ANNEX IV: DATA COLLECTION INSTRUMENTS This annex contains the following basic guides for data collection: a. KII for USAID Officials b. KII for Program Implementers c. KII for Department of State Officials d. KII for National Government Counterparts e. KII for External Stakeholders/Experts f. FGD for Secondary Implementers (Jamaica) g. FGD for Alerta Joven NGO Partners (Dominican Republic) It should be noted that these are general guides with questions to ask across respondents/groups from each category of informant, so that certain results can be compared across these groups. Depending on the stakeholder, particular questions were omitted if they were not relevant. Ultimately, protocols were specifically tailored for each respondent and group as more relevant details became available. The protocols listed here are the ones that were finalized during the pilot stage of data collection. KEY INFORMANT INTERVIEW GUIDES Questions for USAID officials Overall Questions: 1) Which USAID CBSI program(s) are you currently associated with, and what is your role in relation to that program? (Probe for which program(s) they are most knowledgeable about.) 2) What is the primary problem which that program aims to address? 3) What is the major achievement of that program so far? EQ2 Questions: 4) What have been the greatest challenges experienced while implementing that program? (Probe to ask about challenges within USAID and implementer, and external factors.) 5) Were those challenges expected by the designers or implementers of that program (and which ones)? What effect have they had on the program? 73 6) Have there been unanticipated new developments that have presented opportunities for the program? What effect have they had on the program? EQ1 Questions: 7) What are some outstanding examples of current best practices in the Caribbean in the subject area of that program? Why do you consider these best practices? Note: as needed, prompt for the following areas:  Increased economic opportunities for at-risk youth and vulnerable populations.  Improved community and law enforcement cooperation.  Improved juvenile justice sector.  Reduced corruption in public and private sectors. 8) To what extent does the program reflect best practices? (Probe: Are there some elements or activities that are inconsistent with best practices, or simply outdated in their approaches?) 9) (depending on answer to #8) How could CBSI programming (current and future) be brought more in line with best practices in citizen security? 10) Does the program effectively engage with and support vulnerable groups (such as women, persons with disabilities, LGBTI individuals)? Does it use best practices to do so? If not, what could be improved? EQ3 Questions: 11) Besides the program with which you are involved, what other CBSI implementers and/or programs are you aware of? (Probe for USAID and other USG programming.) 12) Do these CBSI programs seem to be complementary, in the sense of supporting each other’s objectives? (Probe for examples.) 13) Have you noticed any overlapping or inconsistencies between the work that USAID is supporting, and what the other agencies are doing? 14) To what extent do those responsible for CBSI programs (in USAID or implementers) share information and try to coordinate with other programs? (Probe for any mechanisms or procedures that may exist to facilitate this.) 15) Generally, how could CBSI programs be more complementary of each other? Closing 16) Do you have any other recommendations or comments that you would like to share with us? 17) Do you have any specific suggestions of people with whom we should speak in relation to this assessment? 18) Any documents that you would like to share with us, related to these questions? 74 Questions for Program Implementers Overall Questions: 1) Which USAID CBSI program(s) are you currently associated with, and what is your role in relation to that program? 2) What is the primary problem which the program aims to address? 3) What is the major achievement of the program so far? EQ2 Questions: 4) What have been the greatest challenges experienced while implementing the program? (Probe to ask about challenges within USAID and implementer, and external factors.) 5) Were those challenges expected by the designers of the program (and which ones)? What effect have the unexpected challenges had on the program? 6) Have there been unanticipated new developments that have presented opportunities for the program? What effect have they had on the program? EQ1 Questions: 7) What are some outstanding examples of current best practices in the Caribbean in the subject area of the program? Why do you consider these best practices? Note: as needed, prompt for the following areas:  Increased economic opportunities for at-risk youth and vulnerable populations.  Improved community and law enforcement cooperation.  Improved juvenile justice sector.  Reduced corruption in public and private sectors. 8) To what extent does the program reflect best practices (in its design and implementation)? (Probe: Which aspects in particular are consistent with best practices? Are there some elements or activities that are inconsistent with best practices, or simply outdated in their approaches?) 9) (depending on answer to #8) How could the program be brought more in line with best practices? 10) Does the program effectively engage with and support vulnerable groups (such as women, disabled, LGBTI individuals)? Does it use best practices in that respect? If not, what could be improved? 75 EQ3 Questions: 11) What other CBSI implementers and/or programs are you aware of? (Probe for USAID and other USG programming.) 12) Do these CBSI programs seem to be complementary, in the sense of supporting each other’s objectives? (Probe for examples.) 13) Have you noticed any overlapping or inconsistencies between the work that USAID is supporting, and what the other agencies are doing? (Probe for examples.) 14) To what extent do those responsible for CBSI programs (in USAID or implementers) share information and try to coordinate? (Probe for any mechanisms or procedures that may exist to facilitate this.) 15) Generally, how could CBSI programs be more complementary of each other? Closing 16) Do you have any other recommendations or comments that you would like to share with us? 17) Do you have any specific suggestions of people with whom we should speak in relation to this assessment? 18) Any documents that you would like to share with us, related to these questions? Questions for Department of State officials Overall Questions: 1) Which CBSI program(s) of Dept. of State are you currently associated with, and what is your role in relation to that program? (Probe for which program(s) they are most knowledgeable about.) 2) What is the primary problem which that program aims to address? 3) In which countries is that program active? EQ3 Questions: 4) Which USAID-funded CBSI programs are you aware of? (Identify those in the same sector as the DoS program.) 5) Do the CBSI programs of USAID and State seem to be complementary, in the sense of supporting each other’s objectives? (Probe for examples.) 6) Have you noticed any overlapping or inconsistencies between the work that USAID and State are supporting, and/or what other U.S. agencies are doing? 76 7) To what extent do those responsible for CBSI programs (in State, USAID or implementers) share information and try to coordinate? (Probe for any mechanisms or procedures that may exist to facilitate this.) 8) Generally, how could CBSI programs be more complementary of each other? EQ1 Questions: 9) What are some outstanding examples of current best practices in the Caribbean in the subject area of that program? Why do you consider these best practices? Note: as needed, prompt for the following areas:  Increased economic opportunities for at-risk youth and vulnerable populations.  Improved community and law enforcement cooperation.  Improved juvenile justice sector.  Reduced corruption in public and private sectors. 10) To what extent does USAID CBSI programming reflect best practices? (Probe: Are there some elements or activities that are inconsistent with best practices, or simply outdated in their approaches?) 11) (depending on answer to #10) How could USAID CBSI programming (current and future) be brought more in line with best practices in citizen security? Closing 12) Do you have any other recommendations or comments that you would like to share with us? 13) Do you have any specific suggestions of people with whom we should speak in relation to this assessment? 14) Any documents that you would like to share with us, related to these questions? Questions for National Government Counterparts Overall Questions: 1) Which USAID CBSI program(s) are you currently associated with, and what is the role of your (ministry/agency/department) in relation to that program? (Probe to determine specific element of program they are involved in. Refer to that element throughout the interview, instead of program as a whole.) 2) What is the primary problem which the program aims to address? 3) What is the major achievement of that program so far? 77 EQ2 Questions: 4) What have been the greatest challenges experienced while implementing the program? (Probe to ask about challenges within USAID and implementer, and external factors.) 5) What effect have those challenges had on the program? 6) Have there been unexpected new developments that have presented opportunities for the program? What effect have they had on the program? EQ1 Questions: 7) What are some outstanding examples of current best practices in the Caribbean in the subject area of the program? Why do you consider these best practices? Note: as needed, prompt for the following areas:  Increased economic opportunities for at-risk youth and vulnerable populations.  Improved community and law enforcement cooperation.  Improved juvenile justice sector.  Reduced corruption in public and private sectors. 8) To what extent does the program reflect best practices (in its design and implementation)? (Probe: Which aspects in particular are consistent with best practices? Are there some elements or activities that are inconsistent with best practices, or simply outdated in their approaches?) 9) (depending on answer to #8) How could the program be brought more in line with best practices? 10) Does the program effectively engage with and support vulnerable groups (such as women, disabled, LGBTI individuals)? Does it use best practices in that respect? If not, what could be improved? EQ3 Questions: 11) What other CBSI implementers and/or programs are you aware of? (Probe for USAID and other USG programming.) 12) Do these CBSI programs seem to be complementary, in the sense of supporting each other’s objectives? (Probe for examples.) 13) Have you noticed any overlapping or inconsistencies between the work that USAID is supporting, and what other USG agencies are doing? (Probe for examples.) 14) To what extent do those responsible for CBSI programs (in USAID or implementers) share information and try to coordinate? (Probe for any mechanisms or procedures that may exist to facilitate this.) 15) Generally, how could CBSI programs be more complementary of each other? Closing 78 16) Do you have any other recommendations or comments that you would like to share with us? 17) Do you have any specific suggestions of people with whom we should speak in relation to this assessment? 18) Any documents that you would like to share with us, related to these questions? Questions for External Stakeholder/Experts Overall Questions: 1) Which USAID CBSI program(s) are you familiar with, and what is the role of you/your organization in relation to that program? (Probe to determine specific element of program they are involved in. Refer to that element throughout the interview, instead of program as a whole.) 2) What is the primary problem which the program aims to address? 3) What do you see as the major achievement of that program? EQ2 Questions: 4) What have been the greatest challenges experienced by that program? (Probe to ask about challenges within USAID and implementer, and external factors.) 5) What effect have those challenges had on the program? 6) Have there been unexpected new developments that have presented opportunities for the program? What effect have they had on the program? EQ1 Questions: 7) What are some outstanding examples of current best practices in the Caribbean in the subject area of the program? Why do you consider these best practices? Note: as needed, prompt for the following areas:  Increased economic opportunities for at-risk youth and vulnerable populations.  Improved community and law enforcement cooperation.  Improved juvenile justice sector.  Reduced corruption in public and private sectors. 8) To what extent does the program reflect best practices (in its design and implementation)? (Probe: Which aspects in particular are consistent with best practices? Are there some elements or activities that are inconsistent with best practices, or simply outdated in their approaches?) 9) (depending on answer to #8) How could the program be brought more in line with best practices? 79 10) Does the program effectively engage with and support vulnerable groups (such as women, disabled, LGBTI individuals)? Does it use best practices in that respect? If not, what could be improved? EQ3 Questions: 11) What other U.S. Government funded programs are you aware of in your sector? (Probe for USAID and other USG programming.) 12) Do these CBSI programs seem to be complementary, in the sense of supporting each other’s objectives? (Probe for examples.) 13) Have you noticed any overlapping or inconsistencies between the work that USAID is supporting, and what other USG agencies are doing? (Probe for examples.) Closing 14) Do you have any other recommendations or comments that you would like to share with us? 15) Do you have any specific suggestions of people with whom we should speak in relation to this assessment? 16) Any documents that you would like to share with us, related to these questions? FOCUS GROUP DISCUSSION GUIDES Questions for Focus Group with (Secondary) Implementers (Jamaica) Introduction: 1) Please go around and briefly identify which group you belong, your role within that group, and the primary activity the group does in relation to ‘X’ (insert specific CBSI activity of focus). EQ2 Questions: 2) Can you share a recent achievement of your activity? 3) What have been the greatest challenges for your activity? (Probe to ask about challenges with USAID, implementer, and external factors). 4) Were some of those challenges unexpected/did they take you by surprise? What effect have those challenges had on the activity? EQ1 Questions: 5) Do you think the activity reflects best practices in its design and implementation? (Are there specific aspects of the activity that strongly reflect best practices? Are there aspects that are inconsistent with best practices, or outdated in their approaches?) 80 6) If so, how do you know those are best practices? Are there ways that the activity can be brought more in line with best practices? 7) How does the activity engage with members of vulnerable groups (e.g., women, differently abled, LGBTI, etc.)? Does it use best practices in that respect? If not, what could be improved? EQ3 Questions: 8) Are you aware of any other CBSI (or US-funded) implementers in the region? (Probe for USAID and other USG agencies) 9) Do these CBSI (or US-funded) programs complement (in the sense of supporting each other’s objectives) or overlap each other’s work? Closing 10) Do you have any other recommendations or comments that you would like to share with us? 11) If you have any suggestions of people we should speak to or have any documents you would like to share with us, please do come talk to us after this meeting. Questions for Focus Groups with Alerta Joven NGO Partners (DR) Please go around and briefly identify which group you belong, your role within that group, and the primary activity the group does in relation to Alerta Joven. 1) What has been the greatest success so far of your work with Alerta Joven? (one thing only) 2) What do you think of the methods and approaches used and promoted by Entrena for this activity? Does the activity use methods that have already been successful (in DR or elsewhere)? screening/targeting methods, monitoring systems, atención integral, etc. 3) Are there ways that the activity can be improved, other successful models that could be tried? 4) What has been the greatest challenge of your collaboration with Alerta Joven? (Probe to ask about challenges with USAID, implementer, and external factors). Subgrant procedures?? 5) Were some challenges unexpected/did they take you by surprise? What effect have those challenges had? 6) How does the activity involve members of vulnerable groups (e.g., women, differently abled, LGBTI, etc.)? Are there ways that this can be improved, other methods? 7) What other U.S. Government funded programs do you know of in your sector (at-risk youth, health, education)? (Probe for USAID and other USG programming.) esp. Casas Comunitarias, peer mediation concept – are they aware? 8) Do these programs seem to be consistent with each other? Have you noticed any overlapping or inconsistencies? (Probe for examples.) Closing 9) Do you have any other recommendations or comments that you would like to share? 81 10) If you have any suggestions of people we should speak to or have any documents you would like to share with us, please do come talk to us after this meeting. ANNEX V: DOCUMENTS REVIEWED95 Arístides, Juan. Plan de Regularización Publicado, 14 May 2015 http://elnacional.com.do/plan-de￾regularizacion/ (accessed April 30, 2016). Balestino, Ramon, Mark Bardini and Lindsay North. A Ganar and Caribbean Youth Empowerment Program: A Mid-term Performance Evaluation. Dexis Consulting Group and Management Systems International, 2013. Britt, Charla, Jason Jackson and Rohan Jeremiah. Gender Assessment USAID/ Barbados and Eastern Caribbean. DevTech Systems, Inc, 2010. Chambers, Victoria. Securing communities and transforming policing cultures: A desk study of community policing in Jamaica. Overseas Development Institute, 2014. Chemonics International. Criminal Justice System Strengthened Project (CJSSP) Workplan (June 2015 – December 2016). USAID/Dominican Republic, 2015. Chemonics International, Inc. Assessment Report of Anti-trafficking in persons in the Dominican Republic. 2011. Coutino, Josefina, Rosalia Sosa, Elizabeth Ventura and Martha Contreras. USAID/ Dominican Republic Justice Project Final Report. Development Alternatives Incorporated, 2012. Democracy International. Caribbean Basin Security Initiative Juvenile Justice Assessment: Final Report, 2011. CARICOM Terms of Reference, CBSI Technical Working Group (TWG) on Preventing Crime by Focusing on At-Risk Youth and Vulnerable Populations, 2014. Caribbean Basin Security Initiative Technical Working Group. Draft Report of the Fifth Caribbean Basin Security Initiative Technical Working Group on Crime Prevention by Focusing on At-Risk Youth and Vulnerable Populations Paramaribo, Suriname, 22-23 October 2014. Caribbean Examinations Council. Monitoring and Evaluation Plans (Fiscal 2015, Fiscal 2016). CXC/USAID Strengthening Second Chance Education Programme Cooperative Agreement. 95 Given the large number of documents received during the course of the assessment, the team has elected to include those that were the most relevant to answering the assessment questions provided. A complete listing can be provided on request. Caribbean Examinations Council. Summary Semi Annual Progress Reports (October 2013, April 2014, April 2015, October 2015). CXC/USAID Strengthening Second Chance Education Programme Cooperative Agreement. Caribbean Examinations Council. Work Plans (Fiscal 2013, Fiscal 2014, Fiscal 2016). CXC/USAID Strengthening Second Chance Education Programme Cooperative Agreement. Chambers, Victoria. ODI Securing Communities and Transforming Policing Cultures: A Desk Study of Community Policing in Jamaica, Overseas Development Institute Report, 2014 Cragin, Kim, Melissa A., Eric Robinson, Paul S. Steinberg. What Factors Cause Youth to Reject Violent Extremism? Results of an Exploratory Analysis in the West Bank. RAND Corporation, 2015. Education Development Center, Inc. Skills and Knowledge for Youth Employment (SKYE) in Guyana: Assessment Report and Program Design. Education Development Center, Inc, 2011. Espinal, Rosario, Jana Morgan, Elizabeth J. Zechmeister. Political Culture of Democracy in the Dominican Republic and the Americas, 2014: Democratic Governance across 10 Years of the Americas Barometer. USAID, 2015. Garcia, Danielle de, Sarah Edith Jones and Mateusz Pucilowski. Evaluation of the Guyana Skills and Knowledge for Youth Employment Program. Social Impact, Inc, 2014. Goggin, Jim, Mansfield Blackwood, Michael Taylor, Judith Hinds and Marguerite Walcott. Rapid Youth Assessment in the Eastern Caribbean. Education Development Center, Inc, 2008. Government Accountability Office. Status of Funding, Equipment, and Training for the Caribbean Basin Security Initiative. U.S. Government Accountability Office, 2013. Government of Grenada. Grenada Country Report: Global Progress Survey on Violence Against Children. Ministry of Social Development, the Government of Grenada, 2012, http://srsg.violenceagainstchildren.org/sites/default/files/global_survey/responses/grenada/unvac_s urvey_grenada.pdf (accessed January 11, 2016). Government of the Republic of Trinidad and Tobago. National Parenting Policy: Trinidad and Tobago, Ministry of Gender, Youth and Child Development, Government of the Republic of Trinidad and Tobago, 2015. http://www.ircp.gov.tt/cms/policy_files/14044066520National%20Parenting%20Policy.pdf (accessed January 8, 2016). Harriott, Anthony A., Balford A. Lewis and Elizabeth J. Zechmeister. Political Culture of Democracy in Jamaica and in the Americas: Democratic Governance across 10 years of the Americas Barometer. United States Agency for International Development, 2014. Harriott, Anthony et al. Caribbean Human Development Report: Human Development and the shift to better Citizen Security. United Nations Development Programme, 2012. Holm, Robert et al. Youth Workforce Development Assessment: For USAID/ Barbados & the Eastern Caribbean. Chemonics International, Inc and Jobs for the Future, 2010. International Monetary Fund. Dominica 2012 Article IV Consultation. 2013. IMF Country Report No. 13/31. https://www.imf.org/external/pubs/ft/scr/2013/cr1331.pdf. International Security Sector Advisory Team. Grenada Country Profile. ISSAT, 2015. http://issat.dcaf.ch/Learn/Resource-Library/Country-Profiles/Grenada-Country-Profile (accessed January 11, 2016). International Security Sector Advisory Team. St. Kitts and Nevis Country Profile. ISSAT, 2015. http://issat.dcaf.ch/Learn/Resource-Library/Country-Profiles/Saint-Kitts-Country-Profile (accessed January 11, 2016). John Buchanan et al., USAID guide for USAID Democracy and Governance Officers: Assistance to Civilian Law Enforcement in Developing Countries, USAID, 2011, Johnson, Rosemarie, Needs Assessment of South Camp Juvenile Correctional and Remand Centre and Metcalfe Street Juvenile Centre, 2015. K’nife, K’adamawe. Regional Law Enforcement and Anti-Corruption Conference: Situational Analysis. 2011. Lemard, G. and D. Hemenway, “Violence in Jamaica: An Analysis of Homicides 1998-2002,” Injury Prevention, 2006. 12:15-18. Lyday, Corbin, Keith Henderson and Sharene McKenzie. Combatting Corruption in Jamaica: Final Performance Evaluation. United States Agency for International Development, 2015. Ministry of Education, Jamaica and U.S. Agency for International Development. Project Implementation Plan for USAID/MoE Partnership for Improved Reading Outcomes (September 2014-March 2016). USAID/Jamaica, 2014. Ministry of Education, Jamaica and U.S. Agency for International Development. USAID/MOE Partnership for Improved Reading Outcomes Activity. Annual Report (September 2013-September 2014). USAID/Jamaica, 2014. Ministry of Education, Jamaica and U.S. Agency for International Development. Closing the Gender Gap: A Guide for Improving the Literacy Performance of Boys and Girls at the Primary Level. USAID/MOE Partnership for Improved Reading Outcomes Activity, 2014. McLean, Andrew et al. Assessment of Community Security and Transformation Programmes in Jamaica. Planning Institute of Jamaica, 2009. McLean, John, Anthony Harriott, Elizabeth Ward, John Buchanan and Roopa Karia. Jamaica – Community￾based Policing Assessment. United States Agency for International Development, 2008. Morgan, Jana, Rosario Espinal and Mitchell A. Seligson. Political Culture of Democracy in the Dominican Republic. United States Agency for International Development, 2010. Office of the Inspector General. Audit of USAID/Eastern and Southern Caribbean’s Youth-related Projects. Audit Report No. 1-534-15-007. USAID, 2015. Organization of Eastern Caribbean States. OECS Juvenile Justice Reform Project Annual Progress Report 2013/2014. OECS Commission. 2014. Organization of Eastern Caribbean States Commission. Juvenile Justice Reform Project Annual Progress Report OECS, 2015. Organization of Eastern Caribbean States. OECS Juvenile Justice Reform Project Workplans (2011 – 2012; 2012 – 2013). United States Agency for International Development. Organization of Eastern Caribbean States. OECS Juvenile Justice Reform Project Extension Workplan (2014 – 2015). United States Agency for International Development. Organization of Eastern Caribbean States. Draft Performance Management Plan 2012- 2015: Juvenile Justice Reform Project. OECS, 2012. Owen, Taylor and Alexandre Grigsby, “In Transit: Gangs and Criminal Networks in Guyana,” Small Arms Survey, 2012. http://www.smallarmssurvey.org/fileadmin/docs/F-Working-papers/SAS-WP11- Guyana.pdf. Pan American Development Foundation. Kari Yu! Annual Report, October 1, 2013 – September 30, 2014. PADF, 2014. Pan American Development Foundation. Kari Yu! Annual Report, October 1, 2014 – September 30, 2015. PADF, 2015. Pan American Development Foundation. Proposed Revised Results Framework. PADF, 2015. Presidencia Republica Dominicana; Plan de Regularización inició hoy; Gobierno confía en éxito del proceso, 2014 accessed from http://presidencia.gob.do/noticias/plan￾de-regularizacion-inicio-hoy-gobierno-confia-en-exito-del-proceso Seligson, Mitchell A., Lawrence A Powell and Balford A Lewis. The Political Culture of Democracy in Jamaica: The Impact of Governance. United States Agency for International Development, 2009. Spector, Bertram, Svetlana Winbourne, and Phyllis Dininio, Management Systems International. U.S. Agency for International Development Practitioner’s Guide for Anticorruption Programming, 2015 Umaña, Isabel Aguilar, Rikkers, Jeanne, Interpeace. Nine Strategies To Prevent Youth Violence In Central America Policy Recommendations for the European Union, 2012. UNICEF TACRO (The Americas and the Caribbean Regional Office). Juvenile Justice Systems: Good Practices in Latin America, Child Protection, 2006. UNDP. Caribbean Human Development Report: Human Development and the Shift to Better Citizen Security, 2012. U.S. Agency for International Development. Access to Justice Activity Fourth Quarterly Technical Report, prepared by Checchi and Company Consulting, Inc. 2013 U.S. Agency for International Development. The A Ganar Alliance: Using Sport to Impact Youth Employment and Youth Engagement FY14 Annual Report, 2014. U.S. Agency for International Development. The A Ganar Alliance: Using Sport to Impact Youth Employment and Youth Engagement Final Report, 2015. U.S. Agency for International Development Caribbean Youth Empowerment Program Associate Award Cooperative Agreement No. 538-A-00-08-00101-00, FINAL REPORT: 2008 U.S. Agency for International Development. A Ganar & Caribbean Youth Empowerment Program Mid-Term Performance Evaluation 2013, accessed from http://pdf.usaid.gov/pdf_docs/pdacy247.pdf U. S. Agency for International Development. The Caribbean Basin Security Initiative, FY 2016 Projected Allocation. USAID, 2015. U.S. Agency for International Development. Caribbean Basin Security Initiative Juvenile Justice Assessment, Final Report. USAID/Barbados, 2011. U.S. Agency for International Development. Caribbean Youth Empowerment Program, Final Report. The International Youth Foundation. USAID. 2014. U.S. Agency for International Development. “CBSI schools: Summary Enrolment by School, and Sex (2014/2015).” Ministry of Education Annual Schools Census. Government of Jamaica, 2014. U.S. Agency for International Development and Checchi and Company Consulting, Inc. Colombia Access to Justice Activity, Third Annual Technical Report 2015 U.S. Agency for International Development and Dexis Consulting and Management Systems International. Combating Corruption in Jamaica Final Performance Evaluation, Jamaica Final Report 2015 U.S. Agency for International Development. Community Empowerment and Transformation (COMET) Program: Final Evaluation. USAID/Jamaica, 2012. U.S. Agency for International Development and Management Systems International: Corruption Assessment for Jamaica, Contracted under DFD-I-01-03-00144, Task Order #01, 2008. U.S. Agency for International Development and Democracy International. Crime and Violence Prevention Field Guide Draft Report 2015 U.S. Agency for International Development. Criminal Justice System Strengthened Project CJSSP Quarterly/Startup Report, prepared by Chemonics, 2015. U.S. Agency for International Development. Dominican Republic Country Development Cooperation Strategy. USAID/Dominican Republic, 2014. U.S. Agency for International Development. Country Development Cooperation Strategy: FY 2014-2018. USAID/Jamaica, 2013. U.S. Agency for International Development and Entrena SRL. Dominican Republic At-Risk Youth Initiative Alerta Joven Project Annual Report 2014-15. U.S. Agency for International Development. Eastern and South Caribbean Youth Assessment (ESCYA) Final Report. USAID, 2013. http://www.undp.org/content/dam/trinidad_tobago/docs/DemocraticGovernance/Publications/ES CYA%20Final%20Report%20(FINAL%20SUBMISSION)%2031%20Oct.pdf (accessed January 8, 2016). U.S. Agency for International Development, Ministry of Education and JA. Empowering Jamaica’s Youth Project Completion Report, 2015 U.S. Agency for International Development. Human and Institutional Capacity Development Handbook, Participant Training team in the Office of Education, Bureau for Economic Growth, Agriculture & Trade.2010. U.S. Agency for International Development. Juvenile Justice Reform for the OECS Summary Sheet. USAID. Accessed on January 5, 2016. U.S. Agency for International Development. PMP Youth Development and Juvenile Justice Program in Suriname. 2013. U.S. Agency for International Development. Program Overview and Activity Profiles: Caribbean Basin Security Initiative. USAID/Jamaica. 2013. U. S. Agency for International Development. USAID ESC Regional Development Cooperation Strategy 2015- 2019 (Public Version). USAID. 2015. U.S. Agency for International Development. Revised Results Framework. USAID/PADF Youth Development and Juvenile Justice Program (Kari Yu!), 2015. U.S. Agency for International Development. Skills and Knowledge for Youth Employment (SKYE). AID-504-LA-11-00001. 08 Aug. 2011. Cooperative Agreement. U.S. Agency for International Development. Support to the Metcalfe Street Secure Juvenile Centre Project. (PowerPoint Presentation). USAID. U.S. Agency for International Development. Suriname “At-Risk” Youth Assessment. USAID and PADF. 2013. U.S. Agency for International Development. Suriname Rapid Labor Market Study. USAID and PADF. 2013. U.S. Agency for International Development. Suriname School Drop-Out Assessment. USAID and PADF. 2013. U.S. Agency for International Development. Suriname Youth in Conflict with the Law Assessment. USAID and PADF. 2013. U.S. Agency for International Development. Strengthening Second Chance Education in the Eastern Caribbean, AID-538-A-12-00005. 26 Sept. 2012. Contract. U. S. Agency for International Development. USAID/ Barbados and Eastern Caribbean Strategic Plan, FY 2011-2015. USAID, 2015. U. S. Agency for International Development. USAID/ Jamaica Community-based Policing Assessment Progress Report. USAID/Jamaica, 2009. U. S. Agency for International Development. USAID/ Regional Caribbean Crime Prevention Needs Assessment – Bahamas, Trinidad and Tobago, Guyana. USAID, 2014. U.S. Agency for International Development. Youth Development and Juvenile Justice Program in Suriname. AID-538-A-13-00001. 07 Feb. 2013. Contract. U.S. Agency for International Development. Towards Understanding Troublesome Youth Groups in the Caribbean. AID-538-F-14-00001. 04 April 2014. Fixed Obligation Grant. U.S. Congress, Senate, Committee on Foreign Relations. The US-Caribbean Shared Security Partnership: Responding to the Growth of Trafficking and Narcotics in the Caribbean. Testimony of Ambassador Liliana Ayalde, Senior Deputy Assistant Administrator, Bureau for Latin America and the Caribbean, United States Agency for International Development, 2011. U.S. Congress, House, Foreign Affairs Committee. Testimony of Julissa Reynoso before the House Foreign Affairs Committee Subcommittee on Western Hemisphere Affairs. U.S. Congress, 2009. Print. U.S. Department of State. 2012 Country Reports on Human Rights Practices—Grenada. U.S. Government, 2013. http://www.refworld.org/docid/517e6e3018.html (accessed January 11, 2016). U.S. Department of State. Saint Kitts and Nevis. U.S. Government, 2013, http://www.state.gov/documents/organization/186748.pdf (accessed January 11, 2016). U.S. Department of State. Saint Kitts and Nevis 2013 Human Rights Report. U.S Government, 2013. http://www.state.gov/documents/organization/220677.pdf. (accessed January 11, 2016). U.S. Department of State. The Caribbean Basin Security Initiative. U.S. Government, 2013. U.S. Department of State. The Caribbean Basin Security Initiative: Dominican Republic. U.S. Government, 2013. U.S. Department of State. The Caribbean Basin Security Initiative: Jamaica. U.S. Government, 2013. U.S. Department of State. The Caribbean Basin Security Initiative: St. Lucia. U.S. Government, 2013. U.S. Department of State. The Caribbean Basin Security Initiative: St. Vincent. U.S. Government, 2013. U.S. Department of State. The Caribbean Basin Security Initiative: Suriname. U.S. Government, 2013. U.S. Department of State. U.S Efforts to Address Citizen Security Challenges in the Caribbean. (PowerPoint). Bureau of Western Hemisphere Affairs – Office of Caribbean Affairs, 2015. United Nations. United Nations Subregional Analysis of the Development Context in Barbados and the OECS. United Nations Subregional Team for Barbados and the OECS, 2011. http://www.undp.org/content/dam/barbados/docs/legal_framework/United%20Nations%20Subre gional%20Analysis.pdf (accessed January 8, 2016). United Nations Children’s Fund. Views on Juvenile Offending in Barbados, Dominica and St. Lucia. UNICEF, 2012. http://www.unicef.org/easterncaribbean/Juvenile_Offing_Study_WEB.pdf (accessed January 8, 2016). United Nations Children’s Fund. Grenada: 2015 Child Protection Statistical Digest. UNICEF, 2015. http://www.unicef.org/easterncaribbean/ECAO_GRENADA_Child_Protection_Statistical_Digest _2015.pdf (accessed January 11, 2016). United Nations Development Programme. Human Development and the Shift to Better Citizen Security, UNDP, 2012. http://www.undp.org/content/dam/trinidad_tobago/docs/DemocraticGovernance/Publications/U NDP_TandT_Citizen_Security_Survey_2012.pdf (accessed January 8, 2016). United Nations Development Programme. Program Description: ‘UNDP Youth Sustainable Livelihood Project.’ UNDP, Jamaica, 2015. United Nations Development Programme. Trinidad and Tobago Juvenile Court Project. UNDP, 2015. United Nations Office on Drugs and Crime. Global Study on Homicide, 2013. UNODC. https://www.unodc.org/documents/gsh/pdfs/2014_GLOBAL_HOMICIDE_BOOK_web.pdf. Vincent, Gina M, Ph.D., Guy, Laura S., Ph.D. Grisso, Thomas Ph.D. National Youth Screening & Assessment Project Risk Assessment in Juvenile Justice: A Guidebook for Implementation accessed from http://www.modelsforchange.net/publications/346 Walter, Amy et al. USAID/ Dominican Republic Cross-Sectoral At-Risk Youth Assessment: Final Report. Education Development Center, Inc. 2010. World Bank. Suriname Overview. The World Bank, 2016. http://www.worldbank.org/en/country/suriname/overview World Bank. Unemployment, Youth Total (% of Total Labor Force Ages 15-24). The World Bank, 2016. http://data.worldbank.org/indicator/SL.UEM.1524.ZS (accessed January 8, 2016). ANNEX VI: Matrix for Analysis of Best Practices CBSI Activity Classification Sources of Best Practices Information Evaluation report of CBSI project or predecessor Research report that informed design or implementation USG guideline or policy that is directly relevant Other document analyzing practices in the region since 2010 (esp. USG-produced) 1 A Ganar and CYEP Midterm Performance Evaluation (2013); SKYE Midterm Evaluation (2014); A Ganar Monitoring and Evaluation Plan SKYE assessment report (2011); USAID/Dominican Republic Cross-Sectoral At￾Risk Youth Assessment (2010) USAID Youth in Development Policy ESC Youth Assessment (2013); OIG Audit of ESC youth-related projects (2015); UNDP Regional Human Development Report: Human Development and the Shift to Better Citizen Security (2012) youth employment and entrepreneurship Empowering Jamaica’s Youth Project Completion Report (2015) educational support, esp. integrated support services for 2 COMET II Final Community Baseline Survey Report USAID A Field Guide for USAID Democracy and Governance Officers: Assistance to Civilian Law Enforcement in Developing Countries (2011) USAID Human and Institutional Capacity Handbook (2010); USAID/Colombia: Mid-term Evaluation of the Access to Justice Activity (2015) enhanced policing (including community-based) USAID/Jamaica Community Empowerment and Transformation (COMET) Program Final Evaluation (2012) CBP Regional Conference Report (2011); Jamaica CBP Baseline Survey (2012); USAID Jamaica CBP Assessment (2008) ODI Securing Communities and Transforming Policing Cultures: A Desk Study of Community Policing in Jamaica (2014) community access to justice and dispute resolution services CJH Project Evaluation, Draft Report March (2016) Colombia - USAID/Colombia Access to Justice Activity mid￾term evaluation (2015), third annual progress report (2015) crime and violence prevention Nine Strategies to Prevent Youth Violence Central America (2012); USAID-2016-What-Works-in-Reducing-Community￾Violence-Meta Evaluation Report criminal prosecution 3 CBSI Juvenile Justice Assessment (2011); Needs Assessment of South Camp Juvenile Correctional and Remand Centre and Metcalfe Street Juvenile Centre (2015) Eastern and Southern Caribbean Youth Assessment (2013); OIG Audit of ESC youth-related projects (2015); UNDP Caribbean Human Development Report (2012); Risk Assessment in Juvenile Justice: A Guidebook for Implementation (2012); Crime and Violence Prevention Field Guide - Draft Report (2016); Juvenile Justice Systems: Good Practices in Latin America (2003); Terms of Reference, CBSI Technical Working Group (TWG) on Preventing Crime by Focusing on At-Risk Youth and Vulnerable Populations (2014) rehabilitation of youth offenders alternative sentencing and diversion methods reducing youth delinquency Nine Strategies to Prevent Youth Violence Central America by InterPeace (2012) 4 USAID/Jamaica Final Report Combatting Corruption in Jamaica Final Performance Evaluation (2015) USAID Practitioner's Guide for Anticorruption Programming (2015) Analysis of USAID Anticorruption Programming Worldwide 2007-2013; USAID Corruption Assessment for Jamaica (2008) money laundering tax administration Reduced corruption in public and private sectors Sources of Best or Promising Practices USAID CBSI Social Justice Objective (and subtopics) Increased economic opportunities for at-risk youth and vulnerable populations Improved community and law enforcement cooperation Improved juvenile justice sector ANNEX VII: DISCLOSURE OF ANY CONFLICTS OF INTEREST CA U.S. Agency for International Development 1300 Pennsylvania Avenue, NW Washington, DC 20523