USAID/JAMAICA ADVANCE FINAL PERFORMANCE EVALUATION The purpose of this performance evaluation is to: (1) Assess the long-term sustainability of Advance program’s interventions on the participating Jamaican Higher Education Institutions’ (HEIs) performance, sustainability, and institutionalization. (2) Identify enabling and inhibiting contextual and institutional factors associated with the HEI’s performance. ADVANCE PARTICIPATING HEIS 1. University of Technology, Jamaica (UTECH) 2. Vocational Training Development Institute (VTDI) 3. Knockalva Polytechnic (KPC) 4. Knox Community College (KCC) EVALUATION METHODOLOGY This evaluation utilized mixed-methods, including: 1) Desk Review 2) Key Informant Interviews 3) Focus Group Discussions 4) Web-based Surveys 51 participants: 12 males, 39 females Including: HEI informants (leadership, faculty, administrators, and students); relevant Government of Jamaica and employing industries representatives. Purposive and snowball sampling were applied to select respondents. Outcome Mapping: The Evaluation Team used Outcome Mapping as the main methodology to assess the changes in institutional performance across HEIs. Systems Approach: The design of this evaluation is based on a systems approach to identify and analyze existing processes that affect or impact performance and institutionalization of the program objectives. SUMMARY OF SUSTAINABILITY AND INSTITUTIONALIZATION OF KEY ELEMENTS OF ADVANCE INPUTS: ADVANCE PROGRAM ACTIVITIES • Curriculum development • Resources for scholarships and for the professional development of faculty and staff • New ways of working and engaging with private sector and disadvantaged youth OUTCOMES: WHAT WAS GAINED BY THE HEIs • New and better curricula meeting the needs of the labor market • Introduction to experiential learning • Labs, student services, and new methods and approaches to marketing, recruitment and scholarship programs IMPACTS: THE DIFFERENCE ADVANCE HAS MADE • Associate degree programs aligned to labor market and youth needs across HEIs • Application and adoption of experiential learning across HEIs • Improved services for students, including advisement centers, scholarship programs, and more disadvantaged students admitted • Increase in internships, partnerships, understanding of labor bridging services within the system MAIN FINDINGS FOR EACH OF ADVANCE’S PROGRAM OBJECTIVES Advance implementation Main results at the systems level Considerations for future programming Program Area 1: Curriculum development • Supported the development and strengthening of new programs. • Built the capacity of faculty in curriculum design and pedagogy. • Introduced new and modern approaches for experiential learning. • The system has six new or improved fully institutionalized associate degree courses that use an experiential learning approach and are tailored to labor market needs. The HEIs are able to offer them to future students without the support of the Advance program. • The system is better positioned to reach students through online alternatives, attracting those for whom online learning is a more suitable and cheaper option than face￾to-face education. • Degree programs developed under the Council of Community Colleges of Jamaica (CCCJ) were transferred to the community colleges (CCs), but CCs did not participate in the development of Advance courses. Future programs should engage CCs to ensure effective collaboration with relevant stakeholders in the development of courses. Program Area 2: Professional development of faculty staff • Supported the development of opportunities for peer-to-peer learning. • Exposed staff to new knowledge and information from experts and technocrats in the field. • Extended professional development opportunities to HEI staff beyond just those involved with Advance. • Organized train-the-trainer activities to support the institutionalization and sustainability of approaches and new knowledge introduced by Advance. • Systems’ faculty and staff demonstrate strengthened skills in curriculum development and design. • Community colleges discuss labor market insertion best practices and lessons learned through recently created communities of practice. • HEIs mainstreaming socio-emotional learning in courses beyond the Advance program. • HEIs integrating academic and professional development activities and practicum training. • A gap between planned opportunities for professional development and what was actually implemented led some informants to consider that capacity building opportunities were limited. Program Area 3: Labor-bridging services • Exposed HEIs to new ways of engaging with private sector, which resulted in an increase in internships and work experiences for their students. • Provided capacity building opportunities for staff and students covering areas such as resume-writing, developing business plans, and interview skills. • Advance enabled HEIs in the system to become more aware of the concept and importance of labor-bridging, and how it could be aligned to an HEI’s mandate and consequently, its courses and degree programs. • Given that labor-bridging services is considered the weakest program area by participants and interactions between HEIs and the private sector were limited in some HEIs, more work is needed to link youths and HEIs with the private sector. Program Area 4: Admissions and recruitment • Assisted HEIs with program marketing and engagement of disadvantaged youth, including preparing advertisements and hosting events to bring together HEIs and the target group. • The system already had good admissions and recruitment processes in place that did not change. However, the system is now able to undertake assessments of vulnerable youth and determine their suitability for higher education. • Working with youths with lower academic levels requires supporting them to adjust to university requirements, including in terms of academic knowledge and social and emotional learning. Program Area 5: Access to scholarships • Provided scholarships to improve access to higher education for disadvantaged and vulnerable youth covering tuition, stipends, transportation, accommodation, and course materials. • Overall, HEIs across the system are adopting and developing their own scholarship programs to continue providing financial aid to vulnerable youth. • The first cohort of scholarship students was required to pay for expenses before being reimbursed at a later date. This model did not work and was adjusted during program implementation.