1 2 CONTENTS EXECUTIVE SUMMARY............................................................................................................................................ 4 1 INTRODUCTION.............................................................................................................................................. 9 1.1 Background to the High Gear Programme..........................................................................................10 1.2 Evaluation Purpose .............................................................................................................................11 1.3 High Gear Programme Interventions and Activities being evaluated in Year 2..................................13 2 EVALUATION METHODOLOGY .....................................................................................................................15 2.1 Limitations to the Study ......................................................................................................................16 3 FINDINGS IN RELATION TO THE PRIMARY EVALUATION QUESTIONS..........................................................18 3.1 Question 1: Is the High Gear programme’s Theory of Change – including its embedded assumptions – accurate, thereby leading to anticipated results?.........................................................................................18 3.1.1 Does the Theory of Change contribute to increased and sustained partnerships between industry and TVET colleges? ........................................................................................................................20 3.1.2 To what extent are the curricula and teaching in TVET systems more aligned to industry needs and being delivered as intended according to project design? ...................................................................21 3.1.3 To what extent did the intended outcome of increasing young women’s and young men’s learning and employment in targeted regions occur over the course of the project?................................22 3.2 Question 2: Is the High Gear programme contributing towards increased ecosystem resilience?....25 3.3 Question 3: Does the High Gear programme project model deliver Value for Money? (VfM) ..........27 4 FINDINGS AGAINST THE HIGH GEAR PROGRAMME OUTCOMES................................................................. 30 4.1 Partnerships have been sustained and deepened ............................................................................ 30z 4.2 Curricula has been strengthened and aligned to industry needs .......................................................32 4.3 Young people have received guidance to better employment outcomes..........................................38 5 IMPLICATIONS FOR FUTURE HIGH GEAR PROGRAMMING..........................................................................43 5.1 The challenging economic environment.............................................................................................43 5.2 Repositioning the High Gear programme ...........................................................................................43 5.3 Managing expectations.......................................................................................................................43 5.4 Intentional focus on soft skills interventions......................................................................................44 6 CONCLUSION................................................................................................................................................46 ANNEXURE 1: VfM REPORTING FRAMEWORK .....................................................................................................47 ANNEXURE 2: FULL RESULTS FRAMEWORK..........................................................................................................51 ANNEXURE 3: STAKEHOLDERS ENGAGED DURING THE RESEARCH FOR THE YEAR 2 FORMATIVE EVALUATION 65 ANNEXURE 4: DETAILS OF THE EVALUATION QUESTIONS ...................................................................................68 3 TABLES, DIAGRAMS & FIGURES Table 1: Scope of High Gear programme activities in 2022 (and 2023 in KwaZulu-Natal)...................................13 Table 2: High Gear programme Technical Assistance Interventions in 2022 (and 2023 in KwaZulu-Natal).........13 Table 3: Summary Table of Achievement against VFM Indicators.......................................................................28 Table 4: Lecturer Attendance of the High Gear programme Lecturer Development Interventions ....................36 Diagram 1: The High Gear Programme Hypothesis & Theory of Change ...............................................................9 Diagram 2: High Gear Programme Outcomes......................................................................................................11 Diagram 3: Primary Evaluation Questions for the High Gear Programme ...........................................................12 Figure 1: Student Views on Workplace Experience/Exposure..............................................................................24 Figure 2: Student Views on Quality of Resources & Teaching..............................................................................33 Figure 3: Lecturer Views on Capacity Development Interventions......................................................................37 Figure 4: Participating in the Programme helped me to increase my connections with employers....................39 4 EXECUTIVE SUMMARY The High Gear programme aims to demonstrate a model for greater industry involvement in Technical, Vocational, Education & Training (TVET) College course design and delivery. Specifically, the programme tests the key assumptions that improved and sustained partnerships between industry lead to curricula and teaching that is better aligned to industry needs, which results in improved learning and employment outcomes for young people. This formative evaluation of the second year of programme implementation is intended to test whether the High Gear programme is on track to achieve its intended impact. Our assessment focused on assessing the programme's key components, including whether the programme is being implemented as planned and ‘testing’ the assumptions within the Theory of Change. This includes an analysis of the extent of system adoption in terms of partnerships, curricula, and training and employment outcomes. We also examined progress against outcomes and evaluated the extent to which the programme is contributing to its intended impact. We found that in Year 2, the High Gear programme in KwaZulu-Natal and the Eastern Cape built on the foundation that was laid in the first year. This allowed the High Gear programme to achieve concrete results in terms of supporting the skills development system to become more responsive to industry needs. In relation to partnerships, we found that the High Gear programme has worked effectively with industry associations to facilitate greater and more effective employer involvement in curriculum review, and to enable greater collaboration between employers and colleges. However, there are some concerns about whether colleges can take forward the effective provision of a revised curricula that continues to be responsive to change, or about whether these partnerships have resulted in sufficient opportunities for students. Respondents from employers, from industry associations and from TVET Colleges all confirmed that they have been able to establish what they consider to be mutually beneficial partnerships and suggest that the High Gear programme has both contributed to the development of these partnerships and that these partnerships will in turn contribute to the overall goal of the High Gear programme. Turning to the curriculum, we found that there have been valuable revisions to the qualifications. This has also allowed for the development of curriculum frameworks that are better aligned to the needs of industry. Stakeholders, including colleges indicate they are more aware of industry needs, but state the process of formally updating curricula is long and complex, which does not accommodate the fast pace of technology change. Different ways of thinking about ensuring ‘relevance’ therefore need to be considered, including practical training for students and workplace exposure for students as well as lecturers. With respect to lecturers, it was found that lecturers are not feeling very confident about their ability to deliver quality learning within the new frameworks, although students suggest that their lecturers are helpful and skilled. And in respect of learning and employment outcomes, there is evidence of improvements in learning, with learner and lecturer respondents noting that the quality of the course was good. The tracer study also showed an increase in the number of students who state that they received guidance in their courses. However, there is a clear need to increase the practical learning and workplace exposure components of the courses. 5 In relation to increased employment, the numbers of graduates who indicated that they are employed in the tracer studies is still low, and most employed young people have contract work or are on learnership agreements. The percentage of employed graduates increased between the two tracer studies, from 4% to 15% of respondents, which is important given the challenging economic context and the depressed labour market. The other key evaluation questions test ecosystem resilience and value for money. There is definite evidence of increased ecosystem resilience, particularly in relation to sustainable partnerships between industry and colleges, and better relationships between employers and the merSETA. Respondents highlight the role of the High Gear programme in achieving this. The value for money question has recently shifted on the basis of discussion between IYF and donors, and we relied on IYF reporting data to review this. It was found that all reports had been submitted and in general indicators were met. Specifically looking at achievement against indicators for the three outcomes of the programme, the table below shows that the programme in the Eastern Cape performed slightly better that the programme in KwaZulu-Natal in relation to achieving targets against indicators for the partnership outcome, while KwaZulu-Natal did better than the Eastern Cape in relation to indicators for the curriculum outcome. Overall, the programme fared poorly in relation to indicators for the learning and employment outcome. Partnerships ► The Year 2 target for the number of beneficiaries engaged in partnerships, engagements and convening activities was exceeded in both KwaZulu-Natal and the Eastern Cape, (target in Eastern Cape was 47 beneficiaries and Year 2 result is 90, target for KwaZulu-Natal was 14 and 56 beneficiaries were engaged). These beneficiaries included government staff, employers, lecturers and college management staff. ► Year 2 targets for the number of High Gear practices and policies that are adopted by industry partners were exceeded in both KwaZulu-Natal and the Eastern Cape: we found that three practices/polices were adopted in each of KwaZulu-Natal and the Eastern Cape. ► In relation to the number of partners who reported that elements of the model are integrated into their institutional strategies: in KwaZulu-Natal we found that the Year 2 target of seven partners was missed, as only two partners reported integrating elements into their institutional strategies. In the Eastern Cape the target was exceeded as four partners reported integrating elements of the High Gear model, against a target of one. ► We also found that Year 2 targets for the number of instances in which industry partners disseminate High Gear learnings was significantly exceeded, with learnings being shared 238 times across both KwaZulu-Natal and the Eastern Cape through a combination of forums, publications and social media. ► The Year 2 target for the number of KwaZulu-Natal industry partners who have accessed additional funding to advance inclusive practices is 8. We found that two KwaZulu-Natal employers have been supported through SETA Advisory services, which is facilitated by Engeli enterprise development, to develop and submit SETA grant applications in 2022, so this target was missed. ► No finding on the indicator on the number of KwaZulu-Natal employer proposals that focus on partnerships that were submitted to the merSETA, as no data available ► The Year 2 target for the number of industry group partners that indicate that High Gear has supported them to access increased skills development funding was missed in KwaZulu-Natal and in the Eastern Cape: each assisted one partner against a target of two. ► One of the overall indicators for High Gear is the amount of public funding that has been secured by partners for implementation of High Gear or related programming. The baseline for this was ZAR 8 661 900 which was secured in Year 1, and the target for this was a 10% increase in Year 2 or R9 528 090. We are unable to make a finding on this as the amounts secured were not made available, though NAACAM respondents have indicated that they are working more closely with the SETA's. 6 Curriculum ► In terms of the number of TVET graduates who receive updated course content, the target for Year 2 were not reached at either of the colleges: at Elangeni the targets was 1 344 and 648 students enrolled, and at EMC the target was 480 and only 296 students enrolled. ► In Year 2 the target for the percentage of graduates that indicate that the quality and resources of the programme is of high standard has been exceeded in KwaZulu-Natal: the target was 65% and 78% was achieved. For the Eastern Cape the Year 2 target was missed: the target was 90% and 71% was achieved. ► Year 2 targets were met for the number of TVET courses aligned to industry requirements – three in each Province. Subjects in the N4-N6 Electronics Engineering, Electrical Engineering and Mechanical Engineering were reviewed with insights from industry partners as well as curriculum specialists at TVET colleges. ► Year 2 targets of #/% of TVET students who access workplace experience were not met in either province. The Year 2 target in KwaZulu-Natal was 25%, however only 5% of the students were placed. In the Eastern Cape the Year 2 target was 38% and 12% of the students were placed. ► In Year 2 the targets for the number of technical assistance interventions was one for KwaZulu-Natal and six for the Eastern Cape, and this target was exceeded in both provinces. In KwaZulu-Natal, 10 technical assistance interventions were delivered and in the Eastern Cape 8 interventions were delivered. These included WIL placements for students and lecturers, career guidance services, lecture capacity development as well as SETA advisory services (only applicable in KwaZulu-Natal). ► The Year 2 targets for the number of individuals affiliated with higher education (tertiary) institutions receiving capacity development support with USG {or UKPF] assistance were met in KwaZulu-Natal (the target was 28 and all 28 lecturers participated in capacity development training), and in the Eastern Cape 16 lecturers received training just missing the target of 19. ► We note that two key indicators on the quality and relevance of the training are that graduates access higher education following participation in High Gear, and an improved completion rate. As neither of the colleges have released 2022 performance data to High Gear yet, we are unable to comment on these indicators. Learning & Employment Outcomes ► When students were asked if they thought that the qualification provided them with the tools and skills to access employment, we found that there was a slight increase in both provinces in the percentage of respondents who selected strongly agree and agree, from 69% in Year 1 to 77% in Year 2 in KwaZulu-Natal, and from 65% in Year 1 to 68% in Year 2 in the Eastern Cape. However, despite this increase, the Year 2 target of 90% was missed in KwaZulu-Natal as 77% was achieved and in the Eastern Cape the Year 2 target of 36% was exceeded as 68% was achieved. ► With regards to improved soft skills, the measure is the % of students surveyed who indicate that they believe that they have improved soft skills because of being in the programme. It is a concern that in both provinces there has been a decrease in the percentage of respondents who believed that participating in the course had improved their soft skills a fair amount or an enormous amount. In KwaZulu-Natal, this decline has been steady, from 93% in the baseline, to 91% in Year 1, to 86% in Year 2. In the Eastern Cape there was a decrease from baseline (94%) to Year 1 (85%) and a slight increase from Year 1 to Year 2 (89%). The Year 2 targets were missed in both provinces: the target was 95% in KwaZulu-Natal and 86% was achieved, and it was 95% in the Eastern Cape and 89% was achieved. ► Employment targets were missed in both provinces. We found that 14% of those that responded from KwaZulu-Natal said that they were in full- or part- time employment (against a target of 26%) and 16% of those that responded in the Eastern Cape said that they were in full- or part-time employment, against a target of 24%. 7 ACRONYMS BBBEE Broad based Black Economic Empowerment CATI Computer Assisted Telephone Interviews DAC Durban Automotive Cluster DHET Department of Higher Education and Training ECAIF Eastern Cape Automotive Industry Forum EC Eastern Cape EMC Eastcape Midlands TVET College GDPR General Data Protection Regulation (GDPR) HE Higher Education KZN KwaZulu-Natal MERL Monitoring, Evaluation, Research, and Learning MIS Management Information System MoU Memorandum of Understanding MSDF Michael and Susan Dell Foundation N Nated NAACAM National Association of Automotive Component and Allied Manufacturers NCV National Certificate Vocational NGO Non-Governmental Organisation OEM Original Equipment Manufacturer POPI Protection of Personal Information QCTO Quality Council for Trades and Occupations SETA Sector Education and Training Authority SOP Standard Operating procedures S4P Skills for Prosperity STEM Science, Technology, Engineering, and Mathematics ToC Theory of Change ToA Theory of Action TVET Technical and Vocational Education and Training UKPF United Kingdom Prosperity Fund USAID United States Agency for International Development VfM Value for Money WIL Work Integrated Learning 8 9 1 INTRODUCTION The High Gear programme is an innovative initiative conceptualised and led by IYF which aims to demonstrate a model for greater industry involvement in Technical, Vocational, Education & Training (TVET) College course design and delivery. The model focuses on equipping industry partners in South Africa with IYF’s globally refined tools, curricula, and processes, which address core technical education challenges, and then working with industry partners to guide the initial implementation of these approaches with partner TVET Colleges. The High Gear programme operates on the following hypothesis: IF industry intermediaries better support and facilitate collaboration and partnerships between employers and public TVET Colleges, AND curricula and teaching in TVET Colleges is aligned to industry needs, THEN public TVET Colleges will become, and remain, more responsive to the needs of young people and industry, providing employers with an improved skills pipeline, and positioning more young people for workforce success. This hypothesis, with the High Gear programme Theory of Change (ToC) are outlined in Diagram 1 below. DIAGRAM 1: THE HIGH GEAR PROGRAMME HYPOTHESIS & THEORY OF CHANGE This hypothesis is being tested by the IYF staff in the High Gear programme. This report focuses on the progress made in the second year of the programme against these objectives and considers the extent to which actions undertaken appear to be resulting in the intended changes within colleges in which the High Gear programme is being implemented. It provides an analysis of the data collected from TVET students and graduates, and from TVET lecturers and college management officials at Elangeni TVET College in KwaZulu￾Natal and at Eastcape Midlands TVET College in the Eastern Cape. It also includes data collected from other stakeholders, such as industry intermediaries, employers from the automotive components manufacturing sector in KwaZulu-Natal and the Eastern Cape, as well as Government stakeholders at national and provincial levels in South Africa. 10 The Year 2 Annual Formative Evaluation Report focuses on the activities that were undertaken in the second year of the High Gear programme and covers the period January 2022 to December 2022. However, due to the fact that implementation of the programme in KwaZulu-Natal will end in March 2023, we have included some employer activities that took place in that province in February 2023. 1.1 Background to the High Gear Programme The High Gear programme was launched in South Africa in February 2020 and runs through March 2023 in KwaZulu-Natal at Elangeni TVET College and through February 2024 in the Eastern Cape at Eastcape Midlands College. The programme aims to create a demand driven public TVET system, which is aligned to industry needs and provides high quality course delivery to generate positive learning and employment outcomes for young men and women in South Africa. To achieve this, the High Gear programme has been guided by TVET Colleges and industry in: 1. Identifying priority industry competencies with industry associations and taking reasonable action to integrate these key industry competencies into the teaching environment at the identified colleges. 2. Designing industry validated TVET course upgrades in partnership with TVET lecturers and industry experts. 3. Supporting TVET Colleges to deliver industry-aligned content through drawing on lessons learnt by TVET and drawing on industry-based needs. 4. Strategically utilising and organising workplace exposure for TVET staff and students, which is co￾developed with employers and implemented by the High Gear programme staff and key implementing partners. 5. Delivering Sectoral Education and Training Authority (SETA) grant advisory services to employers within industry associations in order to assist these industry associations to secure funding from the relevant SETAs. The Government of the United Kingdom’s Skills for Prosperity Programme funded the High Gear programme implementation in KwaZulu-Natal until March 2023, and the United States’ Agency for International Development (USAID) and the Michael & Susan Dell Foundation (MSDF) are funding programme implementation in the Eastern Cape until January 2024. All three funding partners have supported the High Gear programme’s national stakeholder engagement and learning efforts. The design and implementation of the High Gear programme is being managed by IYF, in partnership with the National Association of Automotive Component and Allied Manufacturers (NAACAM) and the Department of Higher Education and Training (DHET). Within this partnership the following roles are allocated: IYF has partnered with Singizi Consulting Africa (Singizi) to conduct research and evaluation activities for the duration of the High Gear programme’s implementation process. IYF manages the High Gear programme partnership and leverages its model and curriculum enhancement tools. NAACAM ensures the High Gear programme remains responsive to the skill needs of employers in the automotive components manufacturing sector. DHET, as the overseer of South Africa’s public TVET College system, informs decisions related to the selection of partner TVET Colleges; the selection of courses/qualifications for upgrading; the review and approval of proposed course upgrades; and the facilitation of TVET College leadership and lecturer participation in the initiative. In addition, DHET will support expansion of successful course upgrades in the TVET system 11 1.2 Evaluation Purpose The purpose of this formative, developmental evaluation is to assess the progress made against outcomes and the extent to which the High Gear programme is contributing towards the intended impact of the programme. In doing so, we have tested key components of the programme’s design and the assumptions underpinning the Theory of Change during the implementation period. Specifically, this formative evaluation focuses on progress towards system adoption in relation to partnerships, curricula, training and employment outcomes: In June 2021 a baseline study was conducted for the High Gear programme by Singizi. The baseline provides a basis against which change could be measured. It reports on the perceived status of TVET curricula and training against programme indicators, it offered more nuanced insights into the training system based on the perspectives of NAACAM and other industry intermediary respondents, employers, college officials, TVET NATED 4-6 engineering graduates (2019 and 2020), lecturers, and other key stakeholders. During the baseline study, programme partners confirmed the logic of the High Gear programme Theory of Change and affirmed the importance of sustainable partnerships to improve the alignment of the TVET Colleges’ curricula and teaching to industry needs. All stakeholders interviewed were confident that with greater levels of alignment the colleges will become, and remain, more responsive to the needs of both young people and industry, which in turn should provide employers with an improved skills pipeline, contribute to improved productivity, and position more young people for access to employment. The objectives of this report are to: •Improved & sustained partnerships between industry Partnerships •Lead to curricula & teaching that is better aligned to industry needs Curricula •Which results in improved learning & employment outcomes for young people Training & Employment Outcomes Determine whether the hypothesised Theory of Change is holding true or not, and the factors that are influencing these results. Inform programme strategy shifts, if necessary, to strengthen implementation and outcomes. Gather preliminary evidence of the High Gear programme’s impact on young people and the TVET system. DIAGRAM 2: HIGH GEAR PROGRAMME OUTCOMES 12 Additionally, the findings of the formative annual evaluation report provide a response to the High Gear programme’s primary evaluation questions, evidence of the robustness of the High Gear programme’s Theory of Change and, where relevant, recommendations about modifications that may be required to the Theory of Change. The primary evaluation questions are presented here, in Error! Reference source not found.Diagram 3 below and are dealt with in more detail in Annexure 4 to this report. DIAGRAM 3: PRIMARY EVALUATION QUESTIONS FOR THE HIGH GEAR PROGRAMME •Does it contribute to increased and sustained partnerships between industry and TVET Colleges? •To what extent are the curricula and teaching in TVET systems more aligned to industry needs and being delivered as intended according to project design? •To what extent did the intended outcome of increasing young women’s and young men’s learning and employment in targeted regions occur over the course of the project? Question 1: Is the High Gear programme’s Theory of Change— including its embedded assumptions—accurate, thereby leading to anticipated results? •Does the High Gear programme contribute toward increases in economic opportunity and inclusive growth, as a result of a higher-quality, industry-aligned TVET system continuum? This relates to the extent to which the High Gear programme has contributed towards: •Increased (or more effective) ecosystem partnerships •A more youth and gender-inclusive system •Improved competitiveness of the automotive manufacturing sector, in relation to skills and productivity •Further embedding TVET partnerships into South Africa’s industrial strategies •Increasing employers’ confidence to make investments in technology upgrades Question 2: Is the High Gear programme contributing towards increased ecosystem resilience? •Does the High Gear programme exhibit programmatic and operational approaches that deliver the most value for money (economy, efficiency, effectiveness and equity) in improving learning and workforce outcomes for young people enrolled in the TVET college system, including for low-income youth, young women and youth with disabilities? Question 3: Does the the High Gear programme model deliver value for money (VfM)? 13 1.3 High Gear Programme Interventions and Activities being evaluated in Year 2 In 20221 , the High Gear programme was implemented in two TVET Colleges, one per Province. Interventions focused on two qualifications namely Electrical Engineering and Mechanical Engineering in each of these colleges and impacted a total of 944 students and 46 lecturers, with a variety of technical assistance interventions, as shown in Table 1 below: TABLE 1: SCOPE OF HIGH GEAR PROGRAMME ACTIVITIES IN 2022 (AND 2023 IN KWAZULU-NATAL) Elangeni TVET College Eastcape Midlands College Province KwaZulu-Natal2 Eastern Cape Qualifications Electrical Engineering & Mechanical Engineering Electrical Engineering & Mechanical Engineering Students 648 296 Lecturers 28 18 Technical Assistance Interventions WIL, ELT, Career Guidance, SETA Advisory Services WIL, ELT, Career Guidance Source: IYF MERL Team These technical assistance interventions are detailed in Table 2 below: TABLE 2: HIGH GEAR PROGRAMME TECHNICAL ASSISTANCE INTERVENTIONS IN 2022 (AND 2023 IN KWAZULU-NATAL) Elangeni TVET College Eastcape Midlands TVET College WIL placements students: 56 (N6) electrical engineering graduates WIL placements students: 54 (N5-6) electrical and mechanical students WIL placement lecturers: 12 electrical engineering lecturers WIL placements lecturers: 18 electrical and mechanical engineering lecturers Career guidance: Industry guest lecturer sessions Career guidance: Yakh’iFuture Activation- In-person student demonstration and activation workshop CV optimisation services (also took place at Coastal College in Feb 23) Career guidance: Industry guest lecturer sessions Career guidance: Yakh’iFuture Activation-In-person student demonstration and activation workshop Mentorship briefing and planning sessions Mentorship briefing and planning sessions Effective teaching Session: focused on sharing methodologies to improve knowledge transfer in the classroom. Yakh’iFuture Platform Demo sessions: focused on upskilling the lecturers on use of gamification function, demo kit assembly and class integration of demokit aids Effective teaching Session: focused on sharing methodologies to improve knowledge transfer in the classroom. Yakh’iFuture Platform Demo sessions: focused on upskilling the lecturers on use of gamification function, demokit assembly and class integration of demokit aids The High Gear programme Working Groups Session: focused on subject matter additions that could support lecturers in facilitating classroom facilitation the High Gear programme Working Groups Session: focused on subject matter additions that could support lecturers in facilitating classroom facilitation SETA Advisory Services info sharing session (Feb 23) Source: The High Gear programme MERL Team 1 As noted above, the project is coming to an end in March 2023 in KwaZulu-Natal, and the only activities held this year, in February, were a SETA Advisory Services breakfast and a CV optimization workshop at Coastal College, and so these have been included in this report. 14 15 2 EVALUATION METHODOLOGY The High Gear programme programme’s Theory of Change includes a detailed Results Framework containing indicators for all the Objectives, Intermediate Results, Short Term Results and Outputs. Each of these indicators has a measure, and during the Baseline Study, baselines and targets were set for the indicators. The baseline provided an understanding of the system and beneficiaries before the intervention, and during the Year 2 formative evaluation, Singizi collected data against these indicators to understand changes that have taken place in the second year of implementation of the High Gear programme, as well as to build on the findings from the Year 1 formative evaluation and to explore preliminary answers to some of the primary evaluation questions. The full Results Framework is attached as Annexure 2, (although, as described below, performance data from the colleges has not been received and is not yet complete) and some of the key findings are integrated into the various sections of this narrative report. Data collection methods focused on gaining as broad an understanding of the programme as possible and included methodologies to assess progress against both the indicators and the three primary evaluation questions. An overview of the data collection methods used, as well as stakeholders with whom we engaged, is reflected in the list below, with more details contained in Annexure 3: A structured CATI process with 249 Elangeni TVET College graduates and 173 Eastcape Midlands College graduates. Focus Group Discussions with 17 Eastcape Midlands College graduates and 11 Elangeni TVET College graduates. A structured CATI process with 13 Elangeni TVET College lecturers and 16 Eastcape Midlands College lecturers. Focus Group Discussions with 17 Elangeni TVET College Lecturers and 17 Eastcape Midlands College lecturers. Interviews with four employers in KwaZulu-Natal and six in the Eastern Cape. Interviews with two members of the College Management team at Elangeni TVET College and three at Eastcape Midlands TVET College. Interviews with key stakeholders including NAACAM and other intermediary bodies, merSETA, DHET etc. Various engagements with the IYF team to reflect on the routine M&E data that they collected. 16 In addition to this fieldwork conducted directly for this Year 2 Formative Evaluation, results from two Tracer Studies conducted at Elangeni TVET College and Eastcape Midlands TVET College have been included in this report. The Tracer Studies reached 46 N4-N6 graduates in June 2022 and 43 N4-N6 graduates in January 2023. We further note that a full Value for Money (VfM) analysis was excluded from the scope of this report, but IYF provided key internal VfM indicators which were analysed and included in Section 3.3 below. 2.1 Limitations to the Study Lecturers were difficult to reach, especially in KwaZulu-Natal, as they indicated they had limited time due to work pressures. In total we reached 63% of the total population of 46 lecturers involved in the High Gear programme at both Elangeni TVET College and Eastcape Midlands TVET College. Employer interviews for the evaluation were also difficult to secure, and some of the employers to whom we were referred by IYF were not sufficiently familiar with the activities of the High Gear programme and so could not provide detailed comment (although they did provide input on skills development in the industry more generally). We planned to reach nine employers in KwaZulu-Natal and ten in the Eastern Cape. Despite repeated attempts to schedule interviews, and the assistance of IYF, many employers were simply not able to make the time to participate in the process, and ultimately, we only reached four employers in KwaZulu-Natal and seven employers in the Eastern Cape. This means that there is limited insight from companies in this report. In KwaZulu-Natal we could not reach officials from merSETA or the regional office of DHET despite numerous appointments and requests for interviews. However, we did secure interviews with national officials from merSETA and DHET and these views are included in this report. 17 18 3 FINDINGS IN RELATION TO THE PRIMARY EVALUATION QUESTIONS This formative evaluation found that the second year of implementation of the High Gear programme in KwaZulu-Natal and the Eastern Cape built on the foundation that was laid in the first year. This allowed the High Gear programme to achieve concrete results in terms of supporting the skills development system to become more responsive to industry needs. Diagram 3: Primary Evaluation Questions for the High Gear ProgrammeDiagram 3 at the beginning of this report summarises the primary evaluation questions against which the data is analysed. Annexure 4 contains a matrix which outlines how each of the evaluation questions relates to the overall purpose of the evaluation and to the Theory of Change. 3.1 Question 1: Is the High Gear programme’s Theory of Change – including its embedded assumptions – accurate, thereby leading to anticipated results? Yes partly: the High Gear programme has facilitated greater and more effective employer involvement in curriculum review, and enabled industry association leadership of the process and co-ordination of the partnership between employers and colleges. However, there are some concerns about whether colleges are able to take forward ongoing revision of curricula, or about whether these partnerships have resulted in sufficient opportunities for students. The High Gear programme’s Theory of Change includes several core assumptions related to the roles and efficacy of stakeholders in the South African skills development system. These assumptions include: The findings of this evaluation suggest that these assumptions partially hold. There is a focus on curriculum reform, which IYF has already been able to support through the High Gear programme in the review of the existing qualifications. Specifically, IYF has supported the Quality Council of Trades and Occupations (QCTO) in designing the curriculum frameworks for new occupational qualifications for electronics, electrical and mechanical engineering studies and two foundational modules: work readiness and business communication. This work on the qualifications and curriculum framework started in mid-2021 and was completed in July 2022. There is recognition of the need for curriculum reform, and an appropriate sense of the urgency of this need, among both private and public sector stakeholders. Industry intermediaries are able to effectively facilitate linkages between industry and TVET Colleges, and that they have the capacity to do so. Employers are willing to invest time and resources in programmes that benefit the sector as a whole, including their competitor firms. The TVET sector has the capacity to implement processes that require the ongoing revision and updating of curricula in line with changes in industry production processes. That students have the means and capacity to access opportunities made available through partnerships between TVET Colleges and industry. 19 The importance of this work was highlighted by stakeholders who indicated that the NATED qualifications were very out of date and required fundamental changes in order to meet the needs of industry. As indicated by one employer respondent from KwaZulu-Natal, “Typically, historically one of the issues around this is TVET Colleges or the Department of Education goes and develops the curriculum. That is why the NCV never got real uptake from the employers because it did not address the employer needs.“ Employer respondents explain that their involvement in the High Gear programme enables them to make considered input into the curriculum. One industry intermediary respondent detailed the manner in which they were able to reflect industry needs into the process of upgrading the course content: “…. through our quarterly automotive labour market survey, [that] our members participated in, we were able to understand … the scarce skills, what are the hard to fill vacancies and what are the challenges with graduates coming in the labour market. We submitted these inputs to the IYF team, and they worked on upgrading course content” (Industry Intermediary). This is seen as a real improvement on previous experiences with the NCV and employers suggest that this has resulted in employers and TVET Colleges working together for greater effect. The revised qualifications and the curriculum frameworks also include a stronger focus on practical training, the need for which also emerged strongly in this evaluation, as well as work integrated learning. We also saw evidence of the value of industry associations playing a stronger coordination role between companies and colleges: as observed by one industry intermediary respondent “NAACAM has the influence to create sustainability when we have access to our members who are employers of these TVET graduates”. Related to this is the finding that employers see the benefit of participating in the High Gear programme, especially in their ability to recruit high quality new entrants: “We received support in a sense of ensuring that the people that we are getting are of the right calibre, they have been screened and they are ready to come in take this opportunity” (Employer, Eastern Cape). This has resulted in employers being more willing to participate in the programme: “Such programmes are allowing youth to see the real world of work in manufacturing and engineering and … [the] level of participation is big from the company side…. [We] do not have to do it as we have our own work to do, but the fact that we have opened the doors and are willing to participate really shows a lot of commitment.” However, we found (including when seeking interviews for this evaluation) that the challenges in the current context have adversely effected the extent to which companies are able to invest as much time and resources in the programme as anticipated. This will need to be considered in the continued implementation of the High Gear programme. Further, there is uncertainty as to whether TVET Colleges have the capacity or flexibility to revise and update curricula as required. Certainly, there appear to be challenges with respect to keeping up with technology, to ensuring that there are the practical resources in the colleges to support applied learning in the updated curricula, and that lecturers have the content related knowledge that they require to deliver the new elements of the curriculum. There have also been challenges in terms of the extent that these partnerships have resulted in direct opportunities for students: Work Integrated Learning (WIL) opportunities for students, based on stronger partnerships between employers and colleges, is a key component of the High Gear programme as they enable students to gain exposure to the tools of trade and workplace culture, as well as possible future employment opportunities. However, for a variety of reasons, including the economic pressure on companies, most students have experienced difficulties in accessing WIL during their studies. It appears to have been easier for some students, particularly in KwaZulu-Natal, to access WIL after they have graduated, which suggests that it might be necessary to shift the focus in the programme to ensuring that young people complete their 20 qualification and are then supported to access both short term WIL opportunities, and more long-term employment opportunities once they have graduated. 3.1.1 Does the Theory of Change contribute to increased and sustained partnerships between industry and TVET colleges? Respondents from employers, from industry associations and from TVET Colleges all confirmed that they have been able to establish what they consider to be mutually beneficial partnerships and suggest that the High Gear programme has both contributed to the development of these partnerships and that these partnerships will in turn contribute to the overall goal of the High Gear programme. The employers interviewed for this evaluation3 indicated that they have engaged with colleges through the High Gear programme in various ways, including through the provision of WIL opportunities for students and lecturers, and through curriculum reform within the Engineering programmes. Employer respondents confirm that through this engagement, their relationships with TVET Colleges have shifted and that this in turn has led to changes in the way they think about skills planning, work integrated learning and career guidance. One employer said that he thought that the High Gear programme and the IYF team “have really been playing an intermediary role and have brought us closer to the colleges and the management of the colleges.” He added that there is now top management support for improved employer relations from the side of the college, and they have bought into the programme, “they have taken on more responsibility and if we’re not getting results from those we work with at the college [then] we pull in management and then we get things done.” Another respondent pointed out that, overall, the relationship between industry and the colleges has improved as “colleges are seeing a great response from industry who are more into the programme – they are seeing the benefit and industry is now enjoying working with the colleges and they are starting to donate equipment to the colleges.” The respondent added that, “we have some companies who are willing to go to the colleges and inform colleges and learners what the companies do - we have had some great host companies who are willing to go extra mile.” Key to these relationships, he indicated, is someone with passion at the college who is dealing with the host companies and is driven to ensure the development of youth (Employer, Eastern Cape). TVET College respondents echoed this view, indicating that they think that through the High Gear programme industry appears to ‘understand’ them better. One respondent from the Eastern Cape stated that “NAACAM partners are exposed to us as colleges and they understand us through the High Gear project (sic); but when the project ends it would be good if the conversation continues with industry” (College Management, Eastern Cape). Another respondent added that, in general, management at the college is excited about the possibilities of the partnerships and ascribed this largely to the presence of a detailed Memorandum of Understanding (MoU) which sets out clear guidelines that govern the relationship, and which is seen as crucial to developing and fostering linkages between TVET and industry. 3 As indicated previously this includes automotive component manufacturers as well as NAACAM members from KwaZulu￾Natal and the Eastern Cape. 21 In summary, with respect to this evaluation question, we found that stakeholders see a clear rationale for their involvement in these partnerships, and respondents suggest that the partnerships are beginning to contribute to the intended results. Critically, employer respondents highlight the importance of the role that IYF team plays in facilitating these partnerships and ensuring the decisive implementation of the High Gear programme. “What I like about this the High Gear programme and the involvement of IYF is that they make decisions and things gets going”. Respondents observe that building strong and sustainable partnerships can be challenging and emphasise that it will be important for the High Gear programme to continue to support these relationships. This sustained support is coupled with a growth in the number of partnerships in the programme: in 2022 the High Gear programme established new partnerships with two TVET Colleges, namely Coastal College in KwaZulu-Natal and Port Elizabeth TVET College in the Eastern Cape. IYF respondents indicated that interventions in these two colleges will focus primarily on support for student and lecturer WIL placement and on the utilisation of the Yakh’iFuture platform. 3.1.2 To what extent are the curricula and teaching in TVET systems more aligned to industry needs and being delivered as intended according to project design? There have been valuable revisions to the qualifications. This has also allowed for the development of curriculum frameworks that are better aligned to the needs of industry. Stakeholders, including colleges indicate they are more aware of industry needs, but state the process of formally updating curricula is long and complex, which does not accommodate the fast pace of technology change. Different ways of thinking about ensuring ‘relevance’ therefore need to be considered, including practical training for students and workplace exposure for students as well as lecturers. With respect to lecturers, it was found that lecturers are not feeling very confident about their ability to deliver quality learning within the new frameworks, although students suggest that their lecturers are helpful and skilled. As outlined above, the High Gear programme has made an important contribution to the reform of qualifications as well as the development of curriculum frameworks. However, the key achievement during Year 1 and Year 2 of the High Gear programme, with respect to the alignment of the curricula to industry needs, has been the fostering of a supportive environment for curricula change. This change includes, to some degree, an improvement in the quality of teaching. Respondents note that, while the upgrading of TVET curricula is a lengthy and complex process, the High Gear programme has been able to communicate industry requirements into the colleges in a manner that bridges the gap between colleges and industry and fosters collaboration: “The fascinating thing with the High Gear programme is [that they have surfaced] input from industry regarding the competence of graduates and curriculum content gaps and have been able to support colleges and DHET to sit down and look into these issues. They have been able to connect colleges with industry and this is a key lesson that can be taken to other qualifications and industries” (DHET). Some respondents from the colleges indicated that they thought that the curricula are now more aligned to industry needs, with one college respondent from KwaZulu-Natal suggesting that “We …[think] that the curriculum is … definitely aligned. A lecturer did a demo using the toolkit from the High Gear programme, if you listen to the practical implementation of that exercise that they did, then you realise that it is applicable” (College Management, KwaZulu-Natal). The majority of lecturers interviewed were more cautious about these changes, with some respondents stating that they do not yet feel that the changes in the curricula address all of industry’s needs. Further, lecturers also indicate that while they have received some training through the High Gear programme, they do not feel fully prepared to offer a changed curriculum. 22 Of import though is that students4 are very positive about the quality of lecturers and the extent that they were willing to assist them to understand concepts that they are struggling to understand. The students also indicated that the lecturers offer them support for tests and exam preparations: “The lecturers are very committed to their work and helping us if we don’t understand”. Students do indicate that they are concerned about the limited access to practical training during the programme, as well as workplace experience during and post the programme. One respondent observed that, “It will be better to gain work exposure next year when we are doing N6, because it’s going to be difficult for us once we have finished the course to be employed without experience or practicals.” Overall, these findings suggest that advances have been made with respect to the qualification and associated curriculum frameworks and in the implementation of the programmes, although there is still a need for sustained efforts to effectively support curricula change. This carries with it the need to grapple with how the process of adapting the curricula can be more responsive given the relatively rapid technological changes that occur in industrial production processes. It will also be critical to ensure that lecturers receive further training to effectively offer updated content. As indicated previously, this should be combined with efforts to enable more students to access practical training and workplace experience, both during and immediately after completion of their qualifications. 3.1.3 To what extent did the intended outcome of increasing young women’s and young men’s learning and employment in targeted regions occur over the course of the project? There is evidence of improvements in learning, with learner and lecturer respondents noting that the quality of the course was good. The tracer study also showed an increase in the number of students who state that they received guidance in their courses. However, there is a clear need to increase the practical learning and workplace exposure components of the course. In relation to increased employment, the numbers of graduates who indicated that they are employed in the tracer studies is still low, and most employed young people have contract work or are on learnership agreements. The percentage of employed graduates increased between the two tracer studies, from 4% to 15% of respondents, which is important given the challenging economic context and the depressed labour market. Increasing learning Respondents indicate that they have acquired learning that challenges their thinking and that will provide the foundation that they require to gain new knowledge. Further, most of the youth who were still studying in Year 2 stated that they think that their qualifications will equip them for further study (86% in Elangeni TVET College and 89% in Eastcape Midlands College)5 . However, as indicated above, exam results have not yet been released by either of the colleges and so we are not yet able to confirm these views in practice. Respondents also indicated that they were able to link their knowledge with their work thereby enhancing their on-the-job performance. A student commented that: “The course was challenging and gave us knowledge that we can use in working in industry”. In the second tracer respondents also stated that the training that they had received through the programme has equipped them with the skills required to find employment and function effectively in the workplace. As observed by one respondent, “The knowledge is useful in performing work tasks, it is foundational knowledge everyone needs when they start working- even in interviews they asked me questions that I was able to answer about electrical fields”. 4 Students interviewed as part of the CATI process as well as participants in the Focus Groups 5 Data from the CATIs reflected that 55% of the respondents from KwaZulu-Natal and 67% of respondents from the Eastern Cape indicated that they were studying in 2021 and that they continued in their studies in 2022. 23 To understand the extent to which the High Gear programme has assisted students to access opportunities, in the tracer study we probed the nature of the guidance that students had received: in June 2022, 24% of Elangeni TVET College and 29% of Eastcape Midlands College respondents indicated that they had received some type of guidance. In January 2023, this percentage had significantly increased to 68% of Elangeni TVET College respondents and 52% of Eastcape Midlands College respondents indicating that they received some type of guidance. Students also indicated that lecturers are also supportive of their transition plans. One student respondent stated that “The lecturers are very supportive because they are advising us to apply for learnerships and internships so that we can get work experience”. Additionally, during the first tracer study it was found that 17% (n=5) of Elangeni TVET College respondents and 18% (n=3) of Eastcape Midlands TVET College respondents indicated that they had accessed workplace exposure/experience linked to their studies. In the second tracer study there was a small increase in this percentage from Elangeni TVET College respondents (23%/n=5), and a slightly larger increase from Eastcape Midlands TVET College respondents (33%/n=7) indicating that they had accessed workplace exposure/experience. The views of the students engaged for this evaluation (through the CATI process) on workplace experience/exposure were also positive, as shown in Figure 1 below. Almost all respondents from Eastcape Midlands TVET College felt that the experience/exposure had enabled them to understand the content of their qualification more effectively, and that the knowledge and skills gained was linked to the course content. Interestingly, almost all the respondents from Elangeni TVET College agreed or strongly agreed that they had acquired new skills during the work exposure, against only half of the respondents from Eastcape Midlands TVET College. The lecturers from Elangeni TVET College indicated that they had been consulted on areas which should be emphasised in the process of setting up the workplace exposure/experience whereas the Eastcape Midlands TVET College lecturers stated that this was not the case for them. 24 FIGURE 1: STUDENT VIEWS ON WORKPLACE EXPERIENCE/EXPOSURE Source: Student CATI, November 2022, Singizi 54% 62% 77% 92% 69% 69% 71% 76% 67% 52% 90% 90% In general, the process of accessing workplace experience/exposure was reasonably easy The college learning (theory and practical) was integrated with the workplace experience In general, the existing employees within companies were knowledgeable and helpful I have acquired new skills during the workplace experience/exposure which increase my chances of finding employment In general, the knowledge and skills gained through workplace experience/exposure was linked to the course content In general, the workplace experience/exposure enabled me to learn and understand the content of my qualification more effectively. EMC Elangeni Monitoring data provided by the MERL team reflects the students' positive feedback regarding their experience in the workplace: •"My highlights were working with an artisan on the floor and being explained the workings of the PLC. Also learning about the workings of … the inner gears”. •“Exposure to daily activities of the workplace gave vast and distinct meanings together with relations concerning skills and competencies”. •“My highlight was when we did the relay circuit and now, I have a better understanding of relays and how they work​”. However, students indicated that it would be preferable if they received more workplace learning opportunities, noting the need for this experience when they graduate. •“It will be better to gain work exposure next year when we are doing N6, because it’s going to be difficult for us once we have finished the course to be employed without experience or practical.” •“The college should facilitate work experience for graduates because a lot of them miss out on job opportunities because of a lack of experience which is required by employers.” These views are echoed by employers.... •"The students really valued the work experience from us because it allowed them to take what they learned from the classroom to what we were teaching them from the boardroom, and then taking it to a plant because they were able to take a cycle and start it from the bottom, programme a robot, get a finished product and get to see how all those things interact." (Employer, KwaZulu-Natal). Who also highlighted the issue of the placements being very short and intense: •"They come in for 5 days and we need about 8 hours on the first day for induction and this means immediately the first day is out, and they are left with only 4 days. We provided them with mentors from our side so that they can guide them. This was most valuable to the students because they were now able to discuss with qualified people on what’s happening in the industry and how to work on the machines." (Employer, Eastern Cape). 25 The imperative to reflect on what is possible with respect to increased access to workplace experience in the context of the current economic crisis is even more evident when reviewing employment outcomes from the tracer studies. Increasing employment In order to develop the sample for the tracer study, we sent a short text survey via bulk SMS to all High Gear graduates to confirm their economic status. This was done for both the 6- and 12-month ex-post tracer studies, in June and December 2022. In June, we found that 6% (N=176 ) of the 264 graduates who responded said that they were employed, and in December we found that 15% (N=44) of the 299 graduates who we reached said that they were employed. And in January 2023, we also called those respondents that we had interviewed in June 2022 to establish whether their circumstances had changed. We found that, of the 21 respondents who were working in June 2022, only 11 were still working in January 2023. Nine of those who are no long working indicated that they are now unemployed and are not doing any other work and one could not be reached. Most of these nine respondents indicated that their change in status is because their contract or their internship/learnership programme ended. And of the two who said that they were self-employed in June 2022, one is now working. On the positive side, three of the 13 who were unemployed in June said that they are currently working, all on fixed term contracts and/or learnerships. Based on this data it is evident that where young people are able to access employment, this is primarily in contract work, or learnerships/internships. In the context of the many points of crisis over the past year it is perhaps not surprising, albeit concerning, that so few young people are able to enter the labour market and that many of those who had work are no longer working. The challenges in finding workplace exposure, coupled with the difficulties that young people are experiencing in sustaining employment, highlights the need for the High Gear programme to consider the implications of this challenging economic context and depressed labour market in the focus of the programme. 3.2 Question 2: Is the High Gear programme contributing towards increased ecosystem resilience? Yes: there is definite evidence of increased ecosystem resilience, particularly in relation to sustainable partnerships between industry and colleges, and better relationships between employers and the merSETA. Respondents highlight the role of the High Gear programme in achieving this. This question examines whether the High Gear programme’s focus on improving TVET quality and industry alignment has led to an increase in the resilience of the automotive component manufacturing sector in South Africa. In Year 2, it appears that there has been movement towards building a foundation of resilience in the sector through purposively implemented interventions. The importance of these shifts are described by respondents in the following ways: “If we as the people who are working in corporate are not the ones driving the transformation or driving the development then we are going to find ourselves in the same situation. I quite enjoy interacting with the staff at NAACAM and the staff at IYF who are the High Gear programme team; because these are the people beyond what they are mandated to do they are really trying to bring industry and education together so that it can develop, and the industry can grow and be more localised” (Industry intermediary). Another industry intermediary respondent observed that, “Our members have realised the value of partnerships and its more than getting young people to work in the automotive pipeline but to understand their role within the skills ecosystem, which involves a community level obligation to work with other employers to achieve skills development change” (Industry Intermediary) 6 Note that this number increased to 21 when we called the respondents, as some of those who had indicated that they were unemployed and doing something to earn money had subsequently been employed. 26 An employer from KwaZulu-Natal stated that, “At the end of the day if you are not going to develop suppliers, the cost is always going to be driven higher because you have to import; but if you develop somebody close to you – yes, it will obviously cost you more now but if you develop that person who is closer to you and develop them for a period of 3 years is an additional cost at that time but it is a return in investment once they can actually supply you to the level of skill that you want. I think we need to broaden our perspective a little bit, and we need to be intentional about really driving transformation so that it not only works for society and the economy, but it works for you as a corporate as well” (Employer, KwaZulu-Natal). Employer respondents repeatedly highlighted that industry associations such as NAACAM and AIDC are playing a vital role in sharing the High Gear programme with companies and in building strong relationships with the High Gear programme team. As observed by employers from the Eastern Cape, “Through this programme NAACAM has been supportive in helping us to understand what is happening in the industry in terms of developments and skills demands” (Employer, Eastern Cape). “NAACAM is heavily involved in the communication with the TVET Colleges, and our HR is also very much hands on, and he was [in] the forefront in making sure that this [WIL placement] happens” (Employer, the Eastern Cape) Another employer respondent also affirmed that NAACAM was important in enabling communication through “driving a lot of marketing”. He added that once employers see endorsements from NAACAM and NAAMSA, and the value-add of the High Gear programme, they start to buy-in and support the programme. Another key play in the ecosystem is merSETA: employer respondents in KwaZulu-Natal indicate that they have good working relationships with merSETA, with one respondent noting: “We have plenty of engagement with merSETA, we see them as our constant partner around skills development because when we host learnerships and internships we are always in constant engagement”. Similarly, employers in the Eastern Cape indicated that engagements with merSETA have also improved since the first online SETA advisory session with employers. Respondents noted that the merSETA has now agreed to have monthly meetings about the allocation of funds with employers and is able to guide the process of implementing skills development programmes that are aligned to SETA targets, as well as provide information about scarce and critical skills. One employer respondent from the Eastern Cape noted that: “We probably got roughly about R230K [from the merSETA]: it helps to run learnerships and the funding covers costs for training and development of the people especially with accredited courses under the merSETA umbrella”. Employer respondents indicated that there are still aspects of this relationship that need to be improved. This includes more consistency in communication and coordination. A respondent commented that: “Our experience with them [merSETA] has been challenging but good, their administration is a challenge, their system is a challenge. but we do engage with them. We do receive useful information from them regarding skills development” (Employer, KwaZulu-Natal) This need is recognised in the High Gear programme design and, as mentioned previously, in the course of 2023 the engagements between employers and the merSETA are intended to be expanded. Specifically, IYF will support eight KwaZulu-Natal employers through SETA advisory services to leverage SETA funding opportunities and increase the quality of proposals that include TVET partnerships. 27 Employer respondents spoke specifically about the importance of the High Gear programme in supporting this work in creating linkages and building a more enabling environment. One employer respondent (from the Eastern Cape) stated that they are receiving information and links to articles on the High Gear programme from which they have drawn lessons on industry practices around TVET collaborations as well as trends within the skills development landscape. Another industry respondent stated that, “IYF plays an important role, and the programme would not be successful without them ... capacitating the key partners going forward.” An implementation partner, providing services as part of the High Gear programme, echoed this view and stated that the High Gear programme has been very important to encouraging collaborative work and that “this programme is brilliant for the future of the country – it is needed.” These findings suggest that the High Gear programme is contributing to ecosystem resilience both through enhancing TVET and industry partnerships and through the provision of information which encourages relationships across the partners. 3.3 Question 3: Does the High Gear programme project model deliver Value for Money? (VfM) In terms of the funding agreement with the Skills for Prosperity Programme, IYF has been required to track a number of Value for Money indicators over the course of programme implementation. IYF’s definition for VfM for this programme is: Programmatic and operational approaches that deliver the most value for money (economy, efficiency, effectiveness and equity) in improving learning and workforce outcomes for young people enrolled in the TVET college system, including for low-income youth, young women and youth with disabilities. 28 Table 3 below summarises the data submitted by IYF against these indicators. The full tables are contained in Annexure 1 to this report. TABLE 3: SUMMARY TABLE OF ACHIEVEMENT AGAINST VFM INDICATORS Indicator Type Indicator Title Total 2022 Economy Indicator 1 Cost Containment of Expenditures 83% This is just under the quarterly burn rate target of 95%. The quarterly target was achieved in one quarter in 2022, it was not met in two quarters and was exceeded in one quarter Economy Indicator 2 Promotion of VfM through competitive procurement of inputs 67% The notes indicate that IYF competitively bids all High Gear contracts, with the exception of technical service providers pre-identified during the proposal stage and included in the donor-approved proposal; and sole service providers in the market. Efficiency Indicator 1 Unit cost per TVET lecturer, per year, capacitated with project-based learning upgrades £2,775 The evaluation team did not have access to any data with which to compare this unit cost, to take a view as to whether it shows ‘value for money’. Efficiency Indicator 2 Direct Yakh'iFuture platform cost divided by estimated number of youth reached through the Yakh'iFuture career portal. £23 A significant decline in the unit cost was noted over the year, indicating that this unit cost will decline further as more youth are reached. IYF notes that the project costs for Q4 include the EV component of work done on the Yakh'iFuture platform. Effectiveness Indicator 1 Benchmarking of youth beneficiary costs with other similar initiatives in SA's TVET system £100 This cost refers to the reach of the Yakh’iFuture platform as well, and there are no other similar initiatives in the SA TVET system Effectiveness Indicator 2 Improved use of state and/or student resources, through improved TVET completion rates (i.e., cost of TVET tuition and/or bursary more effectively used) Unknown The implementation of the POPI Act has delayed the release of the Q4 results by the colleges, so this indicator cannot be reported on. Effectiveness Indicator 3 GBP (cash/or in-kind resources) leveraged by High Gear partners for activity implementation £4,400.00 Although these funds were raised by a High Gear partner from the private sector, they ultimately had to be returned as a contractor to use the funds could not be procured within the given time frames. Equity Indicator 1 % of High Gear inputs sourced from businesses owned by individuals from historically disadvantaged backgrounds and/or consultants from historically disadvantaged backgrounds (per SA government definition – disaggregate by gender and HDI status) 40% These figures are not disaggregated by gender and HDI status. Equity Indicator 2 % of High Gear youth beneficiaries from historically disadvantaged backgrounds (per SA government definition) 100% IYF has noted an increase in student enrolment, following the decline in 2020 and 2021 caused by covid-19 disruptions. To date, the programme has reached 913 students, 46% of the 2000 student target. IYF projects that they will increase this to 1363 during trimester 1 enrolment for 2023 (quarter 1). This will be 68% of the target. Source: IYF 29 30 4 FINDINGS AGAINST THE HIGH GEAR PROGRAMME OUTCOMES In the Year 1 formative evaluation, it was highlighted that the High Gear programme was beginning to have a positive effect on skills development in the automotive components manufacturing sector in South Africa. The first year saw the implementation of important foundational work, including developing coordination mechanisms through industry intermediaries, increasing interest and willingness to develop partnerships between industry and TVET Colleges, improved access to funding for industry and TVET Colleges to implement WIL for lecturers and students, and improvements in some student level outcomes. The evaluation also found that foundational work had been undertaken with the QCTO to both revise and build new qualifications that better reflect the needs of industry. The Year 2 formative evaluation found that this strong foundation has been built upon, and, as noted in Section 3 above, partnerships between industry and TVET Colleges, and amongst employers, have been sustained and deepened, the curricula is more aligned to industry needs and there is an increase in the percentage of students and lecturers accessing WIL, although this percentage is still too low and there is a concern that the WIL placements are too short to be meaningful. Of import is that we are also starting to see better employment outcomes, even within a very difficult economic climate, although as indicated this is not yet fully realising the intended outcome in this regard. This section of the evaluation further reflects on these key findings, for the second year of the implementation of the High Gear programme, against the specific targets that were set for each of the three objectives of the High Gear programme, namely partnerships, curricula, and training and employment outcomes. 4.1 Partnerships have been sustained and deepened The first objective within the High Gear programme Theory of Change speaks to partnerships and is premised on the assumption that sustained automotive manufacturing industry involvement in aligning public TVET courses with employer needs relies on these relationships working for all partners. As indicated previously in this report there is evidence of strong and effective partnerships between industry and TVET Colleges, and improved relationships between employers and merSETA, with partners seeing a clear rationale for their involvement in these partnerships. These positive views about the effectives of the coordination enabled by the High Gear programme are evidenced by the numbers involved in these relationships, and the value of the coordination is further shown by the realisation of the targets with respect to the adoption of practices and policies by industry partners. 31 From the Results Framework The Year 2 target for the number of beneficiaries engaged in partnerships, engagements and convening activities was exceeded in both KwaZulu-Natal and the Eastern Cape, (target in Eastern Cape was 47 beneficiaries and Year 2 result is 90, target for KwaZulu-Natal was 14 and 56 beneficiaries were engaged). These beneficiaries included government staff, employers, lecturers, and college management staff. Year 2 targets for the number of the High Gear programme practices and policies that are adopted by industry partners were exceeded in both KwaZulu-Natal and the Eastern Cape: we found that three practices/polices were adopted in each of KwaZulu-Natal and the Eastern Cape. These practices are in relation to partnerships with TVET Colleges to provide WIL opportunities for TVET staff and students, providing career guidance for TVET students as well as engaging with TVET Colleges to influence the alignment of TVET curricula. In relation to the number of partners who reported that elements of the model are integrated into their institutional strategies: in KwaZulu-Natal we found that the Year 2 target of seven partners was missed, as only two partners reported integrating elements into their institutional strategies. In the Eastern Cape the target was exceeded as four partners reported integrating elements of the High Gear programme model, against a target of one. We also found that Year 2 targets for the number of instances in which industry partners disseminate the High Gear programme learnings was significantly exceeded, with learnings being shared 238 times across both KwaZulu-Natal and the Eastern Cape through a combination of forums, publications and social media. A key focus of these partnerships is the imperative of unlocking funding. This is being enabled through the High Gear programme by the provision of SETA Advisory Services, which seek to improve relations between employers and the merSETA with the intention of increasing the number of employer proposals that include partnerships with TVET Colleges submitted to the SETA. This support to unlock funding is intended to enable industry to advance inclusive workplace practices and policies. From the Results Framework (UKPF ONLY) The Year 2 target (which applies only to UKPF-funded portion of the High Gear programme) for the number of KwaZulu-Natal industry partners who have accessed additional funding to advance inclusive practices is 8. The target was missed: we found that two KwaZulu-Natal employers were supported through SETA Advisory Services to develop SETA grant applications in 2022. However, neither company had submitted their proposals for funding. One of the companies indicated that they had missed the submission deadline while another reported that they will consider submitting their proposals in 2023 as the primary focus was on staying afloat due to economic pressures. Therefore, the result was 0 for Year 2. In relation to the indicator on the number of KwaZulu-Natal employer proposals that focus on partnerships and that were submitted to the merSETA, this data was not available from the merSETA: they indicated that there has not been an assessment of whether there has been an improvement in the quality of proposals and the extent to which they have increased TVET partnerships. The Year 2 target for number of industry group partners that indicate that the High Gear programme has supported them to access increased skills development funding was missed in KwaZulu-Natal and in the Eastern Cape: each assisted one partner against a target of two. (Note that this refers to the finding that NACAAM has accessed funding for skills development which cuts across both provinces). In Year 1, SETA Advisory Services, facilitated by Engeli Enterprise Development, engaged with KwaZulu-Natal employers in a number of forums. However, in Year 2, due to the general economic climate and specific problems in KwaZulu-Natal including recovering from the destruction of the 2021 riots and the widespread flooding in that province in 2022, employers were reluctant to engage in any activities outside of their core business. The first SETA Advisory Services breakfast session for Year 2 was therefore held in February 2023 in KwaZulu-Natal. It was attended by 19 participants and facilitated by Engeli Enterprise Development and the 32 High Gear programme team from IYF. After this breakfast session, an invitation was extended to employers to access free SETA advisory services through the High Gear programme, and 11 companies in KwaZulu-Natal indicated their interest in taking up this opportunity. Ultimately eight companies took up this opportunity and further engaged with the High Gear programme team and will be receiving six hours of free SETA advisory service during the remainder of Q1 2023. Monitoring data collected by the High Gear programme MERL team at the end of the SETA advisory services breakfast indicates that 89% of the 19 employers present rated the workshop positively, 74% of the attendees reported that they will implement and share all the information in their organisations and 59% of the organisations indicated that they would like to receive further SETA advisory services in the categories of support in relation to SETA funding and proposal development as well as BBBEE gap analysis. Attendees also shared their views about what they found most valuable about this workshop indicating that “the information sharing on SETA funding and the BBBEE was excellent” and that they had received information on: “How to apply for grants. How to maximise BBBEE. Working with TVET Colleges to benefit us”. As noted previously, employers from KwaZulu-Natal report that the SETA supports them with funding and in guiding the process of implementing skills development programmes that are aligned to SETA targets and hard to fill vacancies, and employer respondents from the Eastern Cape also indicate that there has also been some progress in this regard and that they are accessing funding and are more involved in discussions about the allocation of funds. These findings suggest that merSETA is more open to working with TVET Colleges and this collaborative approach could result in an additional number of proposals that advance inclusive practices. However, as shown below there is limited data about the proposals and the extent to which the programme is contributing to improvements in quality or a greater focus on partnerships with TVET Colleges. 4.2 Curricula has been strengthened and aligned to industry needs The second objective has two aspects: the first part of this second objective within the High Gear programme Theory of Change is that TVET students have access to high-quality relevant training. With regards to the quality and relevance of training, as noted in the previous section (3.1) in 2021, IYF supported the Quality Council for Trades and Occupations (QCTO) with designing the curriculum frameworks for new occupational qualifications as well as two foundational modules. There was also training of lecturers, which took place within the QCTO process as well as directly through the High Gear programme. The work that was done to develop new curriculum frameworks included: a. IYF created a curriculum working group consisting of the QCTO Curriculum Director, Assistant to Curriculum Director, Electrical and electronics industry expert, Mechanical engineering industry expert and IYF’s Curriculum Manager. b. From August 2021 – June 2022, IYF recruited the services of the two industry consultants above who investigated a) the components of international and local engineering studies, b) skills and knowledge required and trends in industry, and then created a curriculum framework of a list of subjects, topics, equipment, budgets and practical tasks. c. The curriculum working group participated in two 3-day workshops where all members discussed and finalised drafts of the occupational curriculum frameworks which included international syllabus with recommendations, industry trends and skills, a list of subjects and topics, outcomes, purpose, practical tasks, assessment criteria and resources needed and IYF designed a detailed curriculum for foundational modules. The innovative approach that was generated through the High Gear programme was integrated into the QCTO’s redesign of these NATED qualifications. Additionally, IYF worked with the QCTO to assist their efforts to redesign the NATED Electronics, Electrical Engineering, and Mechanical Engineering qualifications. IYF explained at the time that they saw their role in supporting this curriculum working group as an expansion of 33 the High Gear programme’s original SOW, and the request that they assist was seen as a positive unintended consequence of the High Gear programme’s systems change strategy. In 2021 the curriculum working group held workshops to make subject matter additions to electronics and electrical curricula and these additions were coupled with training of the lecturers in the use of demonstration kits that integrated the new content. These updated topics have been taught to students since 2021 and N6 students wrote exams on the reviewed curriculum in the second trimester of 2022. As reflected previously, in the stakeholder engagements for the Year 2 formative evaluation, respondents indicated that they thought that the overall quality of the curricula is better, and the quality of provision is improving. According to DHET, what enabled the success in alignment of NATED curriculum content was the involvement of industry experts, curriculum support teams, and expert lecturers from colleges who work in both industry and colleges. From the Results Framework In terms of the number of TVET graduates who receive updated course content, the target for Year 2 were not reached at either of the colleges: at Elangeni TVET College the target is 1 344 and 648 students enrolled, and at Eastcape Midlands TVET College the target is 480 and only 296 students enrolled. In Year 2 the target for the percentage of graduates that indicate that the quality and resources of the programme is of high standard has been exceeded in KwaZulu-Natal: the target was 65% and 78% was achieved. For the Eastern Cape the Year 2 target was missed: the target was 90% and 71% was achieved. These is the average of respondents that agreed and completely agreed with three statements in the CATI relating to the quality and resources of the programme. While the targets for the number of TVET graduates who received updated course content were not reached, an analysis of the data suggests that students are increasingly positive about the quality and resources of the programme. As shown below there is a significant increase in student level levels of satisfaction. FIGURE 2: STUDENT VIEWS ON QUALITY OF RESOURCES & TEACHING Source 5: Student CATI, November 2022, Singizi 57% 92% 86% 49% 88% 75% The quality of technical equipment was excellent The lecturers were knowledgeable and supportive The quality of the learning materials (textbooks, other handouts etc was excellent) EMC Elangeni However, students indicated that they would prefer to get more practical opportunities within the colleges, •“So far, I am happy with the programme, though it would be great to get practical exposure.” •“I would like to see more practicals at the college from N2 going up.” •“It would be great if the programme would offer more in service training and provide the opportunity for us to work with big organisations to help in gaining practical experience, that would possibly also increase chances of employment.” 34 From the RF Year 2 targets were met for the number of TVET courses aligned to industry requirements – three in each Province. Subjects in the N4-N6 Electronics Engineering, Electrical Engineering and Mechanical Engineering were reviewed with insights from industry partners as well as curriculum specialists at TVET Colleges. Year 2 targets of % of TVET students who access workplace experience were not met in either province. The Year 2 target in KwaZulu-Natal was 25%, however only 5% of the students were placed. In the Eastern Cape the Year 2 target was 38% and 12% of the students were placed. As indicated previously, stakeholders were positive about developments with respect to curricula reform and recognise that this is important to align with the needs of industry. However, concern was expressed that rapid technological advances in industry mean that any curriculum is often quickly out of date, and that this needs to be considered when designing and reviewing TVET programmes. Emerging strongly from engagements with lecturers, students and employers is that increased access to work experience assists students to draw the linkages between the theoretical concepts and the needs of the workplace, and that work experience could also provide a mechanism to ensure that curricula keeps pace with technology. Stakeholders commenting on WIL stated that the timing of the placement did not suit industry as it was often in December when there was limited capacity to host students. It was explained that the college faces constraints in terms of timing and a college official respondent noted that the “class attendance timetable is restrictive and makes rotation between work exposure and learning period tricky” (College management, KwaZulu-Natal). The other major challenge faced was simply finding any WIL placements at all. An industry intermediary respondent said that they could not place the number of students that they had anticipated as they were dealing with the consequences of riots and flooding in KwaZulu-Natal, and employers were focused on “keeping afloat” and therefore could not accommodate requests to host students. This concern is seen in the results achieved with respect to the placement of students. An analysis of the IYF routine monitoring data on the High Gear programme shows that, in KwaZulu-Natal, 56 students were placed with two employers, and in the Eastern Cape, 54 students were placed with four employers. The process was facilitated by IYF, NAACAM (in KwaZulu-Natal) and AIDC (in the Eastern Cape), and SkillSonics. When looking at these numbers as a percentage of the total students enrolled, we noted a decrease in the number of students that have accessed WIL since Year 1 from 10% to 5% in KwaZulu-Natal and a decrease from 13% to 12% in the Eastern Cape. It is noted that while these figures are based on the CATI results, which we have used in previous evaluations, there is a significant difference with the IYF routine monitoring data, which shows that 22% (n=56) individuals accessed workplace experience in KwaZulu-Natal and 31% (n=54) in the Eastern Cape. They also indicated that there is a need for more equipment in the colleges to support these practical activities, •“They need to get more equipment and laboratories for practicals.” •“The overall experience has been okay, but it would be better if they could also provide workshops or expose us to the workplace for practical experience.” •“It seems the college is lacking in resources that would help us gain practical experience, that is something they should try to work on improving.” 35 This internal data could suggest a fairly large increase in access to WIL which is a very positive finding. It is unclear if this means that graduates are entering a more substantive work placement after leaving the college, as compared to the 5 days of WIL during the programme. It is noted that in KwaZulu-Natal students could not be placed in the workplace due to problems with the application of the Compensation of Occupational Injuries and Diseases Act (COIDA). As a result, the focus of the placements turned to graduates that had exited the college. Other explanations for this improvement will be important to explore including the role that the High Gear programme is playing with a view to possibility expanding this support. The possibility of expanding access to placements ‘post’ the programme would be very valuable to explore as it could also enable these graduates to attain their Diplomas and is seen as a national priority (it was announced as a key imperative as part of the President’s State of the Nation Address (SONA). We note that two key indicators on the quality and relevance of the training are that graduates access higher education following participation in the High Gear programme, and an improved completion rate. As neither of the colleges have released 2022 performance data to the High Gear programme yet, we are unable to comment on these indicators. From the Results Framework In Year 2 the targets for number of technical assistance interventions was one for KwaZulu-Natal and six for the Eastern Cape, and this target was exceeded in both provinces. In KwaZulu-Natal, 10 technical assistance interventions were delivered and in the Eastern Cape eight interventions were delivered. These included WIL placements for students and lecturers, career guidance services, lecture capacity development as well as SETA advisory services (only applicable in KwaZulu-Natal). The Year 2 targets for the number of individuals affiliated with higher education (tertiary) institutions receiving capacity development support with USG {or UKPF] assistance were met in KwaZulu-Natal (the target was 28 and all 28 lecturers participated in capacity development training), and in the Eastern Cape 16 lecturers received training just missing the target of 19. With regards to the mentorship component of lecturer development, lecturers shared that for Year 2 they were not completely satisfied with how the mentors were selected as well as how the intervention was implemented. Lecturers from Elangeni TVET College explained their concern stating that the mentors were not relevant to the engineering field in Year 2. They stated that they had expected to be assigned industry experts to them (which was the case in Year 1) but in 2022 lecturers reported that the mentors did not add any value and had no understanding of engineering concepts. Lecturer respondents indicated that they would have preferred the same mentors they had worked with in 2021. In Eastcape Midlands TVET College, fewer respondents reported to have worked with a mentor in 2022. Responses from those lecturers who had received mentorship also reflected some concerns, with one lecturer stating that they felt like that they were not being heard and another expressing concern about the timing of these processes. There were however a number of other initiatives that focused on lecturer development. As part of the High Gear programme Theory of Change, it is anticipated that lecturers should also benefit from workplace exposure, in order to enhance the quality of the provision of the curricula. In KwaZulu-Natal, 12 lecturers were placed with one employer, and in the Eastern Cape, 18 lecturers were placed with four employers. The process was facilitated by IYF, NAACAM (in KwaZulu-Natal) and AIDC (in the Eastern Cape), and by SkillSonics. Responses suggested that this process is helping lecturers to identify gaps in what they teach in comparison to what is required in the industry cycle. For example, in KwaZulu-Natal a college management official shared that now that the staff members have workplace exposure, they can see that they might teach only 40% of what is happening in a 100% cycle in industry. 36 The CATI results showed that the 12 lecturers who had attended the workplace exposure either agreed or completely agreed that the knowledge and skills gained through workplace exposure was relevant to the course content of the subjects that they teach. The workplace exposure has enabled them to improve their ability to link teaching with industry requirements/standard and that the workplace exposure improved the quality of their practical training. Another lecturer commented that it was interesting how much had changed since they had been in industry, and that the workplace exposure had helped them to see some of these new trends in practice: “It’s been 8 years out of companies [so] it has been eye opening to me …there is so much change from my time, like renewable energy, [that I only know] from the textbook”. (Lecturer, Eastcape Midlands TVET College) Employers also saw the value of these lecturer placements, as observed by one respondent: “Industry and other role players are becoming aware of all the problems and are now willing to put in support – there is more of an openness of lecturers to be placed and the ones we have had they are more excited to be placed as they have been exposed to what is happening in industry.” (WIL placement implementer in the Eastern Cape) Another employer indicated that they thought that this had been a positive experience for both parties and suggested that lecturers should gain expertise in the application of technology, noting that it had assisted when they were exposed to the current technology being used at the firms. There were however some employers that indicated that they felt that the duration of the placements was too short, indicating that many lecturers have no practical training and so there is a need for the placements to be longer. Other lecturer development interventions that were supported by IYF through the High Gear programme in 2022, together with the number of lecturers who attended each session, are shown below: TABLE 4: LECTURER ATTENDANCE OF THE HIGH GEAR PROGRAMME LECTURER DEVELOPMENT INTERVENTIONS Elangeni Eastcape Midlands Effective Learning and Teaching Sessions: focused on sharing methodologies to improve knowledge transfer in the classroom. 15 16 Yakh’iFuture Platform Demo sessions: focused on upskilling the lecturers on use of gamification function, demokit assembly and class integration of demokit aids. 10 8 Demokit refresher: The focus of the capacity development sessions facilitated was to upskill the lecturers on demokit assembly and class integration of demokit aids. 12 7 The High Gear programme Working Groups Session: focused on subject matter additions that could support lecturers in facilitating classroom facilitation. 7 4 Source: IYF MERL Data Additionally, all lecturers strongly agreed that the workplace exposure was well structured and managed and that the employees within companies were knowledgeable and helpful. These positive perspectives are highlighted in the following quotes: •“It really has shown me that there is more to what we teach in class and gave me confidence in my teaching”. (Lecturer, Elangeni TVET College) •“I was able to link my subjects with industry requirements and able to give relevant examples of the module”. (Lecturer, Eastcape Midlands TVET College) •“There are a lot of new machines which I was not familiar with but are used in industry and can help with teaching students”. (Lecturer, Eastcape Midlands TVET College) 37 As indicated above, the Yakh’iFuture platform is also a key strategy to support lecturer capacitation and development. In the interviews for this evaluation, a NAACAM respondent said that Yakh’iFuture, with its digital content and teaching aids, was launched after realising that upgrading curriculum cannot be done in isolation and this intervention is intended to provide additional capacity to lecturers in delivering content in a way that is impactful. The mobile demonstration kits are an additional element of this approach. In the lecturer CATIs, lecturers from both colleges rated these capacity development interventions as shown below. Source: Lecturer CATI, November 2022, Singizi The figure above shows that the vast majority of respondents in the CATI either completely agreed or agreed that the capacity development interventions have assisted although responses were least positive about the extent to which the interventions have assisted lecturers to improve their understanding of the course content. During the lecturer focus groups, TVET lecturers said there were timing pressures as the trimesters are very short and there is limited time to teach all the relevant chapters in between tests and exams within the 3- month cycle. However, of concern is that lecturers across both colleges report that there is a disjuncture between lecturer expectations of the High Gear programme and what has been delivered through the capacity development 33% 24% 29% 24% 33% 38% 29% 76% 62% 71% 62% 62% 29% 10% 5% 5% 10% The capacity development intervention/ curriculum training assisted me to improve my understanding of the course content The capacity development interventions/training gave me tools to improve my teaching techniques/methods The capacity development interventions/training gave me tools to improve my assessment techniques/methods The capacity development interventions/training has assisted me to understand industry requirements and standards The capacity development interventions/training has assisted me to better prepare learners to meet these industry requirements and standards The capacity development interventions/training has assisted me to better facilitate the acquisition of ‘soft’ skills that learners need for both their studies and for … 5 - Completely agree 4 - Agree 3 - Neither agree nor disagree 2- Disagree 1 - Completely disagree Lecturers elaborated on how the capacity development interventions had assisted them and their teaching techniques indicating that, •“It has confirmed a lot of things and improved my teaching approach. I have learned to be more patient and accommodating with students”. (Lecturer, Elangeni TVET College) •“They taught us how to use the demo kits. Also, how to be detailed with the lesson plans, going step by step with students. How to engage them and make the classroom a safe learning environment”. (Lecturer, Eastcape Midlands TVET College) FIGURE 3: LECTURER VIEWS ON CAPACITY DEVELOPMENT INTERVENTIONS 38 interventions. Lecturers indicated that the training was too generic and focused on teaching methods and while they valued this training, they would have liked for the capacity development interventions to include engineering concepts. They also indicated that there would be value in bringing in technical experts from industry to facilitate these sessions and engage them on concepts in the curriculum that are updated and to discuss specific subject matters where they need more in depth understanding (TVET lecturers, KwaZulu-Natal and the Eastern Cape). In addition, lecturers from KwaZulu-Natal and the Eastern Cape reported that the use of mobile demonstration kits and gamification platforms in class was limited. The main reason highlighted was time constraints, as lecturers explained that there was a fixed amount of time for lessons and using the demokits required additional time to input formulas, sometimes taking longer to provide the solution. In some cases, the mentor was required to assist if the solution provided by the demokit was inconsistent with the textbook solution. Another challenge was some mechanical concepts were not included the content of the mobile demonstration kits and gamification functions on Yakh’iFuture. Lecturers further reported that in some instances they are struggling to integrate these platforms into their lesson plans or to use these platforms. Some suggested that it would be useful to have an industry mentor to assist them in using the mobile demonstration kits. While acknowledging these concerns amongst lecturers as to whether the quality of their teaching is improving it seems worth reiterating the findings presented in Figure 2 which showed that student respondents were particularly satisfied with the quality of their lecturers. Never-the-less this is an area that appears to require greater focus. One aspect of this could be linking lecturers with other lecturer development interventions that could enhance lecturer support. These included the partnership with colleges that is being facilitated by merSETA, which involves the training of lecturers, and the provision of bursaries, to support the professionalisation of lecturers to use digital learning pathways (through an app). In addition, a DHET respondent shared that they also offer lecturer development opportunities. 4.3 Young people have received guidance to better employment outcomes The second part of this second objective in the High Gear programme Theory of Change speaks to improved employment opportunities for the young people in the programme. It is a core part of the High Gear programme programmes hypothesis that, aside from high quality and relevant training, young people also need skills and tools to access employment, and guidance around how to look for work and what employment opportunities are available. Unsurprisingly the majority of graduate respondents said that they enrolled in the NATED programme because they thought that it would help them gain the technical and soft skills that they needed for employment (60% from Elangeni TVET College and 54% from Eastcape Midlands TVET College), or because they thought it would help them get a job and/or workplace experience (49% of the respondents from Elangeni TVET College and 48% from Eastcape Midlands TVET College)7 . Most respondents said that they thought that their expectations were almost or completely met: 59% of respondents from Elangeni TVET College and 67% from Eastcape Midlands TVET College. In terms of the factors which lead to increased employment, we also asked if respondents thought that the qualification provided them with the skills and tools to access employment. We found a slight increase in the number of those that agreed and strongly agreed with statements related to this in respondents from both Elangeni TVET College and Eastcape Midlands TVET College. However, when asked if they thought that they had improved soft skills as a result of being on the programme, there was a very small decrease in the percentage of Elangeni TVET College respondents who selected agree or strongly agree, and a slight increase from the Eastcape Midlands TVET College respondents. This is shown below: 7 Respondents could select more than one answer 39 From the Results Framework When students were asked if they thought that the qualification provided them with the tools and skills to access employment, we found that there was a slight increase in both provinces in the percentage of respondents who selected strongly agree and agree, from 69% in Year 1 to 77% in Year 2 in KwaZulu-Natal, and from 65% in Year 1 to 68% in Year 2 in the Eastern Cape. The Year 2 target of 90% was missed in KwaZulu-Natal as 77% was achieved and in the Eastern Cape the Year 2 target of 36% was exceeded as 68% was achieved. With regards to improved soft skills, the measure is the % of students surveyed who indicate that they believe that they have improved soft skills as a result of being in the programme. It is a concern that, in both provinces there has been a decrease on the percentage of respondents who believed that participating in the course had improved their soft skills a fair amount or an enormous amount. In KwaZulu-Natal, this decline has been steady, from 93% in the baseline, to 91% in Year 1, to 86% in Year 2. In the Eastern Cape there was a decrease from 94% in the baseline to Year 1 at 85% and a slight increase from Year 1 to Year 2 to 89%. The Year 2 target was 95% in KwaZulu-Natal and 86% was achieved. The Year 2 target was 95% in the Eastern Cape and 89% was achieved. As indicated above, students in the CATIs and Focus Groups generally commented that they would like greater access to practical opportunities as well as WIL opportunities. They observed that this would strengthen their skills and tools to access employment and to an extent strengthen their soft skills to cope in the work environment. Employers suggest that to compensate for this lack of experience it would be useful for graduates to pathway into production roles in industry through participating in programmes such as learnerships: “We do not employ TVET graduates but we do take TVET learners into our Learnership programme and that’s for production qualifications at the end of the learnerships. It will typically be an engineering student, and we will take them in a production Learnership. Then the internships are with University graduates and for qualifications such as Engineering, Finance, Logistics, Supply chain and Quality” (Employer, KwaZulu-Natal). Another key method of getting into the employment market is obviously through connections with employers, so it is of concern that only 45% of Elangeni TVET College respondents and 42% of Eastcape Midlands College respondents completely agreed and agreed that participating in the course helped them to improve their connections with employers as shown in Figure 4 below: Source: Student CATI, November 2022, Singizi 14% 19% 31% 23% 28% 29% 16% 19% 7% 8% 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% Elangeni EMC 5 -completely agree 4 - Agree 3 - Neither agree or disagree 2 - Disagree 1 - completely Disagree FIGURE 4: PARTICIPATING IN THE PROGRAMME HELPED ME TO INCREASE MY CONNECTIONS WITH EMPLOYERS 40 In addition to support in accessing networks and making connections with employers, students also received support other areas: 28 students from Elangeni TVET College received career guidance in the form of CV optimisation workshops. These workshops also took place at Coastal College in early 2023 and will take place at Eastcape Midlands TVET College during 2023. Other forms of guidance included guest lecturing by industry experts across both colleges and delivered by three employers in KwaZulu-Natal and three employers in the Eastern Cape. Topics covered included: How to apply for a job and how to prepare and succeed in interviews, gender and diversity in the workplace, learnerships and apprenticeships, TVET Graduates and the world of work, production and quality control, what to expect in the workplace. 83% (n=5) of the employers indicated that they would like to be invited back to guest lecture. According to IYF internal monitoring data, 33% of the industry guest lecturers rated students’ participation (i.e., asking questions and answering questions during the lectures) as excellent, and 33% rated it as fair. It was suggested that students should be fully briefed before the sessions in order to pique their interest. Again, using IYF internal monitoring data, it was found that 184 students were reached across Elangeni TVET College and Eastcape Midlands TVET College during these sessions. They provided feedback on what they had learned during these sessions and commented that they had acquired knowledge in a variety of key topics. Additionally, during focus groups, students commented that they found these sessions very useful and helped them understand career paths in the engineering fields. These industry guest lecturing sessions are complemented by the Yakh‘iFuture platform, which has been designed for TVET college engineering students to help them explore careers in the automotive manufacturing sector, to practice with tools to help students with their studies as well as find learnerships, apprenticeship and job opportunities. This platform was launched in 2022 and while there are some concerns about its capacity, as mentioned previously, stakeholders are very positive about the way in which it was developed and the extent to which it provides a powerful tool for providing career guidance: “The Yakh’iFuture platform was designed, developed, tested, and launched by NAACAM in collaboration with the IYF. We spent a lot of time with our members … through focus groups with employers and students to inform the platform design and content. We also filmed the career guidance videos … with employers. We are currently engaging with employers to maintain and extend the platform; … and encouraging our members to upload entry level jobs on the platform …” (Industry Intermediary). Stakeholder respondents were particularly enthusiastic about the role of the Yakh’iFuture platform to support guidance: “I was offered the opportunity to attend the launch of the Yakh’iFuture … we had just started with our first cohort and I then spoke to our maintenance manager who was running the programme to say please show the students this Yakh’iFuture platform; there are games that they can use to enhance their learning, there’s a jobs board, there are various careers that they can look at to see what they would like to specialise in and interact with that a little bit” (Employer, KwaZulu-Natal). “I attended the launch of Yakh’iFuture, and we think it’s a good … career guidance platform” (merSETA). Some responses from the Elangeni TVET College students interviewed highlighted some of the lessons learnt: •“That every CV must have a professional profile” ​ •“Always customise CV and cover page​” •“How to prepare for an interview​” •“How to apply online​” •“How to behave in workplaces​” 41 Since the launch of the Yakh’iFuture platform, efforts have also focused on expanding TVET student awareness and engagement. The High Gear programme has attempted to increase youth reach by developing marketing materials and partnering with college career development centres across both colleges to place posters and pamphlets that provide information about Yakh’iFuture, including how to join the platform. The national reach of Yakh’iFuture was 3 125 in the second quarter of 2022. This data is collected through tracking IP addresses that go onto the platform: using the national data and provincial distribution of IP addresses it was found that 444 users come from KwaZulu-Natal while and 137 users come from the Eastern Cape, with the balance being from other provinces or unknown. It is noted that we cannot know whether all of these ‘contacts’ were unique users, nor can it be confirmed whether these young people are from either of the targeted colleges. There is currently a mechanism to review ways to access more detailed user demographics through a data sharing MoU, which is being reviewed between the High Gear programme and Harambee. While the efforts to create awareness and encourage usage of the platform are highlighted, it is important to note that there is a need for further information sharing, as students in the focus groups were not very familiar with the platform. Those that were, indicated that they struggled to access the platform due to their limited access to data and the lack of WIFI at the colleges (the platform is not zero rated). Ultimately, these different interventions are all intended to support the achievement of the key indicator for the High Gear programme, which is the percentage of individuals who access new employment, 12 months ex-post. Because of the nature of the programme, it is difficult to delineate a pure ’12-month ex-post’ group, as participants sometimes return to the colleges for further courses etc. The results presented are from the tracer study done in Dec 2022/Jan 2023, 12 months after the first the High Gear programme cohorts completed their studies. In the tracer study, we sent a short message to all graduates for whom we had contact details, asking about their current economic status. 14% of those that responded from KwaZulu-Natal said that they were in full- or part- time employment (against a target of 26%) and 16% of those that responded in the Eastern Cape said that they were in full- or part-time employment, against a target of 24%. 42 43 5 IMPLICATIONS FOR FUTURE HIGH GEAR PROGRAMMING 5.1 The challenging economic environment The High Gear programme is being implemented in a very difficult economic context and this creates further pressure on the ability of graduates to access and retain work opportunities. These pressures include the impact of Covid-19 combined with the impact of unrest, the floods in KwaZulu-Natal as well as the national energy crisis that have all caused extended pressure on employers. This pressure has meant that many employers are unable to focus on training and recruitment of TVET graduates as their immediate concerns relate to retaining current stuff and resuming and continuing with production. In this context skills development and training interventions were seen as a secondary priority during 2022. This appears to partly explain why employers have not taken up the skills development funding proposal development and submission support through SETA Advisory Services, which are being offered in KwaZulu-Natal. A review of DHET reports highlight that such a low level of economic activity adversely impacts on the colleges’ ability to provide locally relevant work experience. This suggests that there is a need to put in place mechanisms for students to travel to workplaces which can provide opportunities to gain practical skills. This issue is likely to become even more important as the occupational qualifications that have been developed by the QCTO require a workplace component. This means that the capacity of a college to deliver such qualifications will be determined by the availability of such work in the areas surrounding the colleges. The one way that this is being addressed is through graduates finding work opportunities in the form of learnerships and internships once they have completed the NATED programme. These opportunities are short term and do not typically result in long term employment. This is a real concern given the negative changes in the environment and the related reduction in demand for graduates with engineering qualifications across all provinces and NQF levels. Within this context we suggest that the High Gear programme places particular attention on ensuring that there is a sustained focus on partnerships between the colleges and industry, that there is an increase focus on niche areas within the industry and also that ways to ensure that young people have greater access to the Yakh’iFuture platform and there is greater levels of availability of information and resources for young people to apply for jobs. Additionally, a review of the kinds of opportunities that young people are able to access through initiatives such as SAYouth.mobi will be important to ensuring that young people are assisted to access different types of opportunities. 5.2 Repositioning the High Gear programme IYF explains that to improve the ability of the High Gear programme to place young people, the scope of the High Gear programme will be expanded in 2023. Going forward, the High Gear programme aims to broaden its scope to beyond the auto industry with a strong focus on renewable energy. These changes will inform the nature of the partnerships between colleges and industry going forward and will require a focus on the development of technology, engineering and mathematical skills, the facilitation of more WIL opportunities and have more impact on young people within the TVET system. 5.3 Managing expectations We found that lecturers across KwaZulu-Natal and the Eastern Cape have different expectations of the High Gear programme and that some of these expectations cannot be addressed within the scope of the High Gear programme. While lecturers were positive about aspects of the capacity development that they received, they indicated that their expectations regarding curriculum support focusing on engineering concepts and technical content were not fully addressed. 44 Interestingly, while lecturers were not certain that they had improved their abilities to offer the technical aspect of the curriculum we saw that students offered very positive feedback about the lecturers. The High Gear programme plans to continue to focus on the provision of effective teaching techniques to lecturers, and we are therefore suggesting that IYF considers how best to work with initiatives that are offering such technical training to ensure that lecturers receive comprehensive support. 5.4 Intentional focus on soft skills interventions This report has found that the percentage of graduate respondents that indicate that their soft skills have improved has declined. We also found that during this period Passport to Success was not delivered to students and instead the content of these programmes was moved to the Yakh’iFuture platform, in part because of the challenges in finding time in the curricula to offer this training. However, as indicated the platform is not being fully utilised and there is a need to ensure the integration of Passport to Success within the NATED programmes, while seeking ways to expand student access to the Yakh’iFuture platform. These suggestions are in line with the existing plan of the IYF to place more emphasis on the provision of life skills for young people and the DHET mandate to ensure the inclusion of life skills training in the NATED programmes across the TVET system by the end of 2023. 45 46 6 CONCLUSION In conclusion we found that, in Year 2, the High Gear programme in KwaZulu-Natal and the Eastern Cape built on the foundation that was laid in the first year of operation. This allowed the High Gear programme to achieve concrete results in terms of supporting the skills development system to become more responsive to industry needs. In relation to partnerships, we found that the High Gear programme has worked effectively with industry associations to facilitate greater and more effective employer involvement in curriculum review, and to enable greater collaboration between employers and colleges. However, there are some concerns about whether colleges can take forward the effective provision of a revised curricula that continues to be responsive to change, or about whether these partnerships have resulted in sufficient opportunities for students. Respondents from employers, from industry associations and from TVET Colleges all confirmed that they have been able to establish what they consider to be mutually beneficial partnerships and suggest that the High Gear programme has both contributed to the development of these partnerships and that these partnerships will in turn contribute to the overall goal of the High Gear programme. Turning to the curriculum, we found that there have been valuable revisions to the qualifications. This has also allowed for the development of curriculum frameworks that are better aligned to the needs of industry. Stakeholders, including colleges indicate they are more aware of industry needs, but state the process of formally updating curricula is long and complex, which does not accommodate the fast pace of technology change. Different ways of thinking about ensuring ‘relevance’ therefore need to be considered, including practical training for students and workplace exposure for students as well as lecturers. With respect to lecturers, it was found that lecturers are not feeling very confident about their ability to deliver quality learning within the new frameworks, although students suggest that their lecturers are helpful and skilled. And in respect of learning and employment outcomes, there is evidence of improvements in learning, with learner and lecturer respondents noting that the quality of the course was good. The tracer study also showed an increase in the number of students who state that they received guidance in their courses. However, there is a clear need to increase the practical learning and workplace exposure components of the courses. In relation to increased employment, the numbers of graduates who indicated that they are employed in the tracer studies is still low, and most employed young people have contract work or are on learnership agreements. The percentage of employed graduates increased between the two tracer studies, from 4% to 15% of respondents, which is important given the challenging economic context and the depressed labour market. 47 ANNEXURE 1: VfM REPORTING FRAMEWORK Economy Indicator 1 Cost Containment of Expenditures Parameters Q1 2022 Q2 2022 Q3 2022 Q4 2022 Total 2022 % Quarterly burn rate against quarter target 56% 72% 95% 108% 83% Summary of achievement for the indicator and other additional notes High Gear's burn rate for Q4 saw an increase from 95% to 108% exceeding the target of 95% by 13%. The quarterly forecasted amount was GBP288, 486,28 with actuals coming in at GBP31, 166,61. IYF will continue to work to work at achieving the quarterly burn rate target of 95% through to project close out. Economy Indicator 2 Promotion of VfM through competitive procurement of inputs Parameters Q1 Q2 Q3 Q4 Total No. of contracts issued 5 2 1 7 15 No. of contract competitively bid 4 1 0 4 9 % of contracts competitively bid 80% 50% 0% 57% 60% Summary of achievement for the indicator and other additional notes, for examples the areas where cost savings were made and whether or not the savings was re￾invested, and how? IYF competitively bids all High Gear contracts, with the exception of 1) technical service providers pre-identified during the proposal stage and included in the donor-approved proposal (Singizi, Loud Rabbits and uYilo); and 2) sole service providers in the market (Business Management Concepts). All vendors in Q1 for the exception of Meraki and Career Management Consulting, were contractors used in 2021. Meraki and Career Management Consulting, new vendors in 2022, were selected through a competitive bid process. BMC was a sole service provider. Q2 contractors were two vendors used in 2021. All took part in a competitive bid process except for Singizi who was included in the donor-approved proposal. In Q3 AIDC was issued a second sub-award. This was not a competitive process as this was a continuation of the work AIDC supported for their first subaward. In Q4, 7 contracts were issued. 2 were new competitive bids- KatieM Communications and Ngawethu Consulting (EV work). Loud Rabbits was previously sourced through a competitive bid process for the Yakh'iFuture platform and were contracted within the quarter to provide support to changing the Homepage to reduce the drop off rate. Loud Rabbits was further contracted for the EV work through an exempt from bid, as they were listed as a vendor in the proposal. uYilo was also contracted through an exempt for bid, as they were included in the EV proposal. Career Management Consulting received a further contract to provide additional support building on the work they provided in Q2 (previous competitive bid). Mindful Skills was contracted through an exempt for bid as they are currently providing Skills 4 Life with Effective Teaching and Project Based Learning Workshops. These workshops are being mirrored within High Gear in Q4 and Q1 2023. 48 Effectiveness Indicator 1 Benchmarking of youth beneficiary costs with other similar initiatives in SA's TVET system VFM dimension and Output (If applicable) Efficiency (Pillar 3 and Pillar 4) Parameters Q1 2022 Q2 2022 Q3 2022 Q4 2022 Total 2022 Direct project costs (cumulative) £104,717 £ 297,289 £ 452,147 £ 777,031 £ 777,031 Number of youth reached (cumulative) 437 3628 3684 7749 Efficiency Indicator 1 Unit cost per TVET lecturer, per year, capacitated with project-based learning upgrades VFM dimension and Output (If applicable) Efficiency (Pillar 3 and Pillar 4) Parameters Q1 2022 Q2 2022 Q3 2022 Q4 2022 Total 2022 Direct project-based learning delivery costs £12,168 £20,748 £16,897 £22,329 £72,142 Number of lecturers reached during reporting period 26 26 26 26 26 Unit cost per TVET lecturer, per year, capacitated with project-based learning upgrades £468 £798 £650 £859 £2,775 Summary of achievement for the indicator and other additional notes. For example, factors responsible for the budget execution rates The same number of lecturers are reached every quarter therefore reporting is non-cumulative. Efficiency Indicator 2 Direct Yakh'iFuture platform cost divided by estimated number of youth reached through the Yakh'iFuture career portal. VFM dimension and Output (If applicable) Efficiency (Pillar 3 and Pillar 4) Parameters Q1 2022 Q2 2022 Q3 2022 Q4 2022 Total 2022 Direct project costs £47,005 £81,400 £5,700 £10,363 £144,468 Number of youth reached during reporting period (cumulative)through the Yakh'iFuture career portal 735 2417 3152 6304 Unit cost per youth, per year, capacitated through the Yakh'iFuture career portal £111 £2 £3 £23 Summary of achievement for the indicator and other additional notes. For example, factors responsible for the budget execution rates With the number of youth reached significantly increasing the unit cost was negative. The project costs for Q4 include the EV component of work done on the Yakh'iFuture platform. 49 Total Cost per Young Person Reached £680 £125 £211 £100 Summary of achievement for the indicator and other additional notes. Yakh'iFuture will continue to grow as we add more content on the website and implement strategies to promote the platform with our partners (NACAAM and Harambee). Effectiveness Indicator 2 Improved use of state and/or student resources, through improved TVET completion rates (i.e., cost of TVET tuition and/or bursary more effectively used) VFM dimension and Output (If applicable) Effectiveness (Pillar 3) Parameters Q1 Q2 Q3 Q4 Total Number of students with improved completion rates in High Gear assisted courses (based on % of improved completion rates in High Gear assisted courses) 199 POPI Delaying results Unknown Average % of TVET students on NSFAS bursary (51% in 2019) & average value of each NSFAS bursary (R14,700 per TVET student) 85% TBD 85% Value of improved use of state resources (NSFAS bursary) TBD Summary of achievement for the indicator and other additional notes. TBD Effectiveness Indicator 3 GBP (cash/or in-kind resources) leveraged by High Gear partners for activity implementation VFM dimension and Output (If applicable) Effectiveness - cuts across all project pillars Parameters Q1 Q2 Q3 Q4 Total GBP (cash/or in-kind resources) leveraged by High Gear partners for activity implementation £0.00 £0.00 £4,400.00 £0.00 £4,400.00 Summary of achievement for the indicator and other additional notes. For example, factors responsible for the budget execution rates NAACAM ran a Bi-Annual Demokit Donation Campaign within the Quarter and received funds from two companies- Umicore and Adient in Q3. These funds were to be returned to the two companies due to NAACAM being unable to contract and procure a vendor to acquire the additions to the demokits in a short timeframe. 50 Equity Indicator 1 % of High Gear inputs sourced from businesses owned by individuals from historically disadvantaged backgrounds and/or consultants from historically disadvantaged backgrounds (per SA government definition – disaggregate by gender and HDI status) VFM dimension and Output (If applicable) Equity (cuts across all project pillars) Parameters Q1 Q2 Q3 Q4 Total Number of contracts awarded 5 2 1 7 15 Number of contractors who are from historically disadvantaged backgrounds and/or firms with B-BBEE Level 1 or 2. 1 1 1 3 6 % of High Gear inputs sourced from businesses owned by individuals from historically disadvantaged backgrounds and/or consultants from historically disadvantaged background 20% 50% 100% 43% 40% Summary of achievement for the indicator and other additional notes. Within Q4, KatieM, Ngawethu and Mindful Skills are the firms that relate to the historically disadvantaged backgrounds and/or firms with B-BBEE Level 1 or 2. Equity Indicator 2 % of High Gear youth beneficiaries from historically disadvantaged backgrounds (per SA government definition) VFM dimension and Output (If applicable) Equity (cuts across all project pillars) Parameters Q1 Q2 Q3 Q4 Total Number of High Gear youth beneficiaries (cumulative) 437 476 476 1389 Number of High Gear beneficiaries from historically disadvantaged backgrounds (cumulative) 437 476 476 1389 % of High Gear youth beneficiaries from historically disadvantaged backgrounds (per SA government definition) 100% 100% 100% 100% Summary of achievement for the indicator and other additional notes. For example, factors responsible for the budget execution rates We are noticing an increase in student enrolment, following the decline in 2020 and 2021 caused by covid-19 disruptions. To date, we have reached 913 (46%) of the targeted students (2000), which we are projecting to increase to 1363 (68%) during trimester 1 enrolment for 2023 (quarter 1). 51 ANNEXURE 2: FULL RESULTS FRAMEWORK INDICATORS, BASELINE, YEAR 1 AND YEAR 2 RESULTS We have included a preliminary evaluative comment on the achievement of programme targets in KwaZulu-Natal, given that the programme is ending there. However, we note that the disconnect between some Year 2 targets and the endline targets will need to be addressed. Objective 1: Sustained automotive manufacturing industry involvement in aligning public TVET courses with employer needs, in a manner that is consistent with the High Gear model. Indicator # of effective and innovative the High Gear programme model practices and/or policies that are adopted by industry partners Unit of Measure Number of policies and or practices adopted by industry partners USAID/MSDF Eastern Cape UKPF KwaZulu-Natal Baseline E/Line Target Year 1 Target Year 1 Results Year 2 Target Year 2 Results Baseline E/Line Target Year 1 Target Year 1 Results Year 2 Target Year 2 Results 0 3 2 3 1 3 0 4 2 2 2 3 The High Gear programme in KwaZulu-Natal has exceeded this target, with a total of 5 effective and innovative the High Gear model practices and/or policies adopted by industry partners, against a programme target of 4. In the Eastern Cape, the endline target has also already been exceeded. Intermediate Result (IR) 1.1: Increased industry-wide coordination to align TVET courses with industry needs. Indicator # of TVET courses aligned to automotive component manufacturing industry needs (to be defined at qualification level and subject level) Unit of Measure Number of TVET courses USAID/MSDF Eastern Cape UKPF KwaZulu-Natal Baseline E/Line Target Year 1 Target Year 1 Results Year 2 Target Year 2 Results Baseline E/Line Target Year 1 Target Year 1 Results Year 2 Target Year 2 Results 0 3 2 2 2 3 0 3 2 2 1 3 Note this refers to three qualifications (Electronics engineering, electrical engineering and Mechanical engineering) across both colleges. 52 The High Gear programme in KwaZulu-Natal has exceeded this target, with a total of 5 TVET courses aligned to automotive component manufacturing industry needs, against a programme target of 3. In the Eastern Cape, the endline target has also already been exceeded. Indicator # of targeted technical assistance interventions delivered [Technical Assistance] Unit of Measure Number of technical assistance activities USAID/MSDF Eastern Cape UKPF KwaZulu-Natal Baseline E/Line Target Year 1 Target Year 1 Results Year 2 Target Year 2 Results Baseline E/Line Target Year 1 Target Year 1 Results Year 2 Target Year 2 Results 0 6 3 5 6 8 0 6 3 6 1 10 The High Gear programme in KwaZulu-Natal has exceeded this target, with a total of 16 targeted technical assistance interventions delivered, against a programme target of 6. In the Eastern Cape, the endline target has also already been exceeded. Indicator # Of beneficiaries (government staff, institution staff, teachers, trainers, students, private sector organisation staff and/or NGO staff) receiving training, coaching or capacity building support, including strategy, regulatory and policy work [Capacity Building] Unit of Measure Number of people USAID/MSDF Eastern Cape UKPF KwaZulu-Natal Baseline E/Line Target Year 1 Target Year 1 Results Year 2 Target Year 2 Results Baseline E/Line Target Year 1 Target Year 1 Results Year 2 Target Year 2 Results 0 1619 1000 306 498 387 0 2031 1150 410 1263 968 The High Gear programme in KwaZulu-Natal has missed this target, reaching a total of 1 378 beneficiaries, against a programme target of 2 031. In the Eastern Cape, the programme has reached a total of 693 beneficiaries. It will need to reach 926 beneficiaries in 2023 if it is to reach target. 53 Indicator # Of beneficiaries (government staff, institution staff, teachers, trainers, students, private sector organisations, NGOs) meaningfully engaged in partnerships, engagement and convening activities [Partnerships, Engagement & Convening] Unit of Measure Number of people USAID/MSDF Eastern Cape UKPF KwaZulu-Natal Baseline E/Line Target Year 1 Target Year 1 Results Year 2 Target Year 2 Results Baseline E/Line Target Year 1 Target Year 1 Results Year 2 Target Year 2 Results 0 142 46 29 47 90 0 144 60 116 14 56 The High Gear programme in KwaZulu-Natal has exceeded this target, reaching a total of 172 beneficiaries, against a programme target of 144. In the Eastern Cape, the programme has reached a total of 119 beneficiaries. It will need to reach 23 beneficiaries in 2023 if it is to reach target. Short-term Result 1.1.1: Industry partners incorporate TVET partnership objectives into their strategic plans. Indicator # of industry partners (NAACAM, industry intermediaries and employers) that report elements of the High Gear programme model are integrated into their institutional strategies. Unit of Measure Number of industry partners that report the High Gear programme learning, relating to TVET and approaches integrated into their institutional strategies USAID/MSDF Eastern Cape UKPF KwaZulu-Natal Baseline E/Line Target Year 1 Target Year 1 Results Year 2 Target Year 2 Results Baseline E/Line Target Year 1 Target Year 1 Results Year 2 Target Year 2 Results 0 3 2 5 1 4 0 10 2 3 7 2 The High Gear programme in KwaZulu-Natal has missed this target, reaching a total of 5 industry partners that report that elements of the High Gear programme model are integrated into their institutional strategies, against a programme target of 10. In the Eastern Cape, the programme has already exceeded the endline target. 54 Intermediate Result 1.2: Key partners access South African funding and resources to sustain and scale the High Gear programme model. Indicator Amount of public and private South African funding that partners (including NAACAM and DHET) access for implementation of the High Gear programme programming (or related projects) Unit of Measure Amount of money in Rands, USD and GBP USAID/MSDF Eastern Cape & UKPF KwaZulu-Natal Baseline E/Line Target Year 1 Target Year 1 Results Year 2 Target Year 2 Results N/A 20% year-on year increase in funding (R10 480 899) N/A R8 661 900 (total for both provinces) 10% increase (R9 528 090) This is a programme wide indicator. The ‘baseline’ for this indicator was set in Year 1, as the amount that was raised. The target was a 10% increase in that in Y2, and a further 10% increase by endline. As the amounts accessed were not made available, we have no finding on this indicator. Indicator # of industry partners that have accessed funding to advance inclusive workplace practices and policies (indicator applies only to UKPF-funded portion of the High Gear programme) Unit of Measure No of firms USAID/MSDF Eastern Cape UKPF KwaZulu-Natal Baseline E/Line Target Year 1 Target Year 1 Results Year 2 Target Year 2 Results Baseline E/Line Target Year 1 Target Year 1 Results Year 2 Target Year 2 Results 0 10 firms 2 firms 0 8 0 The High Gear programme in KwaZulu-Natal has missed this target, we found that two KwaZulu-Natal employers were supported through SETA Advisory services, which is facilitated by Engeli enterprise development, to develop SETA grant applications in 2022 however, both companies did not submit their proposals for funding. One of the companies indicated that they had missed the submission deadline while another reported that they will consider submitting their proposals in 2023 as the primary focus was on staying afloat due to economic pressures. 55 Short-term Result 1.2.1: Partner industry intermediaries and companies submit funding proposals aligned to the High Gear programme model. Indicator SETA indicates an increase in employer proposals that include TVET college partnerships (indicator applies only to UKPF-funded portion of the High Gear programme) Unit of Measure Review of employer proposals that include TVET partnerships USAID/MSDF Eastern Cape UKPF KwaZulu-Natal Baseline E/Line Target Year 1 Target Year 1 Results Year 2 Target Year 2 Results Baseline E/Line Target Year 1 Target Year 1 Results Year 2 Target Year 2 Results TBD TBD N/A N/A N/A N/A It was originally intended that this review would be conducted by merSETA, and then that it would be included in SETA reports to the Presidency in terms of reporting on the Presidential Youth Employment Initiative, but neither of these things happened during the course of the High Gear programme in KwaZulu-Natal. Indicator # of industry group partners indicate that the High Gear programme has supported them to access increased skills development funding. Unit of Measure Number of partnerships USAID/MSDF Eastern Cape UKPF KwaZulu-Natal Baseline E/Line Target Year 1 Target Year 1 Results Year 2 Target Year 2 Results Baseline E/Line Target Year 1 Target Year 1 Results Year 2 Target Year 2 Results 0 2 2 1 2 1 0 2 2 1 2 1 Note that this refers to the finding that NACAAM has accessed for the expansion of the Yakh’iFuture platform, lecture capacity development and entrepreneurial skills development which cuts across both provinces and is reported as such. The High Gear programme in KwaZulu-Natal has achieved the endline target, with industry group partner having indicated that the High Gear programme has supported them to access increased skills development funding, against a programme target of 2. In the Eastern Cape, the programme has also met the endline target. 56 Short-term Result 1.2.2: Partner industry intermediaries disseminate project learnings. Indicator # of instances (e.g., forums, publications, social media) in which industry partners disseminate the High Gear programme learning. Unit of Measure Number of times learning is shared USAID/MSDF Eastern Cape UKPF KwaZulu-Natal Baseline E/Line Target Year 1 Target Year 1 Results Year 1 Target Year 1 Results Baseline E/Line Target Year 1 Target Year 1 Results Year 1 Target Year 1 Results 0 36 12 123 12 238 (in total) 0 36 6 125 18 238 (in total) Unfortunately, the data for Year 2 of this indicator as provided by IYF is not disaggregated by province. But it is safe to say that the overall target has been exceeded. Objective 2: TVET students have equitable access to high-quality. relevant training and improved employment opportunities. Indicator % of individuals with new employment following participation in donor-assisted workforce development programmes. Unit of Measure Percentage of graduates USAID/MSDF Eastern Cape UKPF KwaZulu-Natal Baseline E/Line Target Year 1 Target Year 1 Results Year 2 Target Year 2 Results Baseline E/Line Target Year 1 Target Year 1 Results Year 2 Target Year 2 Results % of respondents who report new employment, 12 months ex-post. 27% 34% 24% 16% 22% 28% 26 % 14% As the indicator is 12 months ex-post, there was no target or result for Year 1. Because of the nature of the programme, it is difficult to delineate a pure ’12-month ex-post’ group, as participants sometimes return to the colleges for further courses etc. The results here are from the tracer study done in Dec 2022/Jan 2023, 12 months after the first the High Gear programme cohorts completed their studies. In the tracer study, we sent a short message to all graduates for whom we had contact details, asking about their current economic status. The percentages presented above are those who responded that they are in full- or part-time employment. The High Gear programme in KwaZulu-Natal has missed this target, with only 14% of graduate respondents indicating that they are in employment, against a programme target of 28%. In the Eastern Cape, the programme also missed the target for Year 2. 57 Indicator #/% TVET graduates that access higher education following participation in the High Gear programme-assisted courses. Unit of Measure Percentage of TVET graduates and number of TVET graduates USAID/MSDF Eastern Cape UKPF KwaZulu-Natal Baseline E/Line Target Year 1 Target Year 1 Results Year 2 Target Year 2 Results Baseline E/Line Target Year 1 Target Year 1 Results Year 2 Target Year 2 Results (average) of NATED candidates accessed Higher Education (N4 to N5, N5 to N6, N6 to other) 41% 62% 48% 44% 34% 51% 40% 31% 46% As neither of the colleges has submitted performance data yet, we have no finding on this indicator. Indicator #/% of TVET graduates that indicate their qualification provided them with the skills and tools to access employment Unit of Measure Percentage of TVET graduates and number of TVET graduates USAID/MSDF Eastern Cape UKPF KwaZulu-Natal Baseline E/Line Target Year 1 Target Year 1 Results Year 2 Target Year 2 Results Baseline E/Line Target Year 1 Target Year 1 Results Year 2 Target Year 2 Results (average) of resp. who agree & strongly agree with three statements in the CATI pertaining to skills and tools to access employment 65% 100% 80% 65% 36% 68% 69% 100% 80% 69% 90% 77% The High Gear programme in KwaZulu-Natal has missed this target, with only 77% of respondents agreeing or strongly agreeing with the three statements pertaining to the having the skills and tools to access employment, against a programme target of 100%, although there has been an improvement in the results since Year 1. In the Eastern Cape, there was a slight improvement on the results from Year 1. It is unclear why the target for Year 2 in the Eastern Cape is so low. 58 Indicator % of individuals with improved soft skills following participation in USG [or UKPF] assisted workforce development programs Unit of Measure Percentage of TVET students with improved soft skills USAID/MSDF Eastern Cape UKPF KwaZulu-Natal Baseline E/Line Target Year 1 Target Year 1 Results Year 2 Target Year 2 Results Baseline E/Line Target Year 1 Target Year 1 Results Year 2 Target Year 2 Results % (average) of respondents who believe that participating in the course had improved their [listed] soft skills a fair amount or an enormous amount 94% 100% 95% 85% 95% 89% 93% 100% 95% 91% 95% 86% The High Gear programme in KwaZulu-Natal has missed this target, with only 86% of respondents believe that participating in the course had improved their [listed] soft skills a fair amount or an enormous amount, against a programme target of 100%. There has also been a very slight decline in this since Year 1, and indeed since the Baseline was conducted. In the Eastern Cape, there was a slight improvement on the results from Year 1, although still below the baseline %. Indicator Completion rate of TVET students in the High Gear programme-assisted qualifications. Unit of Measure Percentage of students that pass USAID/MSDF Eastern Cape UKPF KwaZulu-Natal Baseline E/Line Target Year 1 Target Year 1 Results Year 2 Target Year 2 Results Baseline E/Line Target Year 1 Target Year 1 Results Year 2 Target Year 2 Results NATED 28% NCV 6.7% NATED 42% NATED 33% NATED 36% NATED 21% NCV 4% NATED 32% NATED 24% NATED 50.3% NATED 29% As neither of the colleges has submitted performance data yet, we have no finding on this indicator. 59 Intermediate Result 2.1 (b): Employers have increased confidence that graduates from upgraded TVET courses meet their skills needs. Indicator #/% of employers who indicate that female and male graduates employed through the college perform better (than other new entrants who enter these occupations) in the workplace. Unit of Measure Number of employers USAID/MSDF Eastern Cape UKPF KwaZulu-Natal Baseline E/Line Target Year 1 Target Year 1 Results Year 2 Target Year 2 Results Baseline E/Line Target Year 1 Target Year 1 Results Year 2 Target Year 2 Results % of interviewed employers expressed a level of satisfaction with the quality of graduates from TVET College 0 50% 17% 83% 34% 0 50% 15% 100% 35% This indicator is difficult to make a finding on: when it was developed, we anticipated running CATI surveys with employers. We were requested to do interviews only, and in the Eastern Cape, employer respondents indicated that they have either not employed TVET graduates or said that they had not made the distinction between people employed through the Colleges and other new entrants or had not monitored this. In KwaZulu-Natal, none of the companies expressed a level of satisfaction with TVET graduates. Short-term Result 2.1.1: TVET Colleges deliver engaging, industry-Aligned & inclusion sensitive education to TVET students. Indicator #/% of TVET graduates that indicate that the quality and resources of the programme was of a high standard. Unit of Measure Percentage of TVET graduates and number of TVET graduates USAID/MSDF Eastern Cape UKPF KwaZulu-Natal Baseline E/Line Target Year 1 Target Year 1 Results Year 2 Target Year 2 Results Baseline E/Line Target Year 1 Target Year 1 Results Year 2 Target Year 2 Results % (average) of respondents who agreed and strongly agreed with all three statements on quality & resources in the learner CATI 86% 100% 90% 66% 90% 71% 82% 100% 90% 65% 65% 78% The High Gear programme in KwaZulu-Natal has missed this target, with only 78% of respondents who agreed and strongly agreed with all three statements on quality & resources in the learner CATI, against a programme target of 100%. However, it is noted that they met the Year 2 target in this regard, and that there was an increase from Year 1. In the Eastern Cape, there was a slight improvement on the results from Year 1, although still below the target. 60 Indicator # TVET students who receive updated course content. Unit of Measure Number of students USAID/MSDF Eastern Cape UKPF KwaZulu-Natal Baseline E/Line Target Year 1 Target Year 1 Results Year 2 Target Year 2 Results Baseline E/Line Target Year 1 Target Year 1 Results Year 2 Target Year 2 Results 0 1600 240 288 480 296 0 2000 1100 356 1344 648 The High Gear programme in KwaZulu-Natal has missed this target, reaching a total of 1 004 TVET students with updated course content, against a programme target of 2 000 students. In the Eastern Cape, the programme has reached a total of 584 TVET students. It will need to reach 1 016 students in 2023 if it is to reach target. Short-term Result 2.1.2: TVET College staff are equipped and motivated to deliver the enhanced curricula and pedagogy. Indicator #/% of TVET College lecturers that indicate the quality of their teaching has improved as a result of the High Gear programme support Unit of Measure Number and percentage of college lecturers who received training USAID/MSDF Eastern Cape UKPF KwaZulu-Natal Baseline E/Line Target Year 1 Target Year 1 Results Year 2 Target Year 2 Results Baseline E/Line Target Year 1 Target Year 1 Results Year 2 Target Year 2 Results % of lecturers agree and strongly agree that the quality of their teaching has improved as a result of the High Gear programme support 0 75% 25% 1 lecture neither agreed nor disagreed 50% 63% 0 75% 20% 29% 60% 69% The High Gear programme in KwaZulu-Natal has just missed this target, with 69% of lecturers surveyed agreeing and strongly agreeing that the quality of their teaching has improved as a result of the High Gear programme support, against a programme target of 75%, although the substantial increase from Year 1 is noted. 61 Output 2.1.2.1: The High Gear programme trains and coaches TVET staff to deliver improved courses. Indicator Number of individuals affiliated with higher education (tertiary) institutions receiving capacity development support with USG {or UKPF] assistance. * Unit of Measure Number of TVET lecturers who have received training on upgraded industry-validated course content USAID/MSDF Eastern Cape UKPF KwaZulu-Natal Baseline E/Line Target Year 1 Target Year 1 Results Year 2 Target Year 2 Results Baseline E/Line Target Year 1 Target Year 1 Results Year 2 Target Year 2 Results 0 19 19 9 19 16 0 52 25 26 28 28 Note that originally it was intended that Life Orientation lecturers would also be part of this capacity development support but on the request of the Colleges, it was limited to Engineering and related lecturers. The High Gear programme in KwaZulu-Natal has exceeded this target, with 54 lecturers having received training on upgraded industry validated course content, against a programme target of 52. In the Eastern Cape, the programme has already exceeded its endline target of 19. Intermediate Result 2.2: Students enrolled in upgraded TVET courses have expanded professional networks. Indicator #% of TVET students that have connections with employers that support their ability to seek employment Unit of Measure Number and percentage of learners USAID/MSDF Eastern Cape UKPF KwaZulu-Natal Baseline E/Line Target Year 1 Target Year 1 Results Year 2 Target Year 2 Results Baseline E/Line Target Year 1 Target Year 1 Results Year 2 Target Year 2 Results % of respondents who agree and strongly agree that participating in the course helped them to improve their connections with employers 59% 100% 80% 39% 14% 19% 46% 80 % 65% 46% 75 % 14% The High Gear programme in KwaZulu-Natal has missed this target. It also appears unlikely that Eastern Cape will meet this endline target. 62 Short-term Result 2.2.1: Employers deliver expanded and more gender-inclusive workplace training opportunities. Indicator # of WIL opportunities delivered to individual lecturers, as part of USG [or UKPF] assisted workforce development programmes) Unit of Measure Number and percentage of lecturers [WIL placements] USAID/MSDF Eastern Cape UKPF KwaZulu-Natal Baseline E/Line Target Year 1 Target Year 1 Results Year 2 Target Year 2 Results Baseline E/Line Target Year 1 Target Year 1 Results Year 2 Target Year 2 Results 0 57 19 14 19 14 0 156 25 25 100 12 The High Gear programme in KwaZulu-Natal has missed this target, with 37 WIL opportunities delivered, against a programme target of 156. The programme in the Eastern Cape 28 opportunities in total, meaning that 29 WIL opportunities for lecturers must be delivered in 2023 if the programme target is to be met in that province. Short-term Result 2.2.2: Improved TVET student utilization of national career guidance and job-search platforms. Indicator % of TVET students on HG-assisted courses that report using the online career experience platform Unit of Measure Percentage of TVET students USAID/MSDF Eastern Cape UKPF KwaZulu-Natal Baseline E/Line Target Year 1 Target Year 1 Results Year 2 Target Year 2 Results Baseline E/Line Target Year 1 Target Year 1 Results Year 2 Target Year 2 Results N/A 1600 N/A N/A 750 N/A 2000 N/A N/A 750 This indicator was introduced in 2022, with the launch of Yakh’iFuture. However, there are difficulties in disaggregating data around users of the platform. We also note that lack of data and wifi, combined with the fact that the platform (unlike SAYouth.mobi) is not zero rated, means that, based on the information collected at Focus Groups, there has been fairly low take-up of Yakh’iFuture. To discuss further. 63 Indicator # of youth from SAyouth.mobi that access HG’s online career experience platform Unit of Measure Number of TVET students USAID/MSDF Eastern Cape UKPF KwaZulu-Natal Baseline E/Line Target Year 1 Target Year 1 Results Year 2 Target Year 2 Results Baseline E/Line Target Year 1 Target Year 1 Results Year 2 Target Year 2 Results N/A 1500 N/A N/A 750 137 N/A 1500 N/A N/A 750 444 Another new indicator, the High Gear programme in KwaZulu-Natal has missed this target, although the disjunct between the Year 2 and Endline Target is noted. The Eastern Cape programme also has a way to go if this target is to be met in that province. Difficulties with collecting this data are noted: these results cannot be confirmed, as it is unclear if the platform was accessed by students from the target colleges as this is a national platform, and so the results provided by IYF here are from the targeted provinces. The platform was launched at the colleges in 2022 but because there is no registration requirement the login details can also not be tracked. Indicator #/% of TVET students who access workplace experience, as part of USG [or UKPF] assisted workforce development programmes. * Unit of Measure Number and percentage of students USAID/MSDF Eastern Cape UKPF KwaZulu-Natal Baseline E/Line Target Year 1 Target Year 1 Results Year 2 Target Year 2 Results Baseline E/Line Target Year 1 Target Year 1 Results Year 2 Target Year 2 Results 29% 48% 29% 13% 38% 12% 15% 30% 20% 10% 25% 5% These results are from the CATI (which is what we have used previously) but there differ from the MERL data, which shows that 54 individuals accessed workplace experience in the Eastern Cape and 56 in KwaZulu-Natal. 64 Output 2.2.1.1: Industry intermediaries facilitate gender-inclusive workplace training opportunities for TVET students. Indicator # of new private sector entities that engage with TVET Colleges to provide students with relevant skills and workplace learning experiences Unit of Measure Number of private sector entities USAID/MSDF Eastern Cape UKPF KwaZulu-Natal Baseline E/Line Target Year 1 Target Year 1 Results Year 2 Target Year 2 Results Baseline E/Line Target Year 1 Target Year 1 Results Year 2 Target Year 2 Results 0 5 2 1 3 4 0 5 2 0 3 2 The High Gear programme in KwaZulu-Natal has missed this target, with only two new private sector entities that engage with TVET Colleges providing students with relevant skills and workplace learning experiences, against a programme target of five entities. In the Eastern Cape, the programme has reached the endline target. Output 2.2.1.2: Industry intermediaries facilitate gender-inclusive workplace training opportunities for TVET staff. Indicator # of new private sector entities that engage with TVET Colleges to provide staff with relevant skills and workplace learning experiences Unit of Measure Number of private sector entities USAID/MSDF Eastern Cape UKPF KwaZulu-Natal Baseline E/Line Target Year 1 Target Year 1 Results Year 2 Target Year 2 Results Baseline E/Line Target Year 1 Target Year 1 Results Year 2 Target Year 2 Results 0 5 3 3 3 2 0 5 2 1 4 1 The High Gear programme in KwaZulu-Natal has missed this target, with only two new private sector entities that engage with TVET Colleges providing staff with relevant skills and workplace learning experiences, against a programme target of five entities. In the Eastern Cape, the programme has reached the endline target. 65 ANNEXURE 3: STAKEHOLDERS ENGAGED DURING THE RESEARCH FOR THE YEAR 2 FORMATIVE EVALUATION Interviews This study focused on engaging stakeholders who were previously interviewed during the baseline study as well as the Year 1 evaluation at provincial and national levels. In this Year 2 evaluation the total number of stakeholders targeted was 39 and the total number of respondents reached was 26. The table below indicates the number of stakeholders targeted and the number of stakeholders reached. Table 5: Stakeholders reached through interviews Respondents KwaZulu-Natal Eastern Cape National Target Reach Target Reach Target Reach DHET Directorate 1 0 1 1 2 2 QCTO Directorate 1 1 College Management 4 2 3 3 Industry Intermediary Body 3 2 3 3 1 1 merSETA Directorate 1 0 1 1 Employers 9 4 9 6 Lecturer CATIs In Year 2 a total of 29 lecturers were reached across both colleges, 13 respondents are from Elangeni TVET College and 16 are from Eastcape Midlands TVET College. The lecturers teach NATED levels 1-6 Mechanical engineering as well as Electrical engineering. Table 6: Lecturer survey respondents Qualification (s) taught Elangeni TVET College Eastcape Midlands TVET College Mechanical Engineering 4 6 Electrical Engineering 9 10 Total 13 16 Graduate CATIs Datasets containing the names and contact details of N4 – N6 graduates were provided to IYF by the two TVET Colleges. A dataset was provided to IYF by Elangeni TVET College containing 648 unique entries and by Eastcape Midlands TVET College containing 296 unique entries. A total sample of 242 names was drawn from Elangeni TVET College while a sample of 168 names were drawn from Eastcape Midlands TVET College. The sample was determined using a 95% confidence interval with a 5% margin of error and was stratified by gender across both colleges. Where graduates selected during the sampling process could not be contacted, further random sampling was conducted until the sample size was attained. This resulted a slight overreach of graduates from both colleges, as shown in the table below. 422 surveys were completed in total, representing 249 Elangeni TVET College (59%) and 173 Eastcape Midlands TVET College (41%) respondents. 66 Table 7: Graduate sample & responses Elangeni TVET College Total Population Females Males 648 356 (55%) 292 (56%) Sample Total Sample Females Males 242 133(55%) 109(45%) Margin of Error Confidence Level Response Distribution 5% 95% 50% Call Results Total Reached Females Males 249 (103%) 142 (57%) 107(43%) Eastcape Midlands TVET College Total Population Females Males 296 133 (45%) 163 (55%) Sample Total Sample Females Males 168 75 (45%) 93 (55%) Margin of Error Confidence Level Response Distribution 5% 95% 50% Call Results Total Reached Females Males 173 (103%) 81(47%) 92 (53%) Table 8: Profile of graduate respondents Elangeni TVET College Eastcape Midlands TVET College Total Qualification registered for in 2022 Mechanical Engineering 67 87 154 (36%) Electrical Engineering 182 86 268 (64%) Race Black 248 155 403(95%) Coloured 0 16 16 (4%) Indian 1 0 1(<1%) White 0 2 2 (<1%) Gender Male 107 92 199 (47%) Female 142 81 223 (53%) Disability Living with disability 4 2 6 (1%) Highest qualification before enrolling in the college in 2022 Grade 12 223 153 376 (89%) NCV Level 3-5 8 4 12 (3%) N level below current level 16 11 27 (6%) National Diploma 1 1 2 (<1%) BA 1 0 1 (<1%) Grade 11 1 5 6 (2%) Higher Certificate 1 0 1 (<1%) Economic activity prior to enrolment Unemployed 97 45 142 (34%) Studying 137 116 253 (60%) Employed 12 9 21 (5%) Self-employed 3 3 6 (1%) 67 As shown in Tables 6 and above the majority (64%) of respondents were studying electrical engineering, with 36% studying mechanical engineering. Most respondents were African (95%) with a small number of Coloured and White respondents attending Eastcape Midlands TVET College. In terms of gender 53% of the respondents are female and 47% are male respondents across both colleges. A total of six respondents were living with disability. Prior to enrolment, the majority of respondents reported that they had been studying (60%), while 34% reported having been unemployed. A small percentage of respondents reported having been employed (5%) or self-employed (1%) prior to enrolment. In terms of educational qualifications prior to enrolling, 89% of respondents reported having passed Grade 12 and 6% reported having attained an N level qualification prior to entering this programme. Focus Group Discussions with Lecturers and Students Focus Group discussions were facilitated with 17 mechanical engineering and electrical engineering lecturers at Elangeni TVET College and 17 lecturers at Eastcape Midlands TVET College. Further focus groups were conducted with 11 Elangeni TVET College students studying electrical engineering N5 as well as 17 students studying mechanical engineering and electrical engineering N5 and N6 at Eastcape Midlands TVET College. College Lecturers Students Elangeni TVET College 17 11 Eastcape Midlands TVET College 17 17 Total 34 28 68 ANNEXURE 4: DETAILS OF THE EVALUATION QUESTIONS Evaluation questions & sub-questions Purpose Theory of Change component(s) evaluated Illustrative data collection method Question 1: Is the High Gear programme’s Theory of Change—including its embedded hypothesis and assumptions—accurate, thereby leading to anticipated results? Does the Theory of Change contribute to increased and sustained partnerships between industry and TVET Colleges? ● Primary: Help determine whether the hypothesised Theory of Change (ToC) is holding true or not, and why; ● Secondary: Inform project strategy shifts, if necessary, to strengthen implementation and outcomes. ● Sustained partnership between industry and TVET colleges to align courses with employer needs; ● NAACAM leads increased industry-wide partnerships with the TVET system; ● Industry groups’ strategic plans incorporate the High Gear programme model; ● Industry disseminates project learning and results. ● Interviews with employer and industry associations representatives To what extent is the curricula and teaching in TVET systems more aligned to industry needs and being delivered as intended according to project design? ● Primary: Help determine whether the hypothesised Theory of Change (ToC) is holding true or not, and why; ● Secondary: Inform project strategy shifts, if necessary, to strengthen implementation and outcomes. ● TVET lecturers adopt and champion course upgrades; ● TVET students have equitable access to high-quality, relevant training and improved employment opportunities. ● Survey of all affected TVET College staff, focus groups or interviews with a sample of staff member; ● Document review of attendance registers. To what extent did the intended outcome of increasing young women’s and young men’s learning and employment in targeted regions occur over the ● Primary: Help determine whether the hypothesised Theory of Change (ToC) is holding true or not, and why; ● Secondary: Inform project strategy shifts, if necessary, to strengthen implementation and outcomes. ● Students demonstrate improved academic performance and have expanded professional networks; ● TVET students have equitable access to high-quality, relevant training and improved employment opportunities. ● Computer Assisted Telephone Interview (CATI) with all graduates from the High Gear programme, focus groups with sample of graduates; 69 course of the project? ● Data retrieval from DHET MIS. Question 2: Is the High Gear programme contributing towards increased ecosystem resilience? Does the High Gear programme contribute toward increases in economic opportunity and inclusive growth, as a result of a higher-quality, industry-aligned TVET system continuum? ● Primary: Gather preliminary evidence of the High Gear programme programme’s contribution towards the development of a more resilient, youth-inclusive skills development ecosystem. ● ‘System Scale and Sustainability’ elements of the Theory of Change. ● National routine data on employment; ● Macro-economic measures such as labour productivity and GDP; ● Interviews with key employers and industry intermediaries. Question 3: Does the High Gear programme project model deliver Value for Money? Based on High Gear's VfM indicators of Economy, Efficiency, Effectiveness and Equity- does the High Gear programme exhibit VfM in procurement, expenditure and the youth and lecturers reached? ● Primary: Gather preliminary evidence of the High Gear programme to assess if the programme exhibit programmatic and operational approaches that deliver the most value for money (economy, efficiency, effectiveness and equity) in improving learning and workforce outcomes for young people enrolled in the TVET college system, including for low-income youth, young women and youth with disabilities. ● ‘System Scale & Sustainability’ and ‘Goal’ levels of the Theory of Change. ● IYF Financial Reports- project expenditure; ● Contract tracker; ● ● Programmatic MERL data￾youth/lecturer data. 70