1 2 EXECUTIVE SUMMARY High Gear is an innovative initiative, conceptualised and led by IYF that aims to demonstrate a model for greater industry involvement in Technical, Vocational, Education & Training (TVET) college course design and delivery. This will be achieved through equipping industry partners in South Africa with IYF’s globally refined tools, curricula, and processes, which address core technical education challenges, and then working with them to guide the initial implementation of these approaches with partner TVET colleges. The High Gear Theory of Change hypothesises that: IF industry intermediaries better support and facilitate collaboration and partnerships between employers and public TVET colleges, AND curricula and teaching in TVET colleges is aligned to industry needs, THEN public TVET colleges will become, and remain, more responsive to the needs of young people and industry, providing employers with an improved skills pipeline, and positioning more young people for workforce success. High Gear thus aims to create a demand driven public TVET system aligned with industry needs, and with quality course delivery that generates positive learning and employment outcomes for young men and women in South Africa. To achieve this, High Gear will be guided by TVET Colleges and industry in: 1. Identifying priority industry competencies with industry associations. 2. Designing industry validated TVET course upgrades together with TVET lecturers and industry experts. 3. Supporting TVETs to deliver industry-aligned content by drawing on TVET input and industry trainers. 4. Organising workplace exposure for TVET staff and students based on TVET demand which will then be managed by industry. 5. Delivering Sectoral Education and Training Authority (SETA) grant advisory services to employers within industry associations. The purpose of this formative, developmental evaluation is to assess the impact and contributions of High Gear and to test key components of the programme’s design and Theory of Change during the initial implementation period. Specifically, the evaluation sought to review system change catalysts and initial system adoption in relation to partnerships, curricula and training and employment outcomes: •Improved & sustained partnerships between industry Partnerships •Lead to curricula & teaching that is better aligned to industry needs Curricula •Which results in improved learning & employment outcomes for young people Training & Employment Outcomes 3 The formative evaluation found that, in relation to Partnerships, there is clear evidence that better and stronger partnerships are beginning to emerge between industry intermediaries and colleges, and that employers are being brought into this process. • Industry Association respondents indicate that they have begun to integrate the concepts from the programme into their own developmental processes: “we didn’t understand the landscape and funding mechanisms” and they are using the learning acquired through High Gear to “explore skills development interventions and apply for additional funding to support High Gear as well as our own initiatives’’. • There is direct evidence of national and regional Industry Associations working with colleges, through High Gear, to upgrade the curricula, to upskill lecturers and to source opportunities for Work Integrated Learning (WIL). Colleges are responding well, laying the foundation for better engagements between industry and colleges going forward. • The High Gear approach resonates well with company strategies – employer respondents note that it “makes sense” for there to be closer working relationships between employers and colleges, and that “the programme … is highly relevant and well received by companies and industry as a whole.” Some employer respondents noted that the impact of COVID-19 was slowing their ability to engage more fully in the programme. These findings are borne out in the achievement of specific indicators: • Year 1 targets were met or slightly exceeded in relation to the number of High Gear practices and policies that are adopted by industry partners; and in relation to the number of partners who report that elements of the model are integrated into their institutional strategies. Targets for the number of times partners share learning from the project were significantly exceeded. • ZAR 8 661 900 (USD 577 460) of public funding has been secured by partners for implementation of High Gear or related programming (this becomes the Baseline going forward). In addition, partners have submitted skills development funding proposals to the value of ZAR 19 902 813 (USD 1,326,854) to public entities. • The Year 1 target for the number of beneficiaries engaged in partnership activities was significantly exceeded in KZN (by 93%), and slightly under in EC (63% of target achieved) but this is not considered a concern at this stage. It is noted that the Year 1 target relating to the number of industry group partners that indicate that High Gear has supported them to access increased skills development funding was not met, but this is not considered a concern at this stage as there is evidence that a great deal of groundwork has been done in this regard and these kinds of funding applications take time. Further, there is no evidence yet that partners have accessed additional funding to advance inclusive practices, but there are promising signs as High Gear has supported two KZN firms to develop and submit SETA grant applications in early 2022 and the firms are awaiting the outcomes from merSETA. IYF is engaging with firms through SETA Grant Advisory Services on this data. Also, the SETA is not yet able to provide data on whether there is an increase in employer proposals that include TVET college partnerships, but as this is also a requirement of reporting into the President’s Youth Employment Dashboard, it is likely that this information will be accessible going forward, and will be tracked. In relation to Curricula, it was found that course material has been developed, technical assistance provided, and College lecturers are receiving training, coaching or capacity building support. • Work has focused on aligning curriculum in the existing N4-N6 Mechanical and Engineering programmes through the High Gear Project Based Learning course upgrades. • There has also been significant work done to review the curricula for new occupational qualifications for Electronics, Electrical and Mechanical Engineering studies: the current engineering TVET N4 – N6 qualifications are falling away, and the new qualifications will replace them from late 2023 onwards. High Gear supported the Qualifications Council for Trades and Occupations (QCTO) in the 4 development of these new qualification between mid-2021 and mid-2022, and all three curriculum frameworks are with the QCTO for finalisation. • A number of technical interventions have been provided, and all qualifying lecturers at both colleges have benefitted from capacity building, including an introduction to project-based learning (PBL), effective teaching techniques. and work integrated learning (WIL). Specifically, Year 1 targets were exceeded for the number of technical assistance interventions delivered – five in the Eastern Cape and six in KZN (target of three in each province), and for the number of TVET courses aligned to industry requirements – two in each Province. In relation to the total number of beneficiaries1 who received training, coaching or capacity building, it is noted that overall programme targets were exceeded for institution staff in both provinces, and for private sector or NGO staff in KZN. The Year 1 targets for students reached with the training were not met as the targets were based on the inclusion of NC(V) students who have since been excluded from the High Gear programme on the basis of feedback from the colleges. In discussion with High Gear, it was agreed that, despite the exclusion of the NC(V) students, the project expects to meet the original targets with the additional students reached through the Yakh’iFuture platform, to go live in mid-2022. And in relation to training and employment outcomes, it is too soon to tell but the Programme appears to be on track to achieve this: an increasing percentage of students are passing, and based on interviews with employers, it was found that they (employers) have very high levels of confidence that students employed through the college perform better than other new entrants. But there are some concerns around student perceptions of improvements and access to workplace experience opportunities. Given the timing of the student Computer Assisted Telephonic Interviews (CATI), the data on the number of students who have accessed employment is preliminary and will be confirmed in the Tracer study. In relation to the percentage of students who pass, Year 1 targets for the percentage of students who pass were exceeded in both Provinces: data on student academic performance received from EMC and Elangeni shows that the average combined pass rate for N4 – N6 Mechanical and Electrical Engineering was 35.6% at Eastcape Midlands College (Y1 target was 33%), and 50.3% at Elangeni TVET College (Y1 target was 24%). Of some concern is that, in the student CATI, questions raised as to whether qualifications provided them with the skills and tools to access employment and the required soft skills, and whether the quality and resources of the programme were of a high quality, were less positive than expected. These responses are concerning but may be explained by the reality that many High Gear upgrades had not yet been implemented and that there were COVID-related disruptions to the teaching and learning process. To understand this better this issue was probed in follow up qualitative interviews, in which it was found that students were specifically concerned that they had not been provided with the requisite practical training, that textbooks were not current or did not have sufficient information and that soft skills were not systematically covered. While some of these issues are outside of High Gear’s control, in Year 2 of the programme, IYF is supporting colleges to address some of these challenges by enabling more access to WIL through partnerships with employers, providing more relevant material through the demokits, and integrating Passport to Success (PTS) into Yakh’iFuture. Year 1 targets for the number of students who received updated course content was slightly exceeded in the EC, but not met in KZN. (EC target was 240 and 262 received, while in KZN the target was 1100, and 433 received.) The Y1 target for KZN was particularly high because it was anticipated that NC(V) students would be reached through PBL interventions, however this was not the case as NC(V) students were excluded from the programme, as described above. IYF is aiming to make up these numbers in 2022 by exposing both NATED and NC(V) students nationally to Mechanical and Electrical Engineering concepts with animated demos, formulas and revision quizzes on the Yakh'iFuture's Discover page.demokitassistance. 1 Note beneficiaries here covers institution staff, private sector or NGO staff and students. 5 The Year 1 target for the number of individuals affiliated with colleges (lecturers) who benefitted from capacity development support was not met in the EC but was (slightly) exceeded in KZN. This is largely due to the removal of the NC(V) qualifications from the programme (i.e., the target was too high in the EC). Surveys sent to lecturers to probe their responses had a very low completion rate – we are working to find a solution to better understand lecturers’ views on whether the quality of their teaching has improved. Another important strategy for improving employment outcomes it to increase student connections with employers. Student respondents in the CATI indicated that they don’t feel that their connections with employers have improved, and targets for students accessing work experience were not met at either college - 13% of students in EC and 10% of students in KZN had accessed workplace experience in 2021. This is a concern, but it is noted that in KZN there was a problem with accessing WIL opportunities as the college did not have sufficient insurance to meet COIDA requirements, and in the Eastern Cape there were delays in contracting with AIDC to facilitate placements. Student perceptions about that lack of access and their views that TVET colleges could still do more to connect them with potential employers are therefore correct. Those who had accessed work experience had generally positive views on the experience, although respondents from KZN returned slightly higher average ratings to all statements than those from the Eastern Cape. In order to support colleges to address this, High Gear has some direct interventions around this planned through the Yakh’iFuture career experience platform and the mobile site SAyouth.mobi which will hopefully address this going forward. 6 CONTENTS EXECUTIVE SUMMARY ..............................................................................................................................1 1 INTRODUCTION...................................................................................................................................9 1.1 Hypothesis and Theory of Change ........................................................................................................9 1.2 Evaluation Purpose .............................................................................................................................10 1.3 Background: High Gear interventions and activities being evaluated in Year 1 .................................11 2 EVALUATION METHODS AND METHODOLOGY.........................................................................15 2.1 Stakeholders Engaged.........................................................................................................................16 2.1.1 Interviews .......................................................................................................................................16 2.1.2 Lecturer Self-Administered surveys................................................................................................16 2.1.3 Graduate Surveys............................................................................................................................16 2.2 Limitations to the Study ......................................................................................................................18 3 FINDINGS AGAINST HIGH GEAR INDICATORS...........................................................................20 3.1 Objective 1: Sustained partnership between industry and TVET colleges to align with employer needs 20 3.1.1 Intermediate Result (IR) 1.1: Increased industry-wide coordination to align TVET courses with industry needs. ............................................................................................................................................21 3.1.2 Short-term Result 1.1.1: Industry partners incorporate TVET partnership objectives into their strategic plans..............................................................................................................................................29 3.1.3 Intermediate Result 1.2: Key partners access South African funding and resources to sustain and scale the High Gear model...........................................................................................................................30 3.1.4 Short-Term Result 1.2.1: Partner industry intermediaries and companies submit funding proposals aligned to the High Gear model. .................................................................................................32 3.1.5 Short-Term Result 1.2.2: Partner industry intermediaries disseminate project learnings. ............33 3.2 Objective 2: TVET students have equitable access to high-quality, relevant training and improved employment opportunities..............................................................................................................................34 3.2.1 Intermediate Result 2.1 (a): Students enrolled in upgraded TVET courses demonstrate improved academic performance ................................................................................................................................44 3.2.2 Intermediate Result 2.1 (b): Employers have increased confidence that graduates from upgraded TVET courses meet their skills needs...........................................................................................................46 3.2.3 Intermediate Result 2.2: Students enrolled in upgraded TVET courses have expanded professional networks. .....................................................................................................................................................51 3.2.4 Short term Result 2.2.1: Employers deliver expanded and more gender-inclusive workplace training opportunities..................................................................................................................................52 4 FINDINGS IN RELATION TO THE PRIMARY EVALUATION QUESTIONS ................................60 4.1 Question 1: Is High Gear’s Theory of Change – including its embedded assumptions – accurate, thereby leading to anticipated results? ...........................................................................................................61 4.1.1 Does the Theory of Change contribute to increased and sustained partnerships between industry and TVET colleges?.......................................................................................................................................62 4.1.2 To what extent are the curricula and teaching in TVET systems more aligned to industry needs and being delivered as intended according to project design? ...................................................................63 7 4.1.3 To what extent did the intended outcome of increasing young women’s and young men’s learning and employment in targeted regions occur over the course of the project?................................63 4.2 Question 2: Is High Gear contributing towards increased ecosystem resilience? ..............................63 4.3 Question 3: Does the High Gear project model deliver Value for Money?.........................................64 5 CONCLUSION ....................................................................................................................................66 TABLES Table 1: High Gear Interventions - Year 1.............................................................................................................11 Table 2: Stakeholders reached through Interviews.............................................................................................16 Table 3: Lecturer Survey Responses.....................................................................................................................16 Table 4: Graduate Survey Sample & Responses...................................................................................................16 Table 5: Profile of Graduate Survey Respondents................................................................................................17 Table 6: Technical Assistance Interventions Delivered.........................................................................................26 Table 7: Training Targets and Year 1 Achievements.............................................................................................27 Table 8: Number of people who meaningfully engaged in partnerships, engagement and convening activities 28 Table 9: Status of Applications for Public Sector Funding ....................................................................................31 Table 10: Current Status of Survey Respondents.................................................................................................36 Table 11: Time Taken to Find a Job ......................................................................................................................36 Table 12: Type of Employment Contract..............................................................................................................36 Table 13: Relationship between Qualification and Job ........................................................................................37 Table 14: Job Titles of those Employed ................................................................................................................37 Table 15: Monthly Income of those Employed.....................................................................................................37 Table 16: Contribution to Household Income .....................................................................................................38 Table 17: Perceived Reasons for Unemployment.................................................................................................38 Table 18: Comparing Baseline & Formative Evaluation Survey Responses to Statements on Improving Skills & Knowledge to Access Employment (Combined Agree and Strongly Agree).........................................................40 Table 19: Reasons for enrolling in Qualification (2021 responses only)...............................................................42 Table 20: EMC Respondents - Views on effect of participation on Soft Skills .....................................................43 Table 21: Elangeni Respondents - Views on effect of participation on Soft Skills................................................44 Table 22: EMC Completion Rates ........................................................................................................................45 Table 23: Elangeni Completion Rates...................................................................................................................45 High GearTable 24: Responses to Survey Questions on Quality & Resources of the Programme .......................47 Table 25: Student Access to Workplace Experience/ Exposure...........................................................................53 Table 26: How students accessed Workplace Experience/ Exposure...................................................................53 Table 27: Respondents' views on Workplace Experience /Exposure (Average Ratings) ......................................53 Table 28: Reasons why Respondents thought that they had not accessed Workplace Experience....................55 8 9 1 INTRODUCTION High Gear is an innovative initiative conceptualised and led by IYF that aims to demonstrate a model for greater industry involvement in Technical, Vocational, Education & Training (TVET) college course design and delivery. This will be achieved through equipping industry partners in South Africa with IYF’s globally refined tools, curricula, and processes, which address core technical education challenges, and then working with them to guide the initial implementation of these approaches with partner TVET colleges. High Gear was launched in South Africa in February 2020 and runs through March 2023 in KwaZulu-Natal (KZN) and through February 2024 in the Eastern Cape (EC). The initiative aims to create a demand driven public TVET system that is aligned with industry needs, and which provides high quality course delivery that generates positive learning and employment outcomes for young men and women in South Africa. The UK Government’s Skills for Prosperity Programme is funding High Gear implementation in KwaZulu-Natal Province, and the United States Agency for International Development (USAID) and the Michael & Susan Dell Foundation (MSDF) are funding project implementation in the Eastern Cape province. All three funding partners are supporting High Gear’s national stakeholder engagement and learning efforts. The design and implementation of High Gear is being managed by IYF, in partnership with the National Association of Automotive Component and Allied Manufacturers (NAACAM) and the Department of Higher Education and Training (DHET). Within this partnership the following roles are allocated: • IYF manages the High Gear partnership and leverages its model and curriculum enhancement tools. • NAACAM ensures the High Gear initiative remains responsive to the skill needs of employers in the automotive components manufacturing sector. • DHET, as the overseer of South Africa’s public TVET college system, informs decisions related to the selection of partner TVET colleges; the selection of courses/qualifications for upgrading; the review and approval of proposed course upgrades; and the facilitation of TVET college leadership and lecturer participation in the initiative. In addition, DHET will support expansion of successful course upgrades in the TVET system. IYF has partnered with Singizi Consulting Africa (Singizi) to conduct research and evaluation activities for the duration of the implementation process. 1.1 Hypothesis and Theory of Change High Gear operates on the following hypothesis: IF industry intermediaries better support and facilitate collaboration and partnerships between employers and public TVET colleges, AND curricula and teaching in TVET colleges is aligned to industry needs, THEN public TVET colleges will become, and remain, more responsive to the needs of young people and industry, providing employers with an improved skills pipeline, and positioning more young people for workforce success. This hypothesis, with the High Gear Theory of Change (ToC) are outlined in the following, Diagram 1. 10 Diagram 1: High Gear Hypothesis & Theory of Change This hypothesis is currently being tested by IYF. This report focuses on the progress made in the first year of the programme against its objectives and the extent to which the actions undertaken appear to be resulting in the intended changes. It provides an analysis of the data collected from TVET graduates, TVET lecturers, and other stakeholders, including industry intermediaries, employers from the automotive components manufacturing sector in Durban and Gqeberha, college officials in colleges in which High Gear is being implemented, and Government stakeholders at national and provincial levels in South Africa. High Gear aims to create a demand driven public TVET system aligned with industry needs, and with quality course delivery that generates positive learning and employment outcomes for young men and women in South Africa. To achieve this, High Gear will be guided by TVET Colleges and industry in: • Identifying priority industry competencies with industry associations. • Designing industry validated TVET course upgrades together with TVET lecturers and industry experts. • Supporting TVETs to deliver industry-aligned content by drawing on TVET input and industry trainers. • Organising workplace exposure for TVET staff and students based on TVET demand which will then be managed by industry. • Delivering Sectoral Education and Training Authority (SETA) grant advisory services to employers within industry associations. The High Gear initiative aims to target 3600 TVET students to receive updated course content across Eastcape Midlands College and Elangeni College over the next 4 years. 1600 of these students will come from Eastcape Midlands College and 2000 from Elangeni TVET College. This Year 1 Annual Formative Evaluation Report focuses on the activities that were undertaken in the first year of the programme and covers the period April 2021 to March 2022. It is noted that the Department of Higher Education and Training postponed the 2021 end of year exams due to disruptions caused by COVID-19. The final data gathering for Year 1 activities could therefore only take place in April 2022, and the first version of this report was submitted on the 23rd of May 2022. This version was reviewed by the IYF team and sent back to Singizi on 03 June 2022. The updated version was used to develop the presentation to donors at the bi-monthly PAC meeting on 15 June 2022. In discussion with IYF, and with the support of donors, it was agreed 11 to probe some of the findings with students. These qualitative telephonic interviews were conducted in August 2022, and the additional data integrated into this report. This report reflects on the extent to which the programme appears to be on track in relation to realising the two key objectives of the High Gear programme: 1. The development of sustained partnerships between industry and TVET colleges to align training with employer needs, and 2. Ensuring that TVET students have equitable access to high-quality, relevant training and improved employment opportunities. 1.1 Evaluation Purpose In June 2021 a Baseline study was conducted for High Gear South Africa by Singizi. This was to provide a basis against which change could be measured. The Baseline reported the status of TVET curricula and training against programme indicators and offered more nuanced insights into the training system based on the perspectives from NAACAM and other industry intermediary respondents, employers, college officials, N4-N6 engineering graduates (2019 and 2020), lecturers, and other key stakeholders. During the Baseline study, programme partners confirmed the logic of the High Gear Theory of Change and affirmed the importance of sustainable partnerships to improve the alignment of the TVET colleges’ curricula and teaching to industry needs. All stakeholders interviewed were confident that with greater levels of alignment the colleges will become, and remain, more responsive to the needs of both young people and industry, which in turn should provide employers with an improved skills pipeline, contribute to improved productivity, and position more young people for access to employment. The purpose of the formative evaluation is to assess the impact and contributions of High Gear to the strengthening of the skills system and to test key components of the programme’s design and Theory of Change during the initial implementation period. The objectives of this report are to: • Determine whether the hypothesised Theory of Change (ToC) and assumptions are holding true or not, and the factors that are influencing these results. • Inform project strategy shifts, if necessary, to strengthen implementation and outcomes; and • Gather preliminary evidence of the High Gear project’s impact on young people and the TVET system. Additionally, the findings of the formative annual evaluation report provide content in response to the High Gear primary evaluation questions, they provide evidence of the robustness of the High Gear Theory of Change and, where necessary, also provide recommendations about modifications that may be required to the Theory of Change. 1.2 Background: High Gear interventions and activities being evaluated in Year 1 This section highlights the activities that have been undertaken by the IYF High Gear programme in the past year. Table 1: High Gear Interventions - Year 1 Intervention Purpose Work-integrated Learning (WIL) co-design workshops with TVET colleges. In March 2021, IYF held workshops with the TVET colleges supported by High Gear to conceptualise and develop initial plans for the implementation of WIL. The session also focused on assessing the needs of the lecturers in relation to the industry knowledge that could be easily translated into valuable classroom insights thus linking lecture WIL placements to the improved teaching pedagogy. Introduction to Project-based Learning (PBL) workshops with TVET colleges. In April 2021, IYF held a lecturer engagement workshop with the TVET colleges to co-design and develop initial plans for the implementation of PBL. These sessions also provided the lecturers an opportunity to highlight key topics or concepts where PBL could be directed, based on the NATED programmes they are teaching. A full PBL cycle could not be 12 implemented because there was no time or resources to embed this learning approach in NATED. Instead, as part of a ‘quick-win’ resource strategy, High Gear chose to develop engineering demokits to help lecturers demonstrate foundational engineering concepts. Demokit testing IYF held a testing session in June where lecturers got to test demokit prototypes (i.e., assembly and use of tools). Lecturer capacitation training2 In August 2021, 25 lecturers from Elangeni TVET College were provided with training on how to use a Milling Machine at the George Campbell Technical School in KZN Lecturer WIL (workplace exposure) In August 2021, 10 lecturers from Eastcape Midlands College received work exposure at Compuscan manufacturing company3 . In September 2021, the second cycle of WIL placements for 14 lecturers in Eastern Cape took place at VW and eNtsa (Innovation through engineering, based at Nelson Mandela University4 ). These were short placements and focused on knowledge sharing around the technological advancements in the automotive industry. Workshops on effective teaching techniques In August 2021, IYF conducted a training session for 26 lecturers in KZN and 9 lecturers in Eastern Cape in effective teaching techniques with the use of engineering demokits for NATED teaching. Mentor training In September 2021, IYF conducted training for 3 mentors based in KZN and 2 mentors based in Eastern Cape from Mindful Skills – giving them practical guidance on how to best support lecturers with effective teaching techniques including supporting lecturers to use the demokits in NATED teaching. Mentoring schedules In September 2021, IYF conducted a session with 26 lecturers from Elangeni TVET College and 9 lecturers from Eastcape Midlands College to schedule mentoring sessions for trimester 3. This session was also used to introduce lecturers and campus managers to the tools that would be used as part of this mentoring intervention. Mentoring was carried out between October and November 2021. High Gear WIL strategy In November and December 2021, IYF held a session with the TVET colleges to introduce them to new WIL consultants (Skillsonic), provide orientation on tools, and develop initial plans for 2022 implementation. Career platform development In October and November 2021 focus groups with college students were facilitated to ascertain the key features that the platform would have to integrate to be valuable to students. PBL course upgrades implementation In October and November 2021 NATED (N4-N6) students benefited from the PBL upgrades. A total of 433 students were reached in KZN. A total of 262 students were reached in Eastern Cape. QCTO technical assistance Workshops to co-design new occupational qualifications for Electronics, Electrical and Mechanical Engineering studies were held in Eastern Cape in November 2021and in KZN in March 2022. These workshops were attended by a working group which includes the Quality Chamber of 2 It is noted that, in the internal IYF MERL reporting system, this lecturer capacitation was captured under lecturer WIL because, at the time, it was challenging for AIDC and NAACAM to secure any work exposure for lecturers. 3 Compuscan is a fully fledged manufacturing company based at ECM College. 4 eNtsa is “an engagement institute within the Nelson Mandela University. As an internationally recognised innovation hub, the group focuses on engineering design, component and material testing, prototyping and industrial R&D.” (https://entsa.mandela.ac.za) 13 Trades and Occupations (QCTO), IYF, and High Gear industry consultants – Business Management Concepts. Labour market intelligence IYF and NAACAM conducted four quarterly labour market skills research processes with employers in the industry between January 2021 and February 2022. This research considers the occupations in demand, critical competencies required from TVET graduates, and skills that employers consider critical for the future. SETA Grant Advisory Services SETA Grant Advisory Services aim to address the poor uptake of discretionary grants by employers in the automotive parts manufacturing sector. The service also aims to advise NAACAM members on SETA grant application processes and encourage a more impactful use of SETA funds to support transformation and localisation in the sector, including through increased linkages to the public TVET college system. In 2021, IYF contracted the services of Engeli Enterprise Development & WomHub to compile the High Gear SETA Best Practice Guide that was shared on NAACAM social media platforms and on both the NAACAM and High Gear webpages. An online Information sharing session was held with 27 employers in KZN to provide a high-level introduction to the Best Practice Guide. The Guide provides companies with practical advice about the funding options available through the SETA; how to successfully leverage them to support transformation (including gender equity) and localisation; and the required governance and reporting procedures that follow. High Gear also extended SETA Grants Advisory Services to two employers in KZN. It is noted that the following activities had not yet been launched at the time of the evaluation: • WIL for students • Yakh’iFuture online platform and dissemination • Promotion of SAYouth.mobi with students (along with Yakh’iFuture) • Industry guest lecturers • SETA Grant Advisory (firm application outcomes) 14 15 2 EVALUATION METHODS AND METHODOLOGY The High Gear Theory of Change includes a detailed Results Framework containing indicators for all the Objectives, Intermediate Results, Short Term Results and Outputs. Each of these indicators has a measure, and during the Baseline Study, Baselines and targets were set for the indicators. The Baseline provided an understanding of the system and beneficiaries before the intervention, and during this formative evaluation we collected data against these indicators to understand changes that have taken place in the first year of implementation of High Gear, as well as to start to generate some preliminary answers to some of the primary evaluation questions. Data collection methods focused on gaining as broad an understanding of the programme as possible, and included: • Interviews with stakeholders from NAACAM, other industry intermediaries, employers, college officials and key stakeholders conducted by Singizi. • A structured CATI process with TVET graduates administered by IYF. • Self-administered surveys by TVET lecturers. • Desk review of relevant documents. Interviews and surveys focused on both the indicators and on the three primary evaluation questions: Diagram 2: Primary Evaluation Questions & Sub-Questions Question 1: Is High Gear’s Theory of Change— including its embedded assumptions—accurate, thereby leading to anticipated results? a) Does the Theory of Change contribute to increased and sustained partnerships between industry and TVET colleges? b) To what extent are the curricula and teaching in TVET systems more aligned to industry needs and being delivered as intended according to project design? c) To what extent did the intended outcome of increasing young women’s and young men’s learning and employment in targeted regions occur over the course of the project? Question 2: Is High Gear contributing towards increased ecosystem resilience? a) Does High Gear contribute toward increases in economic opportunity and inclusive growth, as a result of a higher-quality, industry-aligned TVET system continuum? This relates to the extent to which High Gear has contributed towards: o Increased (or more effective) ecosystem partnerships o A more youth and gender-inclusive system o Improved competitiveness of the automotive manufacturing sector, in relation to skills and productivity o Further embedding TVET partnerships into South Africa’s industrial strategies o Increasing employers’ confidence to make investments in technology upgrades Question 3: Does the High Gear project model deliver Value for Money? a) Based on existing South African labour market data—such as productivity outcomes for successful TVET graduates, and long￾term livelihood impacts for young South Africans that access a first job—does High Gear cost-effectively contribute towards positive livelihood and productivity benefits for youth participants? 16 In addition to these engagements, Singizi also conducted telephonic qualitative interviews with 16 graduates from Elangeni TVET College in KZN and 14 graduates from Eastcape Midlands College in the Eastern Cape. These 30 respondents are currently out of college and completed Electrical and Mechanical Engineering qualifications in 2021. The aim of these interviews was to probe some of the findings in the High Gear Year 1 (2021) Formative Evaluation report. 2.1 Stakeholders Engaged 2.1.1 Interviews This study focused on engaging stakeholders who were previously interviewed during the Baseline study at provincial and national levels. The total number of stakeholders targeted was 28 and the total number of respondents reached was 23. The table below indicates the number of stakeholders targeted and the number of stakeholders reached. Table 2: Stakeholders reached through Interviews Stakeholder KwaZulu Natal Eastern Cape National Target Reached Target Reached Target Reached DHET Directorate 1 1 2 2 QCTO Directorate 1 1 College Management 3 3 3 2 Industry Intermediary Bodies 2 1 2 2 1 1 merSETA Directorate 1 1 1 1 Employers 3 2 8 6 Total 10 8 13 10 5 5 2.1.2 Lecturer Self-Administered surveys At Eastcape Midlands College the survey was sent to 10 lectures and 7 completed the survey while at Elangeni TVET College the survey was sent to 26 lectures and all lectures completed the survey. Table 3: Lecturer Survey Responses Qualification (s) taught Eastcape Midlands College Elangeni TVET College Target Reached Target Reached Mechanical Engineering 3 0 6 6 Electrical Engineering 6 5 17 17 Both 3 2 3 3 Total 11 7 26 26 2.1.3 Graduate Surveys Datasets containing the names and contact details of N4 – N6 graduates were provided to IYF by the two TVET colleges. A dataset of 575 entries was provided to IYF by Eastcape Midlands College and a dataset of 1984 entries was provided by Elangeni TVET College. The datasets were cleaned, resulting in a population of 288 17 unique entries for Eastcape Midlands College and 356 unique entries for Elangeni TVET College5 . A total sample of 166 names was randomly drawn from Eastcape Midlands College while a sample of 186 names were randomly drawn from Elangeni TVET College. The sample was determined using a 95% confidence interval with a 5% margin of error and was stratified by gender across both colleges. Where graduates selected during the sampling process could not be contacted, further random sampling was conducted until the sample size was attained. This resulted a slight oversampling of graduates from Eastcape Midlands College, with a final response rate of 103%, while the response rate for Elangeni TVET College was 88%, as shown in the table below. 335 surveys were completed in total, representing 171 EMC (51%) and 164 Elangeni (49%) respondents. Table 4: Graduate Survey Sample & Responses Eastcape Midlands College Total Population Females Males 288 127 (44%) 162 (56%) Sample Total Sample Females Males 166 73(44%) 93(56%) Margin of Error Confidence Level Response Distribution 5% 95% 50% Call Results Total Reached Females Males 171 (103%) 96 (56%) 75(44%) Elangeni TVET college Total Population Females Males 356 195 (55%) 161 (45%) Sample Total Sample Females Males 186 (52%) 102 (55%) 84 (45%) Margin of Error Confidence Level Response Distribution 5% 95% 50% Call Results Total Reached Females Males 164 (88%) 85 (83%) 79 (94%) As shown in Error! Not a valid bookmark self-reference. below, the majority (77%) of respondents were studying Electrical Engineering, with 22% studying Mechanical Engineering. Most respondents were Black (94%) with a small number of Coloured and White respondents attending Eastcape Midlands College. The sampling process ensured roughly equal representation of male (52%) and female (48%) respondents across 5 These are students who attended classes and does not include students who were registered for exams only. 18 both colleges. A total of six respondents were living with disability. In terms of age distribution, 79% of respondents were aged 19 – 25, while 15% were aged 26 – 30, and 5% were 31 or older. Prior to enrolment, the majority of respondents reported that they had been studying (73%), while 21% reported having been unemployed. A small percentage of respondents reported having been employed (6%) or self-employed (<1%) prior to enrolment. In terms of educational qualifications prior to enrolling, 61% of respondents reported having passed Grade 12 and 32% reported having attained an N level qualification prior to entering this programme. Small numbers of respondents reported passing Grade 11 (2%), NQF level 3 (<1%) and 4 (4%), with one respondent reporting having a B Tech degree, and one reporting having a BA. Table 5: Profile of Graduate Survey Respondents EMC Elangeni Total Qualification registered for in 2021 Mechanical Engineering 52 21 73 (22%) Electrical Engineering 115 143 258 (77%) Prefer not to specify 4 0 4 (1%) Race Black 151 164 315 (94%) Coloured 14 0 14 (4%) White 6 0 6 (2%) Gender Male 96 79 175 (52%) Female 75 85 160 (48%) Disability Living with disability 4 2 6 (2%) Age 19 – 25 130 136 266 (79%) 26 – 30 30 20 50 (15%) >30 9 8 17 (5%) Unknown 2 0 2 (1%) Highest qualification before enrolling in the college in 2021 Grade 11 6 2 8 (2%) Matric (Grade 12) 106 97 203 (61%) NQF level 3 2 0 1 (<1%) NQF level 4 5 8 12 (4%) N level below current level 49 57 106 (32%) B Tech 1 0 1 (<1%) BA 1 0 1 (<1%) Unknown 1 0 1 (<1%) Economic activity prior to enrolment Unemployed 24 47 71 (21%) Studying 132 111 243 (73%) Employed 14 5 19 (6%) Self-employed 1 1 2 (<1%) 2.2 Limitations to the Study Some targeted employers indicated that they would not participate in the evaluation as they had only recently begun to engage with the High Gear project and therefore felt that it would be premature to comment on the interventions. Additionally, even in those companies where employers agreed to participate, some respondents that were part of the interviews were not sufficiently familiar with the activities being undertaken as part of High Gear. However, these respondents were able to comment on skills development in the industry more generally. 19 20 3 FINDINGS AGAINST HIGH GEAR INDICATORS The High Gear results framework contains indicators against each of the objectives, intermediate results, short-term results and outputs. In this section, we show the Baseline findings against these indicators, and the Endline targets that were developed during the Baseline research. We also show the Year 1 (2021) annual targets and the Year 1 results to enable an informed assessment of how High Gear is performing against the Year 1 targets. Note that the full table, containing all this information, is contained in Annexure 1 to this report. In addition, and to give context to these Baseline numbers and targets, we provide the perspectives of the stakeholder respondents interviewed, and share the results of the CATI survey conducted with graduates and the self-administered lecturers’ survey where relevant. 3.1 Objective 1: Sustained partnership between industry and TVET colleges to align with employer needs Indicator # of effective and innovative High Gear model practices and/or policies that are adopted by industry partners Unit of Measure Number of policies and or practices adopted by industry partners USAID/MSDF Eastern Cape UKPF KwaZulu Natal Baseline Endline Target Year 1 Target Year 1 Results Baseline Endline Target Year 1 Target Year 1 Results 0 3 2 3 0 4 2 2 The Baseline figure against this indicator was 0 for both the Eastern Cape and KZN as the High Gear model had not yet been implemented. It was found that the Eastern Cape exceeded the Year 1 target while in KZN the target was achieved. In both provinces, there is clear evidence that better and stronger partnerships are beginning to emerge between industry intermediaries and colleges and that employers are beginning to be brought into this process. Although policies and practices have not yet been formally adopted, the basis for the adoption of such policies and practices is in place and is being facilitated through the sharing of the concept of the High Gear model and through the direct involvement of the intermediary bodies in High Gear interventions such as WIL for lecturers. At this stage the national industry intermediary body as well as two provincial industry intermediary bodies in the Eastern Cape (AIDC and ECAIF) and one provincial industry intermediary body in KZN (Skillsonics) indicate that they have started to integrate High Gear practices into their organisations. The national industry association (NAACAM) also indicates that they have begun to reflect on their current policies and practices because of their engagement in the High Gear project. The respondent explained how they have begun to integrate the concepts from the programme into their own institutional processes. They noted that: “we didn’t understand the landscape and funding mechanisms, so through High Gear we have built this internal institutional knowledge on understanding how these systems work and we’ve taken those learnings and gone out on our own to explore skills development interventions and apply for additional funding to support High Gear as well as our own initiatives’’. During the year 1 evaluation, respondents from the Eastern Cape industry intermediary bodies stated that, in the context of the pressure companies are facing to implement diversity and encourage inclusive practices and policies, they have already started to consider how to integrate elements of the High Gear initiative into their Broad Based Black Economic Empowerment (BBBEE) and other plans. A respondent suggested that, within the next two years, companies will have put in place proper mechanisms to conduct skills audits to support skills planning. 21 Industry intermediary bodies in both the Eastern Cape and KZN indicated that prior to High Gear they had no working relationship with the TVET colleges. Through the High Gear programme, they are now working in partnership with the colleges. The intermediary bodies in the Eastern Cape also shared that they are working with the Eastcape Midlands College to upskill lecturers. The initial focus of these partnerships is on training lecturers, launching work integrated learning, and providing guest lecturers to the colleges from industry. A respondent from an intermediary body in KZN also stated that they have experienced an increased willingness on the part of the colleges to engage with the private sector since the High Gear programme. They suggested that this is largely driven by the colleges’ need to find placements for their graduates. All companies interviewed confirmed that they had an opportunity to interact with the programme strategy, which they think resonates with their own company strategies as well as the broader sectoral strategy. A respondent from one company stated that, “the programme strategy is highly relevant and well received by companies and industry as a whole”. Another company respondent observed that the strategy is very relevant and valuable “as it makes sense to have collaboration between industry and colleges in order to align inputs.” However, respondents indicated that implementation had been delayed by the impact of the COVID-19 pandemic. Only one company respondent indicated that they had met with the High Gear team and the relevant industry body to explore what was needed to support the programme. 3.1.1 Intermediate Result (IR) 1.1: Increased industry-wide coordination to align TVET courses with industry needs. Indicators: # of TVET courses aligned to automotive component manufacturing industry needs (to be defined at qualification level and subject level) # of targeted technical assistance interventions delivered [Technical Assistance] # of beneficiaries (government staff, institution staff, teachers, trainers, students, private sector organisation staff and/or NGO staff) receiving training, coaching or capacity building support, including strategy, regulatory and policy work [Capacity Building] # of beneficiaries (government staff, institution staff, teachers, trainers, students, private sector organisations, NGOs) meaningfully engaged in partnerships, engagement and convening activities [Partnerships, Engagement & Convening] Indicator # Of TVET courses aligned to automotive component manufacturing industry needs (to be defined at qualification level and subject level) Unit of Measure Number of TVET courses USAID/MSDF Eastern Cape UKPF KwaZulu Natal Baseline Endline Target Year 1 Target Year 1 Results Baseline Endline Target Year 1 Target Year 1 Results 0 3 2 2 0 3 2 2 The Baseline for this indicator for both the Eastern Cape and KZN was 0 as the programme had not yet begun. With regards to the 2021 targets for this indicator both the Eastern Cape and KZN have achieved 100% of the target. The programme aims to align three qualifications in each province by Year 3: • N4 - N6 Mechanical Engineering; • N4 - N6 Electrical Engineering; and • NC(V) Engineering & Related Design6 . 6 Note that a decision has been made to no longer work with the NC(V) and therefore the Endline target for this indicator should be revised to only 2 qualifications i.e N4-N6 Mechanical engineering and Electrical Engineering 22 The formative evaluation probed the extent to which stakeholders believe that these programmes are aligned with industry needs, their perceptions as to what will be required from the High Gear programme to realise the necessary changes to the programmes, and the extent to which High Gear interventions have already led to changes. While it is acknowledged that there had been work done between industry, individual colleges and DHET prior to the launch of High Gear, this report is obviously focused on progress made against the targets for the High Gear programme. In the Eastern Cape the college management respondents indicated that the practical component of N4-N6 Mechanical Engineering and N4-N6 Electrical Engineering is perceived to be more aligned to industry when compared to theory component. In KZN, the college management respondents reported that the alignment of TVET courses with industry had already taken place for NC(V) Engineering & Related Design programme. A respondent from the national QCTO reported that some work has been done in aligning curriculum through the support that High Gear provided to them in relation to course upgrades in N5- N6 Mechanical and Engineering programmes, and work is being done to review the curricula for new occupational qualifications for Electronics, Electrical and Mechanical Engineering studies in both provinces. The respondent further stated that they are working with IYF, industry and other stakeholders in this process and have developed full details of two modules that will be embedded as common modules in all engineering studies. The current engineering TVET N4 – N6 studies are falling away, and these new qualifications will take its place in late 2023 onwards. The High Gear/QCTO partnership started mid-2021 and it is planned that all three curriculum framework upgrades will be completed by July 2022. In order to develop a new curriculum framework for the new qualifications, IYF industry consultants investigated a) what works in international and local engineering studies, and b) what is needed at industry level; i.e., where is industry currently and where is it moving to (trends) that will influence needed skills of engineers. The engineering curriculum framework includes details of the international syllabus and recommendations, industry trends and skills, a list of subjects and topics, outcomes resources needed for the new qualification at TVET colleges. High Gear’s industry consultants advise and discuss research and develop the final framework with the support of the Deputy Director: Qualifications Development at the QCTO. Overall, respondents noted that the High Gear programme had resulted in progress being made in terms of increased alignment between TVET programmes and industry needs. The factors that were highlighted as being important to enable this shift are outlined below: Building on the alignment work that took place prior to the High Gear Programme It is acknowledged that the progress outlined above, built on efforts that were already in place to align TVET curricula with industry needs for several years prior to the High Gear programme. A DHET respondent noted that the subject and assessment guidelines for the NC(V) Auto Repair and Maintenance course were reviewed and updated with effect from 2013 for Level 2, 2014 for Level 3 and 2015 for Level 4. A college management respondent said that they thought that the changes made in terms of the practical components of the Mechanical Engineering and Electrical Engineering courses had ensured that the programme is more aligned with industry needs. Further, one of the industry bodies confirmed that they had been part of an advisory body that developed the new curriculum through the QCTO/IYF partnership. However, DHET and TVET college staff noted that despite these improvements the IYF High Gear project is still important to address the challenges that remain in terms of the alignment of courses. Respondents emphasise that programmes such as NATED (N levels 1 – 6) mechanical and electrical courses remain outdated. For example, one DHET provincial respondent from KZN commented that, “there is no alignment …. therefore, as a department we do welcome whoever wants to assist including the IYF and High Gear”. The respondent indicated that, “they will have to assist under the developmental mindset …. in areas such as career guidance, practical learning and work integrated learning. The colleges need to get closer to 23 the point that the students are able to get work. On our side we are ready to push on this area of development”. These views recognise that curriculum change is slow, and efforts need to be sustained. Mutual understanding of structures, policies, and processes The alignment of TVET curricula with industry needs requires a degree of mutual understanding of the operational and policy contexts of each of the stakeholders involved in the alignment process. Respondents reported that building this basic level of mutual understanding was an important aspect of the High Gear Programme. For example, TVET college respondents noted that High Gear had contributed to creating a greater understanding of the TVET sector among industry and industry intermediaries in terms of how the curriculum is designed; where the curriculum stands and how the college operates in terms of DHET cycles. A respondent from the skills system discussed meeting with industry representatives about the role of TVET colleges. They noted that the meeting agreed that since the employers have their own learning academy that is focused on practical skills the TVET colleges can focus more on the conceptual and the foundational knowledge. The respondent commented that: “they are technical programmes and industry should be the key in terms of providing the industry requirements and work together with other stakeholders in the skills ecosystem to formulate the content that will align new qualifications with the industry requirements”. Respondents indicated that industry associations had a key role to play in supporting these processes of building mutual knowledge and understanding. They suggested that the industry associations should support engagements to build a common and shared understanding amongst companies of what they require from graduates of skills programmes and what it will take for these programmes to be aligned to their requirements. In this regard, a respondent spoke to the value of the process that is being championed by industry intermediaries to disseminate project learning and in so doing to strengthen linkages. For example, NAACAM stated that they have undertaken research, including a quarterly labour market survey, to understand the needs of the industry in an integrated manner. This is seen as an important mechanism to aggregate demand in ways that contribute to more coordinated planning. The findings of these surveys are also shared with members together with documents that contain information from IYF, the industry SETA, and DHET directly. In this way they further support the development of a shared understanding of skills development activities in the sector. Communication and coordination One of the key challenges noted by respondents in aligning TVET curricula with industry needs is communication and coordination between the various system stakeholders. This was viewed by some respondents as being one of the areas in which High Gear could have a particularly important impact. For example, a skills system respondent stated that, “IYF through High Gear is more like a catalyst, like a middleman who brings the department and industry together in a way that it was never done before. They bring people who understand the field, facilitators who understand the content and the way of delivering it”. Employer respondents indicated that it has previously been difficult to coordinate effectively between individual companies, industry associations, DHET, and TVET colleges and that, “High Gear is different … it’s very hard to coordinate between individual companies, industry association, DHET and TVET colleges that’s why High Gear is different it stands on a formal platform.” The potential for this 24 development was noted by an industry respondent who stated that the WIL intervention had enabled them to see the potential to improve coordination between the TVET colleges and companies in the sector through the involvement of intermediary bodies. Employer respondents also observed that there are existing relationships between industry bodies and employers that support communication and coordination within the sector and that they are kept informed of developments in a variety of ways, including for example, through the NAACAM LinkedIn page. The existence of structures supporting communication and coordination within industry may be an important basis from which to develop stronger links to the skills system. Increased levels of coordination and partnerships between industry and colleges is thus viewed as critical in ensuring that curricula are linked to industry requirements as well as technology demands. Rapidly changing technologies An important challenge to the alignment of TVET curricula with industry needs is the fact that the technologies employed in industry are continuously changing. A respondent from the skills system indicated that industry wants a partnership with colleges but that they had found that the colleges are not able to keep up with industry technological advancements. This challenge was highlighted by a provincial skills system respondent who stated that business and government tend to work in silos, and that there is a lag between the adoption of technologies within industry and the integration of those technologies into the skills system. A college respondent stated that the new technology takes time to filter into the college syllabuses and indicated that, “we still need to do more because what is happening in the industry is that technology is changing at a very high rate. We need to work with industry more to see how outdated our curriculum is, so that it can be revised to suit industry requirements. There also needs to be more focus on ICT programs related to the automotive industry as now with the current technology when diagnosing a car for example a computer is used and so this needs to be included in the syllabus so that the students can be able to use such technology in industry”. Changing technological requirements therefore present challenges for colleges both in terms of access to new technologies and the knowledge required to train learners to use them. For example, a TVET college respondent noted that, “we also have challenges with training resources; our workshops have outdated equipment and so when we have industry closer to us, they could donate some equipment to the colleges. I remember we visited the UK many years back we found colleges that are linked with industry and there are workshops inside the colleges like Mercedes workshop inside the colleges and the students are able to practice whilst they are still studying in college”. In terms of knowledge and skills, a respondent from the skills system stated that, “we have an old type of lecturing component, people who were trained in the 90’s, so they are not coping with the changes so they also need upgrading so there must be correlation between curriculum review and industry alignment and college capacity building and resourcing”. In response to the difficulty of keeping TVET curricula up to date with technological advancement in the automotive industry, a DHET respondent indicated that industry could play a role in putting more pressure on DHET to include a local content section in the syllabus. This would enable industry and colleges to fast￾track new industry requirements ensuring that staff and students are up to date. 25 The provision of a local content clause in DHET policy means that colleges are able to circumvent some of the long-term formal processes required to change syllabi, by including relevant content in curricula in the form of case studies or other similar structures. A respondent from DHET suggested that this could support the more rapid inclusion of changing technologies into TVET curricula: “there is a long turn around period from agreeing on new content for a subject to implementing this content as a formal part of a syllabus, normally a few years. This could be circumvented by including a “local content” section in the syllabus, maybe 25%, which would allow colleges, through a case study or similar, to teach what is required or requested by the automotive industry in the area. DHET consults with industry during syllabus updates, the problem is the time it takes for new syllabuses to be implemented from start to finish, which takes years”. Respondents recognised the value of High Gear in this regard, noting the usefulness of work that was done in the programme to support project-based learning in October and November 2021. This was coupled with training of the lecturers in the use of project-based learning, the use of demokits as well as in other ways to improve the alignment of the curriculum with industry needs. Timing of the process One respondent from the skills system observed that the timing of this focused effort to support the alignment of TVET programmes is very good as the QCTO is currently running a project to review N5-6 programmes. The respondent stated that they are working with IYF, DHET, industry and other stakeholders to support this review process. The respondent observed that these efforts need to be coupled with on-going training for college lecturers by NAACAM to enable these lecturers to keep up with the new technologies so that they can modify their lessons to align with what is currently happening in industry. Another respondent, from a college, also noted that support is needed to expedite processes of curriculum review so that the lecturers that are participating in the High Gear Programme will not get frustrated by the inability to teach content that is relevant to current industry practices and processes. Additionally, respondents comment that the ability of the TVET system to ensure that the students or graduates can adapt to new forms of technology and support the Automotive Sector requires a focus on addressing the miscommunication challenges between industry and TVET. These miscommunication challenges result in industry “failing to articulate how they (industry) are going to invest in new equipment and hence the TVET system does not know what industry wants”. Likewise, the engagement between TVET colleges and the private sector has been ineffective even though NAACAM has always existed and is highly organised; there have been no instances where TVET colleges approached NAACAM and asked for any guidance. This suggests the need for the implementation of a structured process supporting communication and coordination within the sector, such as regular quarterly meetings between key stakeholders focusing on changes within both industry and the TVET system. 26 Indicator # Of targeted technical assistance interventions delivered [Technical Assistance] Unit of Measure Number of technical assistance activities USAID/MSDF Eastern Cape UKPF KwaZulu Natal Baseline Endline Target Year 1 Target Year 1 Results Baseline Endline Target Year 1 Target Year 1 Results 0 6 3 5 0 6 3 6 The Baseline for this indicator was 0. In the planning, it was anticipated that six technical assistance activities would be delivered in each of the provinces by Year 3 (Endline target): 1. Curricula upgrades; 2. Work-integrated learning for TVET students and staff; 3. Industry-led career guidance services for TVET students; 4. Improved collection and dissemination of labour market data in the automotive components manufacturing sector; 5. Strengthening both TVET and industry’s use of existing job-candidate matching platforms; and 6. Supporting and informing QCTO’s reconstruction of NATED engineering qualifications. After the Baseline, interventions related to mentorship for TVET lectures, and the SETA Grant Advisory Services were added to the programme. In the Year 1 formative evaluation we found that in both provinces IYF has exceeded their targets for Year 1. As detailed in Table 1 in Section 1.2 above, the following technical assistance activities took place in the Eastern Cape and KZN: Table 6: Technical Assistance Interventions Delivered Eastern Cape KwaZulu-Natal Endline target Year 1 target Year 1 achiev. Endline target Year 1 target Year 1 achiev. Curricula upgrades 1 1 1 1 Work-integrated learning for TVET staff 1 1 1 1 Work-integrated learning for TVET students 1 0 1 0 Industry-led career guidance services for TVET students7 1 1 1 1 Improved collection and dissemination of labour market data in the automotive components manufacturing sector8 1 1 1 1 Supporting and informing QCTO’s reconstruction of NATED engineering qualifications. 1 1 1 1 7 Noted that focus groups were held to ascertain what would be beneficial to students in designing the Career Development platform. 8 Noted that IYF and NAACAM have conducted four quarterly labour market skills research processes. These have been split across the two provinces although all four were national. 27 SETA Grant Advisory Services 1 Total 6 3 5 6 3 6 Indicator # Of beneficiaries (government staff, institution staff, teachers, trainers, students, private sector organisation staff and/or NGO staff) receiving training, coaching or capacity building support, including strategy, regulatory and policy work [Capacity Building] Unit of Measure Number of people USAID/MSDF Eastern Cape UKPF KwaZulu Natal Baseline Endline Target Year 1 Target Year 1 Results Baseline Endline Target Year 1 Target Year 1 Results 0 1619 1000 279 0 2031 1150 487 The Baseline for this indicator was 0 across both provinces. Endline targets and the High Gear programme’s achievements in Year 1 are shown in Error! Not a valid bookmark self-reference. below. Year 1 targets were to train 1000 beneficiaries in the Eastern Cape and 1150 in KZN. Based on the data provided by IYF, it was found that High Gear achieved 28% of the Year 1 target in the Eastern Cape, and 42% of the Year 1 target in KZN. As shown below, the targets were exceeded for institution staff in both provinces, and for private sector or NGO staff in KZN, but the targets for students reached with the training were not met as the targets were based on the inclusion of NC(V) students who have since been excluded in the High Gear programme. In 2021 TVET staff participated in a regular series of capacity building workshops, as shown in Table 1 above, which benefitted 17 individual staff members in the Eastern Cape and 26 individual staff members in KZN. NATED students were exposed to the Project Based Learning (PBL) approach and private sector and NGO staff received training as part of the High Gear SETA Grants Advisory Services, as shown below: Table 7: Training Targets and Year 1 Achievements Eastern Cape KwaZulu-Natal Endline target Year 1 target Year 1 achiev. Endline target Year 1 target Year 1 achiev. Institution staff, teachers, trainers 15 17 17 26 Students 1600 262 2000 433 Private sector or NGO staff 4 0 14 28 Total 1619 1000 279 2031 1150 487 28 Indicator # Of beneficiaries (government staff, institution staff, teachers, trainers, students, private sector organisations, NGOs) meaningfully engaged in partnerships, engagement and convening activities [Partnerships, Engagement & Convening] Unit of Measure Number of people USAID/MSDF Eastern Cape UKPF KwaZulu Natal Baseline Endline Target Year 1 Target Year 1 Results Baseline Endline Target Year 1 Target Year 1 Results 0 142 46 29 0 144 60 116 The Baseline for this indicator was 0 in both provinces. The focus of this indicator was on the establishment of TVET – Industry working groups in each province. These working groups aimed to include members from government, TVET colleges, industry and industry intermediaries, and students or recent graduates. The Year 1 formative evaluation found that High Gear achieved 63% of the Year 1 target in the Eastern Cape and exceeded the Year 1 target in KZN by 93%, as shown in Error! Not a valid bookmark self-reference. below: Table 8: Number of people who meaningfully engaged in partnerships, engagement and convening activities Eastern Cape KwaZulu-Natal Endline target Year 1 target Year 1 achiev. Endline target Year 1 target Year 1 achiev. Government staff 5 0 5 6 Institution staff, teachers, trainers 35 16 37 62 Graduates 2 6 2 12 Private sector or NGO staff 100 7 100 36 Total 142 46 29 144 60 116 Over and above the realisation of targets, interviewees in the formative evaluation spoke to the value of partnerships to support improved alignment between TVET courses and industry requirements. One industry intermediary reported that this engagement with the High Gear programme has laid the foundation for them to engage with colleges more effectively and has given them the opportunity to collaboratively review course content with lecturers and identify areas that are outdated. The respondent commented that, “when we started with the High Gear project, they brought in employers from industry and lecturers, and we had to sit down and look at the syllabus and the topics covered and people from industry who have been in industry for many years were able to indicate that some of the topics in the syllabus are no longer used in industry”. 29 3.1.2 Short-term Result 1.1.1: Industry partners incorporate TVET partnership objectives into their strategic plans. Indicator # of industry partners (NAACAM, industry intermediaries and employers) that report elements of the High Gear model are integrated into their institutional strategies. Unit of Measure Number of industry partners that report High Gear project learning, relating to TVET and sex approaches integrated into their institutional strategies USAID/MSDF Eastern Cape UKPF KwaZulu Natal Baseline Endline Target Year 1 Target Year 1 Results Baseline Endline Target Year 1 Target Year 1 Results 0 3 2 5 0 10 2 3 The Baseline for this indicator, ‘number of industry partners (NAACAM, industry intermediaries and employers) that report elements of the High Gear model are integrated into their institutional strategies’ was 0 for both the Eastern Cape and KZN. The target for the number of industry partners reporting that elements of the High Gear model are integrated into their institutional strategies is higher in KZN than in the Eastern Cape as there are many more employers in KZN. The Endline targets are explained as follows: • In Eastern Cape the Endline target is 3, including 2 industry intermediaries and 1 large firm – a supply chain leader. • In KZN the Endline target is 10, including 2 industry intermediaries, 2 large firms – a supply chain leader and one other, and 3 small and 3 medium employers. Year 1 targets were exceeded in both Eastern Cape and KZN. The Year 1 targets for the Eastern Cape and KZN were two industry partners in each province integrating elements of the High Gear model into their institutional strategies, and the formative evaluation found that five industry partners in the Eastern Cape and three in KZN were actively undertaking this integration process. To understand the extent that industry partners are beginning to integrate elements of the model into their institutional strategies we interviewed company and industry association representatives. Over and above the High Gear programme target, respondents from DHET, college management and the SETA also indicated the ways in which the programme has begun to cause shifts in their strategies. Industry partners indicated that they have begun to make changes in their strategies. These include: • An increased willingness among companies to work with TVET colleges and to involve DHET in this process as a way of strengthening these partnerships. • An increased commitment to getting industry subject matter experts to go into the TVET colleges to guest lecture. • An increased recognition within industry intermediaries and companies of the importance of supporting work-integrated learning for lecturers. Employers also confirmed their preparedness to provide specialised input to college lecturers and one indicated that they have held an open day for lecturers to get an opportunity to be exposed to new technology. These initial changes are already being recognised by skills system stakeholders including the TVET colleges. One respondent spoke to the changes already experienced in the wider eco-system because of the High Gear programme. For example, they indicated that the SETA has identified the need to start engaging more systematically with the DHET about ways to capacitate the TVET colleges. However, the respondent observed that there is still a need to involve other SETAs who are involved in other parts of the automotive value chain. 30 Another respondent, from a TVET college, stated that prior to High Gear there had been a reluctance from the employers to work with the TVET colleges and that since the programme’s implementation, industry has become more open to partnering with colleges. They also noted that DHET is becoming more supportive and now visits the college regularly to monitor progress through the High Gear project. While these findings are positive, an industry intermediary respondent cautioned that for these changes to be institutionalised there is a need for certainty and a commitment that the working relationship that has been established as part of the High Gear programme is not “reduced to a once off project”. The respondent commented that, “the nice thing about High Gear is it’s 4 years in duration, there’s guaranteed funding, it’s got full time staff [and] at least there is some certainty that the efforts are going to be consistent for the immediate future”. The respondent reiterated that there is still a need to ensure that these relationships are expanded and sustained post the programme. 3.1.3 Intermediate Result 1.2: Key partners access South African funding and resources to sustain and scale the High Gear model. Indicators: Amount of public and private South African funding that partners (including NAACAM and DHET) access for implementation of High Gear programming (or related projects) # of industry partners that have accessed funding to advance inclusive workplace practices and policies (indicator applies only to UKPF-funded portion of High Gear) Indicator Amount of public and private South African funding that partners (including NAACAM and DHET) access for implementation of High Gear programming (or related projects) Unit of Measure Amount of money in Rands, USD and GBP USAID/MSDF Eastern Cape UKPF KwaZulu Natal Baseline Endline Target Year 1 Target Year 1 Results Baseline Endline Target Year 1 Target Year 1 Results 0 20% year￾on year increase in funding 5% R8 661 900 (total for both provinces) 0 20% year￾on year increase in funding 5% R8 661 900 (total for both provinces) The Baseline for this indicator is 0. The target for the amount of public and private South African funding that industry group partners access for implementation was based on a year on year % increase which will be monitored through NAACAM and AIDC-Eastern Cape reporting going forward. The Year 1 formative evaluation found that some positive developments have already been reported by industry intermediaries. These include: • Funding has been accessed to support additional elements of the High Gear programme: for example, funding from UNDP to build a career experience platform. This is seen as an integral part of the High Gear programme as it aims to assist the employment seeking journey of young people. • NAACAM has applied to merSETA to fund the extension of High Gear into other provinces. • Funding applications have been submitted to other SETAs and agencies. Respondents indicated that efforts to source and find additional funding are driven by their commitment to ensure the sustainability of High Gear. • Most company respondents indicate that they receive funding through merSETA, and a few companies indicated that they received funding from the Jobs Fund and another from the Technology Innovation Agency (TIA). 31 Industry intermediaries reported five applications for funding from South African public sector donors, as shown in the table below. Table 9: Status of Applications for Public Sector Funding Theme Amount requested Date submitted Status Lecturer development R8 250 000 30 June 2021 Under review Bursary, apprenticeships, and re-skilling R8 661 900 30 June 2021 Granted TVET lecturer development R6 336 000 29 April 2022 To be evaluated TVET graduate access to infrastructure R2 835 000 29 April 2022 To be evaluated Supporting the maintenance and iterative ‘build’ of Yakh’iFuture R2 481 813 29 April 2022 To be evaluated One respondent from the skills system stated that this component of the project is important and stressed the need for the High Gear programme to work with the SETAs across the value chain to unlock the funds that are required to support skills development effectively in the auto industry. There were also other recommendations made about possible funding opportunities and respondents suggested that IYF is ideally placed to assist with this process as it has connections globally. NAACAM indicates that they have accessed funding to extend the High Gear programme to additional provinces and are in the process of applying for additional funding from SETAs and international donors. Four companies in Eastern Cape and two companies in KZN also indicate that they have accessed public funding for the implementation of related projects but are unable to disclose the exact amount at this stage. To assist companies in understanding funding mechanisms and application processes, IYF has compiled the High Gear SETA Best Practice Guide that was shared on NAACAM platforms. IYF also held workshops with employers to explore the different funding mechanisms that are in place to support skills development in industry and extended High Gear SETA Grant Advisory services to employers that aimed to assist companies in understanding the compliance requirements, how to apply for funding with merSETA, submission areas that they are having challenges with, and capacity building with personnel who perform these tasks internally. IYF is intending to extend services to other interested companies across all provinces as well as automotive associated sectors. In this regard, NAACAM has identified plastics as the next biggest contributor to the automotive sector to engage further in High Gear. Indicator # of industry partners that have accessed funding to advance inclusive workplace practices and policies (indicator applies only to UKPF-funded portion of High Gear) Unit of Measure Amount of money in Rands, USD and GBP USAID/MSDF Eastern Cape UKPF KwaZulu Natal Baseline Endline Target Year 1 Target Year 1 Results Baseline Endline Target Year 1 Target Year 1 Results 0 10 firms 2 firms 0 The Baseline for this indicator is 0. The target is 10 firms, which will be monitored through the SETA Grant Advisory Services that are being put in place as part of the High Gear programme. None of the employers interviewed from KZN have accessed funding to advance inclusive workplace practices and policies. However, companies indicate that there is a growing awareness of the importance of building a more inclusive workplace. In this regard IYF has engaged employers in KZN in an online information sharing session on issues related to inclusive workplace practices. Additionally, IYF has also provided industry with High Gear SETA Grant Advisory services that aim to advise companies on how they can improve their BBBEE levels. 32 During the Year 1 formative evaluation interviews, an industry intermediary respondent indicated that the area of inclusive workplace practices and policies has not yet been fully explored. They expressed confidence that this work will be supported through the High Gear interventions over the course of the programme’s implementation. Respondents confirmed in the formative evaluation that implementing inclusive workplace practices and policies is a priority but noted that they have not yet achieved the intended targets with respect to young people or women. They also commented that their involvement in the High Gear programme has helped them to think about inclusive workplace practices and policies in the following ways: • The partners involved in the IYF High Gear programme have interacted with the Office of the President to explore how High Gear as a project could assist to promote diversity and inclusion in all sectors with a particular focus on women. • NACAAM’s first quarterly survey in February 2022 focused on diversity and inclusion and specifically explored the extent to which the manufacturing environment can create spaces for persons with disabilities. • One of the members of the intermediary association is exploring the possibility of focusing their work on advising companies on BBBEE optimisation. • Additionally, an industry body is working to establish a recruitment channel for people with disabilities and is developing a database of people with disabilities who have had prior experience in the auto industry, which will be accessible to member companies to employ or train the candidates. • The college respondents also indicated that there has been an increase in the enrolment of young women in the study fields related to the automotive industry. The colleges, with the support of High Gear, are deliberately aiming to recruit women engineering lecturers as well as focusing on the placement of young women students in industry. Respondents noted that the COVID-19 pandemic had presented a challenge to their work on inclusive workplaces, with one respondent stating that: “Companies in the past 2 years are just trying to survive … since the pandemic started companies have taken a huge financial knock and as much as they want to focus on inclusive workplace practices and policies it is difficult for them because it’s about survival to a large extent”. Going forward the IYF will engage with firms to get this level of data through the IYF SETA grant advisory services. Short-Term Result 1.2.1: Partner industry intermediaries and companies submit funding proposals aligned to the High Gear model. Indicators: SETA indicates an increase in employer proposals that include TVET college partnerships (indicator applies only to UKPF-funded portion of High Gear) # of industry group partners indicate that High Gear has supported them to access increased skills development funding. Indicator SETA indicates an increase in employer proposals that include TVET college partnerships (indicator applies only to UKPF-funded portion of High Gear) Unit of Measure Review of employer proposals that include TVET partnerships USAID/MSDF Eastern Cape UKPF KwaZulu Natal Baseline Endline Target Year 1 Target Year 1 Results Baseline Endline Target Year 1 Target Year 1 Results TBD TBD NA NA 33 In relation to this indicator, it is noted that there is no Baseline measure for this as IYF was awaiting details from merSETA. However, during the Year 1 formative evaluation it was found that there is no clear evidence as to whether the number of applications has increased as the SETA indicates that they do not have this data. Further a respondent from the merSETA national office indicated that the quality of employer proposals is determined by a guideline to be used in Discretionary Grant applications and that the proposals received from employers in the automotive sector generally reflect these guidelines. As mentioned previously, in 2021 IYF engaged companies through the introduction of the High Gear SETA Best Practice Guide in KZN. A SETA respondent indicated that there is not really a process in place to review the quality of these submissions, but SETA indicates that employer proposals are generally aligned to the SETA guidelines. The focus of the funding is stipulated in the guideline and include the following: • Higher Education Institution partnerships: interventions will include, but are not limited to graduate placement, work integrated learning, short learning programmes, research, and 4IR. • TVET college interventions: TVET college support for occupational programmes; equipment and workshop infrastructure; bursaries for TVET college lecturers to improve their technical or pedagogy skills through qualifications or part qualifications; TVET college managers training on curriculum related studies; credit bearing or non-credit bearing skills development programmes; TVET college lecturers support on industry exposure skills development programmes, projects, or activities; and research projects for the TVET sector. A respondent at provincial level (KZN) indicated that merSETA funds numerous skills development programmes within the TVET college space including: learnerships, apprenticeships, occupational qualifications and bursaries as well as the development of the institution through the Centres of Specialisation (COS) where they work in partnership with the TVET Colleges, Department of Higher Education and Training and companies. However, the respondents noted that there is always room for improvement in terms of strengthening these partnerships commenting that, “TVET colleges don’t necessarily take advantage of all the opportunities that are out there for them as funding available from SETAs”. As an example of this challenge the respondent noted that merSETA allows TVETs to apply for infrastructure development funding, but they indicated that there were very few applications against this option despite the extent of the need. The SETA respondent noted that they are unsure why the colleges did not take up this offer but suggested that it could be because they did not have the capacity to complete funding proposals or that the communication that was sent to all the college principals was not well understood. In order to track this indicator, the SETAs are being asked to provide this data to the IYF. Indicator # of industry group partners indicate that High Gear has supported them to access increased skills development funding. Unit of Measure Number of partnerships USAID/MSDF Eastern Cape UKPF KwaZulu Natal Baseline Endline Target Year 1 Target Year 1 Results Baseline Endline Target Year 1 Target Year 1 Results 0 2 2 1 0 2 2 0 It was noted during the Baseline that it was premature to evaluate the impact of the High Gear programme in supporting companies to access additional funding. It is also noted that the Endline target is to ensure that two employers or industry intermediary associations in both the Eastern Cape and KZN provinces access increased funding. NACAAM indicates that High Gear has supported them to access increased skills development funding, and they have had one successful application to run a bursary programme for Electrical and Mechanical Engineering students, an apprenticeship programme and a programme for reskilling retrenched 34 workers, which is called the “Road to Reskilling”. This programme was conceived as a means of reskilling and recapacitating 300 previously retrenched workers in the automotive components manufacturing sector to support their re-entry into employment by providing them with priority skills as identified in the sector skills plan Eastern Cape and Gauteng and has therefore been reported under the Eastern Cape in the High Gear Results Framework. None of the employers across Eastern Cape and KZN reported that they have received support to access increased skills development funding through High Gear in 2021. As mentioned previously in 2021 the High Gear SETA Grant Advisory services has initiated its work in KZN in hosting information sharing sessions on SETA discretionary and mandatory grants, SETA applications, Workplace Skills Plans, and Annual Training Report submissions and hopes to extend this work to other provinces in 2022. 3.1.4 Short-Term Result 1.2.2: Partner industry intermediaries disseminate project learnings. Indicator # of instances (e.g., forums, publications, social media) in which industry partners disseminate High Gear learning. Unit of Measure Number of times learning is shared USAID/MSDF Eastern Cape UKPF KwaZulu Natal Baseline Endline Target Year 1 Target Year 1 Results Baseline Endline Target Year 1 Target Year 1 Results 0 36 12 123 0 36 6 125 The Baseline for this indicator is 0, and the Endline target is that new learning is shared 36 times in each province, or approximately once a month in each province for the duration of the programme. In Year 1 the target is that new learning is shared 6 times in each province. These targets, and indeed the overall programme targets, were significantly exceeded in both provinces, which evidences the enthusiasm and buy-in from stakeholders to the programme. A review of IYF media records during the Year 1 formative evaluation reveals that: • In the Eastern Cape High Gear learning has been shared 123 times. High Gear featured in 80 industry￾led social media posts in 2021, along with 32 print and online articles that have national coverage, finally, High Gear also featured in 11 Eastern Cape-based print and online publications. • In KZN High Gear learning has been shared 125 times. High Gear has featured in 80 industry-led social media posts, along with 32 print and online articles that have national coverage as well as 13 KZN￾based print and online publications. In addition to this, High Gear learning has featured in national and provincial media including the NACAAM social media platforms and website, Daily Maverick, Engineering News SA, KZN Industrial & Business News, Nelson Mandela Chamber of Business, Automotive Business Review, and the Retail Motor Industry Association’s newsletter. Some of these articles are provided in the following links: o High Gear aligning TVET system with industry demand (dailymaverick.co.za) o High Gear identifies barriers, opportunities for women to enter automotive component manufacturing (engineeringnews.co.za) o Engineering News - High Gear – SETA best practice o Plan to put auto industry skills into High Gear - KZN Industrial & Business News (kznindustrialnews.co.za) o EASTERN CAPE ENGINEERING STUDENTS MOVE INTO HIGH GEAR (nmbbusinesschamber.co.za) 35 3.2 Objective 2: TVET students have equitable access to high-quality, relevant training and improved employment opportunities Indicators: % of individuals with new employment following participation in donor-assisted workforce development programmes. #/% TVET graduates that access higher education following participation in High Gear￾assisted courses. #/% of TVET graduates that indicate their qualification provided them with the skills and tools to access employment % of individuals with improved soft skills following participation in USG [or UKPF] assisted workforce development programs Indicator % of individuals with new employment following participation in donor-assisted workforce development programmes. Unit of Measure Percentage of graduates USAID/MSDF Eastern Cape UKPF KwaZulu Natal Baseline Endline Target Year 1 Target Year 1 Results Baseline Endline Target Year 1 Target Year 1 Results 27% of respondents reported new employment 34% of respondents report new employment, 12-months ex￾post 29% 6% of respondents reported new employment 22% of respondents reported new employment 28% of respondents report new employment, 12-months ex￾post 24% 5% of respondents reported new employment The Baseline graduate CATI found that 22% of respondents from EMC and 27% of respondents from Elangeni had found employment since completing their qualifications. The Endline targets set are a 25% improvement on Baseline, given High Gear’s limited ability to impact directly on this indicator due to the wide range of factors which affect the likelihood of graduates finding employment9 . With regards to Year 1 targets neither Eastern Cape nor KZN achieved the set targets. Given the context of the challenge of unemployment in SA, the length of time it takes to get a job and the impact of COVID-19 on employment opportunities, interventions are in place in terms of linking young people with job opportunities. The Yakh’ifuture platform will provide access to SA youth.mobi for them to apply for jobs. There was an agreement with NACAAM to get employers to post entry level jobs on SA youth.mobi. Employer guest lecturing, CV building workshops as well as WIL placements are also other High Gear interventions to address this indicator. Additionally, the High Gear tracer study will also contribute to addressing this indicator as graduates will be tracked 6 months post completion. In the 2021 formative evaluation the graduate CATI found that only 6% (N: 10) of respondents from EMC and 5% (N: 8) of respondents from Elangeni that had exited at N4, N5, and N6 levels are in new employment. 71% of the graduates across both colleges are currently studying (EMC N: 122 and Elangeni N:116), and 24% are unemployed (EMC N: 39 and Elangeni N:40), as shown below: 9 The definition of employment includes a contract or indefinite position; full-time or part-time position; and paid learnership, apprenticeship, or internship. 36 Table 10: Current Status of Survey Respondents What are you doing now? EMC Elangeni Total Employed (including self-employed) 10 (6%) 8 (5%) 18 (5%) Studying 122 (71%) 116 (71%) 238 (71%) Unemployed 39 (23%) 40 (24%) 79 (24%) Total 171 164 335 As shown above, only 5% of all learner respondents indicated that they were employed. However, 71% of respondents are still at the college, so 19% of those who have left college are employed (on average 1-3 months after leaving college). It is, though, noted that the 1-year student CATI was undertaken 1 – 3 months after the respondents would have completed a semester. In comparison, the Baseline CATI was undertaken 6-12 months after the students completed, and this Baseline figure determined the targets set. A tracer study will be conducted in June/July 2022 which will provide a comparable number. In terms of time taken to find a job for those that are employed, further analysis of the survey data reveals that of the 18 graduates that are employed, 3 from EMC were already employed (or self-employed) while studying. However, it took 1-12 months for the remaining 7 graduates to get a job. For Elangeni it took 1-12 months for the 8 graduates to get a job after completing their qualifications as shown below. Table 11: Time Taken to Find a Job Time taken to find a job EMC Elangeni Total N/A I was employed (or self-employed) while studying 3 0 3 1 month 2 3 5 2 months 2 0 2 4 months 1 0 1 5 months 1 0 1 12 months 1 5 6 Total 10 8 18 Of the 18 respondents who have found work, three indicated that they are in full-time employment, eight that they are in part time employment, while seven indicated that they are in short-term contracts, as shown below: Table 12: Type of Employment Contract Type of job EMC Elangeni Total Full time 2 1 3 Part time 3 5 8 Short-term contract 5 2 7 Total 10 8 18 37 A closer look at the 18 respondents that are employed further reveals concerning findings that only four of the respondents are working in their field of study: Table 13: Relationship between Qualification and Job Is your job related to your qualification(s)? EMC Elangeni Total Yes 4 0 4 No 6 8 14 Total 10 8 18 Taking this further, Error! Not a valid bookmark self-reference. below shows the job titles of those that are employed: It can be seen that very few respondents are in technical roles. Table 14: Job Titles of those Employed What is your job title? EMC Elangeni Total Teachers Assistant 2 4 6 Salesperson 1 0 1 Musician 1 0 1 Learnership 1 0 1 Occupational Health Inspector 1 0 1 Electrician 1 0 1 General worker 1 0 1 Facilitator 1 0 1 Postman 1 0 1 Hair stylist 0 1 1 Chef 0 1 1 Waitress 0 1 1 Unknown 0 1 1 Total 10 8 18 The evaluation also sought to understand the respondents´ monthly income. Of the 18 respondents who are employed, most indicated that they earned R8 000 per month or less. This is in line with average earnings for young people, which are between R3 648 (min average) and R5 980 (max average)10. Four respondents preferred not to share their monthly income. The table below shows the respondents’ monthly salary per college. Table 15: Monthly Income of those Employed Monthly income EMC Elangeni Total Less than R3000 1 4 5 R3000-R8000 4 4 8 R13000-R18000 1 0 1 Prefer not to say 4 0 4 Total 10 8 18 10 Harambee presentation on income benchmarks, reflecting data from the My Employment Journey survey, between December 2019 and February 2022 38 The 18 employed respondents were asked if they contribute to their household income, as a measure of broader impact of participation in the programme. While 11 of the 18 respondents indicated that they do contribute to their households, only eight of these 11 said that they were contributing more than before they participated in the programme. Table 16: Contribution to Household Income Contribution to Household EMC Elangeni Total Yes, but I contribute less; 1 2 3 Yes, I contribute more; 3 5 8 I don’t contribute; 1 0 1 No, nothing has changed; 4 1 5 Total 10 8 18 For those 79 respondents that are currently unemployed, the survey asked the reasons why they think they are unemployed. Of concern is that 27% of the respondents indicated that they thought that there were no job opportunities for their qualifications. Table 17: Perceived Reasons for Unemployment Why are you unemployed? EMC Elangeni Total There are no job opportunities for my qualification 12 (30%) 9 (23%) 21 (27%) There is not enough information about available jobs 4 (10%) 8 (20%) 12 (15%) I recently graduated and am still looking for a job 8 (20%) 11 (28%) 19 (20%) I don’t know the right people (connections) 7 (18%) 6 (15%) 13 (16%) I was working but my contract ended / I lost my job 4 (10%) 2 (5%) 6 (8%) I am still studying 4 (10%) 2 (5%) 6 (8%) Health Problems 1 (3%) 0 1 (1%) I am helping out at the college with tutoring 0 1 (3%) 1 (1%) Total 40 39 79 39 Unemployed respondents were further asked what they think could have been done to increase their chances of finding employment. The following comments were provided: EMC Respondents Elangeni Respondents • I think if the college can meet us halfway by providing in-service training it will really help us • If I could get experience things will be better • To trade test for free • To submit our CVs at the college for internship opportunities organised by the college • To get a driving licence, • To get practical experience at the college • To get apprenticeship opportunities • To get that 18-month in-service training • The college should make the syllabus to be more relevant like the ones used by universities of technology • To do more practicals in my qualification • To get job exposure • The college should organise partnerships and implement apprenticeship programmes • If the college can assist with workplace exposure or maybe ask for assistance from the government • Exposure to workshops more to prepare us for the workplace, also to get more advanced to equipment in the workshops • The institution should organise workplace experience • The college should post or advertise job opportunities • After results are issued, there should be companies that provide career guidance at the institution • College should give NATED students a chance to go workshops because its only being offered to NC(V) students Indicator #/% TVET graduates that access higher education following participation in High Gear-assisted courses. Unit of Measure Percentage of TVET graduates and number of TVET graduates USAID/MSDF Eastern Cape UKPF KwaZulu Natal Baseline Endline Target Year 1 Target Year 1 Results Baseline Endline Target Year 1 Target Year 1 Results11 41% (average) of NATED candidates accessed Higher Education N4 to N5 – 55% N5 to N6 – 27% N6 to other – 0% NC(V)4 to other – unknown 62% (average) of TVET graduates access higher education, 6- months ex-post 48% 44% (average) of NATED candidates accessed Higher Education N4 to N5 – 24% N5 to N6 – 65% N6 to other – 0% 34% (average) of NATED candidates accessed Higher Education N4 to N5 – 40% N5 to N6 – 28% N6 to other – 0% NC(V)4 to other – unknown 51% (average) of TVET graduates access higher education, 6- months ex-post 40% 31% (average) of NATED candidates accessed Higher Education N4 to N5 – 12% N5 to N6 – 50% N6 to other – 0% The Baseline is the average percentage of NATED N4 and N5 candidates that enrol for the next N level, based on data supplied by the colleges. The Baseline for N6 candidates who enrol in other higher education programmes is 0%, as per the graduate CATI interviews. The target is a 50% improvement on Baseline. 11 The data received from Elangeni only covers Electrical N4 in T1, the Mechanical N4 only start in T2 so we can’t track their completion in the calendar year. 40 The data shows that the Year 1 targets were just missed at both colleges. However, the Elangeni result may be affected by some gaps in the data provided by the college. The Elangeni data seems to only be Electrical N4 in T1, the Mechanical N4 only seem to start in T2 which makes it tricky to track completion in the calendar year. Indicator #/% of TVET graduates that indicate their qualification provided them with the skills and tools to access employment Unit of Measure Percentage of TVET graduates and number of TVET graduates USAID/MSDF Eastern Cape UKPF KwaZulu Natal Baseline Endline Target Year 1 Target Year 1 Results Baseline Endline Target Year 1 Target Year 1 Results 65% (average) of resp. who agree & strongly agree with three statements pertaining to skills and tools to access emp. 100% (average) of resp. who agree & strongly agree with all three statements 80% (average) of resp. who agree & strongly agree with all three statements 65% (average) of resp. who agree & strongly agree with all three statements 69% average) of resp. who agree & strongly agree with three statements pertaining to skills and tools to access emp. 100% (average) of resp. who agree & strongly agree with all three statements 80% (average) of resp. who agree & strongly agree with all three statements 69% (average) of resp. who agree & strongly agree with all three statements The Baseline and targets for this indicator were set from the CATI survey of graduates. The measure is the average percentage of ‘agree’ and ‘strongly agree’12 responses to the following three statements presented during the survey: • The course prepared me for the work environment • The course gave me the necessary skills and knowledge for the workplace, • The course has improved my chances of getting employment, As shown above, the Year 1 target has not been met in either the Eastern Cape or KwaZulu Natal. This is further demonstrated when looking at the individual responses to these three questions: Table 18: Comparing Baseline & Formative Evaluation Survey Responses to Statements on Improving Skills & Knowledge to Access Employment (Combined Agree and Strongly Agree) EMC Elangeni B/Line 2021 Formative B/Line 2021 Formative The course prepared me for the work environment 79% 57% 70% 68% The course gave me the necessary skills and knowledge for the workplace 79% 68% 75% 69% The course has improved my chances of getting employment 70% 70% 69% 71% 12 It is important to note that the Baseline evaluation only tracked ‘strongly agree’ responses. However, in the Year 1 evaluation it has been proposed both agree (4) and strongly agree (5) responses be included in the indicator. 41 These High Gear responses were explored in the qualitative research with High Gear graduates, and it was found that respondents were slightly more positive about the effect of the course on their ability to find employment than these responses show, although the most common concern raised by respondents was the lack of practical opportunities to gain skills, and the lack of workplace experience. In the research, respondents were asked if they thought the course improved (will improve) their chances of getting employment and in which ways did (will) the course increase their chances of finding employment. 12 out of 16 respondents from Elangeni indicated that they believed that the course improved (s) their chances of finding a job. Even those respondents that are currently unemployed said that they think the course will increase their chances of finding employment. Respondents explained that there are a lot of job opportunities in the field of engineering and that having completed a qualification advances their chances of accessing these jobs, adding that having completed a higher NATED level increases their chances even further as they are able to demonstrate their abilities to learn and that they have the foundation knowledge of engineering. A respondent commented that “every profession requires a qualification... I believe my qualification will make me more marketable compared to someone without a qualification”. However, four respondents said that they don't believe that the course alone increased their chances of finding a job, noting that they could have acquired more technical skills and that this was the area that the course lacked as it was only theory based. The issue of lack of technical skills was raised as a concern by those who also said they think their chances of employment are increased commenting that: “I did get the right skills to increase my chances of finding employment. However, the challenge is that I only got theoretical knowledge and I didn’t get practical skills during the course”. “Knowledge and skills work hand in hand and at college we only acquired knowledge and not skills because they never did any practicals”. Respondents further explained that a qualification is not enough as employers need work experience and therefore work experience would increase their chances of finding employment. A respondent commented that “Yes, but only after getting in-service training, the theory was good, but the challenge is that most companies don’t hire us because we don’t have practical experience”. On the other hand, all 14 Eastcape Midlands respondents indicate that while they gained theoretical skills that improved their chances of accessing employment, they felt that they lacked technical skills due to a shortage of practical opportunities. Respondents were also asked if they thought the NATED course prepared (s) them for the work environment. Nine of the 16 respondents from Elangeni indicated that they thought that the course had prepared them for the work environment, stating that they learnt the theory which is applied in the workplace. A respondent commented that “I learnt about safety measures are the first thing we applied when I was working”. Another respondent though also commented about the lack of technical skills and how that undermines their ability to perform in the workplace though the respondent acknowledges that theory is important and provides the necessary concepts that one will come across in the work environment. However, the other seven Elangeni TVET College respondents said that the course only partially prepared them and went on to state that the lack of practical opportunities meant that they cannot apply the knowledge or demonstrate technical skills in the work environment. Respondents further explained that they learnt soft skills such as stress management, time management, communication, teamwork and being reliable: “The lecturers did make us aware of the challenges that we may encounter in the workplace and taught us on what we can do to cope with work-related stress”. Likewise, Eastcape Midlands College respondents indicated that while the course prepared them for the work environment, 11 of the 14 respondents noted that they think they have the right theoretical knowledge but acknowledged that they don’t think they have the right technical skills to apply in the workplace due to lack of practical opportunities during the course as well as lack of workplace exposure opportunities. 42 Respondents were also asked to share reasons why they chose to enrol for their respective courses. It is important to note that 30% of the respondents across both colleges indicated that they thought it would help them get a job and/or workplace experience. Another 25% said that they thought it would help them gain employability skills, both technical and soft skills. Table 19: Reasons for enrolling in Qualification (2021 responses only) Why did you choose to register for this qualification(s) in 2021? EMC Elangeni Total I thought it would help me increase my future earnings 3.8% 8.9% 12.8% I thought it would help me get a job and/or workplace experience 11.2% 15.7% 26.9% I thought it would help me acquire more knowledge and skills to do my job better 5.4% 11.0% 16.4% I thought it would help me gain employability skills (technical and soft skills) 11.9% 13.5% 25.3% I thought it would help me to start a business 2.4% 7.9% 10.3% I thought it would help me build connections with employers in the labour market 1.7% 4.5% 6.3% I had no expectations at all 1.9% 0.0% 1.9% In relation to the reasons offered above respondents were also asked if their expectations had been met: 73% of the 171 EMC respondents strongly agreed and agreed that their expectations were met while 98% of the 164 Elangeni respondents strongly agreed and agreed that their expectations have been met. Indicator % of individuals with improved soft skills following participation in USG [or UKPF] assisted workforce development programs Unit of Measure Percentage of TVET students with improved soft skills USAID/MSDF Eastern Cape UKPF KwaZulu Natal Baseline Endline Target Year 1 Target Year 1 Results Baseline Endline Target Year 1 Target Year 1 Results 94% (average) of resp. believe that part. in the course had improved their [listed] soft skills a fair amount or an enormous amount 100% (average) of resp. believe that part. in the course had improved their [listed] soft skills a fair amount or an enormous amount 95% (average) of resp. believe that part. in the course had improved their [listed] soft skills a fair amount or an enormous amount 85% (average) of resp. believe that part. in the course had improved their [listed] soft skills a fair amount or an enormous amount 93% (average) of resp. believe that part. in the course had improved their [listed] soft skills a fair amount or an enormous amount 100% (average) of resp. believe that part. in the course had improved their [listed] soft skills a fair amount or an enormous amount 95% (average) of resp. believe that part. in the course had improved their [listed] soft skills a fair amount or an enormous amount 91% (average) of resp. believe that part. in the course had improved their [listed] soft skills a fair amount or an enormous amount 43 This indicator is measured by average scores of the percentage of graduate respondents in the CATI survey who indicated that participating in the programme had improved their soft skills in a range of defined areas (listed in Error! Not a valid bookmark self-reference. and At Elangeni TVET College an average of 72% of respondents indicated that participating in the course had improved their listed soft skills an enormous amount; and an average of 19% that participating had improved their skills a fair amount, a total of 91% selecting either of these two options. Table 21 below) – respondents were asked the degree to which they thought that participating in the programme had improved their soft skills. In the Baseline an average of 53% of respondents from 171 EMC respondents and 46% of 164 Elangeni respondents said that they believed that their soft skills had improved ‘an enormous amount’ through participation in High Gear, and an average of 41% and 47% said that they thought that their soft skills had improved a fair amount. The Endline target is a 100% of respondents selecting one of these two options, and Year 1 targets are 95% in both provinces. During the 2021 formative evaluation, the total percentage of respondents selecting one of these two options was slightly below the target in both provinces. In EMC an average of 57% of respondents indicated that they thought that participating in the course had improved their listed soft skills an enormous amount, while 28% thought that it had improved their soft skills a fair amount. Overall, an average of 85% of respondents said that participating in the course had improved their soft skills a fair amount or an enormous amount. Table 20: EMC Respondents - Views on effect of participation on Soft Skills Eastcape Midlands College Ratings Participating in the programme helped me to develop the following competencies / soft skills. Completely disagree Disagree Neutral A fair amount An enormous amount I recognise how my feelings affect my behaviour 8% 5% 18% 36% 33% I complete tasks even when I don’t like them 5% 4% 13% 33% 46% I am able to take steps to reach my goal. 1% 2% 7% 30% 60% I stay calm even when I am stressed 3% 2% 13% 33% 49% I know what my strengths are. 1% 1% 6% 26% 65% I am willing to make changes when I am faced with a challenge 2% 2% 5% 21% 71% I listen actively to understand and learn 1% 2% 9% 27% 61% I value suggestions from others about how I can improve 1% 0% 8% 26% 65% I can work effectively with others in a team to accomplish a task 1% 2% 7% 20% 69% I can make compromises in order to resolve a conflict with others 0% 1% 11% 32% 57% I use technology to deepen my understanding 2% 2% 13% 32% 51% I collect a lot of information when making up my mind about something 1% 1% 11% 27% 60% I try to think of multiple solutions when faced with a challenging problem 2% 1% 13% 31% 54% 44 I apply new ideas in a practical way to improve the way things are done 3% 2% 18% 29% 49% I accept responsibility for my actions when I make a mistake or get in trouble 1% 0% 4% 23% 73% Average 2% 2% 10% 28% 57% At Elangeni TVET College an average of 72% of respondents indicated that participating in the course had improved their listed soft skills an enormous amount; and an average of 19% that participating had improved their skills a fair amount, a total of 91% selecting either of these two options. Table 21: Elangeni Respondents - Views on effect of participation on Soft Skills Eastcape Midlands College Ratings Participating in the programme helped me to develop the following competencies / soft skills. Completely disagree Disagree Neutral A fair amount An enormous amount I recognise how my feelings affect my behaviour 8% 3% 23% 21% 44% I complete tasks even when I don’t like them 5% 2% 8% 27% 57% I am able to take steps to reach my goal. 0% 1% 4% 14% 81% I stay calm even when I am stressed 2% 2% 10% 23% 63% I know what my strengths are. 1% 1% 3% 11% 84% I am willing to make changes when I am faced with a challenge 2% 1% 2% 16% 79% I listen actively to understand and learn 0% 1% 2% 15% 82% I value suggestions from others about how I can improve 1% 1% 3% 20% 76% I can work effectively with others in a team to accomplish a task 0% 0% 4% 13% 83% I can make compromises in order to resolve a conflict with others 0% 0% 8% 25% 67% I use technology to deepen my understanding 0% 0% 5% 22% 73% I collect a lot of information when making up my mind about something 0% 1% 4% 18% 76% I try to think of multiple solutions when faced with a challenging problem 0% 1% 7% 23% 69% I apply new ideas in a practical way to improve the way things are done 0% 1% 7% 29% 49% I accept responsibility for my actions when I make a mistake or get in trouble 2% 1% 4% 16% 78% 45 Average 1% 1% 6% 19% 72% In the qualitative research, six out of 16 Elangeni TVET College respondents indicated that there was no component of the course that formally covered soft skills they could apply in the workplace and two respondents who had completed an NC(V) before enrolling for the NATED programme commented that soft skills were covered during the Life Orientation and English subjects of the NC(V). And 12 of the 14 EMC respondents also said that there were no soft skills formally introduced during the course. Two respondents said that they learned soft skills such as communication, teamwork and self-confidence through class discussions and working with other students during the course. The Yakh’ifuture platform will include Passport to Success (PTS) courses to improve these results. Furthermore, the IYF has included PTS content into the work-readiness and customer service modules as well as relevant elements of the PBL and Career Guidance into the common modules and overall engineering curriculum frameworks. 3.2.1 Intermediate Result 2.1 (a): Students enrolled in upgraded TVET courses demonstrate improved academic performance Indicator Completion rate of TVET students in High Gear-assisted qualifications. Unit of Measure Percentage of students that pass USAID/MSDF Eastern Cape UKPF KwaZulu Natal Baseline Endline Target Year 1 Target Year 1 Results Baseline Endline Target Year 1 Target Year 1 Results NATED 28% NC(V) 6.7% NATED 42% NATED 33% 37%% NATED 21% NC(V) 4% NATED 32% NATED 24% 46% The Baseline for this indictor is an average of 28% of students at Eastcape Midlands College and 21% of students at Elangeni TVET College pass the NATED programme. This average was obtained by using the completion rates at each level of NATED – N4, N5, and N6 – since each level results in a certificate. Data on student academic performance received from EMC and Elangeni shows that Year 1 targets were exceeded at both TVET colleges. The average combined pass rate for N4 – N6 Mechanical and Electrical Engineering was 37% at Eastcape Midlands College, and 46% at Elangeni TVET College. Pass rates at Eastcape Midlands College generally ranged between 30% and 50% across levels N 4 – 6 of both Electrical and Mechanical Engineering courses. However, the pass rate for N6 Mechanical Engineering was only 20% which may warrant further investigation to identify whether there are specific factors at play affecting students’ performance. Table 22: EMC Completion Rates EMC Electrical Eng. Mechanical Eng. Overall Enrolled Passed % Pass Enrolled Passed % Pass Enrolled Passed % Pass N4 75 28 37% 53 23 43% 128 51 40% N5 76 23 30% 26 11 42% 102 34 33% N6 17 8 47% 15 3 20% 32 11 34% Total 168 59 35% 94 37 39% 262 96 37% Pass rates from Elangeni TVET College were also generally between 30% and 50% for both Electrical and Mechanical Engineering courses across levels N 4 – 6, with the exception of N6 Electrical Engineering which 46 had a pass rate of 75%. While this may simply be due to the selection effect of progression through each level of qualification resulting in more capable students remaining in the N6 level, it may be worth further investigating this result to identify any factors at play which could be applied to other courses with lower pass rates. Table 23: Elangeni Completion Rates Elangeni Electrical Eng. Mechanical Eng. Overall Enrolled Passed % Pass Enrolled Passed % Pass Enrolled Passed % Pass N4 167 67 40% 27 14 52% 194 81 42% N5 145 64 44% 18 6 33% 163 70 43% N6 60 45 75% 16 5 31% 76 50 66% Total 372 176 47% 61 25 41% 433 201 46% 3.2.2 Intermediate Result 2.1 (b): Employers have increased confidence that graduates from upgraded TVET courses meet their skills needs. Indicator #/% of employers who indicate that female and male graduates employed through the college perform better (than other new entrants who enter these occupations) in the workplace. Unit of Measure Number of employers USAID/MSDF Eastern Cape UKPF KwaZulu Natal Baseline Endline Target Year 1 Target Year 1 Results Baseline Endline Target Year 1 Target Year 1 Results 0 50% of interviewed employers 17% 83% of interviewed employers in Eastern Cape expressed a level of satisfaction with the quality of graduates from TVET College 0 50% of interviewed employers 15% 100% of interviewed employers in KZN expressed a level of satisfaction with the quality of graduates from TVET College The Baseline for this indicator ‘percentage and number of employers who indicate that female and male graduates employed through the college perform better than other new entrants who enter these occupations’ was 0 (not applicable) as graduates from High Gear assisted courses have not yet entered the labour market. The Endline target was set at 50%, and in fact the Year 1 results exceeded this Endline target in both provinces. Five employers were interviewed from Eastern Cape and two employers interviewed from KZN and this may require further investigation as the sample size is small. However, during the formative evaluation in 2021 a review of how college graduates compare to other graduates found that respondents from industry were generally very happy with the college learners. This may partly be due to their ability to pick and choose the top performing college graduates due to the excess supply of work seekers. Another company respondent pointed out that the challenges they experience with graduates of colleges are consistent with those of other new entrants as they generally do not have the requisite work readiness. 47 Two of the company respondents indicated that graduates of colleges generally have several skills gaps, which they must address when they are in their workplaces. Furthermore, one employer respondent observed that they generally accept that learners coming out of colleges do not have the level of practical exposure required to perform effectively in the workplace, and that this requires an intense induction process. Interestingly respondents from two companies indicated that taking learners from university “is a nightmare” and that “those coming out of the colleges do not mind getting dirty whereas someone from university for example, a mechanical engineer will think ‘I am supposed to be in the office’ – the TVET graduates want to be involved in the shop floor and work on weekends to clean machines and working during shutdown – they want to be where the fire is.” The majority of companies indicated that the turnover is low within this cohort, and this is also considered very positive. In terms of gender, some companies indicated that they have not reflected on the performance difference related to the gender of new entrants. There were a few companies that stated that there was no real distinction based on gender although one respondent stated that young women tend to take on more responsibility and “ask for more work.” The five employers interviewed from Eastern Cape and two employers interviewed from KZN expressed an average level of satisfaction with the quality of graduates from TVET colleges, indicating that they observed that there is a concern in terms of certain gaps in graduates’ technical skills, levels of work readiness and degree of workplace exposure prior to entering the workplace. 3.2.2.1 Short-Term Result 2.1.1: TVET colleges deliver engaging, industry-aligned & inclusion sensitive education to TVET students Indicators: #/% of TVET graduates that indicate that the quality and resources of the programme was of a high standard. # TVET students who receive updated course content. Indicator #/% of TVET graduates that indicate that the quality and resources of the programme was of a high standard. Unit of Measure Percentage of TVET graduates and number of TVET graduates USAID/MSDF Eastern Cape UKPF KwaZulu Natal Baseline Endline Target Year 1 Target Year 1 Results Baseline Endline Target Year 1 Target Year 1 Results 86% (average) of resp. who agreed and strongly agreed with all three statements on quality & resources 100% (average) of resp. who agreed and strongly agreed with all three statements on quality & resources 90% (average) of resp. who agreed and strongly agreed with all three statements on quality & resources 66% (average) of resp. who agreed and strongly agreed with all three statements on quality & resources 82% (average) of resp. who agreed and strongly agreed with all three statements on quality & resources 100% (average) of resp. who agreed and strongly agreed with all three statements on quality & resources 90% (average) of resp. who agreed and strongly agreed with all three statements on quality & resources 65% (average) of resp. who agreed and strongly agreed with all three statements on quality & resources During the Baseline study respondents were asked to rate three statements relating to the quality and resources of the programme. • The lecturers were knowledgeable & supportive • The quality of the learning materials was excellent 48 • The quality of technical equipment was excellent The Baseline represents the average responses of the percentages of respondents who agreed and strongly agreed with these statements, and the Endline target is that a 100% of respondents from both colleges agree or strongly agree with the statements. The Year 1 target is 90% for both colleges. Error! Not a valid bookmark self-reference. shows the formative evaluation survey responses to statements on quality and resources of the programme (Combined Agree and Strongly Agree). In both provinces, the year 1 target was not met. Table 24: High Gear Responses to Survey Questions on Quality & Resources of the Programme EMC Elangeni The lecturers were knowledgeable & supportive 84% 86% The quality of the learning materials was excellent 69% 63% The quality of technical equipment was excellent 46% 47% The qualitative research also sought to understand more about respondents’ perceptions around these issues. In line with the results of the CATI survey, most respondents from both colleges indicated that they found that their lecturers were knowledgeable and supportive. 14 out of 16 respondents from Elangeni agreed that the lecturers were knowledgeable and supportive: respondents referred to the lecturers' ability to explain concepts, the lecturers’ understanding of the trade as well as their willingness to answer their questions. Respondents also added that the lecturers were willing to go the extra mile to help them understand the course and conducted extra classes during the week and sometimes on weekends. The lecturers also collected additional study materials for learners, including downloading videos and other material from the Internet. Finally, the respondents also referred to the WhatsApp groups that they have with lecturers where they are able to ask questions, discuss topics and have consultations with lecturers. Only two respondents indicated that the lecturers are somewhat knowledgeable and are not always supportive: one respondent said that they thought that some lecturers do not understand the concepts fully and struggle to explain and just rely on the textbook. With regards to the 14 Eastcape Midlands respondents, 12 respondents reported that their lecturers were helpful and supportive and always encouraged them in the process and gave them extra lessons in their own time. Only two respondents said that they felt that the lecturers were not knowledgeable stating that they did not think they were trained in industry, they were not able to explain concepts and answer their questions, the N6 respondents further explained that their lecturers were not trained to teach this level of the NATED qualification and referred to the electro techniques lectures specifically. The respondents commented that: “the lectures were not that knowledgeable, and they also struggled to explain what is on the textbook and could not make examples or explain things differently”. When asked about the quality of learning material 11 of the 16 respondents from Elangeni indicated that the learning material was useful and up to standard and helped them to understand the course content and to prepare for exams. However, five respondents were not satisfied with the quality of the learning material, stating that there was not enough learning material and that the information covered by the learning material was not sufficient and, in many instances, they had to source information from Google as well as watch YouTube videos in order to help them understand the course content better. For instance, one respondent commented that the learning material did not cover all the topics, and as a result the lecturers would assist them by getting other learning material as well as previous exam papers to help them prepare for assessments and exams. Respondents also gave examples of subjects such as mechano techniques, power machines as well as electronic techniques stating that the information provided in these textbooks was insufficient for them to understand the lessons. One respondent commented that they had used a textbook for industrial and electronics from five years ago and preferred using this textbook because it had more information compared to their current textbook. 49 Six of the 14 respondents from EMC indicated that they thought that the learning material was useful and helped them to prepare and pass exams. However, eight respondents indicated that the textbooks were not entirely useful saying that the textbooks for electro techniques and mechano techniques were incomplete, commenting that “the examples were incomplete, and most information was missing…”. One respondent noted that “the teacher’s guide was better than the learner’s guide and we relied on copies made by lectures to find solutions to problems to prepare for the exams”. Some Electrical Engineering respondents said that the electro techniques textbook was changed in Trimester 3 of 2021 but not all learners received these new resources/guides, and some had to share these study resources. When asked about the quality of technical equipment concerningly, only one respondent from Elangeni mentioned that they once used a linear scale and that helped them to understand the subject better. The rest of the 15 respondents indicated that they had not used technical equipment as there were no practical opportunities for them and that they have never seen or touched technical equipment: “we learnt about machines and boilermakers, but we never saw them. Right now, I wouldn’t be able to point out a boiler maker” “I did not see any machinery for the duration of my course”. “We never saw or touched anything during the course i.e., gearbox, engine etc” Likewise, 11 of the Eastcape Midlands respondents said that the course did not have any practical opportunities. However, three respondents spoke about a lecturer who took them to the workshop that is designated for NC(V) students and commented that they did well in the subject because they saw and touched technical equipment and the lecture did demonstrations for them. Indicator # TVET students who receive updated course content. Unit of Measure Number of students USAID/MSDF Eastern Cape UKPF KwaZulu Natal Baseline Endline Target Year 1 Target Year 1 Results Baseline Endline Target Year 1 Target Year 1 Results 0 1600 240 262 0 2000 1100 433 The Baseline target is 0 for both Eastern Cape and KZN and the Year 1 Endline target is 1600 in Eastern Cape and 2000 in KZN. The Year 1 targets are 240 in Eastern Cape and 1100 in KZN. The formative findings in 2021 indicate that a total of 262 students from Eastcape Midlands College and 433 students from Elangeni TVET College have benefited from course upgrades. Eastern Cape has exceeded the Year 1 target by 9% while KZN only achieved 39% of the target. The Y1 target for KZN was particularly high because it was anticipated that NC(V) students would be reached, however this was not the case. IYF is aiming to make up these numbers in 2022 by exposing both NATED and NC(V) students nationally to mechanical and electrical engineering concepts with animated demos, formulas and revision quizzes on the Yakh'iFuture's Discover page. 3.2.2.2 Short-Term Result 2.1.2: TVET college staff are equipped and motivated to deliver the enhanced curricula and pedagogy Indicator #/% of TVET college lecturers that indicate the quality of their teaching has improved as a result of High Gear support Unit of Measure Number and percentage of college lecturers who received training USAID/MSDF Eastern Cape UKPF KwaZulu Natal Baselin e Endline Target Year 1 Target Year 1 Results Baseline Endline Target Year 1 Target Year 1 Results 50 0 75% of lecturers strongly agree that the quality of their teaching has improved as a result of High Gear support 25% 1 lecture neither agreed nor disagreed if the quality of their teaching has improved as a result of High Gear support 0 75% of lecturers agree and strongly agree that the quality of their teaching has improved as a result of High Gear support 20% 29% agreed and strongly agreed that the quality of their teaching has improved The Baseline for this indicator is 0 as the High Gear support had not yet begun. The question of whether or not lecturers think that the quality of their teaching has improved as a result of High Gear support was included in the 2021 lecturer surveys. The Endline target is 75% of lecturer respondents strongly agree with the statement that the quality of their teaching has improved because of participating in High Gear. The Year 1 target is 25% and 20% of lecturers from EMC and Elangeni respectively. During the Year 1 formative evaluation survey only one lecturer from EMC and 26 lecturers from Elangeni indicated that they received capacity development support from High Gear. It is important to note that a review of IYF records shows that 14 lecturers at EMC received capacity building support. The reason for this discrepancy in reporting participation in capacity building is unclear and should be further investigated in ongoing evaluation processes. Going forward CATI surveys will be conducted by Singizi to increase the response rates. An analysis of the responses from the one lecturer respondent from EMC shows that the respondent neither agreed nor disagreed if the quality of their teaching has improved as a result of High Gear support. An analysis of Elangeni lecturer respondents shows that an average of 29% of the respondents agreed and strongly agreed that the intervention gave them tools to improve their teaching techniques and methods. 3.2.2.3 Output 2.1.2.1: High Gear trains and coaches TVET staff to deliver improved courses. Indicator Number of individuals affiliated with higher education (tertiary) institutions receiving capacity development support with USG {or UKPF] assistance. * Unit of Measure Number of TVET lecturers who have received training on upgraded industry￾validated course content USAID/MSDF Eastern Cape UKPF KwaZulu Natal Baseline Endline Target Year 1 Target Year 1 Results Baseline Endline Target Year 1 Target Year 1 Results 0 19 Eng. and related lecturers 18 Life Orientation lecturers 19 Eng. and related lecturers 9 Eng. and related lecturer 0 52 Eng. and related lecturers 28 Life Orientation lecturers 25 Eng. and related lecturers 26 Eng. and related lecturers The Baseline for this indicator is 0, and the Endline target is that all the engineering and related lecturers and all the Life Orientation lecturers at each college receive training on upgraded industry-validated course content during the project. For Year 1 the target is that 19 lecturers from EMC and 25 lecturers from Elangeni receive capacity development support. Documents received from High Gear showed that 9 engineering and related lecturers from EMC and 26 engineering and related lecturers from Elangeni received training on upgraded course content, namely Project Based Learning, during Year 1. In addition, 9 lecturers from EMC and 26 lecturers from Elangeni received mentorship support in 2021. The Year 1 target was not achieved at EMC and was slightly exceeded at Elangeni. 51 Going forward the Endline target needs to be updated as the IYF has established that there are 14 Electrical and Mechanical Engineering lectures and in Elangeni there are 26 electrical and Mechanical Engineering lectures which make the initial targets set during the Baseline unachievable. Additionally given the exclusion of NC(V) then the Life Orientation lectures fall away. In terms of rating their experience and views on the mentorship the single EMC respondent strongly agreed that the mentorship was excellent, while 49% of the Elangeni respondents strongly agreed that the mentorship was excellent, and 35% agreed that it was excellent. 3.2.3 Intermediate Result 2.2: Students enrolled in upgraded TVET courses have expanded professional networks. Indicator #% of TVET students that have connections with employers that support their ability to seek employment Unit of Measure Number and percentage of learners USAID/MSDF Eastern Cape UKPF KwaZulu Natal Baseline Endline Target Year 1 Target Year 1 Results Baseline Endline Target Year 1 Target Year 1 Results 59% of resp. strongly agreed that part. in the course helped them to improve their conn. with employers 100% of resp. strongly agree that part. in the course helped them to improve their conn. with employers 80% of resp. strongly agree that part. in the course helped them to improve their conn. with employers 39% of resp. strongly agreed that part. in the course helped them to improve their conn. with employers 46% of resp. strongly agreed that part. in the course helped them to improve their conn. with employers 80 % of resp. strongly agree that part. in the course helped them to improve their conn. with employers 65% of resp. strongly agree that part. in the course helped them to improve their conn. with employers 46% of resp. strongly agreed that part. in the course helped them to improve their conn. with employers The Baseline and targets for this indicator were set from the CATI survey of graduates. The Baseline is the percentage of ‘agree’ plus ‘strongly agree’ responses to the question of whether the respondent thought that participating in the course had helped them to improve their connections with employers, and the Endline target is 100% for the Eastern Cape and 80% for KZN. In 2021 formative evaluation, 19% of respondents from EMC strongly agreed that participating in the course helped them to improve their connections with employers. At Elangeni 32% of respondents strongly agreed that participating in the course helped them to improve their connections with employers. Overall, 39% of EMC respondents and 46% of Elangeni respondents agreed and strongly agreed that participating in the course had improved their connections with employers. Thus, the Year 1 target was not met in either of the provinces. Graduates noted that TVET colleges could still do more to connect them with potential employers. For example, a respondent from EMC suggested that the college could send graduates CVs to employers, while an Elangeni graduate note that the college could try to create direct connections between students and potential employers. High Gear has some direct interventions around this planned through the Yakh’iFuture career experience platform and the mobile site SAyouth.mobi, which hopefully will assist students starting from 2022 onwards. In the qualitative research, when asked if the course had helped them to improve their connections with employers and whether they were opportunities to engage with employers during their programme, the majority of the 16 respondents from Elangeni (14) indicated that there were no opportunities to engage with employers and suggest that the college needs to partner with employers to provide them with workplace 52 experience opportunities as well as to visit the college and provide guest lecturers as well as talk them through career opportunities in engineering fields. Comments included: “The college should communicate with companies on our behalf for potential in-service training opportunities”. “The college should invite companies to schools and host career expos”. Only two respondents reported that they had engaged with employers: one said that they were once visited by Toyota to provide career advice at the college, and the other indicated that the lecturers connected them with an employer to apply for work experience opportunity and they were successful. Similarly, five of the 14 respondents from Eastcape Midlands also indicated that there was less engagement with employers during their studies than they would have enjoyed. However, four respondents were able to give examples of interactions with employers such as VW and Entsa, stating that these employers provided information sharing sessions on career opportunities in Electrical and Mechanical Engineering at the college. All respondents recommend collaboration efforts between colleges and employers need to improve for workplace experience opportunities to be provided to students who need this exposure. Another respondent commented that: “more employers need to add information about available jobs on the college notice boards”. Another respondent commented: “The college should find companies that can potentially employ us and connect us to potential employers”. 3.2.4 Short term Result 2.2.1: Employers deliver expanded and more gender-inclusive workplace training opportunities Indicators: #/% of TVET lecturers who access workplace experience, as part of USG [of UKPF] assisted workforce development programmes. * #/% of TVET students who access workplace experience as part of USG [of UKPF] assisted workforce development programmes. * Indicator #/% of TVET lecturers who access workplace experience, as part of USG [of UKPF] assisted workforce development programmes. * Unit of Measure Number and percentage of lecturers USAID/MSDF Eastern Cape UKPF KwaZulu Natal Baseline Endline Target Year 1 Target Year 1 Results Baseline Endline Target Year 1 Target Year 1 Results 0 57 WIL placements 19 14 0 156 WIL placements 25 25 The Baseline for number of lecturers who access workplace experience is 0 as the programme had not started yet, and the Endline target is 57 WIL placements in Eastern Cape and 156 WIL placements in KZN. This target is based on supporting 3 WIL placements per participating lecturer for the duration of High Gear. The target for Year 1 implementation is for 19 lecturers to access WIL in the Eastern Cape and 25 in KZN. A review of IYF records for the 2021 formative evaluation shows that 14 lecturers from the Eastern Cape and 25 lecturers from KZN have accessed WIL. All 3 EMC respondents either agreed or strongly agreed that the knowledge and skills gained through workplace exposure were relevant to the course content of their subjects; that the workplace exposure had enabled them to improve their ability to link their teaching with industry requirements and standards; that the workplace exposure had improved the quality of their practical training; that the workplace exposure was well 53 structured and managed and the employees within companies were knowledgeable and helpful. 36% of the nine Elangeni lecturer respondents selected either agree or strongly agree in response to these statements. Indicator #/% of TVET students who access workplace experience, as part of USG [or UKPF] assisted workforce development programmes. * Unit of Measure Number and percentage of students USAID/MSDF Eastern Cape UKPF KwaZulu Natal Baseline Endline Target Year 1 Target Year 1 Results Baseline Endline Target Year 1 Target Year 1 Results 29% 48% 29% 13% 15% 30% 20% 10% The Baseline for students who access workplace experience is 29% in Eastern Cape and 15% in KZN. The Endline target is 48% in Eastern Cape and 30% in KZN. For Year 1 of the implementation of High Gear the target is 29% in Eastern Cape and 20 % in KZN. The Year 1 formative evaluation found that 13% of students in Eastern Cape and 10% of students in KZN had accessed WIL in 2021. At the end of 2021 only 23 respondents from EMC and 17 respondents from Elangeni had accessed WIL as shown below. Overall, of the 335 respondents across both colleges only 40 students accessed workplace experience. Table 25: Student Access to Workplace Experience/ Exposure Did you have access to workplace experience/exposure in 2021? EMC Elangeni Total Yes 23 17 40 No 146 146 292 Cannot remember 2 1 3 Total 171 164 335 The survey further probed how respondents accessed WIL and found that 17 students out of 40 across both colleges emailed their CVs to different companies in order to access WIL as shown below. Table 26: How students accessed Workplace Experience/ Exposure How did you access the workplace experience/exposure? EMC Elangeni Total I emailed my CV to different companies 13 4 17 Through the college 6 8 14 Through personal networks (family & friends) 0 4 4 Through social media 3 0 3 Cannot remember 1 1 2 Total 23 17 40 In terms of rating their views on the workplace experience, Error! Not a valid bookmark self-reference. below shows that, overall, respondents from Elangeni returned slightly higher average ratings to all statements that 54 those from EMC (respondents were asked to rate the various statements where 1 is strongly disagree and 5 is strongly agree). Overall, the lowest ratings went to the statement that accessing the workplace experience was reasonably easy, followed by the statement that the college learning was integrated with the workplace experience. The statement that received the highest rating was that the respondent had acquired new skills and knowledge which would assist them in finding employment, followed by the statement that existing employees within companies were knowledgeable and helpful. Table 27: Respondents' views on Workplace Experience /Exposure (Average Ratings) EMC Elangeni Total The process of accessing workplace experience/exposure was reasonably easy 3.43 4.00 3.72 The knowledge and skills gained through workplace experience/exposure was linked to the course content 3.91 4.12 4.02 The college learning (theory and practical) was integrated with the workplace experience 3.78 4.00 3.89 The workplace experience/exposure enabled me to learn and understand the content of my qualification more effectively. 3.78 4.29 4.04 I have acquired new skills during the workplace experience/exposure which increase my chances of finding employment 4.35 4.35 4.35 The existing employees within companies were knowledgeable and helpful 4.09 4.47 4.28 The survey further asked respondents what could be done to improve the workplace experience process for them and indicate who should be responsible for these improvements. It is important to flag that not all respondents who accessed workplace experience responded to this question. EMC Respondents Elangeni Respondents • The communication by the employer • Add latest technology • Ensure we rotate to different departments • Add ICT training • Do not give tasks not linked to the qualification • Give us more time to complete tasks • People in the work industry must work in a team and face challenges must stop complaining and adapt to the challenge and find solutions • The college should do more work readiness programs frequently like every Friday to prepare us for workplace experience. They can also provide the equipment that students use much earlier. They must also have assistants available for when the lecturer is not available. • Provide more and better equipment so that I can learn more. • Bring more equipment, bring more people. We also want to be actively involved during training • They must organise a place for you to do the practical and you want to participate more during the workplace exposure. • Institutions should help in the process of job seeking so that when students are done with their qualifications, they have a bit of experience. • Extend the duration • Remove unqualified people training qualified candidates 55 • Improve the working conditions and also provide for real work experience. And not only reading on the books. Those respondents that did not access workplace experience were asked why they thought that they had not been able to get this opportunity. This was an open-ended question, but the evaluation partner (Singizi) categorised similar comments made by respondents. Table 28: Reasons why Respondents thought that they had not accessed Workplace Experience EMC Elangeni Total I am still studying 38 53 91 I applied but I didn't get any response 38 20 58 It’s because we are doing theory only, we not doing practicals at the college 25 24 49 I didn't finish my studies 15 12 27 Because of COVID many places were not hiring 7 8 15 I have no information on places offering employment and training 3 4 7 Maybe the college is not serious about the workplace exposure, they did not help 7 7 I have never applied 5 5 I am not allowed yet to attend workplace exposure by my college 4 4 Because I did not get my certificate (NSFAS did not pay, and we were unable to get certificates) 3 3 They required a driving licence and experience which I don't have 1 1 2 Health Problems/ Family responsibilities 1 1 2 It's because of a lack of connections/networks 1 1 You have to pay to get them 1 1 Maybe companies cannot afford to provide work exposure to students 1 1 Because of the July political unrest/looting in KZN 1 1 I am studying and working at the same time 1 1 I have not found any interesting workplace that I can work for 1 1 I failed my interview 1 1 There were no factors preventing me from accessing WIL 1 1 I don't know/ cannot remember 1 4 5 Of concern is that 38 respondents from EMC and 53 respondents from Elangeni indicate that they cannot access workplace experience because they are currently studying which raises concerns regarding the respondents’ understanding of the link and relevance between studies and undertaking WIL simultaneously. 56 Another concern is that 25 respondents from EMC and 24 from Elangeni commented that they think they have not accessed WIL because they are mostly doing theory at the college and little or no practical work, which makes it difficult for them to access WIL. Linked to this, it is also worth noting that unemployed respondents commented that accessing workplace experience could have improved their chances of finding a job. Related to these concerns the survey asked graduate respondents what guidance they received when they completed their courses. 81 of the 171 respondents from EMC commented that they did not get any guidance upon completing their qualification(s). The remaining 90 respondents commented that they were advised by either their lecturers or a family member to study further or to look for a job. 58 of the 164 respondents from Elangeni commented that they did not get any guidance upon completing their qualification(s). A few of these respondents also commented that they are pursuing their studies further and are not anticipating any guidance at this stage. Other respondents indicated that they received some guidance from either family member or lecturer. 3.2.4.1 Output 2.2.1.1: Industry intermediaries facilitate gender-inclusive workplace training opportunities for TVET students Indicator # of new private sector entities that engage with TVET colleges to provide students with relevant skills and workplace learning experiences Unit of Measure Number of private sector entities USAID/MSDF Eastern Cape UKPF KwaZulu Natal Baseline Endline Target Year 1 Target Year 1 Results Baseline Endline Target Year 1 Target Year 1 Results 0 5 firms 2 1 0 5 firms 2 0 The Baseline was not applicable for this indicator as the initiative had not yet started. The Endline target is to ensure at least 5 firms offer these opportunities in each province. For Year 1 the target is for 2 firms in each province to offer workplace experience for students. The sources of data to address this indicator were interviews with TVET colleges, employers, as well as industry intermediaries. One firm was confirmed to have provided students with relevant skills and workplace learning experience and relevant skills in the Eastern Cape. No workplace learning experience placements for students were confirmed in KZN. The Year 1 formative evaluation reveals that there is clear evidence that private sector entities have started engaging with TVET colleges to provide students with relevant skills and workplace learning experience. It is however noted that there were some students, who had been placed with a large auto company and they did not access workplace experience as intended, in large part because of the disruptions linked to the pandemic. However, industry intermediary respondents as well as employer respondents commented on their continued commitment to provide workplace training opportunities for TVET students in future and, as indicated previously, to availing subject matter experts to provide guest lectures at the TVET colleges. Linked to this the merSETA also commented on its commitment to support employers to provide students with workplace learning experiences. The respondents indicated that their role as a SETA is to assist companies with funds to enable the effective participation of TVET learners in workplace experience interventions as well as ensuring that these companies can effectively support the learners. Employers highlight that there is also a need to ensure that this process is well managed to ensure that the rotation timing works to ensure that the company is not “in the middle of production” when the learners come. An industry intermediary respondent indicated that through the High Gear programme they have developed partnerships between companies, universities, and TVET colleges which will assist to create an increased number of opportunities for students to complete workplace experience. 57 3.2.4.2 Output 2.2.1.2: Industry intermediaries facilitate gender-inclusive workplace training opportunities for TVET staff Indicator # Of new private sector entities that engage with TVET colleges to provide staff with relevant skills and workplace learning experiences Unit of Measure Number of private sector entities USAID/MSDF Eastern Cape UKPF KwaZulu Natal Baseline Endline Target Year 1 Target Year 1 Results Baseline Endline Target Year 1 Target Year 1 Results 0 5 firms 3 3 0 5 firms 2 1 The Baseline was not applicable for this indicator as the initiative had not yet started. The Endline target is to ensure at least 5 firms offer these opportunities in each province. For Year 1 the target is for 3 firms in Eastern Cape and 2 firms in KZN to offer workplace experience for lectures. The sources of data to address this indicator were interviews with TVET colleges, employers as well as industry intermediaries. A few companies indicated that they provide lecturers access to workplace exposure, which included the use of new technology. A college management respondent commented that: “this IYF project is changing the mindset of lecturers to see themselves as not just lecturers but as facilitators of change in the learner from an academic environment into a work environment; so I think that also inspires them. The other benefit is because the lecturers have had the opportunity to facilitate, they themselves have increased confidence and they believe in themselves. They are now not doubting their ability to reckon with an industry in terms of their knowledge; whereas before they were kind of like in a vacuum within education, but this project has allowed them to see themselves as part of the whole system and I think that is a huge success. This has improved their morale in terms of their contribution”. However, one company respondent indicated that “we have not exposed lecturers but are happy to shadow someone for a few days.” They continued to say that they believe that this would not be sufficient as the lecturers in the workplace needed proper supervision to ensure that the exposure is beneficial. Another company respondent added that they had not yet been approached by a TVET college to provide lecturers with workplace exposure. It is noted that respondents anticipate that this will be addressed in the next phase and one intermediary respondent indicated that they have put out a survey to members to identify the number of lecturers and students they want to invite over for work integrated learning, and which is expected to run over the next 2 years. One provincial level respondent further recommended that the best way to ensure sustainable partnerships and provision of access to WIL would be for companies in the auto industry to adopt a college. Some respondents also shared that the participation of lecturers in this High Gear partnership has resulted in an increased amount of pressure on lecturers although they stated that most of their lecturers have expressed appreciation for the opportunity. College management respondents suggested though that in expanding this intervention it would be useful to explore possible incentives to encourage lecturer participation in workplace exposure interventions. A respondent commented that, “at the moment the project is really making them feel that they are being invested in, but that is because the IYF project is still in its Pilot phase, and it is going to get to a point where it is going to complete”. It is also noted that despite these noticeable developments, the COVID-19 pandemic has disrupted some of these partnerships, particularly in KZN. 58 Three firms in Eastern Cape have confirmed to have provided TVET college staff or lecturers with relevant skills or workplace learning experience and one firm in KZN confirmed to have provided TVET college staff or lecturers with relevant skills or workplace learning experience. 59 60 4 FINDINGS IN RELATION TO THE PRIMARY EVALUATION QUESTIONS The first year of the High Gear programme has laid the foundations for the development of an impactful intervention in the skills development system in KZN and the Eastern Cape and has achieved concrete results in terms of supporting the education system in becoming more responsive to industry needs. This section examines the evaluation results in terms of the primary evaluation questions. The matrix below outlines how each of the evaluation questions relates to the overall purpose of the evaluation and to the Theory of Change and provides examples of data collection methods applied. Diagram 3: Understanding the Evaluation Questions in more detail Evaluation Questions & Sub￾Questions Purpose Theory of Change Component(s) Evaluated Illustrative data collection method Question 1: Is High Gear’s Theory of Change—including its embedded hypothesis and assumptions— accurate, thereby leading to anticipated results? Does the Theory of Change contribute to increased and sustained partnerships between industry and TVET colleges? ● Primary: Help determine whether the hypothesised Theory of Change (ToC) is holding true or not, and why; ● Secondary: Inform project strategy shifts, if necessary, to strengthen implementation and outcomes. ● Sustained partnership between industry and TVET colleges to align courses with employer needs; ● NAACAM leads increased industry-wide partnerships with the TVET system; ● Industry groups’ strategic plans incorporate the High Gear model; ● Industry disseminates project learning and results. ● Interviews with company and industry associations representatives To what extent is the curricula and teaching in TVET systems more aligned to industry needs and being delivered as intended according to project design? ● Primary: Help determine whether the hypothesised Theory of Change (ToC) is holding true or not, and why; ● Secondary: Inform project strategy shifts, if necessary, to strengthen implementation and outcomes. ● TVET lecturers adopt and champion course upgrades; ● TVET students have equitable access to high-quality, relevant training and improved employment opportunities. ● Survey of all affected TVET college staff, focus groups or interviews with a sample of staff member; ● Document review of attendance registers. To what extent did the intended outcome of increasing young women’s and young men’s ● Primary: Help determine whether the hypothesised Theory of Change (ToC) is holding true or not, and why; ● Students demonstrate improved academic performance and have expanded professional networks; ● Computer Assisted Telephone Interview (CATI) with all graduates from High Gear programme, focus groups with 61 learning and employment in targeted regions occur over the course of the project? ● Secondary: Inform project strategy shifts, if necessary, to strengthen implementation and outcomes. ● TVET students have equitable access to high-quality, relevant training and improved employment opportunities. sample of graduates; ● Data retrieval from DHET MIS. Question 2: Is High Gear contributing towards increased ecosystem resilience? Does High Gear contribute toward increases in economic opportunity and inclusive growth, as a result of a higher-quality, industry-aligned TVET system continuum? ● Primary: Gather preliminary evidence of the High Gear project’s contribution towards the development of a more resilient, youth￾inclusive skills development ecosystem. ● ‘System Scale and Sustainability’ elements of the Theory of Change. ● National routine data on employment; ● Macro-economic measures such as labour productivity and GDP; ● Interviews with key employers and industry intermediaries. Question 3: Does the High Gear project model deliver Value for Money? Does High Gear cost-effectively contribute towards positive livelihood and productivity benefits for youth participants? ● Primary: Gather preliminary evidence of the High Gear project’s impact on young people by determining whether the High Gear model is cost-effective, in terms of benefits delivered for target beneficiaries in comparison to resources required. ● ‘System Scale & Sustainability’ and ‘Goal’ levels of the Theory of Change. ● South African labour market data; ● Previous evaluations and longitudinal surveys of workforce development beneficiaries in South Africa. 4.1 Question 1: Is High Gear’s Theory of Change – including its embedded assumptions – accurate, thereby leading to anticipated results? High Gear’s Theory of Change includes several core assumptions related to the roles and efficacy of stakeholders in the South African skills development system. These assumptions include: • There is recognition of the need for curriculum reform, and an appropriate sense of the urgency of this need, among both private and public sector stakeholders; • Industry intermediaries are able to effectively facilitate linkages between industry and TVET colleges, and that they have the capacity to do so; • Employers are willing to invest time and resources in programmes that benefit the sector as a whole, including their competitor firms; • The TVET sector has the capacity to implement processes that require the ongoing revision and updating of curricula in line with changes in industry production processes; 62 • That students have the means and capacity to access opportunities made available through partnerships between TVET colleges and industry. Based on the findings of this evaluation, these assumptions are broadly accurate. • Respondents across the skills system recognised the need to upgrade TVET curricula and understood that the lack of appropriately qualified technical graduates impeded the efficiency of industry and undermined the employment potential of graduates. • Industry intermediary bodies have demonstrated their potential in facilitating linkages between TVET colleges and industry by establishing the basis for ongoing cooperation between sector stakeholders. • Employers generally recognise the need for the TVET sector to provide a strong foundational level of technical and theoretical education to graduates in terms of transferable technical skills, and soft skills. • In terms of TVET colleges, there is recognition in the college system of the need to make curricula more responsive to industry needs, but the capacity to make these changes may be limited by the length of the bureaucratic processes required to revise and certify these upgraded curricula. • TVET graduates, including those continuing with further levels of their NATED qualifications, demonstrated a high degree of desire for WIL, despite the very few opportunities which appear to be available. 4.1.1 Does the Theory of Change contribute to increased and sustained partnerships between industry and TVET colleges? The Theory of Change highlights the role of industry intermediaries as critical to facilitating partnerships between industry and TVET colleges. Respondents from industry, colleges, and industry intermediary bodies reported that the programme strategy was relevant and would support the increased effectiveness of the skills development system. While partnerships between industry and individual TVET colleges have not yet been secured, there is evidence that the basis for these partnerships is being established. After Year 1 these partnerships are still in a developmental stage in which the particulars of how they will function, and the roles and responsibilities of respective partners are being negotiated. An important development in this regard is the establishment of working groups with members from each of the stakeholder groups in the skills system. The establishment of a means of regular contact between industry intermediaries, DHET and the TVET colleges, and employers themselves is essential in addressing challenges around communication and coordination and developing a mutual understanding of the internal processes in each stakeholder group that affect programme implementation. One of the critical roles of industry intermediaries noted by several respondents in relation to High Gear has been in providing information about potential opportunities to change how they relate to the skills system more broadly. For example, some industry respondents reported being unaware of the possibility that they could access SETA funding for learnerships and WIL. Since the establishment of High Gear, and the sharing of this knowledge, several funding proposals have been submitted to South African public funders to support skills development both in TVETs and within industry. The current lack of established partnerships between industry and TVET colleges does not, therefore, negate the role of industry intermediaries in the Theory of Change, but does point to the complexity of intervening in the skills development system. It appears likely that through the combined work of coordinating different stakeholders, sharing relevant information and opportunities, and actively building linkages within the system, partnerships will be established in the near future. 63 4.1.2 To what extent are the curricula and teaching in TVET systems more aligned to industry needs and being delivered as intended according to project design? The upgrading of TVET curricula to be more responsive to industry needs is a core element of the High Gear programme. While there is some evidence of changes beginning to occur, the main achievements during Year 1 of High Gear have been in building a supportive context for curricula change. In part, this has involved building a shared understanding among stakeholders of the contexts and processes in which each partner works, the specific complexities that need to be dealt with in each of these contexts, and the differing priorities among stakeholders that need to be balanced for the upgrading process to be successful. While processes were underway to align industry needs with TVET curricula prior to High Gear’s implementation, the approach used by High Gear in working with industry intermediaries to coordinate this alignment was new. One of the primary challenges to the process of upgrading TVET curricula is the length of time it takes to both develop new curricula, and to attain approval for their implementation, in the context of the relatively rapid technological changes that occur in industrial production processes. Respondents acknowledged this challenge and noted that High Gear was addressing this problem using a multi-pronged approach. Firstly, the partnerships facilitated through High Gear between DHET, TVET colleges, and industry stakeholders led to an increased willingness among individual companies to engage in curriculum development, and to providing experts as guest lecturers at TVET colleges. Secondly, High Gear supports capacity building and work exposure opportunities for TVET lecturers to ensure that they can stay abreast of developments within industry. Thirdly, by providing a space in which to problem solve, High Gear facilitates the development of more creative means to integrate new developments into the curriculum. For example, a respondent from DHET pointed out that one way to quickly include new knowledge in existing curricula is to use the provision for local content in TVET courses, including content such as case studies of particular production plants or processes. There has been progress towards aligning TVET curricula and teaching to industry needs during Year 1 of High Gear. While fundamental changes to course curricula have not yet occurred, High Gear is laying the groundwork for these changes to be implemented and is ensuring that partners are engaged and invested in this process. 4.1.3 To what extent did the intended outcome of increasing young women’s and young men’s learning and employment in targeted regions occur over the course of the project? High Gear ultimately aims to increase youth employment through making the education which students receive at TVET colleges more relevant to industry needs. Although there was no measurable improvement in rates of youth employment among TVET graduates after Year 1 of the programme, graduates generally felt that the training they had received through the programme had equipped them with the skills required to find employment and function effectively in a workplace. Most of the youth who continued studying after Year 1 thought that their qualifications had adequately equipped them for further study, and pass rates among targeted courses exceeded their targets. Accessing WIL remained a challenge for TVET students, with low percentages of graduates reporting having participated in workplace experience programmes during their studies. This suggests a need to increase the focus on systematising WIL programmes in the partnerships between TVETs and industry and will be a key marker of industry commitment to High Gear’s objectives. After the first year of High Gear implementation there is some evidence that High Gear is having a positive impact on youth learning outcomes, but it is currently not possible to identify positive outcomes in terms of employment rates among graduates. While the tracer study will deepen our understanding of these results, it is important to note the very negative impact of the COVID-19 pandemic on industry and unemployment in South Africa more broadly, and that graduates were entering a depressed labour market. 64 4.2 Question 2: Is High Gear contributing towards increased ecosystem resilience? This question examines whether High Gear’s focus on improving TVET quality and industry alignment has led to an increase in the resilience of the automotive component manufacturing sector in South Africa. At a broad level, it appears that there are some shifts towards building a foundation of resilience in the sector because of the intervention. These shifts are currently primarily around the development of partnerships between TVET colleges and industry. The increasing awareness among employers of the funding available for WIL is another important development in terms of increasing the active role of industry in the skills development system. These developments are likely to contribute to the increasing resilience of the sector over the course of the High Gear programme. In relation to this evaluation question, although it is generally too soon to tell whether High Gear is contributing to increased system resilience, it appears that the programme is on track to achieve this as High Gear is improving the eco-system, including enhancing TVET and industry partnerships, and is assisting to create a more youth and gender inclusive workplace. This bodes well for the intended outcomes although the extent of this change will be explored in the next evaluation once the programme has been able to be implemented more widely. 4.3 Question 3: Does the High Gear project model deliver Value for Money? This evaluation question tests whether High Gear cost-effectively contributes towards positive livelihood and productivity benefits for youth participants. It is too soon to tell, given the fact that very few participants have entered the labour market thus far. This question will form part of the graduate tracer study being implemented in September 2022. 65 66 5 CONCLUSION The High Gear programme is beginning to have a positive impact on skills development in the automotive components manufacturing sector in South Africa. The first year has seen the implementation of important foundational work, including developing coordination mechanisms through industry intermediaries, increasing interest and willingness to develop partnerships between industry and TVET colleges, improving access to funding for industry and TVETs to implement WIL for lecturers and students, and improvements in some student level outcomes. It is noted that strong relationships have been established with employers, employer intermediaries and colleges: High Gear is seen as credible and adding value. Specifically, this has resulted in significant work having been done in relation to revising qualifications and building new qualifications (with the QCTO) that takes the needs of industry into account. This evaluation has identified several areas of work that may require an increased focus in order to support programme implementation over the next 3 years. • Securing partnerships between TVET colleges and employers remains a priority given the overall lack of participation in WIL among TVET graduates. • Focusing on both longer-term curricula reform as well as exploring the potential to include industry led training through local content measures that do not require the extensive revision and approval processes of standard curricula. • Targeting students more directly with awareness and education about the role of WIL and how to go about accessing WIL opportunities. Overall, the High Gear Theory of Change appears to be beginning to lead to a shift in the skills development system. The focus on industry intermediaries as coordinating bodies seems to be functioning as intended, and important groundwork has been completed which should facilitate the ongoing development of a more resilient and responsive TVET system. One area in which the Theory of Change may need modification is in its conception of how young people will respond to changes in the skills development system. As it stands the Theory of Change is somewhat deterministic, and perhaps underestimates some of the challenges that young people might face in their ability to respond to changes that occur in TVET colleges and industry. There may be a need to increase the focus on working directly with students to identify challenges affecting their progress and to help them to take advantage of opportunities made available through High Gear. 67