Tertiary Education Linkages Project (TELP) 674-0315 Mid-Term Evaluation Report September 2000 Submitted to USAID/South Africa and the TELP Advisory Panel Academy for Educational Development IQC-674-I-00-00-00006-003 Evaluation Team Gretta Middleton, Senior Evaluation Leader Michael Rice, Evaluator Consultants Philip Christensen, Activity Manager Michael Cross, Policy Specialist Amanda Fitschen, Finance Specialist Howard Williams, Evaluation Methodology Specialist Table of Contents 1. EXECUTIVE SUMMARY 1 2. INTRODUCTION 4 2.1 EVALUATION PURPOSE AND AUDIENCE 4 2.2 METHODOLOGY 5 2.2.1 STRATEGY 5 2.2.2 REFERENCE GROUP 6 2.2.3 IMPLEMENTATION PLAN 7 2.2.4 SAMPLING 8 2.2.5 DATA COLLECTION 9 3. BACKGROUND 11 3.1 HIGHER EDUCATION AND HDIS: CONTEXT AND CONSTRAINTS 11 3.2 RECENT DEVELOPMENTS AND NEW CHALLENGES 11 4. USAID’S ASSISTANCE APPROACH 13 4.1 OVERVIEW OF TELP 13 4.2 TELP IMPLEMENTATION STRATEGY 14 5. FINDINGS 16 5.1 POLICY ANALYSIS AND PLANNING COMPONENT 17 5.2 CAPACITY-BUILDING COMPONENT 18 5.2.1 INSTITUTION-SPECIFIC ACTIVITIES 19 5.2.1.1 Staff development 19 5.2.1.2 Curriculum development 20 5.2.1.3 Student development 21 5.2.1.4 Administration and management development 24 5.2.1.5 Research skills development 24 5.2.2 JOINT ACTIVITIES 25 5.2.2.1 Staff development 27 5.2.2.2 Curriculum development 29 5.2.2.3 Student development 30 5.2.2.4 Administration and management development 31 5.3 INSTITUTIONAL LINKAGES COMPONENT 31 5.4 GENERAL FINDINGS 36 5.4.1 PROJECT SUCCESS FACTORS 36 5.4.2 STAFF DEVELOPMENT 37 5.4.3 TELP COORDINATORS 38 5.4.4 PROJECT MANAGEMENT STRUCTURES 39 5.4.5 MILESTONES 40 5.4.6 COMMUNICATIONS 41 6. CONCLUSIONS 43 6.1 RELEVANCE OF CURRENT TELP STRATEGIES 43 6.2 THE CHANGING INSTITUTIONAL ENVIRONMENT 44 6.3 INTEGRATION OF TELP ACTIVITIES 44 6.4 PROGRESS IN RELATION TO BENCHMARKS 45 6.5 COST-EFFECTIVENESS 46 6.6 PROJECT RESULTS 47 6.7 SUSTAINABILITY 50 7. UNRESOLVED ISSUES 52 7.1 RESULT INDICATOR 52 7.2 TELP ADVISORY PANEL 52 8. ISSUES AND RECOMMENDATIONS 53 8.1 MINDSET 53 8.2 LEADERSHIP 53 8.3 PROJECT PLANNING 55 8.4 MILESTONES AND KEY RESULTS 56 8.5 WORKSHOPS AND QUALITY ASSURANCE 57 8.6 COMMUNICATIONS 57 8.7 TELP COORDINATORS & PROJECT MANAGERS 58 8.8 STAFF DEVELOPMENT 59 8.9 COMPENSATORY PROGRAMMES 59 8.10 LINKAGES 60 8.11 SKILLS AUDIT 61 8.12 PUBLIC RELATIONS 61 9. LESSONS LEARNED 62 9.1 ADDRESSING DIVERSITY EFFECTIVELY 62 9.2 CLEAR ROLE DEFINITIONS AND VISION 62 9.3 THE CAPACITY-BUILDING CONUNDRUM 63 10. ANNEXES 64 10.1 ABBREVIATIONS 64 10.2 TELP PARTNERS 65 10.2.1 THE COLLEGE FUND/UNCF 65 10.2.2 DEVTECH SYSTEMS 65 10.2.3 AFRICAN INSTITUTE FOR POLICY ANALYSIS AND ECONOMIC INTEGRATION 66 10.2.4 INTERNATIONAL FOUNDATION FOR EDUCATION AND SELF-HELP 67 10.2.5 DEVELOPMENT CONSULTANTS NETWORK, INC. 68 10.2.6 THE DESMOND TUTU EDUCATIONAL TRUST 68 10.2.7 ADDITIONAL CONTRACTORS AND GRANTEES 69 10.3 EXPENDITURE ASSESSMENT 71 10.4 PERSONNEL 82 10.4.1 EVALUATION TEAM 82 10.4.2 CONSULTANTS 82 10.5 BIBLIOGRAPHY 84 10.6 INTERVIEW LIST 87 10.7 METHODOLOGY 92 10.8 APPROVED ACTIVITY PLAN 102 10.9 EXTENDED ACTIVITY PLAN 109 10.10 INTERVIEW PROTOCOLS 111 10.10.1 VICE-CHANCELLOR INTERVIEWS 111 10.10.2 TELP COORDINATOR INTERVIEWS 112 10.11 FOCUS GROUP PROTOCOLS 113 10.12 FINANCIAL SURVEY INSTRUMENT 117 TELP MID-TERM EVALUATION: REVISED FINAL REPORT 1 1. Executive Summary Overview Approved in 1995, the Tertiary Education Linkages Project will spend an estimated $50 million through the year 2004 to increase access for black South Africans to tertiary education opportunities and resources, and to improve academic, administrative, and research capacity in historically disadvantaged tertiary education institutions. TELP has already committed more than $22 million through its Bilateral Agreement with the South African government. The TELP Bilateral Agreement specifies that a mid-term evaluation be conducted to determine progress in the achievement of project goals. The evaluation results will be used by historically disadvantaged institutions (HDIs), the Department of Education and USAID/SA to identify and correct weaknesses in the programme. It will also serve as a planning document to guide future project implementation. The evaluation has been divided into three phases: planning, pilot testing and evaluation. As required, the evaluation strategies, methodologies and instruments were tested the pilot phase. The Evaluation Team comprised Dr Gretta Middleton (Senior Evaluator) and Dr Michael Rice (Evaluator). The evaluation was conducted at fifteen historically disadvantaged institutions. Qualitative and quantitative data have been collected. Interviews were held with USAID/South Africa, the Department of Education and the College Fund/UNCF. Key findings and conclusions · The historically disadvantaged institutions are not homogenous not withstanding their structural and historical affinities. The universities and technikons differ in conception and function. Each has a different dynamic, history, character and ethos. · Changes in the higher education environment that could not be anticipated by TELP, and over which it could have no influence, are having an effect on morale in most of the historically disadvantaged institutions, with subsequent negative consequences for the project. Falling student numbers in higher education generally have militated against TELP fulfilling its Strategic Objective of increasing access to these institutions. There has been considerable resistance to gender equity workshops. Section 1 TELP MID-TERM EVALUATION: REVISED FINAL REPORT 2 · Three factors contribute most to the project’s success: a committed and pro-active Vice-Chancellor, a competent TELP Coordinator and the project’s integration into the institution’s strategic plan. The absence of any of these factors or a combination thereof limits potential project impact. · Overall, we found the highest level of satisfaction by historically disadvantaged institutions with the institution-specific activities mechanism and the lowest level with the linkages component. We found a mixed degree of satisfaction and impact for the joint activities mechanism. · It would appear that TELP has been integrated to a greater degree into the strategic plans of the technikons than the universities. One of the areas in which joint activities have been most successful has been in capacity building around strategic planning skills. With the exception of two institutions, the HDIs were on course to meet the June 2000 deadline to comply with South African Qualifications Authority requirements. · Staff development under PILs and joint activities has occurred with regard to curriculum development, and raising awareness and understanding of the National Qualifications Framework and the requirements of the South African Qualifications Authority. The expansion and/or establishment of research units has had an effect on skills development and staff morale. The linkages programme has provided significant opportunities for staff to receive advanced technical and post-graduate training. However, it has often proven difficult to retain staff trained overseas. · Student development is being assisted by the development of bridging programmes and standard testing packages and the growth of five-year student development plans, though much remains to be done in these areas. · Criticisms levelled at workshops run under joint activities reveal that many of them have been organised on the assumption that “one size fits all”. Further, that scheduling of workshops often does not take the institutions’ calendars into consideration. Some criticism was voiced about the expertise of facilitators. · All of the HDIs have now established linkages with universities or consortia of higher education institutions in the United States. A third have established successful linkages. However, the majority of South African institutions reported significant frustration and disappointment with linkage negotiations and proposed technical assistance. · One of the most important findings to come out of the site visits is the pivotal role of the TELP Coordinators to the success of the programme. TELP MID-TERM EVALUATION: REVISED FINAL REPORT 3 · Good communications between all stakeholders and actors are essential for TELP’s success. The desire was expressed several times for better sharing of information within and between institutions. · Utilising project benchmarks as a measurement of performance reveals that TELP definitely is making progress towards achieving its intended results. Most historically disadvantaged institutions are developing and using revised curricula. Approximately 50 per cent are evaluating first-year students on entry. A minority of institutions is providing compensatory/bridging programmes. Thirteen of the 15 historically disadvantaged institutions have developed three-year rolling plans. TELP MID-TERM EVALUATION: REVISED FINAL REPORT 4 2.Introduction 2.1 Evaluation Purpose and Audience The Tertiary Education Linkages Project (TELP) Bilateral Agreement specifies that a mid-term evaluation be conducted to determine progress in the achievement of project goals and to test the validity of underlying hypotheses and critical assumptions. The statement of work specifies the following outcomes from the evaluation:1 1. Determine whether the current strategies supported by TELP are relevant to constraints TELP was designed to address. 2. Determine the extent of the changing environment of the HDIs, both as individual institutions and as a group, and whether or not TELP is addressing those changes. 3. Determine if TELP activities are integrated or linked to achieving stated results and, if not, provide recommendations to do so. 4. Determine whether TELP is achieving progress towards its stated results using programme benchmarks. If not, provide recommenda￾tions for adjustments. 5. Determine whether the achievements of TELP are being produced in cost-effective ways. 6. Determine whether actual TELP results are focused towards intended results. 7. Identify intended and unintended results of TELP funding. 8. Determine whether the accomplishments of TELP are sustainable. If not sustainable, provide recommendations for sustainability. 9. Identify the most effective way(s) to achieve TELP objectives. 10.Identify ways to address problem areas. The results of the evaluation will be used by the historically disadvantaged institutions (HDIs), the Department of Education (DOE), and the United States Agency for International Development Mission to South Africa (USAID/SA) to identify and correct weaknesses in the programme and highlight successes. It will also serve as a planning document. Therefore, the evaluation must identify future actions and make recommendations on adjustments needed to achieve TELP goals. The terms of reference for this evaluation specify a consultative, participa￾tory approach. They require that the activity be supported by qualitative 1 IQC Request Number 1/27/2000C, pp. 5-6. Section 2 TELP MID-TERM EVALUATION: REVISED FINAL REPORT 5 research to illuminate project processes and outcomes, as well as quantitative research to heighten understanding of underlying factors and project results. Together, these approaches must address impact on the ground, document lessons learned and make clear, concise recommenda￾tions. The evaluation must be guided by the 10 general and 22 specific process and outcome questions posed in the statement of work. The Academy for Educational Development (AED) team fully embraced this participatory approach. It recognized the importance of the function of the TELP Executive Committee as the Reference Group for the evaluation, as well as the value of utilizing the historically disadvantaged institutions, as a client group, to serve as a sounding board for the evaluators in developing methodology and reviewing draft evaluation reports. 2.2 Methodology 2.2.1 Strategy The TELP Bilateral Agreement required that the mid-term evaluation focus on progress in the achievement of project goals. The statement of work for the activity emphasised that stakeholders will use its results not just to identify and correct weaknesses, but also as a planning document. Therefore, the evaluation had to contextualise problems and progress to date, note defects in the original design or changes to its underlying assumptions, identify future requirements and make recommendations on needed adjustments that will help to achieve project goals. In other words, this is a formative, not a summative, evaluation. Its purpose is not to uncover failure (although it must clearly note defects in assumptions, strategies or implementation), nor to assign blame (although it must specify who is accountable for necessary changes). Instead, it must map the past in order to chart the future: identifying where the project stands now, explaining how it arrived at that point, and suggesting the best ways forward. Furthermore, the evaluation has focused on the project – its successes and failures, its strengths and weaknesses – rather than on its individual participants.2 This approach informed the methodology for the pilot study. It was accepted as being appropriate for the evaluation by the Reference Group at its first meeting on 9th May and again on receiving the pilot study report at its 28th June meeting. Time and budget constraints did not permit separate evaluations of individual institutions, current or past contractors, USAID or the Department of Education, although of course the methodology took into account their views and performance. Instead, 2 These comprise USAID as the donor, the Department of Education in its dual role of oversight on behalf of the Government of South Africa and implementation of the policy component, the historically disadvantaged institutions themselves as clients and partners, and the contractors and grantees responsible for delivering services. TELP MID-TERM EVALUATION: REVISED FINAL REPORT 6 evaluators assessed TELP itself, with a view towards identifying ways in which the project can better accomplish its objectives and through which its stakeholders can contribute to such improvements. At the request of USAID, the evaluators gave highest priority to the most recent stakeholders and activities, including the current TELP contractor, the College Fund/UNCF, and the original cohort of 15 historically disadvantaged institutions. The team did not carry out any separate assessment of the impact of the previous contractor, DevTech Systems, nor of the two grantees still operating under TELP, the African Institute for Policy Analysis and Economic Integration (AIPA) and the International Foundation for Education and Self-Help (IFESH), relying instead on existing reports and evaluations about the work of these three parties. The two newest TELP stakeholder institutions were excluded from data collection because they have only recently joined the project. As specified in the statement of work, this general strategy was applied to evaluating three specific areas: · a sample of institution-specific activities (including interviews with Vice-Chancellors, TELP Coordinators, Activity Managers, and a sample of participants); · joint activities (collecting data from the Department of Education, Vice￾Chancellors, TELP Coordinators, Activity Managers, participants, and contractors and grantees); · linkages (including interviews with Vice-Chancellors, TELP Coordinators, Activity Managers, and a sample of participants). The evaluation team faced two major design challenges in implementing this strategy: sampling and data collection. Annex 10.7 incorporates the full plan presented by the field team at the beginning of April. This section summarises the methodology and its implementation. 2.2.2 Reference Group The terms of reference established an Evaluation Reference Group as follows: “Given the participatory and consultative nature of TELP, the TELP Executive Committee will serve as a reference group for the evaluation and will serve as a sounding board for the evaluators as they develop their methodology and review drafts of the evaluation report. In addition, the TELP Executive Committee will be consulted regarding any major issues that arise during the course of the evaluation.”3 The Reference Group comprised all members of the TELP Executive Committee and UNCF. Thus the Group included representatives of four 3 IQC Request Number 1/27/2000C, pp. 6. TELP MID-TERM EVALUATION: REVISED FINAL REPORT 7 historically disadvantaged institutions (the Vice-Chancellors of ML Sultan Technikon, Peninsula Technikon, University of Venda and University of the Western Cape), the Department of Education, and USAID/South Africa. The Chairperson of the Executive Committee, Prof. B.C. Goba of ML Sultan, chaired the Reference Group. Consistent with the commitments made in its proposal, AED sought to maximise the Reference Group’s role in the entire evaluation process. The Group met three times in formal session: on 9 th May to review the pilot phase report (at which meeting the approach and methodology to be used in the evaluation were approved), on 28th June to review the preliminary draft of this evaluation report, and on 14th July to review a revised draft. Throughout the entire process the team regularly sought Reference Group feedback: by fax on the initial methodological proposal, then by face-to￾face and telephonic consultation on specific issues such as sampling methodology and revisions to the implementation plan after the withdrawal of one evaluator. The Academy for Educational Development and all members of the evaluation team would like to express their appreciation to the Reference Group for its time and constructive direction throughout the evaluation activity. 2.2.3 Implementation Plan The team implemented the evaluation in three stages: a planning phase, the pilot phase, and the full evaluation phase. Annex 10.8 presents the activity plan as approved by the Reference Group at the start of the activity; Annex 10.9 shows the plan for an extended level of effort as requested by the Reference Group.4 The activity’s first phase focused on specifying evaluation processes, developing methodologies, interviewing national stakeholders for the first time, and clarifying the roles of evaluation team members relative to data collection, case studies and report preparation. Specific tasks included documentation review, preliminary briefings and consultations, drafting the work plan, developing the methodology, finalising the pilot sample, arranging Phase Two visits, and interviewing USAID/SA, the Department of Education and the College Fund. The team presented the deliverable for this phase, the evaluation work plan, to USAID on schedule on 7th April. The second, or pilot, phase tested the evaluation strategies, methodologies and instruments at two TELP institutions (ML Sultan Technikon and the University of Transkei). The team selected these institutions in consultation with USAID and the Department of Education. On 9th May, as scheduled, the evaluators presented the deliverable for this phase, the pilot study report, to a meeting of the Reference Group. 4 The extended activity plan is being modified again because it proved impossible in practice to arrange the stakeholders’ workshop on 31st July. As this report is being submitted, that workshop has been tentatively rescheduled for 16th August, to be followed by a meeting of the TELP Advisory Panel on 17th August. TELP MID-TERM EVALUATION: REVISED FINAL REPORT 8 The culminating evaluation phase encompassed the completion of data collection at the remaining institutions, analysis, report writing, and final consultations with all stakeholders. During this phase evaluators interviewed in person every Vice-Chancellor (or his/her designee) and every TELP Coordinator from all 15 historically disadvantaged institutions. In addition, a two-member team carried out intensive site visits at eight institutions. In addition to the two pilot study visits, this brought the total number of sampled TELP stakeholder institutions to 10. Phase three requires three deliverable: a draft report reviewed by the Reference Group (on 28th June and 14th July), a workshop for all partners (held on 16th August), and the final report (circulated to USAID and stakeholders on 26th July prior to discussion at the 16th August workshop). Participants at the stakeholders’ workshop requested consideration of their input in a revised final report, hence this document. 2.2.4 Sampling AED’s original proposal called for the Senior Evaluation Leader to visit all 15 TELP institutions5 to interview the Vice-Chancellors and TELP Coordinators, while two teams of two specialists each made in-depth, two￾day visits to every campus. This had the advantage of full data collection at every institution, but in the event proved too costly an approach. Therefore, AED suggested moving from this census methodology (visiting every TELP stakeholder institution) to a sampling methodology (visiting as large a representative group of institutions as possible). As noted above, during the planning phase the evaluation team worked with the Department of Education and USAID/SA to draw a stratified sample on the basis of perceived institutional success within the project: low, medium, and high. This sample included the following ten institutions for two-day intensive visits in addition to interviews with the Vice-Chancellors and TELP Coordinators: Eastern Cape Technikon, ML Sultan Technikon, Peninsula Technikon, Technikon Northern Gauteng, University of Durban￾Westville, University of Ft. Hare, University of Transkei, University of Venda, University of the Western Cape, and University of Zululand. The remaining five institutions were scheduled for Vice-Chancellor and TELP Coordinator interviews only. These were Border Technikon, Mangosuthu Technikon, Medical University of South Africa, University of the North, and University of the Northwest. Finally, the working group selected two of the ten sampled institutions for the pilot phase: ML Sultan Technikon and University of Transkei. 5 Seventeen HDIs now participate in TELP. However, for most of the project’s history this number stood at 15. Its terms of reference directed the evaluation team to focus exclusively on these 15 institutions: Border Technikon, Eastern Cape Technikon, Mangosuthu Technikon, Medical University of South Africa, ML Sultan Technikon, Peninsula Technikon, Technikon Northern Gauteng, University of Durban-Westville, University of Ft. Hare, University of the North, University of the Northwest, University of Transkei, University of Venda, University of the Western Cape, and University of Zululand. TELP MID-TERM EVALUATION: REVISED FINAL REPORT 9 2.2.5 Data Collection The evaluation design called for collecting both qualitative and quantitative data. It required qualitative information regarding the perceived worth of activities as well as quantitative data such as financial statements. All data collection depended upon a consultative participatory process. It occurred in two discrete stages. The first data collection took place during the planning phase. One component involved a literature survey and review of background documentation, as provided by USAID. The team paid special attention to the UNCF Progress Report, the DevTech Semi-Annual Report (Appendix No. 4), minutes of relevant meetings, financial reports, and the DevTech Academic Audit. The second component comprised preliminary interviews with key stakeholders. These early discussions with donor agency, contractor, and host government representatives provided critical data and insights for team members prior to the commencement of site visits to institutions and the formal assessment of the overall progress of TELP in achieving project goals. · During discussions with USAID/SA, Mission representatives welcomed team members, provided general orientation, presented an overview of the mid-term evaluation process, and entertained questions. The series of meetings were crucial to establishing an open channel of communications, achieving a common understanding of the scope of work and clarifying the roles of individual team members. · The Department of Education’s Branch of Higher Education helped the evaluation team obtain a general assessment of TELP, learn firsthand about the Department’s vision for the TELP initiative, and ascertain what future role the Department sees for the project in light of the fluidity of the higher education policy environment. To-date, the Department’s TELP activities continue to focus on policy development and implementation as intended. · The team’s meeting with the College Fund provided an opportunity for introductions to be made between team members and key UNCF project managers and accounting personnel, for the team to explain the purpose of the mid-term evaluation. That is, this evaluation is not institution-specific but is an evaluation of the TELP project per se. The opportunity was also used to discuss future input that would be vital to the evaluation work because of the Contractor’s centrality to project implementation. A number of subsequent interviews were also conducted to clarify issues and gather additional information. The second data collection stage, the site visits to sampled institutions, occurred during the pilot phase and the first part of the evaluation phase. As planned, the Senior Evaluation Leader and Evaluator spent two days at TELP MID-TERM EVALUATION: REVISED FINAL REPORT 10 each of the two pilot institutions, and then prepared a pilot phase report for presentation to the Reference Group. The original strategy envisaged the Senior Evaluation Leader moving on to visit all 13 remaining institutions while the Evaluator (originally designated Team Leader), assisted by a colleague (the team member), carried out the two-day visits at the remaining eight institutions in the sample. However, the unexpected loss of the second team member after the first of the site visits required alterations to this plan. Consequently the Senior Evaluation Leader and the Evaluator worked together to carry out all but the first two two-day visits, and the Policy Specialist interviewed the Vice￾Chancellors and TELP Coordinators at those campuses not visited in person by the Senior Evaluation Leader. The team used two primary methods for qualitative data collection during the second stage. First, one team member (the Senior Evaluation Leader or the Policy Specialist) interviewed each Vice-Chancellor and each TELP Coordinator separately. Specific protocols (See Annex 10.10) guided these interviews, although they were carried out in a relatively open-ended manner according to the interests and concerns of each interviewee. Second, the team convened focus groups as follows for each two-day visit: 1. Institution-specific joint activities: a focus group consisting of participants from each of the focus areas. 2. Institutional activities & PILs: a focus group consisting of participants from proposal-writing and implementation workshops, plus representatives from each Project Implementation Letter (PIL) focus area, excluding the TELP Coordinator. 3. Linkages: a focus group from each parameter of the focus. The choice of participants in each focus group was crucial to success. The evaluators made every attempt to include in the relevant focus groups the Dean of Students, the Dean of Science, the Deputy Vice-Chancellor for Academic Affairs/Human Resources and the Project Directors, as well as representatives from the student leadership. Each focus group followed its own specially designed protocol (see Annex 10.11). During this second stage the Finance Specialist worked separately on obtaining the financial data required to support the on-site data-collection process and answer specific questions such as cost-effectiveness. She provided input to the Vice-Chancellor and TELP Coordinator interviews, developed a survey instrument that was distributed to the ten sampled institutions (five of which returned the completed form), and obtained relevant data from USAID/SA and the College Fund. TELP MID-TERM EVALUATION: REVISED FINAL REPORT 11 3.Background 3.1 Higher Education and HDIs: Context and Constraints South Africa is experiencing major transformation. As the nation endeavours to build strong democratic institutions, it must simultaneously address the long-term effects of an inherited system that was built on structural inequality. The legacy of inequity and underdevelopment continue to permeate all spheres of life. Since higher education is universally considered the vehicle through which societies fulfil their needs for professional, technical and leadership skills, the major upheavals in South African tertiary education overall have serious implications for national well-being. The evaluation of TELP and the project’s future implementation must take full cognisance of the state of flux in the sector. Whichever way it is regarded, the size and shape of the higher education sector is likely to undergo considerable change in the next few years in response to the requirements of industry and Government’s economic policies. The majority of historically disadvantaged institutions (with the notable exception of Fort Hare) were established under apartheid for black students. Unarguably these institutions were not autonomous. They were inadequately staffed and isolated not only from each other but also from the sector as a whole. Maintaining effective financial management systems has been a particular problem; some TELP stakeholder institutions have accumulated significant debt. The kind of education students received in these institutions was and is perceived as being inferior to that offered on the more prestigious and better-equipped campuses. The active debate taking place on “size and shape” and the call for the creation of a single system of higher education has contributed to heightened levels of anxiety among historically disadvantaged institutions. 3.2 Recent Developments and New Challenges The White Paper on Higher Education issued in 1997 laid the basis for a unified system of higher education under one “Branch of Higher Education” within the restructured Department of Education. One of the intentions was to reduce the divisions between the two sectors since the creation of technikons in the 1960s. From that time forward higher education has functioned in an environment framed by issues emanating from the inevitability of anticipated change. The uncertainty has centred on how the tertiary education sector would evolve. The situation for Section 3 TELP MID-TERM EVALUATION: REVISED FINAL REPORT 12 historically disadvantaged institutions is particularly vexing. They operate in an atmosphere dominated by fears of possibly losing their identity, of being absorbed, reconfigured or closed. On 7 April 2000, the Council of Higher Education’s Task Team Size and Shape Discussion Document was issued. It is expected to have a significant effect on the size and shape of the sector. The debate underway will be greatly influenced by the suggestion to establish a system based on a “model of differentiation and institutional types.” The Task Team states that “higher education’s primary role is to develop the thinking and intellectual capabilities of (our) society and through such development to address and resolve the range of economic, social, cultural, political and other challenges faced by society as a whole.” The document targets what it refers to as “pervasive dysfunctionality” that characterises parts of the higher education system. It enumerates an extensive list of problems that must be resolved in order for the nation to reach its full potential. The problems that it highlights include the dramatic decline in the rate of enrolments of new students and the decline in retention rates, poor graduation and yearly pass rates, fragile management and administrative capacity, and the persistence of crises in some institutions. The overarching position of Council for Higher Education is that a coherent, coordinated and integrated national education system must be achieved over time. It views the attainment of this goal as essential to achieving cost-effectiveness, efficiency and also equity.6 Additional challenges confronting the policy-makers result from the revolution in student enrolment patterns in higher education that has occurred in the past twelve years. According to David Cooper of the University of Cape Town and Nico Cloete and Ian Bunting in a Centre for Higher Education Transformation (CHET) document, there has been a radical transformation in the racial composition of students in the tertiary education sector. It is reported, for example, that between 1988 and 1998 the percentage of African students attending the six Afrikaans Universities increased from one per cent to 33 per cent, while the percentage of white students declined from 97 per cent to 61 per cent. Moreover, in 1988 at the seven historically Afrikaans technikons, African students only comprised two per cent and white students 94 per cent; by 1998 African students were 53 per cent and white students 37 per cent. If current trends continue, it is conceivable that in the not-too-distant future, African students at a number of traditionally Afrikaans universities will pass the 50 per cent mark. Such developments call into question old values, concepts and rationales that underpin institutional categorizations.7 6 Council on Higher Education Size and Shape Task Team Discussion Document 7 April 2000. 7 David Cooper UCT “The Skewed Revolution: The Selective Africanisation of the Student Profile of Historically White Universities and Technikons in South Africa in the 1990s” (April 2000) and Nico Cloete & Ian Bunting “Higher Education Transformation: Assessing performance in South Africa,” CHET. 2000. TELP MID-TERM EVALUATION: REVISED FINAL REPORT 13 4.USAID’s Assistance Approach 4.1 Overview of TELP In the past the higher education and training system perpetuated an inequitable distribution of access and equity for students and staff along axes of race, class and location. There continue to exist disparities between historically black and white institutions in terms of facilities and capabilities. To address the core issues of equity and access, which directly affect quality, institutions of higher education and training need to: 1) develop innovative ways of increasing access by disadvantaged students; 2) provide programmes to maximize the success of such students; and 3) ensure a smooth induction into higher education and training.8 Approved in 1995, the Tertiary Education Linkages Project will spend an estimated $50 million through the year 2004 to increase access for black South Africans to tertiary education opportunities and resources, and to improve academic, administrative, and research capacity in historically disadvantaged tertiary education institutions. Its specific objectives include supporting the Department of Education, building capacity at historically disadvantaged institutions, and carrying out special programmes at 15 participating institutions (plus an additional two institutions that joined the project subsequently). TELP’s activities can be divided into three components: policy analysis and planning, capacity building, and linkages. Policy analysis and planning involves the detailed examination of critical issues in tertiary education with the aim of presenting options and programmes for ensuring a more equitable and efficient tertiary education system. Some of the policy and planning issues involved include: · student and professional access to and quality within various institutions; · the roles of various types of tertiary institutions; · rationalisation of the system; · financial resource formulae and distributions; · restructuring and curriculum design; · strategic planning; · definition of institutional missions; · degree requirements and accreditation; and 8 IQC Request Number 1/27/2000C, p. 3. Section 4 TELP MID-TERM EVALUATION: REVISED FINAL REPORT 14 · articulation within the tertiary education sector. The project defines capacity building as the process by which historically disadvantaged institutions are strengthened, expanded, and/or made more effective. It supports this process by: · improving administrative, academic, and research infrastructure; · developing and implementing strategic plans; · supporting priority strategic disciplines (mathematics, science and engineering; oral and written English; public administration and management; commerce and economics); · promoting professional development; and · strengthening curriculum development and pedagogy. TELP sees linkages as mutually cooperative activities enabling two or more institutions to address broad-based problems, share resources, and engage in joint activities related to policy analysis, planning, and capacity building. It supports the establishment or enhancement of linkages among South African tertiary institutions, and between South African institutions and colleges and universities in the United States. In pursuit of these objectives, TELP has already committed more than $22 million through its bilateral agreement with the South African government. These funds have supported the National Commission on Higher Education, small capacity-building grants to 15 individual institutions, additional grants to consortia of institutions, and work at TELP stakeholder institutions in five focus areas identified by their Vice-Chancellors and Principals: student academic and social development; staff development; research skills development; programme/curriculum development; and management and administrative development. Those institutions where TELP funds are already supporting specific activities report that the project provides benefits that would otherwise not have been possible. 4.2 TELP Implementation Strategy Since inception, TELP has supported the Department of Education and, within that support, 15 historically disadvantaged institutions. The purpose of TELP was, and still is, to 1) increase access by disadvantaged South Africans to tertiary education opportunities and resources, and 2) to improve academic, administrative, and research capacity of its stakeholder institutions. These institutions were targeted because they were the primary means through which the majority population gains access to higher education qualifications. Its design flexibility has allowed TELP to undergo strategic adjustments since inception. The first adjustment occurred in September 1995, when USAID/South Africa signed a TELP Bilateral Grant Agreement with the Department of Education that changed the administrative nature of TELP from being a unilateral assistance programme to co-operative TELP MID-TERM EVALUATION: REVISED FINAL REPORT 15 programme. The overall purpose of that agreement was to assist the Department to improve and transform the system for higher education and improve the academic, administrative and research capacity of the aforementioned institutions. In 1995, TELP focused on strengthening the following three critical areas: · Policy analysis and planning: aimed at examining critical issues in higher education and developing options and programmes designed to assure a more equitable and effective system of higher education. · Capacity building: through this mechanism individual institutions are expanded and/or made more effective in the ways in which they operate and provide learning opportunities. · Institutional linkages: intended to enable collaborative activities between two or more institutions to jointly address problems, share resources, and engage in activities related to policy analysis and capacity-building. In 1996 the TELP Advisory Panel, consisting of the Vice-Chancellors of the historically disadvantaged institutions, the Department of Education and USAID/SA, identified five focus areas for TELP resources: 1) staff development, 2) curriculum development, 3) student development, 4) management and administrative development, and 5) research skills development. These focus areas (with the exception of research skills development for which an investigation into management of information systems was substituted) were the subjects of an extensive academic audit carried out in 1998. Presently, TELP activities at historically disadvantaged institutions are directed at building capacity and institutional linkages in the focus areas. However, as in 1995, the Department of Education’s TELP activities continue to focus on policy development and implementation. The nature of the changing environment and impending decisions about the ‘size and shape’ of the higher education sector necessitate that TELP continue to approach project implementation in a flexible manner. TELP MID-TERM EVALUATION: REVISED FINAL REPORT 16 5. Findings The mid-term evaluation findings are based on qualitative and quantitative data collected by the evaluation team during the course of interviews with USAID/SA, the College Fund and the Department of Education, as well as site visits to the 15 targeted tertiary education institutions. These findings are the result of the distillation, synthesis and analysis of large amounts of disparate information about and from a diverse group of stakeholders. This section presents the evaluation’s findings according to the three critical areas identified for TELP in 1995: policy analysis and planning, capacity-building and institutional linkages. The first of these focuses on the Department of Education; the last two serve the historically disadvantaged institutions. A final sub-section offers general findings that do not fit exclusively into one of the three critical areas. TELP defines capacity building as the process by which historically disadvantaged institutions are strengthened and/or made more effective in the ways in which they operate and provide learning opportunities. Institutional linkages, on the other hand, are intended to enable collaborative activities between two or more institutions to jointly address problems, share resources, and engage in activities related to project goals. In relation to capacity building and linkages, it should be reiterated that the mid-term evaluation of TELP is intended to assess performance and factors affecting project performance. The Reference Group stipulated that it is not an evaluation of individual historically disadvantaged institutions or the Contractor. Therefore, these findings are not institution-specific, but rather summarise a collective view of how the project is progressing in participating institutions. They must, of necessity, generalise about patterns, trends and common practices, as well as indicating wherever possible the exceptions or points of divergence. With such an approach, generalisations made will obviously apply to some in the group more closely than to others. Efforts are made throughout this report to include examples that amplify or provide insight into project implementation. This section should be considered in conjunction with Section 6 ( Conclusions), which draws on these findings to answer the general questions posed by the evaluation statement of work, and Section 8 (Issues and Recommendations), which identifies specific issues arising out of the findings, then recommends what to do in response and who should do it. The Reference Group meeting of 28th June confirmed that the Section 5 TELP MID-TERM EVALUATION: REVISED FINAL REPORT 17 Findings section should present the data without interpreting it. The Conclusions section is where interpretation should, and does, take place. 5.1 Policy Analysis and Planning Component The overall purpose of the TELP Bilateral Grant agreement signed in 1995 was to assist the Department of Education transform the system for higher education and improve the academic, administrative, and research capacity of the historically disadvantaged institutions. The critical area identified with specific regard to the Department was policy analysis and planning aimed at examining critical issues in higher education as well as developing options and programmes designed to ensure a more equitable and effective system for higher education. Department of Education initiatives under TELP According to a representative of the Department of Education, the funding provided under the TELP contract has “done what we wanted done…. [It] provided us with flexibility and facilitated what we wanted to do (policy development).” That is, a number of studies flowing from the White Paper on Higher Education have been undertaken by specially contracted consultants. · The initial support for policy analysis focused on the support to the National Commission on Higher Education (NCHE). The Commission presented its report in 1996, and from its work flowed many policy milestones, including the policy for the higher education sector, as articulated in the White Paper 3, on Higher Education Transformation. · · Reports are imminent to assist on the location of, and the different options for, the agricultural and nursing colleges, as well as on the legal implications for their incorporation into the higher education system. · A new regulatory framework for private higher education has been developed and will be promulgated shortly. · Research has been carried out into the incorporation of the public distance teacher education colleges into higher education. Teacher education was previously a provincial function and, therefore, not part of higher education. · Research has also been carried out into a new funding formula for higher education. TELP MID-TERM EVALUATION: REVISED FINAL REPORT 18 · Workshops (which are milestone requirements) have been held on, and a full-time consultant contracted to develop, institutional planning process, funding and management information systems, and on mergers and cooperation within higher education. · A number of smaller projects not originally planned, such as the creation of a consolidated faculty of music in KwaZulu-Natal, have also been effected. These initiatives have been possible because funds have been used flexibly, enabling the Department to respond to new circumstances as they have arisen. 5.2 Capacity-Building Component With regard to capacity building within the historically disadvantaged institutions, it must be recognised from the outset that these institutions do not form a homogenous whole. Not only are the universities and technikons different in conception and function, but each also has its own distinct character and ethos. Each responds to a different environment that generates different kinds of problems demanding different kinds of strategies and solutions. In the words of one TELP institution, “One size does not fit all.” Nonetheless, it must be acknowledged these institutions share structural and historical affinities. Some institutions are more disadvantaged than others. They all have different histories. The term “historically disadvantaged,” it was pointed out, has had the negative result of all of the so-called historically disadvantaged institutions being lumped together regardless of their differences. Such labelling has resulted in a mindset from without that sees them all, at one extreme, as “hopeless cases”9 and an equally debilitating view from within that encourages them to act in concert and deny their uniqueness, thereby undercutting their real needs. “HDIs talk with one voice but they are not the same.” Originally, TELP focused primarily on individual universities and technikons through the PIL (Project Implementation Letter) contractual mechanism. Institutions requested funding for specific activities from USAID, which responded with a letter outlining the terms and conditions under which it would provide such support. Section 5.2.1 describes the results of this approach. More recently TELP introduced a second delivery strategy, support for joint activities aimed at multiple institutions, discussed in Section 5.2.2. These two approaches seem to overlap in practice. For example, the institution-specific component now includes activities that are of a joint nature, such as national workshops for proposal-writing and gender issues. The joint activities component, on the other hand, can include activities that succeed or give rise to institution-specific technical assistance. This section reports evaluation findings according to whether interventions are aimed at individual (Section 5.2.1) or multiple (Section 9 Quotations are taken directly from stakeholder interviews. TELP MID-TERM EVALUATION: REVISED FINAL REPORT 19 5.2.2) institutions, rather than differentiating activities according to whether they are funded under the institution-specific or joint-activities mechanism. 5.2.1 Institution-Specific Activities In 1996 the TELP Vice-Chancellors, through the Advisory Panel, identified five focus areas for TELP resources as being central to the rehabilitation of their institutions: staff development, curriculum development, student development, management and administrative development, and research skills development. These focus areas now comprise the core of TELP’s strategy to build capacity and promote institutional linkages. This section examines the impact of institution-specific activities on each of the five focus areas. Overall, we found the highest level of satisfaction by historically disadvantaged institutions with the institution-specific activities mechanism. Project Implementation Letters (PILs) TELP utilises the PIL modality to assist historically disadvantaged institutions achieve project and institutional objectives. In 1998 USAID/SA discontinued the practice of directly selecting, awarding and managing PILs. The process was standardised under new regulations and procedures set forth in the document entitled “USAID Proposal Submission Guidelines for PILS.” As stipulated, a Review Panel composed of two representatives each from USAID, the Contractor and TELP Coordinators review PIL proposals and make recommendations. USAID directly makes PIL awards, the institutions are responsible for implementation and the Contractor provides critical support services. The College Fund is expected to: a) oversee the development, review and implementation of PIL activities, and b) provide technical assistance in support of the development and implementation of all PIL activities. Additionally, the Contractor is tasked with conducting proposal-writing workshops for TELP institutions and building capacity to develop successful proposals. 5.2.1.1 Staff development Staff development overlaps with the other focus areas. For example, one institution sees student development not as the sole prerogative of the student development unit or of Student Affairs, believing instead that “all members of staff are involved in student development” and, where possible, providing them with training under this focus area. Academic staff development occurs in activities concerned with curriculum development, especially materials development in the areas of mathematics, science and English, but also in raising awareness and understanding of the National Qualifications Framework and the requirements of the South African Qualifications Authority. TELP MID-TERM EVALUATION: REVISED FINAL REPORT 20 “We must grow our own timber” The expansion and/or establishment of research units has had a positive effect on skills development and the morale of staff, a number of whom are already beginning to produce their first research articles for publication and/or delivery at conferences. “We have seen stars emerge because of TELP.” With regard to specific academic training, the activities organised under the linkage component have been the chief vehicle in this area and will be dealt with separately in Section 5.3. However, a number of PILs have been specifically targeted at staff development. Some of these include efforts to improve the skills of staff better to equip them to meet teaching and administrative challenges, developing a post￾graduate leadership training programme and improving the professional skills of library staff. Staff release can present problems, as it often results in existing resources being stretched. Frequently staff find that instead of being able to give all their attention to a new project they are required to share their time and effort with existing duties. 5.2.1.2 Curriculum development The history of curriculum development at universities and technikons has been quite different. The universities were all established as autonomous institutions. When the technikons gained autonomy, on the other hand, they retained the national character of their programmes. The Committee of Technikon Principals, through a mechanism called “convenor technikons,” used expert committees to guide curricular changes; these changes were then used by all technikons. Accordingly, a greater degree of uniformity exists among the technikons. In addition, the technikons were used to a higher level of cooperation amongst themselves than were the universities. The universities, on the other hand, all had full control over what they taught and examined. That is, they were under no obligation to coordinate what they offered or what fields of research they engaged in with sister institutions. One of the many results of this lack of communication, encouraged by apartheid, has been subject duplication and thinly spread human and material resources, with concomitant implications for programme delivery, curriculum and staff development. This said, many of our findings cut across this divide. For example, a number of institutions have used PILs to develop curricula in mathematics, science and English. One of the most interesting outcomes has been the establishment of a curriculum to teach English as a multi-cultural language. Other exciting initiatives have been the development of indigenous textbook and teaching materials in TELP MID-TERM EVALUATION: REVISED FINAL REPORT 21 mathematics and science undertaken by a consortium of five of the participating institutions. Few institutions have a language policy. This was felt to be a serious lack. Academic staff must start “asking questions around language,” but there is a lack of capacity in the language areas. It was pointed out that a curriculum development unit can facilitate and initiate ideas and organise workshops, but that it has to work with champions in the various departments. At one technikon the Academic Staff Centre is concentrating on assessment and methodology. Four technikons complained that the Department of Education’s national strategies and policies have often not kept pace with the wish to change in the technikons. The perception is that many of the Department’s committees work so slowly that they hold back the pace of change in the institutions. In the main, the technikons have benefited from TELP interventions (both institution-specific and joint workshops) to promote an understanding of the role of the South African Qualifications Authority, the National Qualifications Framework requirements and the technical needs to meet those requirements. In general terms, there is at present fairly good institution-wide understanding among the technikons of the National Qualification Framework. Their modularisation of courses is well underway. All of these institutions expressed the opinion that they are well positioned to comply with the procedural and technical requirements in time for the June 2000 deadline. The situation in the universities is not as easily described as that in the technikons. Some of the universities are well on track to comply with the June 2000 requirements, while others confessed that they had been given an extension because of the difficulties they are experiencing in transforming themselves. Often retrenchments, student disruptions and a lack of strategic planning capacity have resulted in TELP support being, at best, postponed and, at worst, ignored, as an institution engages with the problems of survival. At least two universities visited during the course of the evaluation were facing the imminent loss of between 25 and 30 per cent of their academic staff as a result of cutbacks. As with student access, curriculum development has been overtaken by external forces that TELP could neither foresee and over which it has no control. There is no doubt that the unique tensions within each institution have had a significant impact on the performance of TELP. 5.2.1.3 Student development Access Firstly, in terms of absolute numbers, it is clear that TELP has not helped to increase access for disadvantaged students to the historically TELP MID-TERM EVALUATION: REVISED FINAL REPORT 22 disadvantaged universities, though this is not necessarily the case at the technikons.10 However, it must be pointed out that student enrolments at universities nation-wide have been falling in recent years and that there has been a shift to the technikons both demographically and in terms of absolute numbers. Falling numbers and shifting populations and demographics are not a consequence of TELP but of larger forces within the economy and education generally. Changing perceptions of the relative worth of university education and technical training have also had a role to play. Nonetheless, the technikons are enrolling more black students in mathematics and science than the universities.11 Nor is this trend limited to the formerly white institutions. The technikons included in this evaluation all reported increased numbers of students in these subjects. One of the universities, which has taken the lead in trying to give access to more students by emphasizing exit points rather than entrance qualifications, is particularly interested in exploring through TELP new insights into admissions policy. Thus, with regard to the key Strategic Objective to increase access to quality education and training, the following developments, independ￾ent of TELP, must be taken into account. Since 1994 the number of candidates obtaining full matric exemption has declined. The decline in the school output has meant that many institutions have not been able to meet their admission targets for new undergraduates since 1996. Retention rates in higher education have also dropped off, particularly in 1998 and 1999. The goal of improved participation for the system was not realised during the years 1997-1999. However, the percentage distribution of the head count enrolment by population indicates that the percentage of black students in all institutions in higher education rose from 53% in 1993 to 71% by 1999. The percentage of students in the system attending historically black institutions in 1993 was 49%. This figure fell to 33% by 1999. Large numbers of black students appear to have moved into the previously white institutions. In general the university sector did not satisfy the White Paper goals concerning the improvement of throughput rates between 1995 and 1998. Nor did the throughput rates of black students improve during this time period. We were unable to obtain comparable throughput data on the technikons. In terms of head count, the percentage of black students at technikons was 65% by 1998. An interesting detail is that gender equity has improved considerably, with 52% of the student body being female by 1999.12 The fact remains that there are vast numbers of potential students emerging from the secondary school system every year who are woefully under-prepared for higher education and for whom special 10 See Section 7 (Unresolved Issues) for a more detailed discussion on the problems achieving a results indicator defined in terms of numbers of black students graduating from historically disadvantaged institutions. 11 See, for example, the recent paper by David Cooper of the University of Cape Town: “The Skewed Revolution: The Selective ‘Africanisation’ of the Student Profile of Historically White Universities and Technikons in South Africa in the 1990s” (April 2000). 12 David Cooper, ibid. TELP MID-TERM EVALUATION: REVISED FINAL REPORT 23 provision has to be made to make the transition from one system to another.13 The need to develop appropriate standardised tests, bridging programmes and other strategies to help them cope with the academic and social demands of higher education is not going to disappear. In addition, the requirements of the South African economy in the midst of the information technology revolution and globalisation must be considered. South Africa’s higher education system must produce graduates who can function in the world economy.14 “What we need to develop is a more socially responsible citizen – flexible and creative in the workplace, a super worker, a super citizen – to move us into a more competitive global economy. Not one who merely satisfies bureaucratic needs.” Taken as a whole, the five sampled technikons appear to have been able to take advantage of the opportunities that TELP is providing for student access in bridging programmes and developing standardised testing packages for mathematics, science and English. For example, TELP has supported an “Access Programme in Engineering and Technology,” providing bridging support in core subject areas and emphasising female students; a multi-media, inter-departmental laboratory; standardised tests for Maths and English developed with support from the Contractor; a draft five-year student development plan; and a work-study programme. Of course this does not mean that technikons have all made equal use of the opportunities provided through TELP, or that the universities have failed in this respect. As with the technikons cited above, examples of similar project-supported activities may also be found at universities, though they are fewer in number. A few institutions have established five-year student development plans, standardised testing, work-study and bridging programmes. Several universities are making meaningful strides in expanding access to quality education and training. One institution in particular stands out as an example of TELP. The provision of technical assistance and programme models enabled it to translate its vision into a fully operational, multi-dimensional student development programme that includes not only student testing in core subjects and bridging, but also student mentoring and work-study. Although the latter has been in place at this institution for many years, its fine-tuning is attributed to TELP. As staff there put it, there is real synergy between project and institutional activities. 13 The percentage of candidates writing matric who obtained matric exemption declined from 31 per cent in 1980 to 13 per cent by 1998 (Future Fact Conference 2000). In 1998, of the 280,000 candidates who wrote mathematics only seven per cent passed on the higher grade. In the case of science, 157,000 wrote matric and 17 per cent passed on the higher grade. 14 It is estimated that there will be a quarter of a million jobs in South Africa in the information technology sector within five years. But,South Africa has only 45 per cent of the maths teachers required (50 per cent of whom have no formal qualification in their subject) and 40 per cent of the science teachers required (60 per cent of whom have no formal qualification in science). TELP MID-TERM EVALUATION: REVISED FINAL REPORT 24 Disrupted campuses In spite of major student disruptions at three universities with some form of bridging programme, it was reported that the pass and admittance rates and subsequent student performance in the first year are notable. Even in an institution where standardised testing and bridging had been initiated but could not be sustained because of a loss of staff and student disruptions, lecturers were philosophical about lessons learned. They recognised the potential and expressed the hope that they will get another chance to strengthen their student development services once the institution has resolved some of its administrative problems. It was also reported at other institutions that have had to cut back on their academic staff that the provision of bridging programmes has suffered as a result. “Fewer people are being asked to do more. There is a limit to what people will give.” 5.2.1.4 Administration and management development In general the historically disadvantaged institutions had weak and under-resourced management and administration systems with little if any experience of drawing up budgets and managing their finances. Information technology was in the main antiquated and inefficient. As a result of initiatives under TELP, this situation has been greatly improved in most cases, though much still remains to be done. The final service under the DevTech contract was a financial management training programme delivered through Deloitte & Touche at 14 TELP institutions. In most cases it was agreed that this had been of great value, especially in the area of budgeting and budget management. Other institution-specific services in this area provided under TELP include training in performance management (to improve individual skills of non-academic staff and to strengthen management operations and support services), time management, leadership training and development, general management training, diversity and conflict management, interpersonal skills development, and senior executive management training (including strategic thinking, team building, partnering, and knowledge management). 5.2.1.5 Research skills development Institutions of higher education have traditionally had two main functions: teaching and research. The emphasis in the past at historically disadvantaged institutions has been on teaching, particularly at the undergraduate level. Research, especially in the sciences and technology, which requires high levels of expertise and expensive resources, has, for the most part, been neglected. One of the initiatives undertaken by TELP was to assist in building research capacity at the universities and technikons it serves. Six of these institutions have expanded and/or established research units of one kind or another. Enthusiasm is high and the results have already been quite startling. TELP MID-TERM EVALUATION: REVISED FINAL REPORT 25 In one technikon, for instance, the research capacity-building programme has been geared to developing staff by combining research training in theory and practice. Originally, it was intended to train 10 members of staff. Enthusiasm was so high eventually 27 members of staff joined the programme. As a result the institution has discovered a great deal of formerly unrecognised talent. In addition, student research assistants have been trained and employed. The programme is using the local community as its laboratory and has created 12 projects with staff working in teams or pairs on topics as diverse as mathematical problems encountered by grade 12 school children to community health care. The programme is now at the end of its first year and data are being written up. By the end of July they will go into the second phase to work as a team doing consultancy work in the real world. “The TELP money will be used as seed money.” 5.2.2 Joint Activities As with the preceding discussion of institution-specific activities, this section examines joint activities in the focus areas, excluding research skills development (which is not addressed through this modality). We found a mixed degree of satisfaction and impact for the joint activities mechanism. In this section we will examine regional and national activities that address higher education needs, and arising out of such joint activity workshops, certain institution-specific activities, as well as milestones associated with proposal writing and strategic planning. Of the three critical areas, joint activities have been perceived by USAID as being the least successful, but this is not necessarily the case. An important consideration when considering delivery in this area is the number of participants (variables), each of which can have an impact on performance independent of the others. Nor is it just their performance that must be taken into account. Their reported perceptions need to be carefully balanced with other evidence. The joint activities component was added to TELP in order for the project to have another strategy for achieving its stated objectives. Whereas PILs are based on proposals developed by an institution to address its own targeted needs, joint activities are designed to build capacity at historically disadvantaged institutions as well as to address sectoral and institutional needs. The workshop format has been adopted as the basic vehicle for service delivery. Workshops, thus far, have been based on milestones that are proposed by the Contractor and approved by USAID. (See Section 5.4.5). UNCF is responsible for planning, organising and conducting joint activity workshops. These are organized on a regional or national basis for the historically disadvantaged institutions, and are supplemented by institution-specific technical assistance in areas that arise out of the sessions. Joint activities workshops may be further divided into two categories: a) those that are devoted to transformation issues and higher TELP MID-TERM EVALUATION: REVISED FINAL REPORT 26 education sectoral requirements, and b) general training, skills development and capacity-building sessions. The complexity of this approach to joint activities does not seem to have been fully appreciated. TELP’s design, based on the principle of homogeneity, impinges upon the component. The Contractor’s role as implementer of the project is affected by the organisation’s interpretation of milestones that were, in the main, established some time ago. The historically disadvantaged institutions, which are the intended beneficiaries, affect component performance by their degree of cooperation and participation. It also appears that some TELP institutions bring different levels of commitment to joint activity workshops. USAID has its own expectations that are influenced by policy parameters and historical experiences, derived from funding and managing similar types of performance-based education contracts. And all stakeholders bring different perceptions, expectations, needs and levels of capacity. Complaints abound, but if they are examined closely many are directed towards operational aspects of the component. When explored on the basis of outputs and the developmental needs of historically disadvantaged institutions the value of the component becomes evident. The more fundamental matter to be addressed is how to make the mechanism more responsive to institutional needs and sectoral requirements, how to raise overall quality, and how to incorporate incentives into the process for the Contractor to provide required technical assistance to TELP institutions even if it does not directly contribute to meeting milestones. Workshops In relation to the mixed degree of institutional satisfaction with the joint activities mechanism that we found, it must be said in mitigation that the onus is not only on the Contractor to ensure the success or otherwise of joint activities. The point has been made that those attending the workshops also need to be committed to the process. The complaint was voiced that there is often a lack of continuity in HDI staff attendance at workshops. Substitutes who have no knowledge of the focus area are sometimes sent to sessions. Report-back meetings often are not held, with the result that information does not get disseminated. These factors, which are outside of the Contractor’s control, have a direct bearing on impact. Some of the problems in workshops of a general nature may have resulted directly from the “one size fits all” assumption (treating all historically disadvantaged institutions as one homogenous group) that has already been described in the context of the differing curriculum development requirements at universities and technikons. Although USAID suggests that greater creativity on the part of the Contractor would have helped, a question may be raised whether the project’s design erected obstacles that were difficult to overcome consistently. TELP MID-TERM EVALUATION: REVISED FINAL REPORT 27 In some cases participants felt that the College Fund had not made much of an effort to find out exactly what sort of workshops a given institution might need in, for example, curriculum development. One university stated that it wants to move beyond issues concerned with the National Qualifications Framework to more substantive educational issues, but “we weren’t asked what we wanted out of the workshops.” UNCF, on the other hand, explains that absent further input from the institutions themselves, it relied on academic audits carried out by the previous TELP contractor. It was also asserted that the Vice-Chancellors did not question the appropriateness of workshops topics at the Advisory Panel meetings. Coinciding with the evaluation exercise, the Contractor began a series of visits to the TELP institutions that may help address this issue. USAID and the College Fund have also been discussing the need for better integration within joint activities, so that they provide value beyond a workshop on the same topic delivered to multiple institutions. In UNCF’s view, the situation is substantially better in this second year of the project than it was in the first. Still missing in this component is a follow-up mechanism to assess the impact of joint activities on specific campuses. The issue of non-responsive workshops is but an aspect of a much larger problem. TELP institutions have expressed concern over the fact that UNCF has not made a conscious attempt to get to know their institutions as individual entities, nor do they feel that the Contractor has developed working relationships with management. Moreover, USAID explained to the evaluators that it has tried to impress upon the Contractor its view that increased project effectiveness is dependent upon the acquisition and analysis of specific data about individual institutional needs, as well as project benefits. The College Fund responds that it has been continuously in contact with all TELP institutions, “assessing their needs for institution￾based focussed interventions and generic commonalities where institutions could share experiences.” Scheduling One cross-cutting problem is scheduling. Joint workshops often clash with other institutional activities and commitments. This was even the case with the mid-term evaluation. A three-day workshop was taking place on the day one site visit was being conducted, with the result that a number of people were not able to attend the focus group discussion. This particular instance, of course, represents a shared problem between the Contractor and the evaluators. However, many comments testify to the fact that this is a common difficulty. “Around the end of the financial year there is an escalation of events.” “Simple compliance with the milestones is not enough. There must be space for flexibility.” 5.2.2.1 Staff development As with the institution-specific approach, staff development through joint activities overlaps the other areas of curriculum, student and management development. It does not have its own separate budget. Nevertheless, several types of joint activities have addressed this focus TELP MID-TERM EVALUATION: REVISED FINAL REPORT 28 area. These include proposal writing, gender/labour equity, grant management and leadership workshops. They were followed up with a number of activities on specific campuses. An illustration of the point made earlier about the difficulty of assigning success or failure may be found in the workshops on proposal writing (actually funded as a joint activity under the institution-specific component). TELP institutions’ complaints about the lack of differentiation around individual and institutional needs and the poor quality of facilitators were widespread. However, most institutions acquired sufficient skills to develop fundable TELP proposals, and several indicated that they were in the process of developing proposals to submit to other funding agencies. The College Fund has also offered joint workshops in the modularisation of courses to comply with South African Qualifications Authority requirements. Such training forms part of capacity building for curriculum development (see Section 5.2.2.2). Equity is “not about replacing numbers of one racial group with numbers of another racial group.” The new Employment Equity Act, designed to promote equal opportunities in the workplace for the previously disadvantaged, is making additional demands on institutions. While it can be argued that legislation is necessary to force some changes, altering deeply held cultural convictions and prejudices is another matter. In this regard the more-urban institutions report having fewer problems than those situated in rural areas. Gender equity, in particular, continues to require attention across the board. It was felt that there is “a fair amount of resistance” regardless of perceptions to the contrary. It was reported that in a number of cases senior management did not attend gender workshops on the campuses of individual institutions. Some even expressed hostility towards the concept of gender equity. Workshops were also provided to develop guidelines to institutions on how to set up gender mentoring programmes. One problem commonly cited by the institutions with regards to gender equity workshops is that they were too theoretical and not tailored closely enough to specific contexts. They were said to lack analysis and information about impact implications. “It is important to remember that we are concerned with developing long-term awareness. This is something that cannot happen in the short term.” “It is much easier to address equity issues in the student body than in the staff.” It is realised that “equity is about institutional transformation… it boils down to resources and deeply ingrained attitudes.” There is a need for a great deal of education and training. Further it was realised that there is a “need to get everyone to buy in to what the equity act is all about.” TELP MID-TERM EVALUATION: REVISED FINAL REPORT 29 5.2.2.2 Curriculum development It was confirmed that TELP has played a major role in curriculum development. Most historically disadvantaged institutions are on course with regard to meeting the June 2000 deadline to comply with National Qualifications Framework requirements. Joint TELP activities have made a significant contribution in this regard. A complicating factor has been the perception that mixed messages have emanated from the South African Qualifications Authority. Although a number of institutions claimed that they are well on track or even “way ahead of most other institutions,” in modularising their courses, it was admitted that within institutions there are different levels of capacity. Part of the problem, it was asserted, is that staff are often not aware of the need to grapple with and reflect on their own practice. It was pointed out that curriculum is not just a TELP initiative. It must also be part of the institution’s strategic plan. It was confirmed that without TELP funding curriculum development could not have happened. In addition to the joint activities workshops, the Contractor provided technical assistance in response to individual requests for further help. Historically disadvantaged institutions probably would not have been able to meet the requirements collectively without such help. Some of the institutions said that they would have experienced serious difficulties in acquiring pertinent information on a timely basis and marshalling the necessary resources to respond effectively to the new higher education demands. There was unanimity of opinion that TELP’s contribution was critical to the process. Notwithstanding problems discussed below, any measurement of the accomplishments in this area of joint activities would result in an example of outright success in a way that is not the case for most other activities. “I wish they could have come up with two sets of objectives for technikons and universities, like in a class with slow learners.” One of the problems arising with joint curriculum development activities involves the historical differences between universities and technikons described above in Section 5.2.1.2. For example, one institution is the only technikon participating in its region. The others are all universities, with different priorities and problems. This technikon felt that it did not need the first curriculum development workshop, which was about building capacity to deal with the NQF/SAQA. It was better able to cope with the National Qualifications Framework than the universities. “TELP concentrated on baby feeding for the universities.” They were more interested in learning how the other technikons were dealing with it. The point was made that technikons have a much more uniform way of dealing with these kinds TELP MID-TERM EVALUATION: REVISED FINAL REPORT 30 of issues as they are used to working together. They also have the mechanism of the convenor technikons to provide leadership. 5.2.2.3 Student development There has been a significant transformation in the racial composition of students at different universities and technikons. One of the major consequences of the shift in demographics has been that the formerly advantaged white institutions are perceived, especially by the historically disadvantaged universities, as taking the best black students while leaving them with the remainder. Hence, it is felt that the specific project targets that have been set to have students in bridging programmes by year three are unrealistic, particularly since most institutions did not have the requisite programmes in place when this phase of TELP started. The point was made that historically disadvantaged institutions are struggling to get students and are reluctant to place students in such bridging courses because it would have a negative effect on their mainstream numbers. That is, bridging courses are not attractive to students because of the additional time added to degree programmes. There is, however, increasing recognition among TELP institutions that many students stand in need of compensatory programmes, so the problems associated with skills deficiencies and poor preparation generally await resolution and will require renewed emphasis if these types of student needs are to be met. Taken as a whole, four of the five technikons in the sample group and one-third of the universities appear to have been able to take advantage of the opportunities that TELP has provided for increased student access by providing bridging programmes and developing standardised testing packages in mathematics, science and English. This does not mean that they have all made equal use of TELP resources; the rates of progress have been different. The College Fund utilises the Desmond Tutu Educational Trust (DTET) to conduct activities in the student development focus area under joint activities. The Tutu Trust has been responsible for conducting student leadership programmes, holding work-study workshops and helping in the development of standardised testing packages. Although the Trust reports “generally positive” feedback from participant evaluation forms, some staff at TELP institutions voiced complaints regarding the quality of DTET-organised workshops. Generally, it was said that problems have been reduced but never fully eliminated. And the larger question raised often revolves around a lack of understanding of the role of the Tutu Trust under TELP. The Trust’s testing workshops, supplemented by limited technical assistance, are credited with the creation or enhancement of institutional programmes. Approximately 50 per cent of TELP institu￾tions are evaluating first-year students on entry. TELP MID-TERM EVALUATION: REVISED FINAL REPORT 31 Work-study has met with little success. Only a few TELP institutions have used the Tutu model to develop or upgrade work-study on campus. Funding to assist with establishing and operating student work-study programmes, expected by participating institutions and sought by DTET itself, has not materialised. Some wonder how work￾study programmes relate to project objectives and why such an activity is being promoted without financial support. Student leadership programmes were generally well received. Several institutions stated that student workshop participants seem much more willing to talk to management about critical issues for the first time. This is considered by them to be a significant development. However, there can be no sustained benefit from these leadership programmes because the numbers of students involved are so small, plus students are only around for a few years. As one Coordinator put it, “Students leave, but staff stay behind.” 5.2.2.4 Administration and management development Joint activities have made important contributions toward the attainment of stated project results and higher education objectives in several areas, one of which is capacity building around strategic planning. The obvious indicator here is the three-year rolling plan now required of every university and technikon. Thirteen TELP stakeholder institutions have now produced such plans. One of the intentions of these activities was the incorporation of gender/labour equity plans and mentoring programmes into the strategic and three-year rolling planning processes. Leadership and institutional change in higher education workshops were also held to assist this process. Institutional strategic planning frameworks and faculty business plans were also developed. These initiatives were followed up with technical assistance to specific institutions. It should be pointed out that not all joint activities have received equal emphasis in each year of the project’s implementation. Administration and management is a case in point. It is anticipated that there will be a greater emphasis in this area than hitherto. To date, the College Fund has provided support services that foster management, leadership and financial skills primarily through workshops. Staff planning, knowledge and skills have been developed, and at some TELP institutions strategic planning systems have been institutionalised. 5.3 Institutional Linkages Component The primary goal of UNCF in this component is to facilitate the design and implementation of linkage grants that are directly responsive to the specified needs of the historically disadvantaged institutions. Linkages are intended to take advantage of expertise at U.S. institutions of higher education and provide the means for two or more linked institutions to TELP MID-TERM EVALUATION: REVISED FINAL REPORT 32 address broad-based problems, share resources, and engage in joint activities related to the five focus areas. There have been two cycles of linkage grants at which a Review Panel selected proposals. All of the TELP institutions have now established linkages with universities or consortia of higher education institutions in the United States. Based on the experiences gained in the first cycle, the Contractor provided applicants in the second cycle with an orientation session, specific negotiating skills across cultures and guidance in the preparation of budgets. It was reported that some of the historically disadvantaged institutions did not take special care in selecting the people they sent to negotiate their agreements. Development Consultants Network The College Fund’s subcontractor for managing linkage grants is Development Consultants Network (DCN). DCN is involved in all aspects of implementation from the pre-application solicitation phase to the awarding of linkage grants and reporting on component activities. In preparation for the final round of linkages, each South African institution was provided assistance in the drafting of a “focus statement” that specified institutional priority areas. Additionally, UNCF and DCN arranged site visits to the United States for South African representatives in order to help the TELP institutions gain first-hand knowledge of potential linkage partners. Successes In general linkages are seen as a particularly powerful tool for staff development. Five TELP institutions report no problems in setting up their linkages, getting them to work and producing the expected results. This is not to say that during negotiations there was a meeting of the minds on all matters, but rather that when differences did arise they were tackled in an open and professional manner, then resolved to the mutual satisfaction of all parties. These historically disadvantaged institutions are excited by the possibilities that appear to be opening up to them. South African educators are enthusiastic about the sharing of resources and the rapport that has begun to develop with their American colleagues. Within the group cited above, the two TELP institutions involved in linkages the longest describe their relationship as excellent. They feel that they are treated as counterparts of equal standing. Further, these historically disadvantaged institutions assert that their US partners have demonstrated a commitment to the linkages programme. The American institutions go beyond the specific language of their agreements and take initiative in making suggestions to enhance programming and to apply expertise to broad-based institutional problems. At each of these South African institutions, mission statements and strategic objectives have guided linkage goals and activities. The technical assistance and programmes being offered by the US institutions cut across several of the five focus areas. They call for the provision of staff development, including academic, administrative and support staff, TELP MID-TERM EVALUATION: REVISED FINAL REPORT 33 management and leadership training, as well as curriculum development. One linkage has a major research component. “Joining hands across the sea… progress through technology.” Another TELP institution that falls within the most recent round of linkage awards reports that, although its linkage is relatively new, implementation is on a fast track. It is pleased with the quality of the initial activities provided by its American partner. Jointly the linked organisations are making progress in finalising their action plan for proposed technical assistance in targeted areas. In addition to the staff development component, which is common to all successful linkages in this group, other institutional priorities are quality assurance and research capacity skills development. For the remaining two South African institutions that are just getting started, negotiations likewise went smoothly and expectations are high for programmatic benefits. Overall the five historically disadvantaged institutions that express high linkage programme satisfaction have each achieved some degree of integration. Synergies are beginning to evolve between TELP and their institutions. They also each have a strong, capable, enthusiastic linkage manager/coordinator strategically placed within the organisation. In these South African institutions designated programmes are in process, staff are being developed, targeted problems are being addressed and capacity￾building efforts have potential for being realised. It should also be mentioned that some among this group are well on the way to ensuring linkage sustainability beyond TELP. Several US-South African linkage partners are already exploring areas for possible future long-term collaboration. Frustrations and disappointments However, the majority of stakeholder institutions reported significant frustration and disappointment over the nature of their linkage relationships and the quality of services provided under it. For example, one South African partner explained, “We are in the second phase.” Initially the institution had difficulty finding a partner in the United States. There was no response to their advertisement in the first round. Finally, a US partner applied, but it appeared to the South Africans that it had been pushed reluctantly into the arrangement. One result was that the US institution appeared to want to keep most of the budget. In their initial draft, according to this South African partner, the Americans proposed reserving 98 percent of the funds for their own use. (It should be noted that the TELP Review Panel did approve initial project budgets, and that the TELP institutions were given the opportunity to make changes during their site visits.) The South African partner was quoted as having said, “The grant was after all aid for them.” Furthermore, the Americans assumed that they were in charge, while the South Africans thought that they were equals. This lack of clarity TELP MID-TERM EVALUATION: REVISED FINAL REPORT 34 regarding their respective roles led to extensive, and for the South Africans exhausting, negotiations. Some TELP institutions assert that “no provision was made for the meeting of two very different cultures.” The partners had two different notions of what a contract means and how it is arrived at. Negotiations were consequently very difficult. From the perspective of these institutions, the prevailing attitude seemed to be that the American way is understood by the South African partners and that that is how things would be done. On the other hand, the Contractor cites numerous attempts to strengthen the position of the historically disadvantaged institutions, including an orientation workshop on negotiating memoranda of understanding and an offer (accepted by only one visitor) to communicate with South African representatives while they were visiting the United States. “US assumptions need to be interrogated. American attitudes are subtle. They want to help but they assume they have all the answers and don’t really hear the South African negotiators/ partners” Further, it would appear that the process assumed South Africans have skills that are taken for granted in the US, even though they do not. In addition, it was asserted that because of their relative lack of negotiating experience many South Africans at the historically disadvantaged institutions do not understand the nature of contracts, that “once the memo is signed, it is signed” and negotiations are at an end. Of course, there are many political, historical and psychological reasons for such perceptions, but the fact is “they exist and should be taken into account in setting up linkages in future.” Not only are linkages where “the two cultures meet,” but there also appears to be considerable confusion regarding what is meant to be achieved, how it is meant to be achieved and the logistics of putting linkages in place. Most of these problems can be attributed to poor communications. The very structure of the linkages component seems to undermine effective communications. The complaint was voiced on the South African side that information provided in documents was often couched in obscure legalese that left the South Africans, at least, in some doubt about what they could expect or demand. The Contractor, on the other hand, notes “an erroneous expectation that the US partner is responsible for initiating communication.” At the April, 2000, post-award conference institutional coordinators were strongly advised to initiate contact, since they are in a better position to know when an issue should be addressed. One South African project manager reported having to provide a member of the academic staff at his institution who was going to the United States with the airfare from his own pocket. A few other institutions that have limited funds and no access to credit have experienced difficulties in TELP MID-TERM EVALUATION: REVISED FINAL REPORT 35 starting up because they do not have adequate financial resources to support activities prior to receiving funds from the U.S. institution. Financial arrangements were further complicated by US financial systems that make it difficult to send money to South Africa. Quality assurance No provision is made for on-site monitoring in the United States. Official reports come quarterly from the US institutions, with no procedures for regular feedback about linkages from the TELP institutions themselves to the Contractor or USAID. The Contractor notes that the linkage component’s travel budget is inadequate for monitoring purposes. No mechanisms exist for assuring the quality of the US inputs or for taking corrective action in the event of unsatisfactory performance from an American (or, for that matter, a South African) institution. The result is a wide variance in perceptions about the component’s success, ranging from positive on the American side to negative on the part of many (although not all) South African institutions. These structural problems with reporting and quality assurance assume particular importance because some of the American partners, according to South African perceptions, do not appear to be taking the linkages seriously. It was claimed that staff on the American side were not always fully committed to the linkage and were putting there own concerns and interests ahead of the work that they should be doing to foster the partnership. Complaints were also voiced that some American institutions had made exaggerated claims about what they could provide and are now unable or unwilling to deliver on those promises. On one site visit we witnessed an example of such problems, attending a workshop run by a consultant from a U.S. linkage university who, in our judgement, delivered a presentation of unacceptably low quality. Unequal partnerships The legal basis for the linkages has also raised concerns. The Contractor signs a sub-agreement with each American partner. The South African and American partners sign a Memorandum of Agreement, as required by USAID regulations, which the TELP institutions feel does not have the same contractual force although, in fact, it is incorporated into the binding contract. Only one institution expressed an understanding of the difference between the memorandum and the contract. They felt that they were locked into a relationship in which they had very little power to change or challenge things not to their satisfaction, in spite of the fact that DCN told TELP institutions that they “had the authority to negotiate a memorandum of agreement which could change everything from budget to timeframes to activities.” DCN notes that misconceptions about linkage grants still abound, with some personnel at both South African and US institutions unclear on many points. Problems often arise because busy faculty and administrators do not always read key documents, share important information with colleagues and raise questions when necessary. TELP MID-TERM EVALUATION: REVISED FINAL REPORT 36 A more basic concern arises around the relationship between two linked institutions. Far from being an equal partnership, which is what the South Africans expected it to be, it seems a highly unequal one in which the South Africans are on the receiving end. Instead of empowering them, the relationship ironically disempowers them. 5.4 General Findings 5.4.1 Project Success Factors Determining project success at any institution at any given time is not an easy task. There are a combination of factors and degrees of success and failure within a single institution. The dynamics underlying them and militating circumstances need to be taken into account. Major factors The factors that appear to contribute the most to TELP success on campus include the following: · a committed and pro-active Vice-Chancellor, · a competent TELP Coordinator, and · the integration of TELP into the institution’s strategic plan. Conversely, the absence of such determinants limits potential project benefits. There is no avoiding the fact that in many institutions there is a leadership crisis which impacts negatively on TELP’s performance at these places. The absence of a committed and pro-active Vice-Chancellor with a clear and uncompromising vision inevitably means that TELP is not integrated into the institution’s strategic plan. In plain terms, this means that the institution is not able to take full advantage of the opportunities that TELP provides for capacity building in the five focus areas, identified by the Vice-Chancellors themselves as being crucial for their institutions’ up-grading. A lack of leadership often results in absent or uncoordinated action, with little follow-through on campuses, and sporadic attendance at TELP Advisory Committee meetings. The quality of the TELP Coordinators is also of vital importance. If Coordinators are incompetent or unmotivated it is obvious that TELP must suffer. For it is they who must provide essential information to their institutions as well as maintaining communications with a host of external and internal individuals and agencies such as project managers, the Contractor and other institutions of higher learning. Further, if they are not supported by their Vice-Chancellor or lack status within an institution their role and, therefore, their performance must be compromised. Lesser factors Lesser factors that may influence performance include, but are not confined to: the presence or absence of good project management and TELP MID-TERM EVALUATION: REVISED FINAL REPORT 37 coordination; the level of and continuity of staff involvement and commitment; the quality of institutional leadership; organisational infrastructure and existing capacity. Some institutions have a greater measure of experience than others in implementing similar projects. It was remarked at several institutions that synergies had developed between TELP, institutional programmes and other development projects to the extent that distinguishing one from the other was difficult. In at least half of the institutions, internal politics have had a debilitating effect on TELP’s ability to function effectively. The assumption under which TELP was conceptualised, that all disadvantaged institutions have similar problems and should be treated the same, is another factor that limits efficacy of the project. Currently, no formal on-site evaluations of the project are taking place. However, there is considerable informal evaluation with lessons learned being reflected in the increased capacity of staff. The point was made that there is, in some cases, “no baseline data” and, therefore, internal impact assessment is impressionistic at best. 5.4.2 Staff Development The training of staff is one of the notable successes for TELP. Institutional linkages have provided significant opportunities for selected historically disadvantaged institution staff members to interact with colleagues at tertiary education institutions within South Africa and the United States, and to receive advanced technical and post-graduate degree training. In addition to these linkages, all of the historically disadvantaged institutions benefited from having staff awarded scholarships for post-graduate degrees. In total from 1996 to 1999, 74 members of staff have been involved in such programmes in the United States. Long-term staff development mechanisms facilitate the continued success of the project and contribute to staff development and capacity building. Retention Those institutions with a low staff turnover benefit the most from the continuity of having a stable staff. Such continuity was especially evident in those institutions where leadership was strong and involved and where top management has been able to inspire staff with a shared vision. On the other hand, many institutions reported that it is difficult to retain staff, especially those trained overseas. One institution went so far as to describe itself as “a staff poaching pool.” A number of historically disad￾vantaged institutions wondered how to tie staff to their institutions by means of service contracts and other inducements once they had received training. It was acknowledged that when staff are being headhunted there is little the institution can do if money is made available to buy them out of their contractual obligations. It was also conceded that while the institution itself may be the loser, the country would not lose the benefit of the investment made in their training. TELP MID-TERM EVALUATION: REVISED FINAL REPORT 38 Some institutions experience problems with staff release. When staff have been sent away for training, existing resources are stretched. Many institutions cannot afford to carry a double salary. In addition to the financial implications, which for some organisations are problematic, those remaining behind often find that instead of being able to give all highest priority to ongoing responsibilities and new project activities, they are required to divide their time and attention more and more. 5.4.3 TELP Coordinators One of the most significant findings to come out of the site visits for this evaluation is the important and pivotal role of the TELP Coordinator. Almost without exception, at institutions where the TELP Coordinator is committed and energetic the programme has achieved meaningful results. This is true even of troubled institutions where the Coordinator not only has the backing of the management team but also has some status and authority in the organisation. The greatest success occurs when Coordinators are good communicators, have an understanding of higher education and are backed by a strong Vice-Chancellor. Roles and duties There is a lack of clarity regarding the roles and duties of the Coordinator. Coordinators report to the Vice-Chancellors, but there is confusion regarding their relationship with the Contractor. The Contractor expressed considerable frustration in this regard. The Coordinators are their “life blood.” Yet the Contractor has an undefined relationship with them. They are appointed by the institutions and their salaries are paid by USAID. The Coordinators also feel frustrated by this undefined relationship. A related issue is that at a number of historically disadvantaged institutions Coordinators have taken study leave. This circumstance, plus high turnover, has had a detrimental effect on the project. The evaluation also uncovered the fact that several Coordinators serve part-time. They manage TELP while continuing to perform other major institutional responsibilities. In other words, the institutions concerned are not fulfilling their obligations to the Donor and are in violation of the Agreement. Logistics Another challenge concerns logistics. TELP Coordinators indicated that they often have difficulty arranging project activities because requested dates frequently conflict with the institution’s academic timetable, coming at times when heavy demands are made on staff (such as during examinations). The requirement for extended semi-annual reports in June and December adds an additional burden, as these are the times when students and marking place unusually heavy demands on staff time. TELP MID-TERM EVALUATION: REVISED FINAL REPORT 39 5.4.4 Project Management Structures In our view appropriate basic management structures exist at USAID/SA, the College Fund and the historically disadvantaged institutions to accommodate project implementation. USAID/SA USAID has set up a TELP management team within the Mission’s higher education division that has overall responsibility for managing TELP, including oversight of contractors and grantees. This team is composed of a Higher Education Team Leader and a Project Development Specialist. Senior Management has taken an active role in monitoring TELP. The Team Leader and the Deputy Team Leader for Education support the Higher Education Team. They participate with them in providing technical guidance to UNCF. Meetings between USAID and the Contractor are held on a regular basis to evaluate project progress, to address implementation issues, and to discuss strategies for problem resolution. Additional USAID Management Team responsibilities include, but are not limited to, reporting to the TELP Advisory Panel, the procurement of services outside of the contract and the proper stewardship of public funds. The College Fund The College Fund, the current prime contractor for the implementation of TELP, has established a structure in which designated positions are aligned with key project components. The Chief of Party is responsible for overall project leadership, liasing with USAID and the Department of Education, supervision of staff and fiscal administration. The Deputy Chief of Party’s portfolio includes general project implementation duties and coordination of linkages. Completing the key staff complement are Project Managers for TELP Joint Activities and Institutional Development, financial officer and general support staff. The Advisory Panel The Advisory Panel has an important monitoring function. It is composed of all the Vice-Chancellors of the historically disadvantaged institutions, representatives from the Department of Education and USAID/SA. UNCF serves as the secretariat. Its effectiveness has been compromised by a lack of commitment by some Vice-Chancellors who attend irregularly. Historically disadvantaged institutions All TELP institutions have organised basic project management structures. It should be noted, however, that some institutions have elaborated much more sophisticated project management structures than others. In addition to a general campus advisory committee and the positions of Coordinator and project managers, some have developed TELP manage￾ment forums that meet regularly and have direct access to the Senate and other pertinent decision-making bodies. At one historically disadvantaged institution organisational stakeholders elected the Vice-Chancellor to the Chair, at two the Deputy Vice-Chancellor has been chosen to lead TELP, TELP MID-TERM EVALUATION: REVISED FINAL REPORT 40 and at several others faculty and other participant representatives are elected. Such formal structures have significant impact on TELP’s visibility and success on campus. This does not mean to say that the entire question of leadership and the management of TELP should be reduced to the Vice-Chancellors only. It is an institutional issue. Structures, however, are not enough to ensure effectiveness. Even when organisational structures are in place, if communications within the institution are poor and there is limited dissemination of information, project implementation is impeded. At one historically disadvantaged institution the TELP Coordinator has set up eight committees, but they do not communicate with each other; there is no overall coordination. With each committee acting in isolation, project activities are fragmented and entire segments of the management and academic staff have little or no knowledge of TELP opportunities. Another factor hindering TELP is that changes in top management at some institutions have had a deleterious effect. One institution, for example, has seen three Vice-Chancellors since TELP began. In the early stages of the project, there were significant communications problems between USAID and the historically disadvantaged institutions. These have since been largely resolved through several interventions: strengthening the Advisory Panel, appointing the TELP Coordinators, and internal changes at USAID to improve staffing and management systems. Similar problems bedevilled the early relationship between USAID and its current TELP contractor. The College Fund asserts that both parties have made significant strides towards improving their communications. However, some USAID staff members continue to express negative judgements about the Contractor’s capabilities. This perspective tends to impede USAID’s ability to manage and assess the project. 5.4.5 Milestones TELP services are delivered by the College Fund under a performance￾based contract awarded by USAID/SA. Theoretically, this type of contract requires less supervision from USAID and affords the implementing agent greater flexibility. The actual implementation of TELP through a milestones approach has, however, been problematic. Milestones, proposed by the UNCF home office and incorporated in their original contract, are intended to demonstrate key accomplishments that indicate progress towards the achievement of results specified by USAID. During the life of the project, milestones may be renegotiated but results are cast in concrete. USAID/SA expresses concern that the Contractor interprets milestones literally and not in the broader spirit intended. Milestones are said to be routinely delivered as narrowly defined, discrete activities, not as requisite building blocks in a progressively constructed approach to capacity building. USAID states that while focusing on meeting specific milestones the Contractor misses opportunities to achieve other equally important outcomes, such as finding synergies in joint TELP MID-TERM EVALUATION: REVISED FINAL REPORT 41 activities rather than merely delivering events such as individual workshops. The Contractor’s past progress reports are described by USAID as essentially lists of activities completed, devoid of analysis or discussion of individual institutional needs and project benefits. UNCF/South Africa, on the other hand, expresses considerable frustration with the milestone mechanism, arguing that it is particularly problematic in a transformation environment. The Contractor acknowledges difficulties in the early stages of project implementation, but believes that over time it has made meaningful adjustments in service delivery. The College Fund is of the view that, although they are accused of being too focused on milestones that are specified deliverables approved by USAID, they have worked hard in the past two years to get selected milestones changed. UNCF/SA staff express consternation over what they perceive as a lack of recognition for their contributions to TELP, and feel that they are being unfairly blamed for the shortcomings of a project design that is based on the underlying assumption that “one size fits all.” According to the Contractor, all stakeholders should assume some responsibility for TELP’s successes and failures. The TELP institutions are the most negative about the practical implications of milestones, which they sometimes call “tombstones” or “millstones.” The institutions voice the general complaint that milestones are not responsive to their needs, and that deadlines are frequently very tight. Milestones have to be met in a given time frame, and because there appears to be inadequate planning on the part of organisers, institutions are given little notice, ending up with the feeling that “…we are no longer in control and we are taken over by the process.” One interviewee said: “it feels like you have to have a workshop because you need to have a workshop” – not because a workshop on a particular topic is appropriate, but because it was necessary in terms of meeting milestones. It was felt that quality is compromised as a result. Questions were also raised about whether or not sufficient cognisance is taken of academic calendars when activities are planned. Of all the issues examined during this evaluation, those surrounding milestones are some of the more contentious and complex, yet none are potentially more vital to project progress. The above discussion points up only a few of the differences in perceptions about milestones. Other related issues concern an obvious discrepancy gap in performance expectations held by the various stakeholders, and the lack of international development and project management/implementation experience on the part of the Contractor. 5.4.6 Communications There is a definite need, as well as an expressed desire by project managers and Coordinators, for better sharing of information among institutions. All participating institutions appear keen to interact with each other. They view TELP as a natural vehicle through which strong, shared felt needs could be met. First, they want to have organised opportunities TELP MID-TERM EVALUATION: REVISED FINAL REPORT 42 created that would enable them to meet, develop and cultivate collegial relationships. These kinds of interactions may be taken for granted by educators in other societies, but under apartheid each institution was required to operate in isolation. Therefore, the facilitation of structured forums dedicated to the exploration of philosophical and intellectual pursuits, especially in the midst of transformation, would contribute immensely to the development of professionalism and the broadening of horizons, while building capacity across the sector. Second, institutions are interested in having TELP create communication channels that would encourage and facilitate regular exchanges of information about concrete programme developments. Innovative initiatives, worthy of sharing, exist on every single campus, no two of which have strengths in the exact same areas. Examples that lend themselves to adaptation by others include a community-based research model, technology-based instructional methodologies and teaching tools, a fund-raising programme, a student affairs conflict-resolution programme, and guidelines on how to publish. Without question, all historically disadvantaged institutions would be enriched by shared information about such models. A broader issue concerns communications between the historically disadvantaged institutions, the Contractor and USAID. Some TELP institutions have complained about a perceived lack of feedback from the Contractor on the semi-annual reports. However, UNCF provides written feedback to the institutions and forwards the reports to the Advisory Panel. TELP MID-TERM EVALUATION: REVISED FINAL REPORT 43 6. Conclusions It may be said with certainty that TELP has already made considerable contributions to the historically disadvantaged institutions. As discussed in the Capacity-Building Component and General Findings sections (5.2 and 5.4, respectively), there have been notable and exciting achievements which, it is acknowledged, could not have happened without TELP. Contributions have been made by the project in all five focus areas, contributions that have substantially expanded individual capabilities and enhanced institutional development. This section, Conclusions, further elaborates upon other TELP accomplishments and provides an overall status report on project strategies and benchmarks. It also takes cognisance of some project shortcomings, and examines the effect of transformation and the rapidly changing higher education environment on participating institutions. 6.1 Relevance of Current TELP Strategies The South African higher education system, including the historically disadvantaged institutions within it, is without question still in a state of transition. The constraints that TELP was designed to address have yet to be resolved. Therefore TELP strategies to develop capacity and foster institutional linkages are still relevant. These development constraints had their origins broadly in apartheid and more specifically in the education system designed to support it. In the past the system perpetuated an inequitable distribution of access and equity for students and staff. There continue to exist disparities between historically black and white institutions in terms of faculties and capabilities. TELP now supports the Department of Education and, within that support, 17 historically disadvantaged institutions. Its purpose is to strengthen policy development capacity at the Department of Education, to increase access of disadvantaged South Africans to tertiary education, and to build capacity and institutional linkages at the historically disadvantaged institutions. Those institutions receiving TELP support report that the project provides benefits that would otherwise not have been possible. The five focus areas are no less relevant now than when the Vice-Chancellors first identified them. Further, it is our impression that where TELP is having an effect it is contributing to the psychological well￾being of participants as much as it is to their professional development. Motivation and morale have improved in those institutions where TELP is Section 6 TELP MID-TERM EVALUATION: REVISED FINAL REPORT 44 having a positive impact. As one member of the academic staff put it: “TELP offers opportunities for (achieving) some of our wildest dreams.” 6.2 The Changing Institutional Environment The environment of the historically disadvantaged institutions, jointly and severally, continues to evolve rapidly. If anything, the pace of that evolution is increasing. As this report is being written, the higher education sector awaits the imminent publication of the ‘Size and Shape’ policy document, which undoubtedly will inaugurate a new round of change. The new gender/labour equity laws are also having an effect. We found that current TELP mechanisms and activities are able to respond to these changes, although the institutional capacity still varies substantially. TELP strives for flexibility in its programmatic responses to the changes wrought by the transformation process still underway. Institutional adaptations vary considerably. At one end of the spectrum there are some historically disadvantaged institutions that are excited by the challenges produced by the ‘Size and Shape’ document. They appear to feel secure, are likely to remain so and are in the forefront of harnessing TELP capacity-building opportunities: “each institution should be future focused.” At the other end of the spectrum, there are significant numbers that believe their very existence is threatened. Two such institutions are in a state of upheaval because they face massive retrenchments. As a consequence, it has been very difficult to get their personnel interested in or committed to TELP. Staff are concerned with survival more than anything else. One institution has had three Vice-Chancellors in almost as many years. Morale is low and rumours abound that they will close by the end of the year. As one member of staff expressed it, “What’s the point?” On the other hand, a senior administrator at a different institution expressed the opinion that regional political representatives would not allow anything adverse to happen to their institution. The perception that political loyalty will protect them persists. Many institutions are facing financial and management crises. The financial crisis is exacerbated by the decline in enrolments. Student disruptions are a continuing challenge. There is a need for TELP to continue targeting resources to address these issues. 6.3 Integration of TELP Activities At some institutions TELP is firmly integrated into institutional plans and strategies. At others the linkage is much weaker. What stands out clearly is that integration correlates strongly with project success. Where TELP is perceived as part of the institution and functions as such, TELP inputs have been most successful. TELP MID-TERM EVALUATION: REVISED FINAL REPORT 45 “TELP works when you blur the boundaries between what is TELP and what is not.” Some of the historically disadvantaged institutions have achieved such a high degree of integration of TELP activities that they reported it is not always possible to tell what is a TELP initiative and what is not. The arena in which integration has taken place most successfully is at the technikons. However, it must be noted that such integration is not uniform and, in fact, a few instances may be found where there is a low level of integration yet specific project activities are succeeding. The same can be said to a lesser degree of universities as a group. Those institutions where TELP activities have been guided by their mission statements and integrated into their strategic plans have had the greatest amount of success in achieving project results and in strengthening human resource and organisational capacity. In approximately one-third of institutions, where TELP is performing successfully, its campus profile is high, its management is broad-based, and there is a strong correlation between the successful implementation of TELP activities and its integration into three-year rolling plans. A case in point is an institution that reports that its whole campus, including students, was involved in deciding what the focus of their first PIL should be. Since their initial involvement in TELP regular meetings have been held to keep everyone informed of developments. Activity leaders report to their Deans who in turn report to the Deputy Vice￾Chancellor, and the Vice-Chancellor is actively involved in the whole process. Consequently, TELP has been completely institutionalised and its activities are believed to be sustainable after funding is withdrawn. TELP activities have been targeted at specific areas in the strategic plan, thereby ensuring that lessons learned can more readily be integrated into future decision-making. This example, however, although important to demonstrate possibilities, is the exception rather than the rule. 6.4 Progress in Relation to Benchmarks Utilizing project benchmarks as a measurement of performance reveals that TELP definitely is making progress towards achieving its stated results. As noted above in Section 6.3, the degree of this progress depends largely on the institution itself. We estimate that TELP can currently be judged a success at approximately 40 per cent of the institutions it serves. That is, these institutions have, in some combina￾tion, integrated TELP into their strategic plans while tending to have committed, pro-active Vice-Chancellors and effective Coordinators. Technikons comprise a disproportionate proportion of this group, although it does also include universities. This does not mean that TELP has failed in all of the remaining institutions, only that its success there varies, with TELP MID-TERM EVALUATION: REVISED FINAL REPORT 46 several institutions demonstrating pockets of value even in the face of broader problems. Progress toward stated results has been made through joint activities. The following data provide an overview of progress toward achieving benchmarks. Most historically disadvantaged institutions are developing and using revised curricula in order to conform to the requirements of the National Qualifications Framework. Approximately 50 per cent are evaluating first-year students on entry. A minority of institutions is providing compensatory/bridging programmes, with varying degrees of success. Thirteen of the 15 historically disadvantaged institutions have developed three-year rolling plans, while two have been given an extension because of the difficulties they are having transforming themselves. A small minority of institutions has implemented viable work￾study programmes. All have signed memorandums of agreement with educational institutions in the United States. 6.5 Cost-Effectiveness Annex 10.3 provides a detailed report and analysis of findings about TELP expenditures, drawn primarily from data available through USAID and from the 50 per cent of the sampled TELP institutions that returned completed surveys. It reports how the total TELP budget of $51.2 million will be expended across various categories. Key findings in this regard include the following. · $44 million is covered by bilateral agreements between USAID and the Department of Education and a further $7 million takes the form of unilateral USAID funding (for expenditures prior to the Bilateral Agreement and ongoing USAID management costs). · The largest proportion of the funding is to flow to the institutional contractors (57.6 per cent) and to PILS (30.2 per cent) – together accounting for 87.8 per cent of total funding. · At the close of the project the largest proportion of project funding (44 per cent) will have been channelled through the current contractor, UNCF. · Most expenditure (83 per cent) in the first two years of the College Fund contract has been on the joint activities component. Of the $6,330,210 provided, $5,256,338 was spent on this component. From these funds 40.2 per cent supported curriculum development, 28.9 per cent student academic development, and 30.9 per cent management and administrative development. · TELP funding includes amounts to $6,925,000 across the life of the project to support the linkages programme. TELP MID-TERM EVALUATION: REVISED FINAL REPORT 47 · From the 14 linkages set out it is evident that USAID funding of almost $6 million attracts further funding to the extent of $3.6 million from the U.S. counterpart institutions. Ignoring the cost of administration, this amounts to a 60 per cent matching proportion. When considering the cost effectiveness of the project the fact that the USAID funds facilitate further funding must not be over looked. A 60 per cent matching factor is significant. · The performance-based nature of the UNCF contract precludes any real determination of the cost-effectiveness of project achievements even when activities are considered successful in terms of outcomes that satisfy benchmark indicators. The specific costs for a particular activity or series of activities may be known, but because of the unavailability of baseline data on overhead and associated management costs little light can be shed on cost-effectiveness. However, determinations can be raised about general worth. In general, all stakeholders agree that TELP can deliver significant value to historically disadvantaged institutions. When asked in the survey whether, if given the opportunity to apply for further funding from USAID for TELP￾supported projects, the institution would do so, the responses were unanimously positive. An often-expressed desire for sharing information and expertise between institutions has the potential, if organised properly, of tremendous cost savings, hence increased value for funds committed. There are highly innovative initiatives that have been developed as a result of TELP, but which in the main are unsung. Two such examples that would lend themselves easily and advantageously to replication include a research model developed by one of the historically disadvantaged institutions involving 27 members of staff, plus students, in a collaborative community needs-based research project, and a materials development project undertaken by a consortium of five institutions to produce indigenous mathematics and science textbooks and teaching materials. Both have contributed significantly to the development of research skills, fostered professionalism, enhanced collegiality, increased learning and teaching opportunities, produced new authors, and added immensely to personal and organisational capacity. 6.6 Project Results Since inception the purpose of TELP has been to 1) increase access by disadvantaged South Africans to tertiary education opportunities and resources, and 2) to improve academic, administrative, and research capacity of the historically disadvantaged institutions. In 1995 TELP focused on strengthening the following three critical areas: · policy analysis and planning · capacity building TELP MID-TERM EVALUATION: REVISED FINAL REPORT 48 · institutional linkages. Since 1996 an additional result has been included: increasing access and addressing issues of equity by designing and implementing programmes in the five focus areas. All final results were intended to enhance 1) the development process of the higher education and training sectors, and 2) the linkages between those sectors. Intended results Policy: the intended result was to develop and enact policies for transformation. Systems: The intended result was the preparation and design of three￾year rolling plans for historically disadvantaged institutions. Capacity: The intended results were to strengthen human and organizational capacity and to develop capacity as it relates to the development of policy and the design of systems. With regard to the key Strategic Objective to increase access to quality education and training, the indicator for this overarching result is: “Increased percentage of graduates that are black from HDIs as a percentage of all university and technikon graduates.” Actual results Policy: The result achieved was the development and enactment of the National Policy on Higher Education. Reports are imminent to assist with the location of, and the different options for, the agricultural and nursing colleges, as well as on the legal implications for their incorporation into the higher education system. A new regulatory framework for private higher education has been developed and will be promulgated shortly. Research has been carried out into the incorporation of the public distance teacher education colleges into higher education. Teacher education was previously a provincial function and, therefore, not part of higher education. Research has also been carried out into a new funding formula for higher education. Workshops have been held on, and a full-time consultant contracted to support, institutional planning processes, funding and management information systems, as well as on mergers and cooperation within higher education. Systems: In all but two institutions three-year rolling plans have been drawn up and submitted to the Department of Education. All the historically disadvantaged institutions have upgraded their financial systems, while some have put new management information systems in place. Capacity Building: Capacity building is essentially an on-going activity. Nonetheless, while results are not uniform, there is no doubt that significant capacity building is taking place. Academic staff development occurs in activities concerned with curriculum development, especially TELP MID-TERM EVALUATION: REVISED FINAL REPORT 49 materials development in the areas of mathematics, science and English, but also in raising awareness and understanding of the National Qualifications Framework and the requirements of the South African Qualifications Authority. The establishment of research units and/or projects has had a remarkable effect on the morale of staff, a number of whom are already beginning to produce their first research articles for publication and/or delivery at conferences. With regard to specific academic training, the activities organised under the linkage component have been the chief vehicle in this area. However, a number of PILs have been specifically targeted at staff development. Some of these include efforts to improve the skills of staff to better equip them to meet teaching and administrative challenges, developing a post-graduate leadership training programme, and improving the professional skills of library staff. TELP has supported an “Access Programme in Engineering and Technology,” providing bridging support in core subject areas and emphasising female students; a multi-media, inter-departmental laboratory; standardised tests for Maths and English; a draft five-year student development plan; and a work-study programme. Key Strategic Objective: Shifts in demographics and absolute numbers within the higher education sector are a consequence of forces beyond TELP and over which it could have no influence. As stated earlier, there has been a simultaneous decline in absolute numbers at the historically disadvantaged institutions and a shift of population to the formerly white institutions. The conclusion is, therefore, that the key Strategic Objective of increased access and percentages of black graduates is no longer achievable. Unintended results A few results stand out in particular that were not intended. The first of these concerns the effect of the commitment to build research capacity. The enhancement of morale and self-image among staff at historically disadvantaged institutions, as well as an increased sense of professionalism as a consequence of the opportunities provided in building research capacity and post-graduate study, were unforeseen. Secondly, the disempowering nature of some linkage relationships, in which some South African institutions have been frustrated at discovering that far from being equal partners with the US institutions they have in fact been cast in the role of clients, was unintended. In addition, the performance of US institutions is not monitored and this has contributed unintentionally to some degree to allowing them to dominate the relationship to the detriment of some of the South Africa institutions. Lastly, the milestones which were intended to facilitate delivery have, in fact, had the unintended result of frustrating a number of the role players who have felt hamstrung by them, because this mechanism appears resistant to modification and, therefore, unresponsive to individual needs and realities. TELP MID-TERM EVALUATION: REVISED FINAL REPORT 50 6.7 Sustainability The issue of sustainability cannot be separated from the level of integration that TELP has achieved in any given institution (see Section 6.3). Those historically disadvantaged institutions where there is a high degree of integration, where TELP becomes part of the strategic plan and has the active support of the Vice-Chancellor, have a much greater chance of sustaining TELP activities when funding is exhausted at the completion of the project. The ultimate objective of TELP is that it should achieve such a degree of integration into the life of the universities and technikons that it should be taken virtually for granted as part of the normal way of conducting business. Those institutions that are approaching this ideal have a better chance than those that are not. Our findings suggest that, at this stage, the prognosis for sustainability for TELP at institutions collectively is uncertain. However, if the issue is explored according to the levels of success achieved within the project, a more precise forecast can be made. It can be said that for approximately one-third of the historically disadvantaged institutions TELP activities are sustainable. This optimistic view is based on the notable achievements of these institutions. Within this group, project activities are generally integrated into strategic plans and institutional activities. There are many areas of synergy between TELP and such institutions. Key staff are being developed and greatly improved systems, practices and procedures have been institutionalised. The successful TELP institutions have raised professional standards and achieved new ways of thinking about themselves and their potential. Some are now being recognised by others as experts in given fields of endeavours. Horizons are expanding and institutional dreams and goals are being adjusted accordingly. As a group, the TELP achievers have strengthened management systems, improved the quality of educational opportunities being offered to the next generation of leaders, and are well on the way to redefining their institutions. “As projects mature they start to have cross-linkages with other projects.” TELP, of course, cannot and should not be able to take full credit for the accomplishments because of integration. But the project is credited with contributing significantly to the development of these historically disadvantaged institutions by bringing direction and exposure to new knowledge, as well as by providing resources and mechanisms that facilitate broad-based problem resolution and the achievement of desired goals and objectives. TELP has helped historically disadvantaged institutions realise dreams that might not otherwise have been possible. These institutions have experienced a paradigm shift as a consequence of the project. They report that a new, positive institutional culture has emerged. In such institutions staff tend to reflect on their practice. For TELP MID-TERM EVALUATION: REVISED FINAL REPORT 51 example, they see curriculum development as an on-going process and express excitement about the vast possibilities that research activities contribute to academic life. Further research at several institutions is on a trajectory that appears to have literally changed lives, built capacity and strengthened institutional and vocational loyalties. Other achievements common to most of the successful institutions are new and/or substantially improved communications, decision-making and management information systems, and targeted management and leadership training for staff throughout the organisation. For the majority of TELP institutions, however, this is not the case. In nearly all institutions selected individual activities may continue to exist beyond the project. Nevertheless, the overall sustainability of TELP for most institutions will be dependent upon the nature of future project support in the five focus areas and the institutions’ own willingness and commitment to change, grow and make the necessary concerted efforts to enhance organisational strengths while addressing their own problems and weaknesses. The process of building requisite capacity to achieve sustain￾ability among the majority will be determined, or at least impacted, by TELP’s strategic decision-making about future programming and service delivery to the stakeholder institutions. Our findings lead us to conclude that any possibility of project sustainability for most historically disadvantaged institutions will depend on whether or not TELP decides to begin differentiating and delivering its assistance based on levels of HDI capacity, specific institutional needs and realities. TELP MID-TERM EVALUATION: REVISED FINAL REPORT 52 7. Unresolved Issues 7.1 Result Indicator During USAID/South Africa’s last strategic review in 1998 the key Strategic Objective result to be achieved was refocused to increasing access to quality education and training. The indicator for the overarching result is: “increased percentage of graduates that are black from HDIs as a percentage of all university and technikon graduates.” What was not predicted at the time was the dramatic shift in student demographics that would occur throughout tertiary education institutions nationwide. Historically disadvantaged institutions continue to enrol a majority of black students, but this figure as a percentage of all students is shrinking because increasing numbers of black students are enrolling in historically white universities and technikons. The result indicator has not been, and cannot be, achieved under current conditions. This circumstance, however, is through no fault of TELP per se. Transformation and the changing higher education environment have rendered the indicator unattainable as stated. 7.2 TELP Advisory Panel The TELP Advisory Panel functions as a monitoring body whose efficiency and effectiveness are determined by the commitment and participation of its members. If their commitment and participation are lacking it is difficult for the Advisory Panel to fulfil its function. Rules and expectations regarding membership need to be resolved. Section 7 TELP MID-TERM EVALUATION: REVISED FINAL REPORT 53 8. Issues and Recommendations 8.1 Mindset Issue TELP was designed and continues to operate on the premise that historically disadvantaged institutions form a homogenous group. This mindset within and outside of these institutions not withstanding their structural and historical affinities has a profoundly negative impact on how they are perceived and undercuts their ability to come to terms with, own and effectively address their unique limitations. Recommendation The project design for TELP should recognise that participating institutions are heterogeneous. They share both similarities and differences. USAID/SA, in consultation with the Advisory Panel, should ensure that TELP strategies reflect an understanding of and appreciation for individual organisational uniqueness, as well as the inherent differences between universities and technikons. Specifically, the TELP design assumption that “one size fits all” should be replaced with an appropriate heterogeneity principle and approved by USAID within six months of the acceptance of this report. This issue should be included on the project’s strategic planning agenda. All TELP stakeholders should use the opportunity of the proposed “re-visioning” conference (see Section 8.3) to interrogate not only underlying project assumptions and principles, but also as a forum for differentiating their individual institutional strengths, weaknesses and circumstances in order to own their limitations, exploit their strengths and prepare for transformation challenges. 8.2 Leadership Issues a. The quality of institutional leadership, particularly as embodied in the Vice-Chancellor, represents one of the most significant factors differentiating institutions that have been successful in capitalising Section 8 TELP MID-TERM EVALUATION: REVISED FINAL REPORT 54 on TELP support from those that have not been successful. Without immediate support in this area, some institutions will not be able to benefit from the project. There is a crisis of leadership in many institutions. b. Erratic attendance of some Vice-Chancellors at the Advisory Panel meetings has a detrimental effect on the performance of TELP. c. Challenges confronting senior management of tertiary education institutions are daunting at any time. Few senior managers have training to prepare them for these challenges. It is often assumed that academics who are promoted to senior management posts have the requisite technical and leadership skills. More often than not, they do not have such abilities; institutions suffer as a consequence. Currently educational leadership is rendered even more difficult because of sector transformation and the added pressures of change wrought by fast-paced global developments. These challenges will persist beyond the conclusion of TELP funding. Recommendations a. Within six months of the Advisory Panel reviewing this report the Contractor should deliver rigorous leadership training for senior management at TELP institutions on an ongoing, in-service basis. The Contractor should develop carefully crafted, professionally designed and expertly conducted training programmes for senior officials in contemporary leadership skills development and change management. b. The Advisory Panel should devise appropriate action to be taken against those Vice-Chancellors who do not attend meetings consistently and/or who send surrogates in their place. c. The Advisory Panel should, prior to the conclusion of TELP, encourage a consortium of institutions to explore the development of pre-service, post-graduate qualifications in university/technikon management and student development in order to equip senior and middle management for the tasks that confront them. Such initiatives would also go a long way towards identifying and preparing the next generation of university/technikon leaders with important implications for the sustainability of the TELP ideals. TELP MID-TERM EVALUATION: REVISED FINAL REPORT 55 8.3 Project Planning Issues a. TELP is a process. The project’s evolution is reflected in the modifications to specific objectives and the programmatic adjustments that have been made over the years. In order to remain relevant and maximize benefits for all stakeholders, TELP must continue to respond to changes in its operating environment. b. The benchmark indicator for the attainment of the project’s Strategic Objective, “increased percentage of graduates that are black from HDIs as a percentage of all university and technikon graduates,” is unobtainable. c. Some institutions have not succeeded in integrating TELP activities into their strategic plans. Recommendations a. Using this evaluation as the basis, all stakeholders should embark upon a coordinated mid-term TELP planning process that includes rigorous self-assessment focused on the production of outcomes and data. This process should take into account the differences among individual institutions, enable the project to address new transformation challenges, lay a foundation for future programmatic developments, and re-position the project for ever￾increasing effectiveness. We recommend that this approach include the following individual elements and/or activities: i. At the workshop to be organised for the presentation of the mid-term evaluation report, the Advisory Panel should devise strategies that will enable it to act on recommenda￾tions in the report and map the way forward. In addition, quality assurance mechanisms should be created to monitor progress. ii. Within three months of the next Advisory Panel meeting the Advisory Panel, supported by the Contractor, should undertake a “re-visioning” conference for all stakeholders to review TELP’s design, purpose, strategies and proposed activities. iii. TELP institutions, supported by the Contractor, should develop individual institutional TELP work plans within three months of the re-visioning conference. These work plans should be integrated into the overall institutional strategic plan. TELP MID-TERM EVALUATION: REVISED FINAL REPORT 56 b. Within three months of the re-visioning conference, the Advisory Panel should propose to USAID any necessary changes to TELP’s strategic objectives, as well as any new indicators required by such changes. USAID should approve any required changes to the strategic objectives and indicators within six months of the conference. c. Where institutions have not yet succeeded in integrating TELP activities into their strategic plan, the Contractor should provide technical assistance to enable them to do so within one year. 8.4 Milestones and Key Results Issues a. There is general dissatisfaction among all stakeholders with the present performance-based contract that is determined by the seven specific project key results and is milestones-driven. b. As presently set forth, milestones do not demonstrate a clear progression of actions and/or developments that lead toward the achievement of stated TELP objectives. Many milestones are so narrowly focused that broad project goals may not be achieved even if all deliverables are satisfied. Overall, milestones are not driven by institutional needs and realities. Recommendations a. TELP stakeholders should undertake a comprehensive review of the milestones mechanism as part of the re-visioning process recommended in Section 8.3. This process should include an examination of the relevance of the stated seven project key results and the responsiveness of milestones to institutional needs. b. USAID and the Contractor should review within one month of the re-visioning conference the current project implementation mechanism with a view to devising a new “outcomes-based” approach to milestones and intended results that takes into consideration higher education sectoral needs, transformation realities, institutional needs and realities, TELP’s vision and goals, and USAID requirements. c. USAID should develop and incorporate a revised mechanism for project implementation into its contract with UNCF. Reformulation consultations should include UNCF and the Advisory Panel and be completed within two months of the review process specified in point (a) above. TELP MID-TERM EVALUATION: REVISED FINAL REPORT 57 8.5 Workshops and Quality Assurance Issues a. Joint activity workshops are not differentiated according to levels of need and expertise and are not sufficiently focused on specific needs at TELP institutions. They are generally content-driven rather than outcomes-based. Facilitators often do not have the required expertise. b. Except for post-workshop questionnaires, which have limited utility, there are no assessment mechanisms in place to provide insight into the effectiveness of given workshops. c. TELP institutions sometimes undercut the effectiveness of workshops by not selecting and sending appropriate staff, or by not ensuring that the same participants attend all related workshop sessions. Recommendations a. Within six months the Contractor should submit to the Advisory Panel a revised joint activity workshop strategy that is based on consultation with the TELP institutions. The strategy should deliver workshops that are responsive to the needs identified in the TELP institutional plans and ensure that all facilitators have the necessary expertise. b. The Contractor should develop and implement within three months a quality assurance mechanism for workshops. Instruments should be designed to generate data on pre-workshop preparation, programme content appropriateness and delivery, responsiveness to institutional needs given differentiated levels of institutional capacity, effectiveness of facilitators, attendance continuity and impact. c. Within three months the Advisory Panel should issue clear guidelines requiring regular attendance by properly qualified staff at all workshops. 8.6 Communications Issue Communications within the majority of institutions are insufficient to ensure widespread understanding and acceptance of what TELP is doing, TELP MID-TERM EVALUATION: REVISED FINAL REPORT 58 how it is doing it and what opportunities it offers. All too often when TELP fails it is because a paucity of information has been disseminated about the project. Recommendation Within six months the Contractor should provide targeted assistance where it is needed to TELP Coordinators and key staff members to enable them to communicate project attributes more effectively, to raise TELP’s profile and visibility at the institution, and to facilitate the sharing of information between HDIs. 8.7 TELP Coordinators & Project Managers Issues a. Some Coordinators occupy part-time posts. This is in violation of the TELP Agreement. b. The core duties enumerated in the Coordinator’s job description do not reflect the broad scope of duties and the true nature of responsibilities necessary to manage TELP successfully on campus. c. Some Coordinators are not performing, thereby substantially limiting potential project benefits at their institutions. d. TELP Coordinators do not always have the authority and status within institutions to enable them to carry out their functions effectively. Recommendations a. Every HDI should honour the TELP Agreement and ensure Coordinators are serving in a full-time capacity. The Advisory Panel should monitor this matter and take appropriate action where necessary within three months of the next Advisory Panel meeting. b. The Advisory Panel should decide within six months of the re￾visioning conference on a new detailed job description that adequately defines the TELP Coordinator position. Lines of authority for Coordinators should be developed and communicated to all stakeholders. c. Within three months of the re-visioning conference the Contractor should develop and propose to the Advisory Panel a performance￾appraisal system for TELP Coordinators. Upon approval by the Advisory Panel, all TELP institutions should institute this system TELP MID-TERM EVALUATION: REVISED FINAL REPORT 59 within six months. The Advisory Panel should also decide upon appropriate action to be taken against those Coordinators who are not performing adequately. d. Within six months of the re-visioning conference the Advisory Panel should examine mechanisms to ensure that the Coordinators have direct access to the Vice-Chancellors. They should also explore options and determine where Coordinators should be placed within sub-management units. 8.8 Staff Development Issue Faculty and administrators are often under-prepared for the skilled resolution of student development issues. Recommendation The Advisory Panel, working with the TELP institutions and supported by the Contractor, should prepare a plan within six months to support the development of systematic management and technical training for student affairs staff based on the best international practices available. 8.9 Compensatory Programmes Issue Secondary school preparation will continue to present HDIs with educational problems for the foreseeable future. No single bridging programme should be seen as a panacea. On-going research will reveal new insights and new solutions to what is a worldwide problem as well as a special South African challenge. Recommendation The Advisory Panel, working through the TELP institutions, should support continuing research into compensatory programmes. Using a similar model to that developed for the production of mathematics and science teaching and learning materials, consortia of historically disadvantaged institutions with particular expertise should be established within the next six months to undertake research into specific subject areas that, in collaboration with the Department of Education, should be presented at an annual workshop to the higher education sector as a whole. TELP MID-TERM EVALUATION: REVISED FINAL REPORT 60 8.10 Linkages Issues a. The primary source of information about the performance of the US linkage institutions is the reports that they prepare and submit themselves. The absence of data from other sources precludes effective evaluation of linkage performance. b. The use of the term ‘partnership’ to describe linkage arrangements between United States and South African institutions has proven counterproductive, raising expectations that cannot be fulfilled. c. No provision is made for leave substitutes to replace staff members who are away from the institution on training for an extended period of time. This negatively impacts how some organisational responsibilities and tasks are carried out, and overloads staff. Recommendations a. The Contractor should be given the responsibility and necessary resources to develop and institute within three months a quality￾assurance system for the linkages component. The system should include, at a minimum, the following elements: i. The annual reports submitted by United States institutions should be verified and supported through an on-site monitoring and evaluation system. ii. All South African institutions should be required to submit formal semi-annual reports on the status of linkage activities. These reports should also be subject to an on-site monitoring and evaluation system. iii. United States and South African institutions should be notified of new programme review procedures. b. Through the TELP Advisory Panel, emphasis should be placed on correcting the incorrect assumption that the linkages relationship is one of equal partnership and replacing it with the same service￾delivery model that TELP as a whole uses. The Advisory Panel should certify within one year that this has been accomplished. c. USAID/SA should consider funding leave substitutes under TELP. The Advisory Panel should explore options for this, including costs and the trade-offs that would be required by using project resources for this purpose, within six months. Assuming that the Advisory Panel develops and approves a workable system, USAID should approve it within an additional three months. TELP MID-TERM EVALUATION: REVISED FINAL REPORT 61 8.11 Skills Audit Issue There is no accurate information currently available on the human resources and skills that reside within TELP stakeholder institutions. The absence of such data inhibits optimum utilization of staff, as well as, making the process of identifying and preparing the next generation of university and technikon leaders more difficult. Recommendation A formal skills audit, supported by the Contractor in collaboration with the Department of Education, should be carried out within one year at each institution, in order to identify and catalogue existing skills and expertise. An inventory of available talent should make it possible to develop a professional personnel data base upon which historically disadvantaged institutions, TELP and the national sector could readily draw. 8.12 Public Relations Issue TELP does not have a national profile. There is little public awareness of the successes of the project and its contributions to higher education and society. Recommendation The Advisory Panel, supported by the Contractor, should launch within six months of the re-visioning conference a public relations campaign to raise awareness of TELP nationally and to highlight the achievements of historically disadvantaged institutions. TELP MID-TERM EVALUATION: REVISED FINAL REPORT 62 9. Lessons Learned Three key lessons derived from the findings and conclusions presented above offer implications for similar programmes in different settings as well as for future activities in support of South African higher education. 9.1 Addressing Diversity Effectively The full diversity of institutional needs and realities were not known when TELP was designed. This has resulted in significant implementation challenges and variable impact levels. South Africa’s historically disadvantaged institutions are not a homogen￾ous group. There are significant differences among them beyond those of function and geographic location, in terms of human, financial and material resources, ethos, and history. An understanding of such differ￾ences should inform every level of project planning, implementation, monitoring and evaluation. It cannot be taken for granted that all disadvantaged institutions have equal capacity to negotiate with the Contractor, USAID or their US partners, or that they will interpret contracts, memoranda of agreement, and bureaucratic regulations in the spirit in which they were created. Different cultural frameworks, different experiences of authority and autonomy, a lack of bureaucratic experience, a lack of confidence – all of which are defining characteristics – will have a role to play in their interactions with the Contractor, the Donor and their US partners. 9.2 Clear Role Definitions and Vision Role clarification of all actors is essential for successful project implementation. Many of the problems associated with the implementation of TELP are the result of people and institutions not knowing precisely what is expected of them, to whom they are responsible, and when they are expected to deliver. Beyond this, a shared project vision is vital. Even clarity about who should do what cannot help without equal clarity about the fundamental mission and aims of a project. Too often projects such as TELP begin without such a unified vision among all stakeholders, or without translating that vision Section 9 TELP MID-TERM EVALUATION: REVISED FINAL REPORT 63 into practicable action plans. Even when these requirements are met, in a project that extends over as many years as TELP the need for re-visioning and re-planning is almost inevitable. Project designers and managers must build the necessary time and resources for such strategic planning into the project lifecycle. 9.3 The Capacity-Building Conundrum TELP powerfully demonstrates a classic ‘catch 22’ in offering services to historically disadvantaged people or institutions. Such projects exist because capacity is lacking. Yet the absence of such capacity too often precludes recipients from taking advantage of the very support they need. For example, ultimately the success or failure of TELP has been defined by the quality of institutional leadership. Where there is vision, energy and commitment at the highest level, success follows. The absence of such leadership diminishes TELP’s impact. Yet one key purpose for the project is to develop precisely such management capacity at historically disadvantaged institutions. This challenge threatens to create wider, not narrower, gaps among various institutions. It requires a clear understanding of the issues involved plus the ability to tailor different types of assistance, different levels of support, and different mechanisms for delivery for different situations. TELP MID-TERM EVALUATION: REVISED FINAL REPORT 64 10. Annexes 10.1 Abbreviations AED Academy for Educational Development AIPA African Institute for Policy Analysis and Economic Integration CHET Centre for Higher Education Transformation CUP Committee of University Principals DCN Development Consultants Network DevTech Development Technologies DOE (National) Department of Education DTET Desmond Tutu Educational Trust EFA Educators for Africa HDI Historically Disadvantaged Institution HDU Historically Disadvantaged University PIL Project Implementation Letter SA South Africa SAQA South African Qualifications Authority SAUVCA South African Universities Vice-Chancellors Association TELP Tertiary Education Linkages Project UNCF (United Negro) College Fund UNCF/SA (United Negro) College Fund South Africa Office UNCFSPC United Negro College Fund Special Programs Corporation US United States USAID/SA United States Agency for International Development Mission to South Africa Section 10 TELP MID-TERM EVALUATION: REVISED FINAL REPORT 65 10.2 TELP Partners This annex summarises the role of the different agencies and organisations through which TELP funding has flowed. As explained in Section 2.2.1 above, this mid-term evaluation did not directly examine the role of contractors or grantees beyond the current contractor, the College Fund/UNCF. However, the evaluators did obtain some information about these other TELP participants, primarily from the documentation review and occasionally from comments during interviews or focus groups. These data are included below as appropriate. 10.2.1 The College Fund/UNCF The United Negro College Fund (UNCF) was awarded a five-year contract (1998 – 2003) in the amount of $22,907,300 to implement the Tertiary Education Linkages Project (TELP). The College Fund is responsible for planning, organising and delivering all activities under the project’s joint activity component. These include workshops that are organised on a regional or national basis for TELP institutions, plus institution-specific technical assistance in areas that arise out of workshop sessions. UNCF is also tasked with managing linkage grants, and supporting the development and implementation of Project Implementation Letters. UNCF is the United States’ oldest and most successful African-American higher education assistance organisation. It has been named as the number one education non-profit organisation in America by the Chronicle of Philanthropy. All programs of the College Fund, domestic and international, are designed to enhance the quality of education for the historically disadvantaged and range from scholarships for students to curriculum development, college preparation and faculty development. Effective April 1, 2000, UNCF’s federally funded programs combined to form the United Negro College Fund Special Programs Corporation (UNCFSP). 10.2.2 DevTech Systems DevTech Systems signed a five-year Technical Services Contract under TELP in September 1995 to coordinate delivery of technical assistance and training support in activities carried out under the three components of policy analysis and planning, capacity building and linkages. In March 1997 USAID/SA revised its scope of work to significantly expand DevTech’s responsibilities to the project, including organising and supporting the TELP Executive and Policy Advisory Panel meetings, organising and implementing the National Qualifications Framework Conference, sending 15 mid-level South Africans to US higher education institutions for four-week intensive training programmes, conducting audits of four of the five focus areas, and providing financial management training to the 15 stakeholder institutions. USAID provided $3.76 million in TELP funding through the DevTech contract. TELP MID-TERM EVALUATION: REVISED FINAL REPORT 66 The mid-term evaluators had access to DevTech’s Completion Report, as well as to reports on its academic audit and financial management training activities. However, no independent evaluation of DevTech’s performance was available. In 1998 USAID engaged a new contractor, the College Fund/UNCF, to assist historically disadvantaged institutions increase their capacity and to support the Government of South Africa in implementing its national higher education policy. After a brief overlap period during which DevTech (through its local sub-contractor, Deloitte & Touche) completed its financial management training activity, the College Fund became the primary TELP contractor.15 Therefore, the findings of this mid-term evaluation reflect the contributions of both DevTech and the College Fund to TELP, although by design the emphasis rests with more recent experiences and, therefore, primarily on the College Fund’s role. 10.2.3 African Institute for Policy Analysis and Economic Integration After a small initial seed grant, in 1995 USAID gave the African Institute for Policy Analysis and Economic Integration (AIPA) a larger grant of $386,940 through TELP to create a secretariat for the HDI Forum and to carry out related research. Incorporated in 1992 as a non-profit, independent organisation, AIPA defines its mission as engaging in high￾level, non-partisan, interdisciplinary economic policy analysis. Prior to assuming its Secretariat responsibilities, AIPA had collaborated with the Forum in reviewing policy options and strategic analyses on tertiary education. According to its terms of reference, the evaluation team was not asked to assess independently the contributions of TELP grantees such as AIPA and the International Foundation for Education and Self-Help (discussed below in Section 10.2.4). Consequently, the team did not undertake any independent review of AIPA’s work in the Secretariat or the papers it has produced, nor did it have access to any formal assessment of AIPA’s role in TELP. It did include a question about AIPA’s role in the interview protocol for Vice-Chancellors. This question was asked of most, but not all, Vice-Chancellors. Most respondents did not provide any substantive input, preferring to focus instead on the current contractor, the College Fund. Those Vice-Chancellors appointed more recently seemed uncertain about how AIPA fits into TELP activities. Two of the longer-serving Vice-Chancellors, however, did describe AIPA as “ineffective” in its current work for the HDI Forum. They attributed its problems on the one hand to limited resources and an organisational focus on priorities other than historically disadvantaged institutions, and on the other hand to the growing influence of the South African Universities Vice￾Chancellors Association (SAUVCA). This second factor may explain this more negative view of AIPA’s contributions than found in an 1997 informal 15 USAID projects expenditures of $22.91 million through the College Fund contract by the end of TELP. TELP MID-TERM EVALUATION: REVISED FINAL REPORT 67 assessment of several TELP-supported initiatives in which the HDI Forum Chairperson cited AIPA’s grant as “very helpful.” 10.2.4 International Foundation for Education and Self￾Help An evaluation of the International Foundation for Education and Self-Help (IFESH) TELP grant was submitted to the TELP Advisory Panel in March of this year. This section draws from that document. In September 1996, with a grant of $1,288,005, IFESH implemented the Educators for Africa (EFA) programme at six historically disadvantaged technikons. In August 1997, several additional University Vice-Chancellors indicated an interest in accessing the services of the EFA programme. As a result, another USAID Co-operative Agreement was signed with IFESH that increased the number of institutions to fifteen (15) but without a corresponding increase in funding. The IFESH programme is based on the assumption that strengthening the professional capabilities of staff in critical areas will lead to the strengthening of the institution. The EFA programme identifies and recruits American University professors and academics, preferably PhD’s, who are retired, on sabbatical or otherwise temporarily free of academic duties, to serve as IFESH Scholars to teach and improve educational practices at historically disadvantaged institutions. IFESH Scholars are required to spend a minimum of one academic year, preferably two, at their assigned institutions. The EFA programme concept was for the IFESH Scholars to aid in faculty development, research, and classroom teaching in mathematics, engineering, science, and/or management for institutions that had difficulty in finding staff or where staff were away furthering their own education. Stakeholder institutions provide a job description and preferred qualifications for each position required. IFESH then recruits educators based on the information provided by the institutions. The EFA programme requires that institutions provide housing, transport, office supplies. It also requires them to assign a counterpart to collaborate with the IFESH Scholar in a team-teaching mode. This was planned to encourage skill enhancement of the historically disadvantaged institution faculty members. The intended result was that faculty would be better trained in their respective fields. To date EFA has placed 30 IFESH Scholars at 15 institutions. Stakeholder institutions have requested an additional 50 educators through the programme. The IFESH evaluation found that overall the historically disadvantaged institutions believe that the programme has significant value to institutions, both in terms of strengthening the capabilities of staff and students in varied areas, and also in terms of providing an international perspective and expertise to strengthening academic programmes. It TELP MID-TERM EVALUATION: REVISED FINAL REPORT 68 identified certain structural weaknesses around variables affecting the performance of the IFESH Scholars, such as selection, length of stay, orientation to the institution, advance planning, support structures and communication. One institution visited during the TELP mid-term evaluation volunteered that it is pleased with the EFA Scholars received through this programme. 10.2.5 Development Consultants Network, Inc. Development Consultants Network, Inc. (DCN) is an international corporation made up of professionals with national and international experience in training, development, research, business, management, evaluation and planning. Established in Southern Africa in 1986 by African and African-American women, it offers a variety of specialized services for local, national and global corporations, governments, universities, NGOs and individuals. Areas of specialization include: · Systems Transformation · International Business Services · Professional Placement · Globalisation Workshops · Conference Planning · Short-term Training DCN is The College Fund’s subcontractor for the linkages component. The objective of linkage activities is to foster partnerships between the historically disadvantaged institutions and United States institutions of higher learning that will support the HDIs in achieving objectives in the five focus areas. Under its subcontract, DCN is charged with responsibility for managing TELP Linkage Grants. Activities involved in implementation include development and issuance of applications for linkage grant competition; review and ranking of applications; awarding linkage grants; negotiating budgets; approving Memoranda of Agreement between linkage partners; conducting post award conferences; monitoring programmatic and financial aspects of linkages, and reporting on component activities. 10.2.6 The Desmond Tutu Educational Trust The Desmond Tutu Educational Trust (DTET) was founded, in July 1990, by a group of educators and community leaders in Western Cape Province to solicit funds for tertiary education programs in the region. Based in Cape Town, the Tutu Trust set a goal of assisting black students in redressing the educational disadvantages they experienced as a result of the structural and legal discrimination of the apartheid system. Its objective was to work with tertiary institutions in providing academic enrichment resources, and to facilitate cooperation among Western Cape higher education institutions with which it cooperatives. TELP MID-TERM EVALUATION: REVISED FINAL REPORT 69 DTET is The College Fund’s subcontractor for student academic and social development focus area within the joint activities component. Since 1988, the Tutu Trust has provided basic interventions in the following areas: · Facilitation of the development of student development plans; · Development and administration of student assessment testing; · Training of staff for establishment and administering bridging programs; · Development and assessment of student work study programs, and · The provision of student and staff leadership training. 10.2.7 Additional Contractors and Grantees USAID has used the TELP mechanism to provide funding through five additional organisations. Because these further contracts and grants are less relevant to the purpose of this evaluation, the evaluators made no attempt to collect data on them. However, since they do appear in TELP budgets they are noted here briefly. The following three contracts and grants contributed directly to TELP’s Strategic Objective and operational requirements. · The Academy for Educational Development received a task order through its Education Indefinite Quantities Contract to carry out this mid-term evaluation, budgeted at $287,000. · Howard University received a grant of $270,000 to help upgrade math, science and engineering tuition at a consortium of technikons. In addition to the funding granted directly to the US institution, $682,353 was provided directly to the technikon consortium for the project through PIL funding. This brings the total of USAID funding to $952,353 for this activity. · USAID contracted with Institute of International Education (IIE) to provide academic training in the United States for staff from TELP stakeholder institutions. In conjunction with the African-American Institute (AAI), 74 participants were placed from 1996 to 1999. The following two contracts and grants used the TELP funding mechanism to provide management support and further sectoral support, respectively. However, they were not designed to contribute directly to TELP’s Strategic Objective. · Macro International received $62,000 to help develop performance monitoring systems and indicators for TELP, as part of a standing contract to provide such services to USAID. TELP MID-TERM EVALUATION: REVISED FINAL REPORT 70 · In 1999 the Department of Education initiated a project for building governance capacity at all 36 higher education institutions. The Department of Education and USAID identified the Centre for Higher Education Transformation (CHET) as the agency to provide technical assistance to the project. USAID provided a grant of $1.2 million to CHET under terms determined by the Department of Education and USAID. The funding for this activity was made available by the Education for Development and Democracy Initiative (EDDI). These monies are beyond TELP’s original $50 million budget. Although TELP provides the funding mechanism for this programme, the grant’s scope extends beyond the historically disadvantaged institutions and, therefore, it does not truly fall within the project’s primary goals. TELP MID-TERM EVALUATION: REVISED FINAL REPORT 71 10.3 Expenditure Assessment 1. INTRODUCTION The Tertiary Education Linkages Project (TELP) Bilateral Agreement specifies that a mid-term evaluation be conducted to determine progress in the achievement of project goals. Amongst other, the evaluation calls for answers to the questions of whether the achievements of TELP are being produced in cost-effective ways and whether the accomplishments of TELP are sustainable. In order to properly assess the cost-effectiveness and sustainability of activities within the project, an analysis of expenditure and financial administration was considered necessary. This report sets out the results of that investigation, detailing also the methodology and constraints of the exercise. TELP makes use of a number of different funding mechanisms. The financial review aimed to examine these mechanisms with the view to determining whether the funding mechanism impacted on the project itself. Was one form of linkage more or less cost-effective than another? Did the nature of a specific funding contract influence the amount of counterpart funding flowing to the institution? Answering these types of questions could throw light on the cost-effectiveness of the project. 2. METHODOLOGY 2.1 Data Collection The financial review required input from the Donor (USAID) as well as the major contractor employed to carry out the project, The United Negro College Fund (UNCF). Visits to both institutions and interviews with relevant officials helped the researcher become acquainted with the individual institution’s expenditure and financial control processes; a factor which has potential implication for the effectiveness of the project. The visits also provided opportunity to assess the nature and extent of financial data available to carry out the study, and to gather the available data. 2.2 Questionnaire Survey The tertiary institutions participating in the TELP historically disadvantaged institutions were potentially also a source of information regarding the finance and expenditure process. A questionnaire (see Annex 10.12) was drawn up and distributed to the HDIs in the survey sample. Responses to the questionnaire provide a useful secondary source of data on projects specific TELP MID-TERM EVALUATION: REVISED FINAL REPORT 72 to institutions. Information regarding record keeping and administrative processes was collected by means of the questionnaire. The results of the questionnaire survey are reported separately in Section 4. 3 FUNDING MECHANISMS 3.1 Unilateral/Bilateral Funding Table 1 below provides a summary of funding for the Tertiary Education Linkages Project (TELP) over the life of the project. Approximately $44 million of the project’s $51,2 million total is covered by bilateral agreements between USAID and the Department of Education and a further $7 million takes the form of unilateral USAID funding. Unilateral expenditure made by USAID, without necessary oversight by Department of Education, can be divided into two categories. The first, expenditure undertaken at the inception of the project, prior to the signing of the Bilateral agreement included the DevTech contract and expenditure in the category Invitational Travel. The second is on-going and is confined to programme management and the salaries of USAID project officials. 3.2 Major Expenditure Categories It is evident from Table 1 that the project funds have been allocated to seven categories: Contractors/Technical Assistance, Grants, Programme Management, Project Implementation Letters (PILS), Invitational Travel, Miscellaneous and Financial Reviews. Across the life of the project the largest proportion of the funding is to flow to the institutional contractors (57.6 per cent) and to PILS (30.2 per cent) - together accounting for 87.8 per cent of total funding. Expenditure to date on these two categories is of the same relative proportion. TELP MID-TERM EVALUATION: REVISED FINAL REPORT 73 TABLE 1: TELP Funding 3.2.1 Contractors At the close of the project the largest proportion of project funding, namely $22,907,378 (44 per cent of total funding), will have been channelled through the main contractor, the College Fund. This funding covers a number of components – Linkage Grants, Joint Activities, Institution Specific Activities and support for Policy Implementation and Systems Development. In addition to these four components, the College Fund is tasked with administering the funds provided to US institutions to fund linkages between United States and South African institutions. This latter funding amounts to $6,925,000 across the life of the project. Included in the College Fund contract fee is an annual amount of $74,561 to administer the linkage grants. TELP Future Funding Projection As at 04/2000 FY2000 FY2001 FY2002 FY2003 TOTAL Insti. Contractors/TA 20,326,427 3,000,000 3,000,000 3,000,000 191,862 29,518,289 UNCF Contract 13,715,516 3,000,000 3,000,000 3,000,000 191,862 22,907,378 IIE 2,500,945 2,500,945 DevTech 3,760,329 3,760,329 Macro International 62,251 62,251 Evaluation Contractor 287,386 287,386 Uncommitted - Grants 3,144,945 3,144,945 AIPA 386,940 386,940 IFESH 1,288,005 1,288,005 Howard Uni-HDI Conso. 270,000 270,000 CHET 1,200,000 1,200,000 Program Management 1,003,632 100,000 100,000 100,000 1,303,632 PILS 11,915,973 200,000 1,314,588 1,030,000 1,000,000 15,460,561 NCHE/DOE/CHE 787,728 787,728 HDI Project Coordinators 1,866,117 300,000 30,000 2,196,117 HDI Activity PILs 8,579,775 200,000 1,014,588 1,000,000 1,000,000 11,794,363 HDI Tech Consortium 682,353 682,353 Invitational Travel 284,502 284,502 Miscellaneous 266,174 266,174 Financial Reviews 21,897 21,897 GRAND TOTAL 36,963,550 3,300,000 4,414,588 4,130,000 1,191,862 51,200,000 Bilateral 29,808,279 3,300,000 4,414,588 4,130,000 2,391,862 44,044,729 Unilateral 7,155,271 7,155,271 51,200,000 TELP MID-TERM EVALUATION: REVISED FINAL REPORT 74 To assess cost-effectiveness in the main contract it was envisaged that the exact nature of expenditure would be analysed. By comparing the proportion of funds spent on specific activities (for example general administration of the project versus that specifically on activities furthering the aims of the project such as workshops), we may have had answers to the cost-effectiveness question. Due to the fact that USAID has concluded a milestone contract with the College Fund to deliver the services on the TELP, and the College Fund were, therefore, not required to disclose their expenditure detail, the data necessary for this type of analysis were unobtainable. The analysis cannot establish the exact nature the monies spent to date and therefore cannot judge cost￾effectiveness by these means – for example it is not clear what amounts were spent on salaries and what amounts on the provision of workshops and training. Within the main components as listed above, the project has identified five focus areas. From the data that was available it was possible to carry out an analysis of expenditure on the components and the focus areas. Table 2 below sets out the expenditure on the four main components for the first two years of the contract. [Table 5 at the end of this report sets out expenditure in the two financial years separately]. TABLE 2: Milestone Expenditure Summary – June 1998 to May 2000 Most expenditure in the first two years of the contract has been on the component Joint Activities. Of the $6,330,210 funding, $5,256,338 (83 per cent) was spent on this component. Function Focus Area National Regional Institution Total LINKAGES 154,872 JOINT ACTIVITIES 1,054,865 546,998 3,654,475 5,256,338 (1) Curriculum Development 101,518 0 2,013,968 2,115,486 (2) Student Academic Development 289,421 430,928 798,603 1,518,952 (3) Management and Administrative Development 663,926 116,070 841,904 1,621,900 INSTITUTION SPECIFIC 365,695 365,695 POLICY IMPLEMENTATION 553,305 553,305 TOTAL EXPENDITURE (Years 1 and 2) 6,330,210 Level of Participation TELP MID-TERM EVALUATION: REVISED FINAL REPORT 75 Within the Joint Activities component expenditure was confined to three of the five focus areas, curriculum development (40.2 per cent), student academic development (28.9 per cent) and management and administrative development (30.9 per cent). The three focus areas are in many respects interdependent, and the pattern of expenditure reflects this. A calculation of the nature of the spending in the focus areas to determine whether the funds spent were provided for national, regional or institution￾specific activities reflects some interesting results. Almost 70 per cent of the expenditure provided services at the institutional level. In all three focus areas, the trend was to carry out institution-specific activities, with the case being most pronounced for curriculum development (95.2 per cent of expenditure). This would support the unique circumstances applicable at each of the individual institutions. Expenditure listed for Linkages is the spending utilised to manage the linkage grant process. The contract provides for an annual sum of approximately $75,000 for this management function. Given that in the first two years 15 linkages were established, this amounts to $5,000 per linkage per year. 3.2.2 Linkage Grants Assessing actual linkages between SA and US institutions is important, as it is one area where USAID funds provide the foundation for greater flows of funds brought about by matching or counterpart financing agreements. The linkages financed by TELP aim to enable two or more institutions to address broad based issues, share resources and engage in joint endeavours. US institutions participating in the linkages receive USAID funds on the basis of matching fund agreements. Assessing expenditure on this function calls for an analysis of not only the USAID funds but more importantly the extent of matched funds provided by institutions on the back of the USAID funds. In Table 3 below the funding situation as applicable to linkage grants at a number of the participating institutions is set out. The amounts shown in the table reflect totals for the duration of the linkages and not just funding for the first two years, as is the case in the discussion above. It was not possible to extract matching funding for the largest number of the grants on an annual basis. From the 14 linkages set out it is evident that USAID funding of almost $6 million attracts further funding to the extent of $3.6 million from the U.S. counterpart institutions. Ignoring the cost of administration, this amounts to a 60 per cent matching proportion. When considering the cost effectiveness of the project the fact that the USAID funds facilitate further funding must not be over looked. A 60 per cent matching factor is significant. TELP MID-TERM EVALUATION: REVISED FINAL REPORT 76 TABLE 3: Linkage Grant Funding (June 1998 to May 2000) As mentioned earlier it was not possible to examine the spending pattern of the main contractor across the functions – that is to compare expenditure on personnel with other forms of expenditure. The exercise was, however, carried out on details of five of the existing linkage grants where the level of detail was available. The five selected linkage grants were not chosen for any reason other than the fact that the required detailed information was available. This would support the notion of an unbiased sample and allow the results to be considered indicative of all the linkage grants. The total expenditure of $2,728,013 covering the grants in the sample represents 28.6 per cent of total linkage grant funding as summarised in table 3 above. The data is summarised in Table 4 according to the source of funds (USAID TELP funds or matching funds from the linkage institution) and then according to the function it was spent on. Of total spending 48.3 per cent is to be used to fund personnel and fringe benefits. When the cost of travel - which is also ostensibly to fund personnel￾related activity - is added to this amount, the proportion rises to 59.4 per cent. The smallest portion of the funds is to be spent on equipment (6.1 per cent) and supplies (3.8 per cent). The results reflect a personnel-intensive expenditure pattern. This is not necessarily harmful to the project but must send out warning signals to those tracking expenditure. INSTITUTION LINKAGE INSTITUTION Federal Funding Matching Grants Total Funding Border Technikon University of Maryland Eastern Shore 460000 213137 673137 Eastern Cape Technikon Texas Southern University 459998 330538 790536 Fort Hare University University of Connecticut 460000 416255 876255 Mangosotho Technikon Florida Agricultural and Mechanical University 460000 329876 789876 ML Sultan Technikon Savannah State University 460000 288000 748000 Peninsula Technikon University of Michigan 459768 161052 620820 Technikon North West Tennessee State University 460000 115500 575500 University of Durban Westville Pennsylvania State University 459920 240695 700615 University of North West Office of International Education, Policy and Leadership 460156 238753 698909 University of the Western Cape University of Missouri 454613 115537 570150 University of Transkei The City College of CUNY 457180 170372 627552 University of Zululand Mississippi Consortium for International Development 460000 645615 1105615 Vista University University of Central Florida 197000 59100 256100 Vista University Southern University and A&M College 262816 246792 509608 GRAND TOTAL 5971451 3571222 9542673 TELP MID-TERM EVALUATION: REVISED FINAL REPORT 77 TABLE 4: Detail of Selected Linkage Grants Funding Source USAID Funds Matching Grant Total Function: Personnel 623,327 472,927 1,096,254 Fringe Benefits 107,545 113,230 220,775 Travel 295,848 7,300 303,148 Equipment 121,900 45,000 166,900 Supplies 51,000 51,757 102,757 Contractual 299,700 128,140 427,840 Other 336,575 73,764 410,339 Grand Total 1,835,895 892,118 2,728,013 Due to the fact that detailed data was not available to carry out these comparisons across a number of different activities it is impossible to draw project-wide conclusions. The ability of the evaluation to effectively pass judgement on the cost-effectiveness of the project was impeded by the lack of detailed expenditure data. 3.3 Grants Overall funding to “Grants” accounts for only 6.1 per cent of the project’s budget. Although this is a small proportion of the total, the success of the functions carried out and funded by grants is important to the success of the entire project. 3.3.1 AIPA An amount of $386,940 was made available to AIPA. Although this is a relatively small proportion of the total project cost, the role AIPA was to play in the project was very important. AIPA was contracted to facilitate the interaction between the Department of Education and Vice-Chancellors of the institutions. Specifically, one of AIPA’s roles is to ensure that Vice-Chancellors are well informed on legislation and the likely impact thereof, on the institutions participating in the project. To judge the cost-effectiveness of the AIPA grant, the opinion of the Vice￾Chancellors on the assistance AIPA provided in keeping them abreast of all relevant developments facing tertiary institutions, must be considered. The dollars spent on the service may not have been large but the outcomes have significance for the success of the entire project. Unless Vice-Chancellors are well informed on a timely basis of developments likely to impact on their institutions their role in the transformation process may be severely eroded. TELP MID-TERM EVALUATION: REVISED FINAL REPORT 78 3.3.2 International Foundation for Education and Self-Help (IFESH) An agreement was signed with IFESH in 1996 to implement the Educators for Africa (EFA) programme in six technikons participating in the TELP. The initial concept of the EFA programme was to provide the technikons with the opportunity to obtain Masters and PhD level faculty to help alleviate staff shortages, particularly in the areas of science, engineering, math and management. Site visits led IFESH to be more flexible in the request for placements. In addition to the broader scope of the disciplines, later in 1996 IFESH was informed that their agreement would be amended to make all of the then fifteen TELP-related institutions eligible to participate in the programme. The amendment did not provide additional funds to facilitate working with the increased number of institutions. IFESH received $1,288,000 of USAID funding through TELP. In addition to this funding, counterpart contribution from US sources supporting the EFA programme, as of the date 31 March 1999, of $1,471,410 was provided. These funds were used to provide books ($1,217,940) and to place experts at the participating SA institutions. Contracts covering 328 man months will have been completed by June 2000. The USAID funding of $1,288,005 over the 328 contract months computed to a monthly fee of $3,926 (which does not account for the books provided by IFESH). Comparisons of similar costs on the VOCA Volunteer (Volunteers for Overseas Co-operative Assistance) programme reveal a cost in the region of $5,000 for a month, excluding overhead management time. The TELP project is receiving services of personnel at a price 20 per cent lower than that of comparable US AID programmes. 3.3.3 Howard University HDI Consortium The amount of $270,000 was granted to Howard University to upgrade math, science and engineering tuition at a consortium of technikons. In addition to the funding granted directly to the US institution, $682,353 was provided to the technikon consortium for the project through PIL funding. This brings the total of USAID funding to $952,353 for this activity. Grant financing was used to finance Howard University’s involvement in the project. In addition to the USAID funds, the counterpart contribution from Howard University amounts to $302,571. This equates to a three to one match of USAID funds to private institutional funds. Due to the fact that the Howard University contract spans activities at a number of technikons, not a single institution, it will not be meaningful to draw direct cost comparisons of the individual institution linkage grants with this particular multi-institutional grant. TELP MID-TERM EVALUATION: REVISED FINAL REPORT 79 4. RESULTS OF FINANCIAL QUESTIONNAIRE 4.1 Survey Sample In order to add a further dimension to the financial analysis a questionnaire survey was conducted amongst the institutions in the survey sample. We solicited responses from the ten surveyed institutions; five submitted completed questionnaires. 4.2 Results of the Survey A copy of the questionnaire is attached as Annex 10.12. When asked whether or not the institution identified the five focus areas (Staff Development, Curriculum and Programme Development, Student Development, Management and Administrative Development and Research Skills Development) when accounting for expenditure, only one of the five respondents said they did. This may be as a result of the fact that in many instances projects cover more than one focus area. In contrast, however, the institutions did distinguish between the three main mechanisms to provide funds to institutions: institution-specific activities (through PILs), joint activities and linkage grants. The nature of the funding would make it easier to distinguish between the expenditure. Joint Activities are funded through The College Fund contract and the Institution Specific activities by PILS – both financing mechanisms clearly identifiable. Institutions do not have a direct role in the financing of linkage grants nor in expenditure on this activity. All institutions supplement TELP funds with their own resources to achieve the overall objectives of the programme. The form of the supplement included carrying the cost of providing office space from which the project operates, providing access to the use of equipment such as computers and telephones, providing workshop venues, funding postal and telephonic services, and in one instance even funding a member of staff to carry out secretarial services for the project. Understandably in most instances it is very difficult to apply a cost in terms of rand to these contributions. Question six probed the TELP funding process. The answers to whether the process was efficient were mixed with a number of institutions replying yes and no. It was evident from the responses that the process of reimbursement had improved tremendously over time. Incorrect completion of claim documents slow the process down. Shortage of staff at the institution to carry out the reimbursement function was noted by two institutions to slow the process down. From the responses it was also evident that as the learning process had continued such delays have been reduced. TELP MID-TERM EVALUATION: REVISED FINAL REPORT 80 Of the respondents, three had only recently concluded linkages with US institutions. The need for access to advance funding on linkages was mentioned in a number of cases. Certainly the funding of TELP-related travel within South Africa was not an issue. The normal reimbursement procedures appear to be understood by all and functioning well. Some institutions did mention that it is difficult to carry the cost of travel during the reimbursement process. Question nine dealt with record keeping and auditing. All responding institutions submit their TELP records for audit by external companies. The TELP Coordinator and the institutions’ finance departments are responsible for overall co-ordination and management of TELP funding, with all institutions using computer-based accounting systems to record TELP expenditures. When asked whether, if given the opportunity to apply for further funding from USAID for TELP-supported projects, the institution would do so, the responses were unanimously positive. Three institutions said there is nothing at all that would deter them from applying for further funding. One institution mentioned that the “one size fits all uncompromising criteria” set for institutions was worth reassessing as it currently did not take account of individual institution’s competencies. Another drew attention to the two-year time period for PILS, saying it found the period short. With the academic year impacting on activities the two years actually translate into a year and a quarter. Another issue raised by this institution concerns the measurement of success or otherwise of a project. Considering the expenditure on a project to determine if a project is active and meeting objectives ignores the project cycle and the implementation and monitoring plans. Judging a project by monies spent is not a good measure of success. TELP MID-TERM EVALUATION: REVISED FINAL REPORT 81 Table 5: MILESTONE EXPENDITURE YEARS ONE AND TWO Milestone Expenditure Year One (June 1998 to May 1999) Milestone Expenditure Year Two (June 1999 to May 2000) Function Focus Area National Regional Institution Total LINKAGES 74,561 JOINT ACTIVITIES 852,656 0 1,664,297 2,516,953 (1) Curriculum Development 101,518 0 951,171 1,052,689 (2) Student Academic Development 289,421 0 474,828 764,249 (3) Management and Administrative Development 461,717 0 238,298 700,015 INSTITUTION SPECIFIC 136,413 136,413 POLICY IMPLEMENTATION 178,671 178,671 TOTAL EXPENDITURE FOR THE YEAR 2,906,598 Level of Participation Function Focus Area National Regional Institution Total LINKAGES 80,311 JOINT ACTIVITIES 202209 546,998 1,990,178 2,739,385 (1) Curriculum Development 0 0 1,062,797 1,062,797 (2) Student Academic Development 0 430,928 323,775 754,703 (3) Management and Administrative Development 202209 116,070 603,606 921,885 INSTITUTION SPECIFIC 229,282 229,282 POLICY IMPLEMENTATION 374634 374,634 TOTAL EXPENDITURE FOR THE YEAR 3,423,612 Level of Participation TELP MID-TERM EVALUATION: REVISED FINAL REPORT 82 10.4 Personnel The following professionals carried out the TELP mid-term evaluation. The background descriptions come from AED’s proposal document. 10.4.1 Evaluation Team Dr. Gretta Middleton, as the Senior Evaluation Leader, brings the leadership skills, breadth of tertiary education experience and deep understanding of institutional transformation that will make the difference between a TELP evaluation that is insightful and a real planning document as opposed to being merely adequate, even if thorough. Through her association with historically black colleges and universities and mainstream higher education in the United States, and with universities in Africa, Dr. Middleton knows the challenges of transformation and capacity building. She has introduced innovative programmes and revamped course offerings in line with socio-economic trends. Her evaluation experiences have been in the areas of human resource development and institutional capacity building—both central objectives of TELP. Dr. Michael Rice, the second Evaluator, has extensive experience of education in South Africa having taught in virtually all the sectors from primary school to teacher training and master’s level at university. An experienced and sought-after evaluator and researcher he has been involved in some of the major investigations into the transformation of education in South Africa. He was a major evaluator and researcher for the National Audit on Teacher Education, and a researcher into the transformation of technical colleges in Further Education. He served on a previous audit team of the historically disadvantaged institutions. These projects had many of the same logistical and methodological challenges as the TELP evaluation. Recently, he was responsible for establishing the community higher education service partnership programme at the University of the Witwatersrand. He has also had considerable experience in evaluating mathematics and science projects at teacher up-grading and initial training levels, and micro-enterprise training programmes for marginalised youth. He is an acknowledged expert on language and cognition across the curriculum. He has also edited several White Papers for the departments of Social Welfare and Population Development, and Housing. 10.4.2 Consultants Dr. Philip Christensen is the Activity Manager. He has worked both long- and short-term with AED for 20 years on USAID activities. He is familiar both AED and USAID procedures thoroughly and his conscientiousness and efficiency are well known to both AED and USAID. In addition to strong management skills, he also has evaluation expertise, TELP MID-TERM EVALUATION: REVISED FINAL REPORT 83 HDI experience, and in-depth experience with the curriculum development focus area. Dr. Christensen will serve as the point person for USAID. He will provide quality assurance and ensure that all stakeholder views are represented in the methodology and final product. Dr. Michael Cross fills the Policy Specialist position, providing at critical junctures during the evaluation the connections to the broad education policy developments in education, especially at the tertiary level. He has been a consistent contributor to government policy on higher education, at the provincial and national levels. Issues of diversity and gender representation have been part of his research and policy work. Ms. Amanda Fitschen, the Finance Specialist, is an economist and financial analyst with experience in the higher education sector. Ms. Fitschen currently lectures full-time at the University of Cape Town School of Economics. She is familiar with tertiary education financial issues, has been a lead actor in developing provincial-level budgets, and has addressing many other public finance issues. A respected researcher and evaluator, she has conducted numerous consulting assignments to study project sustainability, assess the impact of donor programmes, and evaluate the financial management of donor programmes. In 1997 she analysed the funding of tertiary education in South Africa on behalf of the Mellon Foundation and the Department of Education. This research developed a model for implementing a sustainable loan scheme to fund needy students at tertiary institutions to the year 2010. Dr. Howard Williams, the Evaluation Methodology Specialist, is a US￾based AED staff member who consulted with the team by email, fax and telephone. A recognized international expert in evaluation theory and practice, Dr. Williams has provided long- and short-term technical assistance in monitoring and evaluation (M&E), research, EMIS, and testing and measurement across several settings and countries. His work includes M&E design and implementation in Nepal, Egypt, Liberia, Ghana (USAID), Bangladesh (ADB) and Florida (U.S.); research in Nepal, Bangladesh, and Florida; EMIS in Nepal, Bangladesh, and Lesotho; and instruction and technical assistance for testing and measurement in Botswana and Ghana. TELP MID-TERM EVALUATION: REVISED FINAL REPORT 84 10.5 Bibliography Academic Audit of Historically Disadvantaged Tertiary Institutions in South Africa. Overall Report. DEVTECH. May 1998 Council On Higher Education. Size and Shape Task Team: Discussion Document. 7 April 2000 Higher Education Transformation: Assessing performance in South Africa, Nico Cloete & Ian Bunting. CHET. 2000. Inaugural Address: Tertiary Education Linkages Project (TELP) Opportunities and Challenges. Prof. C Z Gebeda. 23 June 1999 Futurefact 2000 Conference Proceedings 13th June 2000 Leadership and Institutional Change in Higher Education. CHET & UNCF. 2000 Project TELP: Attachment – Deloitte & Touche Financial Management Training for Historically Disadvantaged Universities and Technikons in South Africa. January 2000 Project TELP: Semi-Annual Report – appendix No. 4. 1 May 1998 – 20 November 1998 TELP Governance Document as approved by the TELP Advisory Panel. October 1999 TELP Progress Report 1998-2000. The College Fund/UNCF-South Africa. March 1, 2000. Tertiary Education Linkages Project (TELP). Project Number 674-0315. PROJECT PAPER. November 1993 “The Skewed Revolution: The Selective ‘Africanisation’ of the Student Profile of Historically White Universities and Technikons in South Africa in the 1990s”. David Cooper. University of Cape Town. 5 April 2000 UNCF Contract. 26 May 1998 Agendas and minutes of meetings Agenda for Meeting and Workshop of the TELP Policy Advisory Panel 22 February 1995 Executive Summary of the Minutes of the TELP Policy Advisory Panel 29- 30 June 1995 TELP MID-TERM EVALUATION: REVISED FINAL REPORT 85 Executive Summary of the Minutes TELP Policy Advisory Panel 16 February 1996 Minutes of the Policy Advisory Panel Retreat 18-20 April 1996 Minutes of a Meeting Held by the TELP Policy Advisory Panel 14-15 August 1996 Minutes of the TELP Working Group Meeting 19 & 20 October 1996 Minutes of a Meeting Held by of the TELP Policy Advisory Panel 24-25 November 1996 Minutes of the Meeting of the Executive Committee 31 January 1997 Minutes of the TELP Policy Advisory Panel Meeting 27 February 1997 Minutes of the TELP Executive Committee Meeting 13 & 14 April 1997 Minutes of the Executive Committee Meeting 25 June 1997 Minutes of the TELP Policy Advisory Panel Meeting 26 June 1997 Minutes of the Executive Committee Meeting 24 November 1997 Minutes of the TELP Policy Advisory Panel Meeting 25 November 1997 Minutes of the TELP Executive Committee Meeting 5 June 1998 Minutes of the TELP Executive Committee Meeting 11 June 1998 Special Meeting the TELP Executive Committee 21 August 1998 Minutes of the Meeting of the Executive Committee of the TELP Policy Advisory Panel. 27 September 1998 Minutes of the Meeting of the Executive Committee of the TELP Policy Advisory Panel. 28 & 29 September 1998 Minutes of the Meeting of the Executive Committee of the TELP Policy Advisory Panel 11 March 1999 Minutes of the Meeting of the Executive Committee of the TELP Policy Advisory Panel 12 March 1999 Minutes of the Meeting of the Executive Committee of the TELP Policy Advisory Panel 5 June 1999 Minutes of the College Fund Quarterly Report Meeting 21 October 1999 TELP TASK TEAMS Report of Working Group Discussions 27 June 1996 TELP MID-TERM EVALUATION: REVISED FINAL REPORT 86 Grant Agreements Tertiary Education Linkages Project (TELP) Grant Agreement; Project No. 674-0315, Project Implementation letter (PIL) No.1 Tertiary Education Linkages Project (TELP) Grant Agreement; Project No. 674-0315, Project Implementation letter (PIL) No.6 Tertiary Education Linkages Project (TELP) Grant Agreement; Project No. 674-0315, Project Implementation letter (PIL) No.8 Tertiary Education Linkages Project (TELP) Grant Agreement; Project No. 674-0315, Project Implementation letter (PIL) No.26 Tertiary Education Linkages Project (TELP) Grant Agreement; Project No. 674-0315, Project Implementation letter (PIL) No.29 Tertiary Education Linkages Project (TELP) Grant Agreement; Project No. 674-0315, Project Implementation letter (PIL) No.42 Tertiary Education Linkages Project (TELP) Grant Agreement; Project No. 674-0315, Project Implementation letter (PIL) No.77 Tertiary Education Linkages Project (TELP) Grant Agreement; Project No. 674-0315, Project Implementation letter (PIL) No.79 Tertiary Education Linkages Project (TELP) Grant Agreement; Project No. 674-0315, Project Implementation letter (PIL) No.84 Tertiary Education Linkages Project (TELP) Grant Agreement; Project No. 674-0315, Project Implementation letter (PIL) No.85 Tertiary Education Linkages Project (TELP) Grant Agreement; Project No. 674-0315, Project Implementation letter (PIL) No.94 University and Technikon Calendars · University of Zululand 2000 · ML Sultan Technikon 2000 · University of Transkei 1999 University and Technikon reports and reviews · ML Sultan Annual Review1998 · University of the North Research Report 1997 · University of the North Research news May 1999 · MEDUNSA Exploration Vol.2 No.1 April 1999 · MEDUNSA Caladrius March 2000 · MEDUNSA Annual Report 1997 · Peninsula Technikon Annual Report 1998 · Eastern Cape Technikon Annual Report 1998 TELP MID-TERM EVALUATION: REVISED FINAL REPORT 87 10.6 Interview List USAID/South Africa Dr K Nlovu Lisa Franchett Patrick Fine Futi Umlaw Eileen Oldwine USAID/Washington Henry Reynolds Department of Education Nasima Badsha, Deputy Director General, Higher Education Molapo Qhobela, Chief Director, Higher Education Policy Development and Support Nazeama Mohamed, Director, Higher Education Policy Development and Support Themba Mohambi Ahmed Esso,p Chief Director Higher Education Planning UNCF/College Fund (South Africa) Ms Thandiwe January-Mclean (Chief of Party) Prof Mike Phala (specific achievements) Dr Gloria Braxton (Deputy Chief of Party) Dr Rumilla Naran (TELP resource centre) Ms A van der Vyfer (finance) UNCF/College Fund (USA) Ms Liz Lowe (CEO, Special Programmes) Dr Shelby Lewis (President, Development Consultants Network – Linkage Grants Subcontractor) ML Sultan Technikon Prof Goba Vice-Chancellor Prof Bharuthram (Director of Research and Development) Linkages Robin Toli TELP Coordinator Lesley Cooke Acting HOD Curriculum Development Delysia Timm Senior Lecturer Chemistry Dept. Shubnam Rambharos Lecturer Educational Development & Gender Forum Haj Vahed Lecturer Educational Development Core Curriculum Programme Sipho Zulu Lecturer Dept. Medical Sciences Mdu Mokoena Staff Development Facilitator Dept Bio Sciences Mpho Rakgotho HOD Student Life Osman Seedat HOD Educational Development Mal Reddy Senior Lecturer Law & Administration Faculty of Commerce. Mrs Krish Chetty HOD (retired) Communications TELP MID-TERM EVALUATION: REVISED FINAL REPORT 88 Lesley Cooke Acting HOD Curriculum Development M Moodley Human Resources University of the Transkei Dr Nkosi Administrator of the University of the Transkei Prof Z Gebeda TELP Coordinator Mrs Barbara Nomsa Mkosi (Health Sciences Deputy Director) Dr MN Jadezweni Education Foundation Mr LA Monyooe Education Foundation Prof T Dye Zoology Dept Mr KK Govender Dept of Business Management Prof K Mfenyama Faculty of Health Sciences Mr J Nornwand Vice-Principal Mr Matiwana Deputy Register Academic Ms Nschtlhelani Acting Director of Human Resources Mr PK Gqulu acting Deputy Registrar Administration Dr Guwa Dean of Students Ms Z Dotwana Acting head of Student Councelling David Abbey – Programme & curriculum development Prof M Mahabir Dean of Economic Sciences Prof Mwamwenda Coordinator of LINKAGES Prof A Dye Dean of Research Prof M Glencross HOD Maths and Science Education Director of the Research Resource Centre. Peninsula Technikon Prof Brian Figaji Vice-Chancellor Merle Hodgson TELP Coordinator Prof J Tromp Deputy Vice-Chancellor Academic Affairs Prof L V Tanga Deputy Vice-Chancellor Student Affairs and International Relations Thelo Wakefield Paul Tennant Bennett Alexander Cecilia Jacobs James Garraway Norman Jacobs Dean faculty of Business Johnny Basson David Bleazard Lorenzo Himunchul Hilton Fransman Educational Development Centre V der Linde Shirely Levendal Siyabonga Ndabezithe Student Development Dr D Gihwala Dean Faculty of Science A P Staak Linkages Faculty of Engineering DR B Sun Linkages Thabong Molusi (student) Arele Foster (student) Michael Foster (student) Wessel Badenhorst TELP MID-TERM EVALUATION: REVISED FINAL REPORT 89 Tobias Titus Faculty of Business Nicky Arends Faculty of Business Graham Renecke Faculty of Business Lynn Coleman Materials Development University of the Western Cape Prof Cecil Abrahams Vice-Chancellor Prof Peter Vale Deputy Vice-Chancellor Muhammed Haron TELP Coordinator Lionel Nicholas Tahir Wood Natalie Pekeur-Engelbrecht Kailene Merawr E R Delves Larry Pohpas Rensche Bell Ruth Wills Hermine Engel Writing Centre Venda University Prof. D R Thakhathi Deputy Vice-Chancellor Republic Monakedi – TELP Coordinator Prof Nkomo Linkages Dr Brian Brink Library Nic Hoffmann Dept Religious Studies Vincent Bvuma – English Murray Hofmeyr – Religious Studies Ben Siyakwazi – Educational Foundation Bao Davhana – Student Affairs Marhudu Maselerel – Nursing Science Thidi Tshiguvho – Biological Sciences Technikon Northern Gauteng Prof G Lenyai – Vice-Chancellor Ntsatsi Mantsho – TELP Coordinator Dr M Collen – Vice Principal Student Services Ms T Maja Research Mr Mukhola Student Leadership Mr C Baloyi Student Leadership Ms M Seerane Student Academic Development Mr X Mankazana Student Academic Development Mr T Abraham Student Academic Development Ms C Selepe Curriculum Development Mrs A van der Westhuizen Curriculum Development Mr T Pudikabekwa - Dean of Engineering Mr J Makina Research Ms N Ngozwana Research University of the Durban Westville TELP MID-TERM EVALUATION: REVISED FINAL REPORT 90 Nadia Minty TELP Coordinator Ms Leanne Browning Student Academic Development Ms Rajie Tudge Strategic Planning Mr Nelson Moodley Research office Mrs M Doorasamy Research office Ms N Tengimfene Student Leadership & Enrichment Prof Dsoni Linkages Ms S Beerilall Workstudy Mr B Hadebe Science Education Project Ms S Seetal HOD Career Counselling Ms V Govendar Student Employment Office Mr B Nel School of Education Studies: Educational Technology University of Zululand Prof Dlamini Vice-Chancellor Ms G Gugule TELP Coordinator Prof H Glass Dept Sociology Ms T Dlamini Warden of Residences Prof P Dlamini Dean of Students Ms J Kiamba Dept Consumer Science University of Fort Hare Mr Z Hloma TELP Coordinator Miss N G Scott Training and Development Officer Mr J Mhaga Dean of Students Ms Nokweti Finance Jeremy Punt Strategic Planning sub-committee convenor Petrus Strydom Special assistant to the VC Eastern Cape Technikon Mr A Bomvu Vice Principal Mr V Mapolisa TELP Coordinator Irene Harvey Staff Development & Research Lindewa Mpono Student Councellor Faith Sigaba Student Affairs N Zulu HOD Quality Management Chris van Wyk Staff Development University of the North Prof Golele Acting Vice-Chancellor Morwamoche Ntwampe TELP Coordinator Mangosuthu Technikon E Zingu Acting Vice-Chancellor Mrs N Mkhize Border Technikon Prof Brunyee Vice-Chancellor M A Dandala TELP Coordinator TELP MID-TERM EVALUATION: REVISED FINAL REPORT 91 Medunsa University Prof T P Masihleho Deputy Principal for Administration Frank Daniels TELP Coordinator University of North West Prof Nico Gouws Vice-Chancellor Ms Masabata TELP Coordinator Dr M W Legotto Deputy Vice-Chancellor Linkages Coordinator Prof Simeon Taohe Dean of Science & Technology Mr Mkosama Mavuso Project manager Mr M A Makgwa Finance officer TELP MID-TERM EVALUATION: REVISED FINAL REPORT 92 10.7 Methodology The following document comprises the methodology plan submitted on 3rd April 2000. See Section 2.2 for details about subsequent changes to this plan. TELP MID-TERM EVALUATION: REVISED FINAL REPORT 93 TELP Mid-Term Evaluation DRAFT Methodology as of 3 April 2000 Purpose: This mid-term evaluation of TELP will “determine progress made in the achievement of the project goals and will test the validity of the underlying hypotheses and critical assumptions of the programme.” The evaluation will identify future actions and make recommendations so that it can be used as a planning document. It will focus on the 15 Historically Disadvantaged Institutions (HDIs) that have participated throughout the project. The evaluation will employ a highly consultative participatory process with key individuals who will be interviewed in their own right, key institutions and discussions with a number of focus groups. A reference group consisting of members of the TELP Executive Committee (the four Vice-Chancellors plus representatives from the Department of Education and USAID) will be used to test the methodology. All interviews and the focus group discussions will have prepared protocols. Deliverables: 1. A workplan which includes the method of evaluation and evaluation instruments 2. A pilot study in two institutions to test the instruments. The reference group will be instrumental in approving the methodology. A report will be presented to the reference group at the end of the pilot study with the object of presenting preliminary findings and of reviewing the methodology if necessary. 3. A draft report with recommendations on the evaluation. 4. A workshop with partners on 10 July 2000. 5. A final report. Research Team: Dr Gretta Middleton (Senior Evaluation Leader) Dr Michael Rice (Team Leader) Mr Thozamile Nomvete (Team Member) Dr Howard Williams (Evaluation Specialist off-site) Dr Michael Cross (Policy Specialist) Ms Amanda Fitschen (Finance Specialist) Dr Philip Christensen (Activity Coordinator) Data collection: The information to be collected will encompass quantitative and qualitative data. Data will be collected by means of: 1. A literature survey. A survey of the relevant documents will be undertaken, with special reference to the UNCF Progress Report, the DEVTEC Semi-Annual Report – appendix No. 4, Minutes of meetings, financial reports, the Devtech Academic Audit and so on. TELP MID-TERM EVALUATION: REVISED FINAL REPORT 94 2. Interviews. Individuals to be interviewed: · Vice-chancellors of each institution · TELP Coordinators at each institution where there is one · UNCF/College Fund personnel · Ms Thandiwe January-Mclean (Chief of Party) · Prof Mike Phala (specific achievements) · Dr Gloria Braxton (linkage grants) · Dr Stanley Van Harte (institution specific achievements) · Dr Rumilla Naran (TELP resource centre) Special interviews will be conducted with representatives from: · The Department of Education · USAID · Other contractors and grantees 3. Focus groups 1. Institute-specific joint activities: a focus group consisting of two participants from each of the Focus Areas. (Curriculum development with particular reference to the NQF and SAQA.) 2. Institutional activities & PILS: a focus group consisting of two participants each from proposal-writing and implementation workshops, plus representatives from the PIL Focus Areas, excluding the TELP Coordinator. (Ensure that students are included where relevant, at least two including the SRC President). 3. Linkages: a focus group from each parameter of the focus. The choice of participants in each focus group will be crucial to its success. Certainly, every attempt will be made to include in the relevant focus groups the Dean of Students, the Dean of Science, the DVC Academic Affairs/HR and the Project Directors, as well as a representative group from the student leadership. Each focus group will have a specifically designed discussion schedule. 4. Questionnaires. Questionnaires will be designed to capture the relevant quantitative data, especially the financial data. Process: The evaluation of the institutions will take the following form: · The methodology will be referred to the Reference Group for comment and amendment. · Interviews to be conducted with the Department of Education, USAID and UNCF. · A literature survey of available documents will be carried out. · The Finance Specialist will conduct a financial analysis of the whole TELP operation. · The Senior Evaluation Leader and the Team Leader will pilot the methodology in two of the HDIs. Each institution will be visited for two days. TELP MID-TERM EVALUATION: REVISED FINAL REPORT 95 · A report on the initial findings of the pilot will be written and submitted to the Reference Group, then discussed by the Executive Committee at a workshop on 9 th May. · The Team Leader and Member will conduct interviews and focus group meetings at eight additional HDIs. Each institution will be visited for two days beginning 15th May. · The Senior Evaluation Leader will conduct interviews with the Vice-Chancellors and TELP Coordinators at all 13 remaining HDIs. · A draft report will be submitted to the TELP Advisory Panel which, when approved by USAID, will be sent to the Vice-Chancellors, the Department of Education and USAID contractors and grantees. · A one-day workshop will be held to discuss the draft report with the Vice￾Chancellors on the 10th July. · The final report will be submitted to USAID by 14th July. Scope: The evaluation shall: 1. Determine whether current strategies supported by TELP are relevant. 2. Determine the extent of the changing environment of the HDIs and whether or not TELP is addressing these changes. 3. Determine if TELP activities are linked to achieved stated results, and if not provide recommendations to do so. 4. Determine whether TELP is achieving progress towards its stated results using programme benchmarks. If not provide recommendations. 5. Determine if the achievements of TELP are cost effective. 6. Determine whether actual TELP results are focussed towards intended results. 7. Identify intended and unintended results. 8. Determine if TELP accomplishments are sustainable. If not provide recommendations. 9. Identify the most effective ways to achieve TELP objectives. 10. Identify ways to address problem areas. Areas to be evaluated: 1. Institution specific activities: a sample of institution specific activities funded through PILs. Specific activities to be decided in consultation with USAID. Interviews will be conducted with Vice-Chancellors, TELP Coordinators, Activity Managers and a sample of persons participating (in focus groups) in the activities. 2. Joint activities: A sample of activities supported or funded through contractors or grantees. This is a departure from the original terms of reference necessitated by the decision to sample 10 institutions rather than survey all 15. Data will be collected from the DOE, Vice-Chancellors, TELP Coordinators, Activity managers, persons participating (in focus groups) and contractors and grantees. 3. Linkage activities: The linkages funded through PILs and contractors. Interviews will be conducted with Vice-Chancellors, TELP Coordinators, Activity Managers and a sample of persons participating (in focus groups) in the activities. Evaluation questions: Overall questions: TELP MID-TERM EVALUATION: REVISED FINAL REPORT 96 1. Are the current development strategies supported by USAID/SA relevant to address development constraints TELP was designed to address? 2. To what extent has TELP been responsive to the changing environment of the HDIs and to the critical challenges facing the HDIs? 3. Are TELP activities sufficiently integrated to achieve the stated results? 4. Is TELP achieving satisfactory progress towards its stated goals and objectives? 5. Are TELP activities cost effective? 6. What are the intended and unintended impacts of TELP? 7. Are the accomplishments of TELP sustainable? Process questions: 8. Do appropriate management structures exist at USAID/SA and the HDIs to facilitate the implementation of TELP? 9. Have TELP Project Coordinators been able to coordinate TELP activities at their institutions? 10. Is TELP supporting the appropriate mix of activities to build capacity at HDIs? What other activities or approaches should TELP consider? 11. Are long term staff development mechanisms appropriate to facilitate the improvement of capacity building at HDIs? 12. To what extent are TELP initiatives integrated into the institutional development plan or three year rolling plan? 13. Are TELP activities and strategies responsive to the needs of individual institutions? 14. Have synergies being actively explored between TELP and other development activities at their institution? 15. To what extent are institutions evaluating progress of ongoing TELP interventions and applying the lessons learned? Outcome questions 16. Which TELP activities are contributing the most to the achievement of TELP objectives? 17. Which activities are contributing least to the achievement of TELP objectives? 18. To what extent are returning TELP candidates contributing to institutional strengthening? 19. To what extent are recipients or participants of TELP staff development initiatives retained by institutions? 20. What factors account for the success or failure of TELP to achieve its objectives? Impact questions 21. Do TELP initiatives have institution-wide impact? 22. Is there a discernable correlation between training received and subsequent performance of training participants at their institutions? Institutions: The institution evaluation will be preceded by a pilot evaluation to test the methodology. Pilot institutions: University of the Transkei TELP MID-TERM EVALUATION: REVISED FINAL REPORT 97 ML Sultan Technikon Institutions to be sampled: Zululand University University of Fort Hare Eastern Cape Technikon Venda University Technikon Northern Gauteng Peninsula Technikon University of Durban Westville University of the Western Cape Rationale for the selection of the institutions: Budget and time constraints have resulted in the original strategy, of carrying out in￾depth evaluations at each of the 15 historically disadvantaged institutions (HDIs), being modified. It is no longer possible to produce institution-specific reports to support the main evaluation. This does not mean that the different experiences of the various institutions will be ignored. On the contrary, such variety will inform the project-wide findings and recommendations. However, it will not be possible to examine every HDI in depth. Each HDI will be visited during the course of the evaluation, but a sample of 10 (two for the pilot, eight for the evaluation) will be examined more carefully. In other words, the methodology has moved from being a census to a sample. The sample has been stratified on the basis of perceived institutional success within the TELP context: low, medium, high. Accordingly, it can be seen that those institutions that have a high or low success rate are to receive the most attention. Further, an attempt has been made to balance the technikon/university representation. The same principles apply to the pilot. A constraining factor was the necessity to choose institutions that had a functioning TELP Coordinator. However, it was also agreed that it would be useful to see how the project had fared at an institution where there is at present no TELP Coordinator, and accordingly one such institution was chosen. INSTITUTION FOCUS GROUP PROTOCOLS The following presents the generic focus areas for discussion in all focus group meetings at all the institutions. Issues specific to individual institutions are detailed below. GENERAL QUESTIONS: · Why only three focus areas? · To what extent are TELP initiatives integrated into mainstream initiatives? Synergy? · What lessons have been learned? · Are lessons learned being applied? TELP MID-TERM EVALUATION: REVISED FINAL REPORT 98 · Which activities are contributing most/least to the TELP objectives? · To what extent are TELP candidates contributing to institutional strengthening? · To what extent are recipients of staff development retained by the institutions? · What factors account for the success or failure of TELP? · Do TELP initiatives have institution wide impact? · Is there a correlation between training and subsequent performance? · Comment on the quality of presentation and organisation at workshops. JOINT ACTIVITIES: Curriculum design: · understanding of the role of SAQA, NQF requirements and the technical needs to meet NQF requirements · understanding of the NQF requirements · understanding of the technical needs to meet NQF requirements · technical assistance on writing of learning outcomes · build capacity towards uniform understanding of NQF technical requirements · usefulness of workshop on where institutions should be by June 2000 · understanding of the interplay between SAQA, DOE and CHE re DOE and SAQA to comply with June 2000 and June 2003 NQF requirements Student development · new challenges to student development and facilitate the development of draft institutional five year student development plans · work study model that will provide financial assistance to students which encouraging cooperative learning · leadership training · 5 year student development plans · train staff to set up and administer bridging programmes for under prepared first time entering undergraduate students · assess the status of work study and development of and implementation of modified DTET work study models. Management & administration development · first round of three year rolling planning process, problem areas and capacity building strategies · gender · awareness of national planning issues and the facilitation of the process of developing a planning framework; enhance management capacity to develop three year rolling plans · role of leadership in institutional change; facilitate awareness of best practice. · equity INSTITUTION SPECIFIC ACTIVITIES & PILS: all institutions received workshops in: · Proposal writing · Implementation LINKAGES are specified individually TELP MID-TERM EVALUATION: REVISED FINAL REPORT 99 SPECIFIC INSTITUTIONAL ACTIVITIES Eastern Cape Technikon JOINT ACTIVITIES: specific workshops and areas to probe: Curriculum design: · writing of learning outcomes Student development · clarify SATS/Version 2 of diagnostic tests Management & administration development · equity INSTITUTION SPECIFIC ACTIVITIES & PILS · Staff dev & linkages · Eastern Cape LINKAGES: Texas Southern University Parameters of the focus: Engineering, Business Science, Applied Technology Academic, Administrative and Support Staff development Research, curriculum development Programme design M L Sultan Technikon INSTITUTION SPECIFIC ACTIVITIES & PILS · Student & staff development · Programme & curriculum dev. · Research LINKAGES: Letter of Intent with Savannah State University Parameters: Science , Engineering Staff development and infrastructure in: · Waste Water Technology · Food Science · Renewable energy for rural development Peninsula Technikon JOINT ACTIVITIES: specific workshops and areas to probe: Curriculum design: · language support into curriculum · development of qualification specifications Student development · clarify SATS/Version 2 of diagnostic tests Management & administration development · equity INSTITUTION SPECIFIC ACTIVITIES & PILS · Programme, Staff & Curriculum Development TELP MID-TERM EVALUATION: REVISED FINAL REPORT 100 · Staff and student development LINKAGES: University of Michigan Parameters: Mechanical Engineering · Cleaner production technology research · Smart Materials and Structures University of Durban-Westville INSTITUTION SPECIFIC ACTIVITIES & PILS · Curriculum, Staff Development & Linkages · Programme and curriculum development LINKAGES: Letter of Intent with Penn State Consortium Paremeters: All leadership levels · Management and leadership · Staff Development University of the Transkei JOINT ACTIVITIES: specific workshops and areas to probe: Curriculum design: · understanding of OBE · development of qualification specifications Student development · clarify SATS/Version 2 of diagnostic tests INSTITUTION SPECIFIC ACTIVITIES & PILs · Programme and curriculum dev. LINKAGES: Letter of Intent with University of Connecticut Parameters: Central Admin · Strategic, institutional and organisational development · University-based business development and income generating strategies through research · Management and the development of quality assurance academic programmes University of the Transkei INSTITUTION SPECIFIC ACTIVITIES & PILS · Programme and curriculum development LINKAGES: City College of New York Parameters: Law, Medicine and Health Sciences, Arts and Social Sciences and Economic Sciences. · Staff development · Student development · Curriculum development TELP MID-TERM EVALUATION: REVISED FINAL REPORT 101 University of Venda INSTITUTION SPECIFIC ACTIVITIES & PILS · Programme and staff development LINKAGES: Letter of Intent with Georgia State University Parameters: School of Business, Economics and Administrative Sciences · Tourism and Hospitality, Business Information Systems, Centre for Entrpreneurship University of the Western Cape JOINT ACTIVITIES Management & administration development · equity INSTITUTION SPECIFIC ACTIVITIES & PILS · Linkages and staff development · Student development · Programme and curriculum development LINKAGES: University of Missouri System Parameters: Science, Community and Health Sciences, Economics and Management Sciences, Education, Law, Humanities · Staff development University of Zululand INSTITUTION SPECIFIC ACTIVITIES & PILS · Programme and curriculum development LINKAGES: Mississippi Consortium for International Development Parameters: Chemistry, Home Economics, Business Economics, Social Work · Research, curriculum design and development, Finance, Management, Information Systems, alumni networks, student exchange, library development. TELP MID-TERM EVALUATION: REVISED FINAL REPORT 102 10.8 Approved Activity Plan This annex presents the activity plan approved by the Reference Group at the beginning of the evaluation activity. It shows scheduled tasks and indicates anticipated participation (level of effort) by specific team members. Mr. Nomvete’s withdrawal (see Section 2.2.5 above) caused the most significant changes to this plan, with Dr. Middleton joining Dr. Rice for the two-day site visits and Dr. Cross substituting for Dr. Middleton on five of the single-day interview sessions with Vice-Chancellors and TELP Coordinators. The Reference Group also added one meeting on 28th June to review the preliminary draft of this report prior to its circulation to stakeholders. With the exception of the above-note changes the following plan was implemented up to the end of June with only minor changes to dates and participation schedules. TELP MID-TERM EVALUATION: REVISED FINAL REPORT 103 TELP EVALUATION ACTIVITY PLAN: REVISION 2 Notes: · Deliverables noted in italics · All dates mm/dd/yy format · Time d=days, w=weeks · Level of effort o n = discrete person-days for category o 0 = task assigned but person-days included in another task · “ExCo” = TELP Executive Committee Key to Consultant Level of Effort · 1: Activity Manager (Christensen) · 2: Senior Evaluation Leader (Middleton) · 3: Team Leader (Rice) · 4: Team Member (Nomvete) · 5: Evaluation Specialist (Williams, off-site) · 6: Policy Specialist (Cross) · 7: Finance Specialist (Fitschen) TELP MID-TERM EVALUATION: REVISED FINAL REPORT 104 Activity Time Start End L O E Notes 1 2 3 4 5 6 7 1. PLANNING PHASE Mon 3/27/00 Fri 4/7/00 2. Review project documentation 2d Mon 3/27/00 Tue 3/28/00 0 0 0 3. Initial USAID briefing 0.5d Mon 3/27/00 Mon 3/27/00 .5 .5 .5 4. Draft strategies, evaluation meth. & instruments, financial review model, data collection plan, work plan 4d Mon 3/27/00 Thu 3/30/00 3.5 3.5 3.5 2 2 2 5. Select 2 pilot institutions 0.5d Fri 3/31/00 Fri 3/31/00 .5 .5 .5 6. Hold planning meeting with USAID 0.5d Fri 3/31/00 Fri 3/31/00 .5 .5 .5 7. Solicit feedback from ExCo 3d Mon 4/3/00 Wed 4/5/00 Office staff distributes and follows up 8. Arrange pilot HDI visits 1d Mon 4/3/00 Mon 4/3/00 Office staff 9. Preliminary visits DOE, key contractors, grantees 3d Mon 4/3/00 Wed 4/5/00 3 3 2 Financial specialist gathers macro￾level data 10. Continue documentation review, refine methodology & instruments 3d Mon 4/3/00 Wed 4/5/00 2 0 0 3 Financial specialist gathers macro￾level data 11. Revise instruments, work plan 2d Thu 4/6/00 Fri 4/7/00 2 2 12. Print instruments, work plan 1d Fri 4/7/00 Fri 4/7/00 OFFICE STAFF 13. Submit & distribute work plan 0d Fri 4/7/00 Fri 4/7/00 TELP MID-TERM EVALUATION: REVISED FINAL REPORT 105 Activity Time Start End L O E Notes 1 2 3 4 5 6 7 14. PILOT PHASE Mon 4/10/00 Thu 5/11/00 15. Distribute questionnaires 13 HDIs 3w Mon 4/10/00 Thu 5/4/00 Office staff 16. Arrange visits 13 HDIs 1w Mon 4/10/00 Fri 4/14/00 Office staff 17. Visit 1st pilot institution 2.5d Mon 4/10/00 Wed 4/12/00 2.5 2.5 18. Visit 2nd pilot institution 2.5d Wed 4/12/00 Fri 4/14/00 2.5 2.5 19. Draft pilot study report & begin methodology revisions 4d Mon 4/17/00 Thu 4/20/00 4 4 1 2 20. Print copies draft report 1d Thu 4/20/00 Thu 4/20/00 Office staff 21. Submit 10 copies draft report USAID 0d Thu 4/20/00 Thu 4/20/00 22. Hold briefing session for USAID 0.5d Tue 4/25/00 Tue 4/25/00 .5 .5 23. Revise draft report, strategy & instruments as required 1.5d Tue 4/25/00 Wed 4/26/00 1.5 1.5 1 1 24. Print & distribute revised report for ExCo review 6d Fri 4/28/00 Mon 5/8/00 Office staff distributes and follows up 25. Interview remaining partners 4d Fri 4/28/00 Thu 5/4/00 4 4 26. Analyze questionnaire data 13 HDIs 2d Fri 5/5/00 Mon 5/8/00 1 2 2 2 TELP MID-TERM EVALUATION: REVISED FINAL REPORT 106 Activity Time Start End L O E Notes 1 2 3 4 5 6 7 27. Workshop to solicit feedback from ExCo 1d Tue 5/9/00 Tue 5/9/00 1 1 1 1 28. Final revisions to report & instruments 2d Wed 5/10/00 Thu 5/11/00 2 2 2 1 29. Reprint report (as req’d) 1d Thu 5/11/00 Thu 5/11/00 Office staff 30. Submit and distribute final pilot￾phase report 0d Thu 5/11/00 Thu 5/11/00 31. EVALUATION PHASE 48d Fri 5/12/00 Fri 7/14/00 32. Print HDI visit instruments and docs 1d Fri 5/12/00 Fri 5/12/00 Office staff 33. Orient remaining team member 1d Fri 5/12/00 Fri 5/12/00 1 1 1 1 34. Team leader visits HDIs 1-4 + travel & write-up 6d Mon 5/15/00 Sat 5/20/00 1 6 Team members work one extra day/week (Sat.) for write-ups 35. Field team visits sample HDIs 1-2 + travel & write-up 6d Mon 5/15/00 Sat 5/20/00 6 6 Team members work one extra day/week (Sat.) for write-ups 36. Team leader visits HDIs 5-8 + travel & write-up 6d Mon 5/22/00 Sat 5/27/00 1 6 Team members work one extra day/week (Sat.) for write-ups 37. Field team visits sample HDIs 3-4 + travel & write-up 6d Mon 5/22/00 Sat 5/27/00 6 6 Team members work one extra day/week (Sat.) for write-ups 38. Team leader visits HDIs 9-12 + travel & write-up 6d Mon 5/29/00 Sat 6/3/00 1 6 Team members work one extra day/week (Sat.) for write-ups 39. Field team visits sample HDIs 5-6 + travel & write-up 6d Mon 5/29/00 Sat 6/3/00 6 6 Team members work one extra day/week (Sat.) for write-ups TELP MID-TERM EVALUATION: REVISED FINAL REPORT 107 Activity Time Start End L O E Notes 1 2 3 4 5 6 7 40. Team leader visits HDI 13 + write￾up 3d Mon 6/5/00 Wed 6/7/00 1 3 41. Field team visits sample HDIs 7-8 + write-up 4d Mon 6/5/00 Wed 6/7/00 4 4 3 Financial specialist completes sustainability analysis on sample of institutional activities 42. Follow-up w/ HDIs & other parties as necessary 2d Thu 6/8/00 Fri 6/9/00 2 1 1 2 Financial specialist completes sustainability analysis on sample of institutional activities 43. Completion field work 0d Fri 6/9/00 Fri 6/9/00 44. Preliminary results analysis 2d Mon 6/12/00 Tue 6/13/00 2 2 2 2 1 2 45. Write draft report 7d Wed 6/14/00 Fri 6/23/00 7 7 7 7 1 3 46. Submit 1 copy draft report USAID 0d Fri 6/23/00 Fri 6/23/00 47. USAID reviews draft report 3d Mon 6/26/00 Wed 6/28/00 48. Complete assignments 3d Mon 6/26/00 Wed 6/28/00 3 3 49. Revise draft report as required 2d Thu 6/29/00 Fri 6/30/00 2 2 2 50. Print revised draft report 1d Fri 6/30/00 Fri 6/30/00 Office staff 51. Submit revised report to all partners 0d Fri 6/30/00 Fri 6/30/00 TELP MID-TERM EVALUATION: REVISED FINAL REPORT 108 Activity Time Start End L O E Notes 1 2 3 4 5 6 7 52. Partners review revised draft 5d Mon 7/3/00 Fri 7/7/00 53. Prepare for workshop 5d Mon 7/3/00 Fri 7/7/00 1 5 54. 1-day workshop w/ partners 1d Mon 7/10/00 Mon 7/10/00 1 1 1 1 1 55. Revise report as required 3d Tue 7/11/00 Thu 7/13/00 3 3 3 1 1 56. Print final report 1d Fri 7/14/00 Fri 7/14/00 Office staff 57. Submit final report to USAID 0d Fri 7/14/00 Fri 7/14/00 TOTAL LEVELS OF EFFORT 16w 84 129 124 61 4 4 52 TOTAL LOE: 0 PERSON-DAYS TELP MID-TERM EVALUATION: REVISED FINAL REPORT 109 10.9 Extended Activity Plan At its meeting of 28th June the Reference Group asked the evaluation team to provide greater specificity in several areas of the draft report, especially in terms of findings about the role of different partners (USAID, the historically disadvantaged institutions as a group, and the Contractor) in various issues encountered so far in the project. The Reference Group also requested another meeting to review the revised draft prior to its circulation to all stakeholders. It was accepted that the evaluation team would require additional time to respond to these requests. This decision required postponing the stakeholders’ workshop originally planned for 10th July and extending the evaluation team’s work. On 7 th July AED submitted to USAID a revised activity plan for the concluding five weeks of the activity, as shown on the next page. The plan includes revised levels of effort for five of the principal consultants, labelled as follows in the following table: 1: Christensen 2: Middleton 3: Rice 4: Cross 5: Fitschen At the 10th July Reference Group meeting it was decided to further postpone (until August) the stakeholders’ workshop, due to lack of available Vice-Chancellors on 31st July. TELP MID-TERM EVALUATION: REVISED FINAL REPORT 110 Activity Time Start End 1 L 2 O 3 E 4 5 58. Complete reanalysis and section rewriting. 5d Mon 7/3/00 Fri 7/7/00 4 5 5 59. Final rewriting and editing of second draft. 3d Mon 7/10/0 0 Wed 7/12/00 3 3 2 60. Circulate second draft to Ref. Group 1d Thu 7/13/0 0 Thu 7/13/00 1 1 61. Ref. Group meets again 1d Fri 7/14/0 0 Fri 7/14/00 1 1 1 62. Revise report 3d Mon 7/17/0 0 Wed 7/19/00 3 3 2 1 63. Print revised report 1d Thu 7/20/0 0 Thu 7/20/00 1 1 64. Circulate final report to all stakeholders 1d Fri 7/21/0 0 Fri 7/21/00 1 1 65. Prepare workshop 5d Mon 7/24/0 0 Fri 7/28/00 3 66. Hold final workshop 1d Mon 7/31/0 0 Mon 7/31/00 1 1 1 1 1 67. Make final revisions 3d Tue 8/1/00 Thu 8/3/00 3 3 3 68. Print final report 1d Fri 8/4/00 Fri 8/4/00 1 69. Submit final report to USAID 0d Fri 8/4/00 Fri 8/4/00 Totals (July/August) 5w 19 22 14 1 1 REVISED TOTALS 19w 1d 46 90 82 10 20 Original Totals 16w 32 77 72 4 26 TELP MID-TERM EVALUATION: REVISED FINAL REPORT 111 10.10 Interview Protocols 10.10.1 Vice-Chancellor Interviews · What is the role of the Advisory Panel? · Are relations between the Advisory Panel and USAID/SA satisfactory? · Are relations between your institution and USAID/SA satisfactory? · How are your relations with contractors and grantees? · How has participation in TELP benefited your institution? · Is TELP contributing to building capacity? · Has TELP contributed to institutional empowerment? · To what extent has TELP been responsive to the changing higher education environment at your institution? · Is TELP responsive to the critical challenges that you face? · Does the existing TELP structure facilitate or hinder programmatic implementation during this time of transformation? · Are TELP activities and strategies responsive to institutional needs? · Are TELP activities sufficiently integrated to achieve stated goals? · What are the impacts of TELP? · Do TELP initiatives have institution-wide impact? · Has the TELP Project Coordinator been able to effectively coordinate TELP activities? · Does the TELP Coordinator have adequate support to perform designated duties? · Are staff development mechanisms appropriate and adequate to foster capacity building? · Are TELP initiatives integrated into the institution’s development plan and rolling plans? · Are synergies being actively explored between TELP and other development activities at the institution? · Is TELP achieving satisfactory progress? · Which activities contribute the most/least to the achievement of TELP goals and objectives? · What factors contribute to the success or failure of TELP activities? · Are TELP funded training participants contributing to institutional strengthening? · To what extent are recipients or participants of TELP staff development initiatives being retained? · Is there a discernible correlation between training received and subsequent performance of training participants? · How do you measure performance? · Comment on the implementation of Joint Activities, PILs and Linkages. · Are the accomplishments of TELP sustainable? · What are the TELP Project problem areas that need to be addressed? TELP MID-TERM EVALUATION: REVISED FINAL REPORT 112 10.10.2 TELP Coordinator Interviews · Are you engaged full-time in TELP Project tasks? · Do you have other institutional responsibilities? If yes, how is your time divided? · Do you have a good working relationship with USAID/SA? · How are your relations with TELP funded organizations? · Do you receive full cooperation and support from key personnel at your institution? · Is your position self-contained? · Do you regularly interface with persons at the institution not involved in TELP activities? · Do you serve as a primary point of contact for TELP on campus? · Is the training provided by USAID on procurement, financial management and other relevant topics adequate for effective job performance? · Do you collaborate and network with other TELP coordinators? · Are you provided secretarial support by the institution? · Does the TELP project make enough resources available for you to do a satisfactory job? · Are there any obstacles to carrying out TELP related activities? · Do you receive requisite data from the institution and participants on a timely basis? · Are you generally able to meet USAID deadlines for submission of vouchers, reports and other documentation? · How long have you been a TELP Project Coordinator? · Have other persons previously occupied your post? · What are the most/least satisfying aspects of your job? · What are the major accomplishments of TELP? · Are there any problem areas that need to be addressed? TELP MID-TERM EVALUATION: REVISED FINAL REPORT 113 10.11 Focus Group Protocols INSTITUTION FOCUS GROUP PROTOCOLS The following presents the generic focus areas for discussion in all focus group meetings at all the institutions. Issues specific to individual institutions are detailed below. Please circulate to the relevant people. If anything has been overlooked please include it as appropriate. GENERAL QUESTIONS: ß To what extent are TELP initiatives integrated into mainstream initiatives? Synergy? ß What lessons have been learned? ß Are lessons learned being applied? ß Which activities are contributing most/least to the TELP objectives? ß To what extent are TELP candidates contributing to institutional strengthening? ß To what extent are recipients of staff development retained by the institutions? ß What factors account for the success or failure of TELP? ß Do TELP initiatives have institution wide impact? ß Is there a correlation between training and subsequent performance? ß Comment on the milestones. Have they facilitated implementation? ß Comment on the quality of presentation and organization at workshops. ß Comment on the changing HE environment and how it is affecting TELP. ß Comment on financial management traing JOINT ACTIVITIES: Curriculum design: ß understanding of the role of SAQA, NQF requirements and the technical needs to meet NQF requirements ß understanding of the NQF requirements ß understanding of the technical needs to meet NQF requirements ß technical assistance on writing of learning outcomes ß build capacity towards uniform understanding of NQF technical requirements ß usefulness of workshop on where institutions should be by June 2000 ß understanding of the interplay between SAQA, DOE and CHE re DOE and SAQA to comply with June 2000 and June 2003 NQF requirements Student development ß new challenges to student development and facilitate the development of draft institutional five year student development plans ß work study model that will provide financial assistance to students which encouraging cooperative learning ß leadership training ß train staff to set up and administer bridging programmes for under prepared first time entering undergraduate students ß assess the status of work study and development of and implementation of modified DTET work study models. Management & administration development ß first round of three year rolling planning process, problem areas and capacity building strategies ß gender & equity ß awareness of national planning issues and the facilitation of the process of developing a planning framework; enhance management capacity to develop three year rolling plans ß role of leadership in institutional change; facilitate awareness of best practice. ß Financial management training INSTITUTION SPECIFIC ACTIVITIES & PILS: all institutions received workshops in: ß Proposal writing ß Implementation TELP MID-TERM EVALUATION: REVISED FINAL REPORT 114 LINKAGES are specified individually below. SPECIFIC INSTITUTIONAL ACTIVITIES Eastern Cape Technikon JOINT ACTIVITIES: specific workshops and areas to probe: ß Curriculum design: ß writing of learning outcomes Student development ß clarify SATS/Version 2 of diagnostic tests Management & administration development ß equity INSTITUTION SPECIFIC ACTIVITIES & PILS ß Staff dev & linkages ß Eastern Cape LINKAGES: Texas Southern University ß Parameters of the focus: Engineering, Business Science, Applied Technology ß Academic, Administrative and Support Staff development ß Research, curriculum development ß Programme design M L Sultan Technikon INSTITUTION SPECIFIC ACTIVITIES & PILS ß Student & staff development ß Programme & curriculum dev. ß Research LINKAGES: Letter of Intent with Savannah State University Parameters: Science , Engineering Staff development and infrastructure in: ß Waste Water Technology ß Food Science ß Renewable energy for rural development Peninsula Technikon JOINT ACTIVITIES: specific workshops and areas to probe: Curriculum design: ß language support into curriculum ß development of qualification specifications Student development ß clarify SATS/Version 2 of diagnostic tests Management & administration development ß equity INSTITUTION SPECIFIC ACTIVITIES & PILS ß Programme, Staff & Curriculum Development ß Staff and student development LINKAGES: University of Michigan TELP MID-TERM EVALUATION: REVISED FINAL REPORT 115 Parmeters: Mechanical Engineering ß Cleaner production technology research ß Smart Materials and Structures University of Durban-Westville INSTITUTION SPECIFIC ACTIVITIES & PILS ß Curriculum, Staff Development & Linkages ß Programme and curriculum development LINKAGES: Letter of Intent with Penn State Consortium Paremeters: All leadership levels ß Management and leadership ß Staff Development Technikon Northern Gauteng INSTITUTION SPECIFIC ACTIVITIES & PILs ß Student development – academic support ß Research – design and implement a research programme LINKAGES: Letter of Intent with Tennessee State University Parameters: Engineering and Built environment, Finance and management, agriculture ß Mechatronics, Architecture and Art design ß Food engineering ß Finance and management ß Research information and communications technology University of the Transkei INSTITUTION SPECIFIC ACTIVITIES & PILS ß Programme and curriculum development LINKAGES: City College of New York Parameters: Law, Medicine and Health Sciences, Arts and Social Sciences and Economic Sciences. ß Staff development ß Student development ß Curriculum development University of Venda INSTITUTION SPECIFIC ACTIVITIES & PILS ß Programme and staff development LINKAGES: Letter of Intent with Georgia State University Parameters: School of Business, Economics and Administrative Sciences ß Tourism and Hospitality, Business Information Systems, Centre for Entrpreneurship University of the Western Cape JOINT ACTIVITIES Management & administration development ß equity INSTITUTION SPECIFIC ACTIVITIES & PILS ß Linkages and staff development ß Student development ß Programme and curriculum development LINKAGES: University of Missouri System TELP MID-TERM EVALUATION: REVISED FINAL REPORT 116 Parameters: Science, Community and Health Sciences, Economics and Management Sciences, Education, Law, Humanities ß Staff development University of Zululand INSTITUTION SPECIFIC ACTIVITIES & PILS ß Programme and curriculum development LINKAGES: Mississippi Consortium for International Development Parameters: Chemistry, Home Economics, Business Economics, Social Work ß Research, curriculum design and development, Finance, Management, Information Systems, alumni networks, student exchange, library development. TELP MID-TERM EVALUATION: REVISED FINAL REPORT 117 10.12 Financial Survey Instrument TELP MID-TERM EVALUATION: REVISED FINAL REPORT 118 TERTIARY EDUCATION LINKAGES PROJECT (TELP) MID TERM REVIEW FINANCIAL QUESTIONNAIRE Institution: ……………………………………………………………………………………………….. Respondent Details: Name: …………………………………………………………………………………………….. Telephone: ………………………………………………………………………………………. Fax: ………………………………………………………………………………………. E-mail address: …………………………………………………………………………………. The aim of this instrument is to get information on how your institution uses the funds it receives through TELP and how you manage those funds. Where questions ask for detail on actual expenditure please provide as detailed as possible breakdown of expenditure to date. Where specific detail is not available, aggregate figures will be a valuable second best. If you have any questions or require assistance in completing this instrument, please ring Amanda Fitschen on (082) 457-1299 or email her at amfit@yebo.co.za. Please return the completed questionnaire to the evaluators when they visit your campus. If you wish to submit additional information subsequently, please fax it to Amanda Fitschen on (021) 685-1604 or send it by overnight courier to: Amandla Consulting 2 Alfred Road Newlands 7700 Thank you for you assistance with this important input to the TELP mid-term evaluation. TELP MID-TERM EVALUATION: REVISED FINAL REPORT 119 Question 1: TELP Expenditures We would like to know as much as possible about how you are spending TELP funds on specific projects and activities. This will help us analyse the different categories of expenditure across all institutions and focus areas. Please attach to this questionnaire copies of your summary expenditure records for each TELP activity that your institution has managed. We imagine that you might have a one-page summary for each activity showing how much money you received from USAID, when you received it, and how you spent it by broad category (such as project management, travel, supplies, communications, commodities, etc.) However, if your records are in a different format simply provide copies of whatever you have. If you do not have activity-by-activity records, please explain your record-keeping system below and give us whatever information you have on TELP expenditures: when funds were received, how much you received, and how you spent the monies. Use additional pages as necessary to answer this question. _________________________________________________________________ _________________________________________________________________ _________________________________________________________________ _________________________________________________________________ _________________________________________________________________ _________________________________________________________________ _________________________________________________________________ _________________________________________________________________ _________________________________________________________________ _________________________________________________________________ _________________________________________________________________ _________________________________________________________________ _________________________________________________________________ _________________________________________________________________ _________________________________________________________________ _________________________________________________________________ _________________________________________________________________ _________________________________________________________________ _________________________________________________________________ _________________________________________________________________ _________________________________________________________________ _________________________________________________________________ _________________________________________________________________ _________________________________________________________________ TELP MID-TERM EVALUATION: REVISED FINAL REPORT 120 Question 2: Focus Areas TELP focuses on five areas - Staff Development, Curriculum and Programme Development, Student Development, Management and Administrative Development and Research Skills Development. a) Does your institution identify the five focus areas when accounting for expenditure? YES NO b) If yes, please provide the breakdown (in Rand) in the table below for you expenditures to date in each focus area. FOCUS AREAS EXPENDITURE (RAND) Staff Development Curriculum and Programme Development Student Development Management and Administrative Development Research Skills Development TOTAL TELP MID-TERM EVALUATION: REVISED FINAL REPORT 121 Question 3: Funding Mechanisms TELP uses three main mechanisms to provide funds to institutions: institution￾specific activities (through PILs), joint activities and linkage grants. a) Does your institution identify the three different mechanisms when accounting for expenditure? YES NO b) If yes, please provide the breakdown (in Rand) in the table below for you expenditures to date for each mechanism. We understand that money for joint activities and linkage grants also goes to other institutions (the TELP Contractor and co-operating U.S. institutions). In this question we only want to know how TELP funds that have come to your particular institution have been spent across these categories. FUNDING MECHANISMS EXPENDITURE (RAND) Institution-Specific Activities Joint Activities Linkage Grants TOTAL TELP MID-TERM EVALUATION: REVISED FINAL REPORT 122 Question 4: Institutional Funds a) Does your institution supplement TELP funds with its own resources to achieve the overall objectives of the programme, through in-kind or cash contributions? YES NO b) If YES, please report the level of your in-kind and cash contributions to TELP￾supported activities in the table below. If you do not have records of your contributions by activity, you may report by focus area or just report your overall contributions in the “Total” row. Use additional pages as necessary to answer this question. TELP ACTIVITY OR FOCUS AREA IN-KIND CONTRIBUTION (RAND EQUIVALENT) CASH CONTRIBUTION (RAND) TOTAL TELP MID-TERM EVALUATION: REVISED FINAL REPORT 123 Question 5: External Funds a) Does your institution obtain funding from external sources (e.g., other than USAID contributions and your own budget) to supplement TELP funds to achieve overall objectives of the programme? YES NO b) If YES, what is the extent and source of such external funding? (If the funds are spent in specific focus areas please supply the details in the third column below, as show.) Use additional pages as necessary to answer this question. SOURCE OF FUNDS AMOUNT (RAND) FOCUS AREA TOTAL c) If NO, please give reasons why your institution does not use external funding to supplement TELP funds. REASON YES NO TELP funds are sufficient – no need to supplement No funders available Do not have the human capacity to make application for funds Funders not prepared to supplement TELP projects Other reason(s): (please provide details) _________________________________________________________________ _________________________________________________________________ _________________________________________________________________ _________________________________________________________________ _________________________________________________________________ TELP MID-TERM EVALUATION: REVISED FINAL REPORT 124 Question 6: Expenditure Reimbursement Process a) In general, does your institution feel that the TELP funding process is efficient? YES NO Please explain below: _________________________________________________________________ _________________________________________________________________ _________________________________________________________________ _________________________________________________________________ _________________________________________________________________ _________________________________________________________________ _________________________________________________________________ b) Have you experienced delays in the receipt of funds owed to your institution for TELP? FUNDING SOURCE YES NO N/A USAID Contractor (UNCF) Linkage Grants (U.S. Institutions) Other: ____________________________________________ Please comment below as necessary: _________________________________________________________________ _________________________________________________________________ _________________________________________________________________ _________________________________________________________________ _________________________________________________________________ _________________________________________________________________ _________________________________________________________________ TELP MID-TERM EVALUATION: REVISED FINAL REPORT 125 c) What factors (if any) slow the reimbursement process down? REASON YES NO Time-consuming process set down by funders Staff shortage at the institution to carry out the function Lack of clear guidelines to support the process Other factors: (please provide details) _________________________________________________________________ _________________________________________________________________ _________________________________________________________________ _________________________________________________________________ _________________________________________________________________ _________________________________________________________________ _________________________________________________________________ _________________________________________________________________ _________________________________________________________________ d) How might the flow of TELP funds to your institution be improved? _________________________________________________________________ _________________________________________________________________ _________________________________________________________________ _________________________________________________________________ _________________________________________________________________ _________________________________________________________________ _________________________________________________________________ _________________________________________________________________ _________________________________________________________________ TELP MID-TERM EVALUATION: REVISED FINAL REPORT 126 Question 7: Linkage Funds a) Has your institution experienced any difficulties accessing the linkage funds you need? YES NO b) If YES, please explain below: _________________________________________________________________ _________________________________________________________________ _________________________________________________________________ _________________________________________________________________ _________________________________________________________________ _________________________________________________________________ _________________________________________________________________ TELP MID-TERM EVALUATION: REVISED FINAL REPORT 127 Question 8: Travel Funds a) Who provides funding for TELP-related travel within South Africa? REASON YES NO USAID Contractor (UNCF) Your Institution b) What mechanism do you use for claiming reimbursement for travel funds? _________________________________________________________________ _________________________________________________________________ _________________________________________________________________ _________________________________________________________________ _________________________________________________________________ _________________________________________________________________ _________________________________________________________________ c) Have you encountered any problems with travel reimbursements? YES NO If YES, please explain below: _________________________________________________________________ _________________________________________________________________ _________________________________________________________________ _________________________________________________________________ _________________________________________________________________ _________________________________________________________________ _________________________________________________________________ TELP MID-TERM EVALUATION: REVISED FINAL REPORT 128 Question 9: Record keeping and Auditing a) Who is responsible for the overall coordination and management of TELP funding? b) Does your institution keep accounting records concerning TELP activities? YES NO If NO, please explain below and skip to Question 10. _________________________________________________________________ _________________________________________________________________ _________________________________________________________________ _________________________________________________________________ _________________________________________________________________ _________________________________________________________________ _________________________________________________________________ c) What accounting records concerning TELP activities are kept by your institution? NATURE OF RECORD PERSON RESPONSIBLE FOR RECORD KEEPING RECORDS NOT KEPT Detailed itemised records Project level records (i.e. per PIL) Aggregate/global expenditure records Other – specify _______________________________________ TELP MID-TERM EVALUATION: REVISED FINAL REPORT 129 d) Does your institution use a manual or computer-based accounting system for TELP expenditures? MANUAL COMPUTER If your institution uses a computer-based system, what software package do you employ for TELP purposes? _________________________________________________________________ _________________________________________________________________ e) Are the TELP accounting records audited? YES NO If so, who carries out the audit and what is the nature of the audit process? _________________________________________________________________ _________________________________________________________________ _________________________________________________________________ _________________________________________________________________ _________________________________________________________________ _________________________________________________________________ _________________________________________________________________ _________________________________________________________________ _________________________________________________________________ _________________________________________________________________ TELP MID-TERM EVALUATION: REVISED FINAL REPORT 130 Question 10: Further Funding a) Given the opportunity, would your institution apply for further funding from USAID for TELP-supported projects? YES NO b) What, if anything, would deter you from applying for further TELP institution￾specific funded projects (PILs)? _________________________________________________________________ _________________________________________________________________ _________________________________________________________________ _________________________________________________________________ _________________________________________________________________ _________________________________________________________________ _________________________________________________________________ _________________________________________________________________ _________________________________________________________________ _________________________________________________________________