EVALUATION OF THE DISTRICT DEVELOPMENT SUPPORT PROJECT USAID/SOUTH AFRICA (USAID Contract No. 674-0314-C-00-8009-00) Submitted to Sibusiso Sithole, Contract Technical Officer U.S. Agency for International Development Mission Pretoria, South Africa Prepared by Aguirre International Richard Dye, Team Leader Nancy Horn, Ph.D. Jordan Naidoo Everard Weber, Ed.D. Joy Wolf, Ph.D. April 2003 This evaluation was conducted under GEM Contract No. FAO-I-806-99-00010-00 Task Orders 806 and 809. The findings, conclusions, and recommendations are those of its authors and do not reflect the policies of USAID/South Africa. 1 ACKNOWLEDGMENTS The success of this review of the District Development Support Program in South Africa is due to the cooperation and collaboration of a great many dedicated people. · Mission Director, Dirk Dijkerman, and the Deputy Director, Eileen Oldwine, who gave generously of their time and provided excellent suggestions on the team’s draft conclusions and recommendations. · All the Mission staff that directed and supported the project; Ms. Sharon Harpring and Mr. Sibusiso Sithole of the mission’s Basic Education Team deserve special thanks for the guidance, insight, and information they so freely provided, as well as for their assistance with myriad practical arrangements. · National, provincial, and district education officials and staff who very graciously made themselves available for interviews and questions. · The Chief of Party for the Research Triangle Institute, Mr. Richard Cartier, and his excellent staff who cooperated fully with information, suggestions, and other assistance as needed. · The directors, managers, and staffs of LINK, MSTP, and READ who generously shared the perspectives of the grantee organizations. · The directors, managers, and staffs of the subcontractors: CEPD, JET, Khulisa Education Foundation, and HSRC, who similarly gave freely of time and effort. · The school principals, teachers, managers and staff, who are on the front lines of the project, were most helpful in sharing their experiences and their expertise. · Our thanks also to the members of the school governing boards, students, parents, and others in the various school communities who participated in this study. May their dreams of providing quality education to current and future generations of the community’s children become reality. Last, but not least, very grateful appreciation is given to all the people of South Africa who received us so graciously and whose dedication, hard work, and desire for educational improvement and reform inspires us all. 2 TABLE OF CONTENTS Page Acknowledgments ...…………………………………………………………………… ii. Glossary of Terms …………………………………………………………………….. iii. Executive Summary…………………………………………………………………….. v. I. Introduction………………………………………………………………………1 II. Background ……………………………………………………………………… 4 III. Findings ………………………………………………………………………….. 6 IV. Evaluation Questions/Conclusions …………..…………………………………26 V. Recommendations ……………………………………………………………….34 VI. Unresolved Issues ………………………………………………………………..38 VII. Lessons Learned/Best Practices ………………………………………………...39 Annexes Page A. List of Contacts ...………………………………………………………………... 2 B. DDSP Evaluation Team Work Plan …………………………….……………… 9 C. Methodology and Field Work Plan ……………………………………………. 12 D. School Sample.…………………………………………………………………... 25 E. Stakeholders’ Workshop ……………………………………………………….. 29 F. Education and HIV/AIDS....……………………………………………………. 34 3 GLOSSARY OF TERMS AMI Assessment Modeling Initiative ARB Assessment Resource Bank ARP Assessment Resource Person C2005 Curriculum 2005 CA Curriculum Advisor CEPD Centre for Education Policy Development, Evaluation, & Management CIE Catholic Institute of Education CM Circuit Manager COUNT Co-operative Organization for Numeracy Training DCP District Contact Person DDO District Development Officer DDOE District Department of Education DDSP District Development Support Program DFID Department for International Development DIP District Improvement Programme DM District Manager DOE Department of Education EC Eastern Cape Educator Teacher EMD Education Management Development EDO Education Development Officer EF Education Foundation EMIS Education Management Information System ESST Education Support Service Trust FDE Further Diploma in Education GSA Government of South Africa HOD Head of Department HR Human Resources HSRC Human Sciences Research Council IEB Independent Examination Board INSET In-Service Education and Training IQC Indefinite Quantity Contract JET JET Education Services KMS Khulisa Management Services KTP Kimberley Thusanang Project KZN KwaZulu-Natal Learner Pupil LINK Link Community Development MC Maths Centre (formerly MCPT) MIET Media in Education Trust MSTP Management of Schools Training Programme NC Northern Cape NDOE National Department of Education 4 NGO Non-Governmental Organization OBE Outcomes-Based Education PCV Peace Corps Volunteer PDOE Provincial Department of Education PPC Provincial Project Coordinator PPD Provincial Project Director PROTEC Programme for Technical Careers PST Project Steering Committee RAC Regional Advisory Committee READ READ Educational Trust RFA Request for Application RIEP/UOFS Research Institute of Education/University of Orange Free State RST Replication Support Team RTI Research Triangle Institute International SA Subject Adviser SABER South African Basic Education Reconstruction Program SASA South African Schools Act SCF School Change Facilitator SEM Superintendent of Education SGB School Governing Body SMATE Science, Mathematics, and Technical Education (University of Port Elizabeth) SMILE St. Mary’s Interactive Learning Experience SMT School Management Team SQAF School Quantity Assurance Framework TA Technical Assistant USAID United States Agency for International Development 5 EXECUTIVE SUMMARY Purposes of the Evaluation The United States Agency for International Development/South Africa (USAID/SA) and the South Africa National Department of Education (NDOE) commissioned Aguirre International to conduct this evaluation to: (1) Obtain reliable information about the performance of the District Development Support Program (DDSP) in achieving its goals and objectives; (2) Generate suggestions for utilizing the time remaining in the existing contract to reinforce sustainability of successful initiatives and interventions; and (3) Obtain comprehensive information for USAID to be used for planning purposes, including consideration of the nature and context of education sector activities during and beyond 2003. Methodology The in-country portion of the work was conducted from late July to early September 2000. A consultative and participatory approach was followed throughout. A representative sample was developed of 6 districts and 26 schools in the four provinces where DDSP is active, and site visits were made to all of them, as well as the four provincial education departments. Interview protocols were developed for key categories of interviewees, in order to assure consistency and comparability of data. Wherever possible, classroom observations were made. Extensive additional data collection was conducted in Gauteng, including interviews at USAID, the National Department of Education (NDOE), the contractor, RTI International (RTI), and the headquarters of the project grantees and subcontractors. The culminating event was the holding of a Stakeholders’ Workshop, at which the team’s key findings were presented and suggestions regarding future USAID work in basic education were obtained. All concerned cooperated fully with the evaluation. Key Findings The team’s principal findings concerning DDSP are that: · For a number of reasons, including problems with the contractor’s initial staffing and program redesign by USAID and NDOE, it took a year and a half for DDSP to get off the ground; therefore, at the time of the evaluation, it had been in operation for just 30 months (and for some components, even less). · While the results have varied by province, largely as a result of different levels of development of education in each, a remarkable amount of work has been accomplished during the two and a half years the project has been in operation. 6 · District offices have been strengthened, through training and technical assistance, in establishing more effective management and school support systems. · School management, teaching, and governance structures also have been strengthened, through training and school support, with the management area showing the greatest progress and governance the least. Progress in the curricular area is somewhere in between, but nevertheless significant. · A DDSP Education Management Information System (EMIS), linked to the national EMIS, has been created and installed; the major achievement is that a comprehensive record-keeping system has been developed for use at the school level, where none existed before. Short-term challenges remain, however, especially increasing the focus on analysis of the data produced by the system and its use for strengthening district and school performance, as well as completing installation of the electronic version of the system. There is also a need for the NDOE and the Provincial Departments of Education (PDOEs) to integrate and institutionalize the system nationwide. · An extensive performance monitoring system has been installed, and the data it has generated confirms that, on the whole, project goals, as measured by the indicators, are being met. Nevertheless, the team received complaints that the very large number of indicators (currently 45) on which data is being collected is unnecessarily burdensome. · The Grade 3 test is arguably the single most important output indicator identified by USAID. A number of concerns about it were expressed to the team, mostly by educators. · The team developed serious doubts about DDSP’s sustainability and replicability, at least as it is currently designed and operating. The main reason is the lack, so far, of adequate integration of the project with the education structure in all but one of the four provinces in which it is operating, and the limited involvement to date of the NDOE. · Not withstanding the fact that the RTI contract has been utilized to channel policy assistance to the NDOE in such non-DDSP areas as economics, finance, and national HIV/AIDS policies, DDSP has from the outset lacked a national policy component designed to help assure that the project, if successfully implemented, has long-term, national impact. · The argument made by RTI, among others, that DDSP is not really a pilot project in a true sense, but rather a model-developing project, strikes the team as having considerable merit. If correct, it lends importance to the need to begin work immediately on the next stage, i.e., the development and implementation of a sustainable model, so as not to lose the momentum created by DDSP. Implementation The implementation of the project, due to its NGO grantee-based design, has varied by province and grantee. In many ways, as is often the case with experimental project designs, the last two and a half years have been a learning experience for everyone: provincial and district official, grantees, RTI, and USAID. As a result, adaptations have emerged, based on that learning. Many of these adaptations are on their way to becoming “best practices,” especially in the areas of development of new models for school support, increased focus on developing key human resources, notably district managers and key teachers, and enabling districts to take training and other school support to the schools, utilizing a cluster concept. Other best practices to date include the Assessment Resource Banks and the provision of box 7 libraries. (The latter was supplementary and complementary to DDSP, though managed USAID.) In part because of its reliance on independent grantees (and their multiple partners) rather than contractor staff and subcontractors, DDSP, particularly at the start of the project, experienced occasional coordination problems with education officials. In addition, the project’s rapid pace of installment of multiple, complex systems and execution of a large training effort and other project components in a short period of time has sometimes been more than the participating districts and schools could absorb. As a result, considerable follow-up and reinforcement remains to be done, if the benefits achieved are to be consolidated. Main Conclusions The team’s three core conclusions are that: · Given the fact that for a variety of reasons it has only been in operation on the ground for two and a half years, DDSP has accomplished a great deal and laid the groundwork both for completion of the current scope of work and future activity. · DDSP’s sustainability and replicability objectives are threatened by its incomplete integration into the national education system and the lack of development, to date, of a fully sustainable model supported by RSA education authorities. · Working closely with the NDOE, PDOE’s and other relevant bodies, a new project should be designed, which, building on DDSP and other appropriate South African primary education development experience, will facilitate development of a model for replication on a national scale, as resources permit; care should be taken to prevent any gap in coverage between DDSP and the new activity. The new project, which would be developed in complete partnership with and fully integrated with the national education system at all levels, would be charged with: (1) Responsibility for consolidating the work begun under DDSP; (2) working with the national and provincial departments on extracting and utilizing the experience and best practices of DDSP (and other key primary education improvement projects) to develop a national primary school improvement model, which will be sustainable under South African conditions and progressively replicated, as national resources permit; and (3) Helping the national and provincial departments, as needed, to strengthen their program evaluation capacities. In order to assure full control over project resources and delivery mechanisms, the new project’s management structure should be contract-based, at all levels. 8 Key Recommendations The team recommends that the proposed follow-on project, to begin approximately October 1, 2003, focus on the following objectives. · Fully integrating the project within the South African education sector, at all levels, while maintaining its administrative flexibility. · Consolidating the work begun by DDSP, with the emphasis on: leadership development at provincial, district, and school levels; strengthening of district school support teams to assist schools in assuring that, in particular, curricular and classroom changes become self-sustaining; strengthening school governance; and developing and introducing appropriate HIV/AIDs components in the program. (An estimated 24 months would be required to complete the consolidation phase, following which the original districts and schools would “graduate.”) · Working closely with national education officials at all levels, developing and testing a pilot replication model, based on the concept of using the enhanced human and organizational capacities of the original pilot districts to help other districts and schools in each of the four provinces to apply the model. · Working with the national and provincial departments of education and other donors to develop a national, formative evaluation system to analyze the lessons and best practices derived from DDSP and other pilot primary education improvement projects, including activities conducted under the follow-on project, and strengthening the permanent evaluation capacities of the respective departments. · Finally, as noted earlier, care should be taken to ensure a smooth transition between DDSP and the new project; if necessary, the RTI contract and the four NGO grants should be further extended, to avoid a gap in coverage. As regards the existing project (DDSP), the team recommends that: · The lessons learned and best practices of the past two and half years be carefully studied and documented, in close cooperation with the national and provincial departments and using both external and internal analysts; · The increased focus on district support and development that was introduced in 2002 continue through the end of the current contract/grant periods, with priority given to achieving a higher degree of integration of the project with the districts, providing higher level training for district managers and key educators, and putting greater emphasis on building school support teams and systems; · School support teams go beyond “school visits” and assign increased priority to site￾based training, classroom demonstrations, and parent and School Governing Body (SGB) training, preferably at the cluster level; emphasis should be given to those schools found to be lagging behind on various indicators; · HIV/AIDS components of DDSP be evaluated with a view to providing an input into the development of enhanced efforts in this area, as part of the follow-on project, in this effort, the focus should be on seeking ways to (1) help the districts and schools 9 mitigate and cope with the impacts of HIV/AIDS on the education system and (2) utilize the schools as agents for assisting local governments and communities to help prevent HIV transmission and provide social support for those affected by the disease, either directly or indirectly; and · The cumulative experience with the Grade 3 test be evaluated, employing internal and external experts as needed, to serve as an input into the planning and development of the assessment components of the follow-on project. 10 CHAPTER I. INTRODUCTION A. Background and Purposes of the Evaluation In 1995, the newly elected democratic Government of South Africa (GSA) and the United States Agency for International Development (USAID) signed a Primary Education Bilateral Agreement. This agreement enabled the two governments to work together to improve the quality of education for disadvantaged South Africans and formalized a partnership for the transformation and development of the education system, based on the goals of equity of access (redress), improvement of quality, and democratic participation. Four of the poorest provinces were identified to receive assistance: Northern Province (now Limpopo); KwaZulu-Natal; the Northern Cape; and the Eastern Cape. As a result of this agreement, the South African Basic Education Reconstruction Program (SABER) was launched, with an authorization of $55.3 million over ten years. Under this agreement, a series of activities focusing on basic education (pre-primary, primary, secondary, and adult education) were carried out in pursuit of the goals of the agreement. The District Development Support Program (DDSP), initially known as Saber Phase III, is a continuation of these efforts and, thus, is a joint education initiative of the GSA and USAID/South Africa. In January 1998, USAID awarded a contract to manage a portion of the activities under the DDSP to the Research Triangle Institute (RTI) International and a number of partners. Among the tasks to be implemented by the contractor were assisting the GSA in (1) formulating, disseminating, and enacting policies for transforming the education sector; (2) creating systems by which the transformation could be implemented; and (3) developing the human and organizational capacity to undertake transformation. The contract amount was $24.27 million, later reduced to $20.43 million. GSA counterpart funding was set at one-third the amount of the U.S. contribution. As part of its scope of work, RTI was to provide long- and short-term technical assistance at the national, provincial, and NGO (Non-Governmental Organization) levels and grants management services for a period of five years (since extended to June 30, 2003). A design team commissioned by USAID/SA and the National Department of Education (NDOE) further refined the scope of work, recommending a focus on education districts within the four provinces and motivating the change in name from SABER to DDSP. The goal of DDSP became “improved quality of educational delivery for Grades 1-9 in the DDSP target areas.” For a number of reasons, one of them problems associated with initial RTI staffing, implementation of the project was delayed for approximately 18 months, including a startup period which followed the arrival of a new RTI Chief of Party (COP). Aguirre International was asked to conduct an evaluation of DDSP, which would provide reliable information about the performance of DDSP in achieving its goals and objectives and make recommendations and suggestions to assist RTI and its partners make maximum use of the time remaining under the extended contract. In addition, the evaluators were asked to provide information and make suggestions to assist USAID and its counterparts in planning future 11 education activities, in support of the continued transformation of the education system, through the period 2003-2010. B. Evaluation Questions Seven evaluation questions were posed by USAID/SA to guide the evaluators in their work. 1. To what extent were planned objectives of DDSP met? What contributed to objectives being met, and what, if anything, hindered progress? Reference each province. 2. What was the role and relative importance of implementers, e.g., the lead contractor, subcontractors, grantees, district offices, etc? Consider elements such as integration, coordination, cost-effectiveness, and sustainability. 3. What evidence is there to indicate that integration of key components of the provincial programs is taking place at the school and district levels, i.e., curriculum, management, governance, district support? Is a holistic picture emerging of district offices and schools? 4. How satisfied are beneficiaries (sample all levels) of the program? Consider the services provided to national, provincial, district, and school level beneficiaries. To the extent they were not satisfied, what is the exact nature of their dissatisfaction? 5. What modifications would be feasible to recommend increasing the success and impact of the program during its remaining life and/or extension period? Present supporting evidence to substantiate any recommendation. 6. What are the pros and cons of replication of all or some of the DDSP beyond the project’s current time frame and scope including the extension period? Please consult with key stakeholders. 7. What are the pros and cons of extending all or some of the DDSP project activities beyond the extension period? Please consult with key stakeholders. These questions are addressed, together with the team’s other conclusions, in Section IV. C. Evaluation Team Aguirre International’s efforts on this evaluation were managed by Roger Rasnake, Director of the firm’s Washington office. Team members included: Richard Dye, Education Administration Specialist/Team Leader Joyce Wolf, Evaluation Specialist Everard Weber, Educational Planning/Policy Analysis Specialist Nancy Horn, International Development Specialist Jordan Naidoo, Instructional Systems/Education Specialist D. Methodology 12 The in-country portion of the evaluation was carried out over a six-week period from late July to early September, 2002. The means used by the Aguirre team for gathering information and insight into the components of DDSP included review of documents and reports, interviews and focus group meetings with program partners, and stakeholders at all levels of the program, and two intensive weeks of field work in the four target provinces. During the field phase, provincial and district officials, RTI field staff, project grantees and their partners, school management teams, educators, parents, and school governing boards in six project districts and 26 project schools were consulted. Wherever possible, classroom observations were conducted. In all, some 300 individuals were contacted. (See Annex A, List of Contacts; Annex C, Methodology; and Annex D, School Sample.) E. Other Donors A number of other bilateral donors besides USAID are involved in basic education development in South Africa. Notable among them is the British Department for International Development (DFID), which is assisting two large projects in Limpopo and the Eastern Cape. Others include Belgium, Finland, Ireland, Japan, the Netherlands, Norway, and Switzerland. In most cases, the target provinces are Limpopo and the Eastern Cape. Japan’s current assistance is focused largely on infrastructure, but reportedly it is considering expanding the size and scope of its education assistance. Coordination among donors and NDOE is facilitated by informal, but structured (agendas, minutes, etc.), quarterly meetings, at which issues of interest to both the Department and the donors are discussed. Follow-up action is taken as determined at the meetings, but they are used primarily for information and exchange of views. 13 CHAPTER II. BACKGROUND A. Context The period after the democratic elections of 1994 in South Africa was characterized by the rapid formulation of policies in all areas of government, not least in the education sphere, which was faced with the task of providing education with a focus on redress, equity, quality, and democratic participation. Major policy reforms affecting the structure and processes of education and training were introduced. Legislation including the White Paper on Education and Training (DOE, 1995), the South African Qualifications Act (DOE, 1995) and the National Education Policy Act (DOE, 1996), gave legal effect to the new policies, especially the integration of education and training. In relation to governance and finance, the central legislation to direct the implementation of educational reform, the South African Schools Act, SASA (DOE, 1996a), was promulgated on the heels of the second White Paper on Education (DOE, 1996b). Subsequently, the National Norms and Standards for School and Funding (DOE, 1998), which spelled out in detail new norms and standards for school funding, was released, partly in response to the survey Register of School Needs (Education Foundation et al., 1996c), which highlighted the glaring inequities in educational opportunities. In addition, a new outcomes-based curriculum was launched with the release of the Curriculum 2005 Framework (DOE, 1997). This curriculum views the learning process as being as important as the learning content, spelling out the outcomes to be achieved at the end of the teaching and learning process. These outcomes include: identifying and solving problems; working effectively with others; and collecting, analyzing and critically evaluating information. In terms of the new curriculum, assessment should be continuous and designed to allow students to demonstrate the achievement of learning outcomes. Initially, attempts to address the crises in education were through educational policy reform and investment in centralized (at national or provincial level) programs to support these reforms. Around 1996, the focus of the new bureaucracy shifted to implementation and delivery of educational services at provincial and sub-regional levels. There was also an ongoing debate about the actual impact of these education reforms on schools and classrooms, and their value in terms of transformation of society more broadly. There was a growing consensus that there needed to be much more work focused on the massive challenge of implementing policy goals in the context of the economic and psychological footprints of apartheid, and sustained economic deprivation and rural under-development. A large number of public schools especially those serving low income communities continued to be characterized by low morale, poor attendance, a poor learning environment, low levels of parental involvement, and poor learning outcomes. B. District Development Support Project 14 V. The DDSP, the origins of which are described in Section ?, was aligned to the national Department of Education's District Improvement Programme (DIP). DIP was intended to operationalize the policy frameworks around governance and curriculum, by defining the roles and strengthening the capacity of district education offices to provide effective support to schools, while simultaneously mobilizing schools to engage in self-improvement processes. Within this context, DDSP's resources were to be directed at supporting education transformation at the level of district, circuit and school. At a broader system level, activities included those that could impact directly on district capacity and dissemination of lessons learned system-wide. VI. The goal of the DDSP is "improved quality of educational delivery for Grades 1-9 (basic education)" 1 in the DDSP target areas. A longer-term goal is to create an environment that pulls into the system good educational and organizational practices of all kinds: research-proven methodologies and models, reflection, innovation, sharing and communication, team building, etc. Four sub-goals were agreed upon. Sub-Goal 1: Improved quality of Curriculum Practices Sub-Goal 2: Improved quality of District and School Management Sub-Goal 3: Enhanced School Governance Sub-Goal 4: Developed theory and best practice for whole school and whole district development. In accordance with the Whole District focus, DDSP sought to facilitate the development of approaches, practices and structures that modeled good district organization and practices for effective school support. In line with the Whole School focus it tried to promote effective in-school development programs and practices for improved teaching and learning, and school organization, governance and management. DDSP was implemented through the awarding of grants to NGOs and their local partners in the four provinces to design and implement activities that would achieve the program’s goals. Four projects were implemented by three grantees and their respective partners: · READ (Isithole Project, Eastern Cape) · LINK (Kimberley Thusanang Project, Northern Cape) · MSTP (the Mthonjeni Project, KwaZulu Natal) · MSTP (Fanang Diatla Project, Limpopo Province) 1 The Basic Education band is divided into 3 phases: Foundation Phase (Grade R-3); Intermediate Phase (Grade 4 - 6); Senior Phase (Grade 7-9). 15 CHAPTER III. FINDINGS A. Introduction In this section, the findings derived from the team’s fieldwork in the four project provinces, as well as in Gauteng, are presented, in two groupings. Part B contains findings specifically related to the four provinces. Findings on eight cross-cutting issues are presented in Part C. B. Findings by Province Given the many differences among the four provinces and also in the DDSP activities carried out in them, the provincial level data collected by the team are critical to understanding the current status of DDSP. 1. Eastern Cape Eastern Cape is one of the poorest provinces in South Africa. The population in 1999 was approximately 6.7 million, with 70 percent living in the former Xhosa homelands of Ciskei and Transkei. In 1997, school enrollment was estimated at 2.3 million, the second largest in the country, with 1.7 million primary and over 600,000 secondary learners. These learners were enrolled in 6,126 schools: 2,927 primary, 760 secondary and 2,439 combined. A total of 61,764 educators (30% males and 70% females) were employed. Approximately 42 percent of educators are under qualified. Sixty percent of the primary learners and 66 percent of the secondary learners were accommodated in the available classrooms with a backlog of 40 percent at the primary level and 34 percent at the secondary level. There are limited teaching and learning materials, and a need to provide running water to 34 percent, electricity to 77 percent, and telecommunication services to 81 percent of the schools.2 In 2000, just as DDSP was getting underway, the PDOE eliminated the regional level of administration, and consolidated 41 districts into 24. In October 2001, 24 District Managers (DMs) were appointed, all of whom report to the Acting Chief Director, Districts, at the Provincial level. Services to the schools were to be provided by Educational Development Officers (EDOs) and Subject Advisors (SAs). DDSP was implemented in EC by READ Educational Trust and its partners: The Maths Centre (MC, formerly MCPT), and the University of Port Elizabeth’s Department of Science, Mathematics, and Technical Education (SMATE). In 2000, district level training was provided to a relative handful of officials, as the EDOs and SAs were not yet in place. It was only in November 2001 that the new districts were finalized, and not until April 2002 that staff were appointed, and that DDSP training at the district level began in earnest. 2 Paul Musker, District Development Support Programme (DDSP) Situational Analyses, Paul Musker and Associates for the Research Triangle Institute, August 1999, pp. 42-43. 16 Coordinating mechanisms were developed to facilitate the implementation of DDSP, e.g., the establishment of a provincial coordinating forum and the appointment of an RTI Provincial Project Director (PPD). Later, the PPD left, and a District Contact Person (DCP) was named by the project to assume similar functions. But, as the districts themselves were undergoing reorganization, there were significant implementation problems. For example, at the outset of the project, at the suggestion of the Regional Advisory Committee (RAC), READ and its partners, in addition to training those district officials, who were already in place, began to work directly with the schools. While it was not possible to undertake comprehensive training of large numbers of district officials because of the on-going reorganization, READ continued with the training of District Managers (DMs) throughout the life of the project. At the same time, training was provided directly to School Management Teams (SMTs), School Governing Bodies (SGBs), and educators in English, Math, and Science. The schools welcomed the help of DDSP. Confused about how to translate district-provided Outcomes Based Education (OBE) training into classroom practice, educators were relieved that the training they were being provided by READ and its partners was practical, clear, and helpful. School managers also welcomed the DDSP training, as district reorganization and staffing shortfalls made it difficult for them to provide much assistance. School level training was initially delivered in central locations to very large groups of individuals, with follow-up support provided through a system of on-site visits. But, as this approach proved not to be very effective, it was later changed to one of training at the “cluster” level. Clusters included approximately 10 schools in more or less close proximity to each other, some of them DDSP and some not. The DDSP identified and trained “key” educators for each phase at each DDSP school, so that they could cascade the training at their schools, and also to offer support to other non-DDSP schools in the cluster. Cascading, however, also turned out to be less effective than anticipated, due largely to a lack of time to practice what educators had learned before being required to cascade and a lack of materials to pass on to non-DDSP educators. As a result, an enhanced program of on-site follow-up support to trainees was developed and implemented, which proved to be successful. Management training for SMTs and SGBs included: 1) Strategic Planning for Whole School Development – resulting in school policies and a school development plan; 2) Communication and Conflict Management – resulting in skills to resolve interpersonal issues; 3) Human Resource Management – resulting in skills to hire additional staff, in mentoring and coaching, and to assess the performance of educators; and 4) Financial Management – resulting in SMTs and SGBs ability to budget and determine new school fee levels. As a result, some schools were able to apply for and be awarded Section 21 status.3 3 Section 21 of the South African Schools Act provides for the allocation of additional functions for SGBs, over and above the minimum responsibilities listed under Section 20 of the law. The Section 21 functions include the maintenance and improvement of the school’s property; determining the extra-mural curriculum and choice of subjects; purchasing textbooks, materials and equipment; and the payment for services to the school. The Provincial HOD has the responsibility of determining the school’s capacity to perform the Section 21 functions. 17 The results of the curriculum training were more visible in schools where educators consistently attended training and where there was follow-up support for what they had learned. Impact was also found in the increased enthusiasm of learners. In most schools where OBE was being implemented, children enjoyed participating in class activities, interacting in their small groups, and undertaking projects. Educators commented that learners’ ability to communicate in different languages had increased, as had their willingness to ask questions and think critically. The uniform message received by the evaluators in the Eastern Cape is that if DDSP as such is to end and be replaced by a follow-on project, great care should be taken to avoid a gap between the two projects, so that critical momentum is not lost. 2. Northern Cape The lowest population density in South Africa is found in the NC, where only six persons per square kilometer are spread over the largest province in the country.4 While the province has the third highest per capita income in the country, it is also ranked as having the lowest economic growth.5 The high disparities in wealth, with more than 70 percent of earnings and disposable income of the NC in the hands of only 15 percent of the population,6 and the uneven population density have led to wide variation in the education offered to learners. The DDSP in the NC has benefited from a number of factors. All of the DDSP schools were located in a single district, which allowed the project to focus on a single bureaucratic and geographic location. Working with 62 schools and focusing only on the Foundation Phase (Grades 1-3) facilitated successful implementation. In addition, the district offices are located in the same town as the provincial office, aiding communication and cooperation. The DDSP also benefited from strong provincial support and a relatively stable and well-developed district office. The PPD in the NC was the joint choice of the department and contractor in 1998 and has remained in that position during the life of the project. In sum, the DDSP has been seen as a very important part of department activities, and district officials are anxious to replicate the experience in additional schools. The DDSP in NC also faced a few challenges that were relatively unique. Elements of the three school administrative systems that had been combined to form the current provincial government included both extremely small, disadvantaged farm schools and well￾resourced, high performing ex-Model C schools. Schools of all types were included in DDSP. The presence of the well-off schools inevitably led to distortion in the performance data. Because of their greater resources, however, they have not been large consumers of DDSP services. The large number of exceptionally small farm schools, on the other hand, has presented a special set of problems to the project, because of their inadequate teaching practices and materials for multi-grade classrooms, management problems associated with efforts to amalgamate some of the smaller farm schools into larger ones, and school ownership issues (farm schools traditionally were owned and operated by farm owners). 4 Musker 1999:44 5 ibid. 6 ibid. 18 In NC, DDSP was implemented by LINK, aided by five partners. The training in three curriculum areas and management was carried out by members of the LINK staff, plus personnel from national training NGOs among the partners: COUNT (Cooperative Organization for Numeracy Training); ESST (Education Support Service Trust); Molteno; RIEP/UOFS (The Research Institute of Education, University of the Orange Free State); and CIE (Catholic Institute of Education). The governance training was sub-contracted to a small local NGO, which conducted training until last spring. At that point a LINK staff member took over the governance training, adding a school level element to the approach. The two Provincial Project Coordinators (PPCs) who worked with the DDSP facilitated its integration into department activities. The primary PPC role has been to monitor school performance on project indicators and communicate this information to district officials and project staff. DDSP adopted an integrated planning process to assist district development. In this approach, schools were supported in the completion of school development plans, which were integrated into a district plan; the proposed roles of the district office were, therefore, defined and shaped by the needs of the schools. One of the major challenges in following this approach lies in the wide range of schools (described above) that are found in both the DDSP sample of schools and the total schools in the district, differences which require a range of strategies. While a strategy for assisting very small farm schools has been developed, models for the larger farm schools facing integration and amalgamation problems, and where the majority of the learners in the district are found, have not. DDSP in the NC displays two other major characteristics: production of massive amounts of project documentation and extensive use of the DDSP database monitoring system as a management tool. The grantee has produced an extensive library of manuals, papers, forms, and checklists that will remain beyond the life of the project. To some of the participants, this large amount of documentation has been seen as overwhelming, but the dissemination of these products could assist in the expansion of DDSP practices into other districts. The project has attempted to use the DDSP database, created with the assistance of Khulisa Management Services, for both project management and as a development tool. The tool has been used very successfully in this province to monitor the USAID indicators on a school level. On the other hand, attempts to use this database as a development tool to guide school support and/or district approaches to schools have been limited. Educators have been trained in numeracy, literacy and life skills. The service providers say more work is required in school development, educator appraisal and further consolidation is necessary in educator subject knowledge, especially numeracy. Several interviewees indicated that educator morale has improved and that they appear more optimistic about their work now than a few years ago. The team’s observations suggest that inside the classrooms, change has occurred but is in many cases still not well￾established. Most educators perform competently enough, but there is evidence of traditional teacher-centered methods co-existing alongside the new student-centered approaches mandated by OBE. 19 3. Limpopo Limpopo comprises about one tenth of South Africa’s land area. Ninety-seven percent of the roughly 5 million people are African. The school system must cater to almost half of the population, which is under the age of 15 years and is growing fast. Drop-out rates are especially significant between the primary and secondary phases.7 Far more people live in rural areas in Limpopo than in most of the other provinces. Because of the low rate of urbanization, farm schools are especially important. Among the special problems of these schools are the great distances learners have to travel, unqualified educators, multi-grade classes, the absence of teaching materials, and lack of basic facilities such as running water, telephone lines, and electricity. Most of the economically active young adults work in the richer provinces. Unemployment is high. In 1995 Limpopo contributed only 3.7 percent to the national GDP, making it one of the country’s poorest provinces. Limpopo has gone through a long and difficult process of educational restructuring, since 1994. It inherited five Education Departments, including three former Homelands, which within a short space of time had to be amalgamated. The resulting reorganization is not complete, in part because it still has to be brought in line with new local government demarcations, which are still unsettled. In addition, uncertainties regarding the roles of different education units remain. In Limpopo, for example, the province is not directly involved with the districts; which, in fact, have no legal status. Instead, the province and the districts communicate with each other through mediating, regional structures, which control the funds. While it is still far from clear what will happen, current indications are that the regions, not the districts, will in the future become the key implementation bodies. Complicating the picture is the possibility that in the process, their names may be changed to “districts,” with the current districts being incorporated into the new districts, to provide outreach in their respective areas. The appointment of two DDSP personnel as coordinators has improved relationships. Despite initial problems regarding coordination, the province and the districts are now said to be cooperating much better. Support at the higher tiers of government, however, has often depended upon the individual the grantee deals with and has not always been forthcoming. While NGOs may do the same work as the DOEs and collaborate with them, they are not embedded within this structure, nor do they share a common way of operating. The Limpopo Province Education Support Project (Fanang Diatla Project) seeks to link curriculum implementation, the development of teaching and learning materials, and education management development through a Whole School approach in a vast area comprising 255 schools in six districts (one in each region). The project, which is being implemented by a consortium headed up by the Management of Schools Training Programme (MSTP), has concentrated on education managers at schools and in the DDOE, school governing bodies, and teaching and learning in Grades 1-9 within the Curriculum 7 Ibid. pp 43-8. 20 2005 (C2005) framework. Counting MSTP, the consortium originally included 14 partners, six of which subsequently were dropped for poor performance or insolvency. MSTP’s remaining partners included: The Siayakhula Trust; PROTEC; PROMAT; COUNT; MIET; Molteno Trust, and IEB. The consortium got off to a difficult start, due primarily to a number of staff changes in the early years. In addition to the service providers, a number of Peace Corps Volunteers (PCVs) were assigned by the PDOE to assist in DDSP schools. The PCVs are educational professionals and live in the local communities. An important aspect of DDSP in Limpopo was the employment and training of District Development Officers (DDOs). The appointment of DDOs resulted from the need to have a mechanism in place in each of the participating districts to continue the DDSP in the non￾DDSP schools when the project ended. DDOs were chosen from internal DDOE professionals via a stringent selection process. The DDOs have worked with circuit managers (CMs) and have co-facilitated training workshops, supported educators in classrooms, and helped SMTs to implement what the project has taught them. The 11 DDOs appointed in Limpopo will be accredited as School Change Facilitators (SCFs) through the School Change and Facilitation Program run by DDSP. The DDOE is also presently looking at ways of best utilizing DDO skills and services in leveraging DDSP best practices. Overall, the appointment and use of DDOs in Limpopo bears testimony to the involvement of the PDOE in the project, the attempts at sustainability of the project, the ownership of the project by the DOEs and the level of coordination that is presently enjoyed between the DOEs and the project. Their employment also enabled the project to deal with the large number of schools and the large territory to be covered. Another strategy employed to help deal with the large number of schools in Limpopo was clustering. MSTP clustered groups of 10 schools that could share resources and attend the same meetings and workshops. Facilitators typically spent two weeks in one district and then moved on to the next. The Grade 3 assessment test helped prioritize schools and target areas that needed special attention. There were not enough district officers to accompany facilitators. On average, only two visits per year per school were possible. Staff shortages constrained the work the NGOs were able to complete. RTI’s role in the field, through the PPD, was mainly to monitor the existence and quality of policy documents, as required by the Schools Act. The size of the MSTP consortium and difficulties in coordinating the work with the DOE meant that, in practice, schools and educators were often confused by the mixed messages they received. Among DDSP’s greatest achievements in the province was the management training it provided DDOE officials and SMTs at schools. Generally these officials praise the project. They typically refer to changed perceptions about the nature of their jobs, teamwork, and more participatory modes of management. Conversations with educators show that more democratic SMT practices were evident at some, but not all schools. There has also been a shift at circuit and district levels from only performing administrative functions to supporting schools. The PDOE stresses its meager resources before DDSP started and the degree to which the project helped build its capacity and human resources. 21 SGB training did not receive the attention or the funding of the other categories. MSTP and its partners acknowledge this. Training modules were in English, instead of the mother tongue, so simultaneous translations were needed. Parent illiteracy is common. There have been educators who attended training sessions, those who attended in part, and those who did not attend at all. This, together with the extent of SMT support, appears to determine the depth and degree to which change in teaching practices has taken root. There is evidence from the classroom observations of the use of more progressive pedagogies, as well as the persistence of educators encouraging rote learning and memorization. Educators generally criticize the cascade method and say it does not work. There are further changes to OBE in the pipeline at the national level, and this may be problematic. There were three years of OBE training by the DDOE and other NGOs before DDSP began. There are better academic results in the Foundation Phase, most likely because they had more training than the other phases. As noted above, there have been problems with the MSTP consortium. An underlying reason for this is the fact that there are few NGOs in Limpopo, and the ones that exist are inexperienced. Another is that, in retrospect, the consortium may have been too large to operate with maximum efficiency. Among the problems experienced have been difficulty in building trust among the different partners, dissatisfaction with the allocation of resources within the consortium, and arguments over roles and division of labor. Some of the partners did not perform as expected or became insolvent and withdrew from the project. 22 4. KwaZulu-Natal KZN, the country's third smallest province, has the largest population of approximately 9.3 million. Though not the poorest province, it is relatively poor, despite being relatively urban (42%) compared to provinces such as the EC (37%) and Northern Province (11%). It is ethnically diverse: 76 percent of the population is African (largely Zulu-speaking), 14 percent Indian, 7 percent white and 3 percent colored (Statistics SA). There are 5,734 schools in the province, with a learner enrollment of 5,174, 000 (DOE, 2002). The KZN DOE determined that the DDSP Mthonjeni Project should focus its activities in the Ulundi Region, with a concentration in the Nkandla district, one of the most impoverished areas in South Africa. It is also one of the most disadvantaged school districts in the province, with an alarmingly high learner: classroom ratio of more than 50:1 (HSRC). It contains 180 schools, of which 130 are Primary and Intermediate Schools (Grade R – 9), 48 are high schools, and 2 are schools for learners with special needs. The aim of the project is quality education delivery in all schools in the Nkandla District as a sustainable model for development in KZN. A different MSTP Consortium is implementing the project in this province. The members of the consortium are the Human Sciences Research Council (HSRC), the Independent Examinations Board (IEB), the Programme for Technical Careers (PROTEC), St. Mary’s Interactive Learning Experience (SMILE), Cooperative Organisation for the Upgrading of Numeracy Training (COUNT), and Media in Education Trust (MIET). The MSTP consortium model in KZN presents both strengths and challenges. It has encouraged a number of NGOs in the area to work together in support of district development. It has also presented a substantial challenge with regard to coordination and a consistent message in working with individual educators and schools. The project leadership structure in KZN is somewhat different from the other provinces. The RTI PPD is also the Provincial Director for Educator Development in the province and is based in Durban and not at the Project Office in Nkandla. The PPC is responsible for the day-to-day project management duties. The PPD provides conceptual direction while the PPC is mostly involved in monitoring at school and district level, reporting to the DM and RTI, keeping the PPD informed, and acting as a liaison between the grantee and the district office. The regional structure, with functions such as Curriculum Advisory Services located in Ulundi, has affected the implementation of DDSP in terms of project/district relations, and resource allocation to the Nkandla District. For example, the project assisted the Nkandla District with developing its integrated plan. However, despite the planning processes introduced by the project, the Ulundi Region did not accept the integrated plan concept, and, as a consequence, not much credence has been given to the district’s plan. As a consequence, regional directives have taken precedence over the district’s plan. This lends importance to current efforts in the province to resolve and rationalize the respective roles of the regions and the districts. 23 District support of DDSP was limited initially, contributing to tensions among district officials and project staff over resource use, reporting, and the design of training and school support. Subsequently, relations improved. Recently, the project has begun to support the capacity building of DMs, and Superintendents of Education (SEMs) through the School Change Facilitator course, which is accredited through the University of Witwatersrand and aimed at officials (24) across the Ulundi Region. Officials praise the course for skill development in managing change, organizational culture, collecting and analyzing data, problem solving, and collaborative planning. The Education Management Development (EMD) component of the programme is the direct responsibility of MSTP. It was expected that the principal and another member of staff from each of the 180 project schools would register for the Further Diploma in Education (FDE). The FDE is offered over two years through a mixed-mode model of delivery, i.e., materials-based with eight five-day teaching sessions. Principals and educators who hold a management portfolio and have a minimum M+3 qualification may enroll for the course, run by MSTP and accredited by the School of Education, Training and Development, University of Natal, Pietermaritzburg. This component appears to be effective in developing theoretical understanding and practical skills. There is much evidence of SMTs being organized, and undertaking structured planning, engaging their staff and employing more consultative management practices. The school management and curriculum development training model in KZN has been adapted over time. Initially, most educators, principals and other school management staff were trained in large groups from across a circuit. There was a reliance on some form of follow-on cascade training by these participants in their individual schools. Recently training has been done at cluster level, with participants from no more than 11 schools. Follow-on by consortium members is now more school-based and includes working with individuals and groups of educators on planning, and demonstration lessons. Most educators value the follow-on sessions greatly. Curriculum-related training by the various partners appears to have had variable impact. Most educators praised the additional skills they have been able to develop, especially in implementing OBE practices, group work, and learner centered teaching strategies. Observation of classes indicates that actual practices vary. While there is positive change in educator practices, including more structured and collaborative lesson planning, grouping of learners, and greater learner involvement, educator-centered instruction remains quite prevalent. A common feature is the minimal use of learner materials, and continued use of Zulu as the medium of instruction beyond the Foundation Phase, even in cases where the SGB had decided that the medium of instruction should be English.8 In many cases, learners are given little practice in English except to repeat in rote form instructions from the educator. 8 Schools surveyed indicated that they had exercised their authority to set the medium of instruction per the powers granted to the SGB in Section 6.2 of the SASA, which states that: The governing body of a public school may determine the language policy of the school subject to the Constitution, this Act, and any applicable provincial law. 24 Assessment related training and practices appear to be weak. The IEB provided specific assessment training during 2000 and early 2001, but discontinued it when assessment training was incorporated into curriculum training provided by other providers. Subsequently, the HSRC was the sole provider of dedicated assessment training. While mixed messages appear to be frequent, and educators report they are often confused about assessment issues, closer monitoring and better coordination by MSTP has improved the situation. Curriculum related training by the various consortium members appears to have had variable impact. Most educators praised the additional skills they have been able to develop, especially in implementing OBE practices, group work, and learner centered teaching strategies. Observation of classes indicates that actual practices vary. While there is positive change in educator practices, including more structured and collaborative lesson planning, grouping of learners, and greater learner involvement, educator-centered instruction remains quite prevalent. A common feature is the minimal use of learner materials, and continued use of Zulu as the medium of instruction beyond the Foundation Phase. In many cases, learners are given little practice in English except to repeat in rote form instructions from the educator. Assessment related training and practices appear to be weak. The IEB does assessment training as its core function, and at the same time the HSRC as well as the other providers involved in specific learning area or phase also provide assessment support. Mixed messages appear to be frequent, and educators report they are often confused. Closer monitoring and improved coordination by MSTP has improved the situation. Governance training has focused on training principals and the chairperson of the SGB. This was done initially in large training sessions, focusing on basic planning, resource management, and budgeting skills. The impact of the governance training appears to be minimal in empowering the SGB, and particularly parent members, to play the roles envisioned in the SASA. The indicator framework is both valued and derided as a major constraint. District staff has been able to develop a School Quality Assurance Framework (SQAF) for the Ulundi Region, which serves as a powerful tool to benchmark key indicators against which the performance of districts and schools can be gauged. However, at school level it is seen as responding to a checklist of external requirements with little time to reflect on what is actually being done. The establishment of Resource Centers at 12 KZN nodal schools has affected the project in a number of ways. The PPC is responsible for overseeing this process with the result that the Resource Centers are closely identified with DDSP, even though they are outside the ambit of the project. This has put additional strain on district office/project relations. Despite some problems with the administration of the Resource Centers, the concept of support to educators through the development of support teams at nodal schools offers promise for strengthening program delivery and sustainability of DDSP work in the province. 25 C. Cross-Cutting Issues Eight cross-cutting issues are also important for understanding DDSP. They are: District Management and Support; School Management; Curriculum; School Governance; Monitoring and Evaluation; Assessment; Education Management Information; and Program Implementation. Summaries of the team’s findings on each follow. 1. District Management Development and Support Improved district management has been one of DDSP’s successes. This is particularly noteworthy, when one considers that during the time that the project has been active, three of the four provinces’ education departments (EC, LP, and KZN) have been involved in on-going reorganization. In general, the DDSP and the various forms of support that have been provided through the work of the service providers, RTI, and USAID have been welcomed and appreciated by district, regional and, provincial education officers. However, there have been challenges, as well as successes. For example, there has been some confusion and debate arising from the lack of clear definition of roles and relations of the various DDSP participants vis-à-vis the district office and from a perceived overload of new programs on the part of some districts. Also, because of the provincial reorganizations, it was difficult for district capacity-building to receive the same attention as capacity-building in the schools. Despite this situation, the achievements of the DDSP with regards to district level capacity building have been substantial and should have continued long term benefits for the district administrations themselves, as well as for the schools they serve. Perhaps the greatest achievement of DDSP’s district support initiatives, present to some extent in all four provinces, is that district officials have begun to develop a range of new skills - or strengthen existing skills - to improve their interaction with educators and school managers. Officials have been able to change attitudes and adopt a more collaborative and integrated approach to school development and improving instruction and learning in their districts. The success of DDSP at this level, however, varies significantly across the four provinces. In Limpopo, for example, the vastness of the province, the large number of schools, cultural differences (language and ethnicity), and district relationships to their respective regions made it difficult for DDSP to focus to the planned degree on the district offices. While capacity building and support for schools proceeded as anticipated, district development often lagged behind. (This situation should improve with the additional emphasis on the districts in the current RTI work-plan.) In the EC the project has started facilitating the process of strategic planning for the districts, with two sessions completed for the Queenstown and Cofimvaba Districts. These sessions have helped refine the job functions of the EDOs and SAs, assisted these districts develop their mission statements, and encouraged them to look critically at their strengths and weaknesses. 26 In KZN, while the project assisted the Nkandla District with developing its integrated plan, it has not gone far, because the Ulundi Region did not accept the concept of the integrated plan. In the NC, the project has helped to consolidate the district office and its agency role in school development. DDSP has worked with the Kimberley District Office to develop a District Integrated Plan that ensures that CMs and CAs focus their efforts on school development in a holistic way. Some district level training has focused on content and not on training and mentoring skills that district officials need to become trainers with the capacity to develop their own school support interventions. In EC, however, the opposite approach has been taken, with positive results. There have been mixed results, as well, in development of district-level curriculum management skills. In part, this is due to the fact that in some provinces curriculum advisory services are provided by the regions rather than the districts, but district officials involved in providing curriculum support management (e.g., CMs and SEMs) clearly would benefit from having greater curriculum management skills. In some cases, tensions between district officials and project staff around issues of resource use, reporting, and the design and scheduling of school support training reportedly existed. Where this happened, it is said to have led to reduced engagement between project and district staff. In retrospect, on occasion, perhaps not enough attention was paid to the bureaucratic practices and internal dynamics of a given district. In some cases, engagement was limited to written reports and standard reporting at weekly or other scheduled meeting, which some of the district officials involved did not feel constituted meaningful involvement in program activities or in decision￾making. Most of these differences were eventually ironed out, however, and the majority of district officials see a clear role for the DDSP in providing them with needed skills in district and curriculum management. Despite limited initial progress in some instances with district level capacity building, as a result of a great deal of mutual learning, the DDSP in all provinces is at a stage where, provided the issue of district ownership of the project or its successor is properly addressed, it is in a position to provide more comprehensive and in-depth support and development to district officials. Capacity building and professional development needs for district officials have become clearer, as districts have become more settled with regard to staffing, individual responsibilities, and district priorities. 2. School Management The training provided to SMTs (comprised of principals, deputy principals, and heads of department) focused on: Whole School Development; Strategic Planning; Communication and Conflict Management; Team Building and Problem Solving; Human Resource Management (including Performance Assessment and Staff Development); Financial Management and Fundraising; and Instructional Leadership. These are all critical areas. The content was confined for the most part to training in practical, rather than leadership or conceptual skills. 27 A factor limiting the impact of the training for many SMT members was that, because most of them have teaching as well as administrative responsibilities, they were unable to attend all the training sessions. Many SMT members attended just one or two sessions, with only a few principals attending all. This problem eventually was resolved – although not to everyone’s satisfaction – by offering training during holiday time. The most visible results came from the workshops on planning. Visibly displayed in each principal’s office now are the vision, mission, learner code of conduct, educator code of conduct, timetables and policies on admissions, security, HIV/AIDS, language, and other matters. Some SMT members indicated they had delegated the production of these documents to teams of educators, or they developed them with SGBs. In most cases, the documents were presented to the SGB and parents for ratification. This is a significant DDSP success. SMT members indicated that they had no knowledge before of policies or how to generate a school development plan. The training materials guided participants through the meaning and the development of these documents, and their manuals guided their staffs through policy development and approval. SMT members, however, indicated that they had problems in translating the vision and mission statements into practice. While most SMT members found the Human Resources (HR) workshop helpful, few in any province had been involved in hiring new staff or in conducting educator performance appraisals. In general, SMTs did not feel confident in judging educator performance, nor were they clear on what the outcomes should be. One principal asked: Will this lead to a promotion? Another asked: What will I do with it after it is done? There was little understanding of how performance appraisals can be used for professional development. The outcomes of the Financial Management course included an understanding of the budgeting process, the principles of cash flow, balanced budgets, and fundraising, as well as an increase in school fees. Understanding the basics of budgeting has led a number of schools to apply for Section 21 status and some to achieve it. Others, however, need more assistance in gaining an understanding of Section 21 issues. The Communications and Conflict Management modules were lauded for the insights they provided into interpersonal relations. SMTs had a clearer understanding of the need to include members of staff in different processes. Still, there are principals who have made it a conscious decision not to share power, and at these schools educators, in particular, expressed discontent. Educators want the opportunity to call a staff meeting or to raise issues. In some cases, this was not possible and business went on “as usual” with the principal calling the staff meeting and setting the agenda. SMTs’ management style has also changed. SMTs reported opening up, delegating more, listening more, and becoming more genuinely interested in staff. Those who were genuinely interested in sharing power were documented as creating a democratic environment, used a more consultative approach – including with the SGBs – were more open, and transparent in their activities. For others, outside of the creation of the policy documents, there was little evidence of leadership. In one case, the principal sat in the interview and was unable to answer any of the questions posed, deferring always to his deputy or HOD. 28 Before DDSP, SMT activities reportedly were mostly directive, limited to transmitting information from the DOE, and disinterested in educator issues. SMTs are now said to be “involved” in a much deeper way in the daily life of the school and better addressing the personal and professional concerns of educators. SMTs have divided their work by committees and have enlisted the assistance and input of educators to make the school run smoothly. SMT members now see it as their responsibility to hold staff meetings monthly or quarterly, and to discuss more substantive issues. Confusion remains about the SMT role as instructional leader. For HODs who had also attended curricular training, there was significant understanding of OBE and they are more capable of instructional leadership. Principals who had attended only selected management workshops had a more limited understanding of the instructional leadership role. For the most part, SMTs feel much more confident now in managing their schools. Their relationship with the SGBs has improved considerably, and most work together with the SGBs in addressing school problems. As SGBs take on a greater stewardship role over the school, by holding “voluntary” days to repair windows, toilets, paint different offices, and install security systems – and, most of all, contribute to fund raising activities – the SMTs are feeling less isolated in solving the issues of school maintenance and development. Some issues, however, cannot be resolved through training, but are critical in determining quality school management. The poverty of the community surrounding the school in many cases complicates school management. In a large number of cases, children are being cared for by grandparents, who pay the school fees and other costs out of their modest state pensions. In many cases, families do not have the funds to buy shoes or school uniforms, and many children lack “implements” for learning. At one school, we were told that many children (orphans?) were staying on their own, with a relative checking in “now and again.” Many of these children come to school hungry and unable to concentrate on schoolwork. For them, the feeding program is a reason to come to school (and attendance drops when government funding for food runs out). Poverty and distance combine to create a spotty attendance record, as well as chronic lateness. Other extraordinary management circumstances have evolved as the result of farm school amalgamation, giving rise to conflict over language and the desire to have a single language taught at Foundation level. In larger schools, this was handled by having two streams, but in smaller schools this was not possible. A further stumbling block in being able to manage a school effectively – to plan, to generate professionalism, to conduct performance assessments – is the “acting” or “temporary” nature of many of the positions held by both SMT members and educators alike. Until permanent appointments are made, the SMT will have to work with individuals who may be “here today but gone tomorrow.” Under the circumstances, it is noteworthy that they have been able to incorporate as much as they have of the learning they have gained through DDSP. 3. Curriculum 29 VII. DDSP has provided a variety of curriculum training and related inputs at the district and school levels. A large output of learning materials accompanied this training and varying degrees of follow-up and on-site consolidation were carried out at schools, with positive results. VIII. IX. For example, educators now have a general understanding of at least some of the key elements informing the new ways of teaching and learning and the core philosophies underpinning OBE and Curriculum 2005 (C2005) and positively contrast the old transmission model of learning with OBE. They refer, for instance, to the need to replace passive learners with active learners; educator￾centered and textbook-bound methods with learner-centered work; and rigid syllabi with learning programs that lead to creative activities. Educators also are curious about the innovative curricula, including those aspects they acknowledge they have not fully grasped. They emphasize that there is a need for more training, more information, and on-site, follow-up support at schools that includes practical components, like lesson demonstrations. An educator in the EC raised the need for more training in subject matter knowledge. This probably applies to the other provinces as well. Many educators have changed their practices in specific ways, for instance in the use of continuous assessment. They are eager to show records of test scores and learners’ profiles and portfolios. A group of five educators in Limpopo, however, had difficulty in distinguishing between peer assessment, group assessment, and individual assessment. They said too much material was covered at the training workshops. The fact that learners have portfolios does not speak to their content or the quality of the work. Almost all educators praised the training they received in lesson preparation and lesson planning. It helps them carry out their day-to-day work and they did not view the extra effort involved as unnecessary administrative burdens. A possible exception is KZN; more than in the other provinces, educators there complained about there being too much change, too fast. Few educators were able to showcase how their more abstract understandings about good pedagogical practice translate into purposeful and planned classroom and lesson activities. Four educators in KZN mirrored the overall picture in the four provinces. While their classroom practices had shifted, they felt the process had not gone far enough. The team came across several examples of lessons designed around the much-maligned use of rote learning. There were other lessons, however, that showed that the training and technical support received through the project were being successfully applied. These the team found enjoyable, and the pupils enjoyed them as well. Change of this kind was especially evident in the EC, where educators in the Foundation Phase had developed their own learning materials and where the sharing of lesson plans and materials was impressive. The models of training used in the curriculum area by a few NGOs were problematic, in that they contained pronounced features of the transmission teaching OBE seeks to replace, e.g., large lectures with little opportunity for give and take. During a discussion with a group of four educators, one said, “They don’t know our problems. They don’t listen to us. They say, no, you must do it [this way].” In some cases, the expected outcomes were listed at the start of each 30 lesson. This had the effect of pre-empting the results of what is supposed to be a collective process generating new knowledge. In a few cases, as well, confusion was created by different approaches taken by different training service providers to the same schools. Educators at several farm schools argued that the lack of basic infrastructure and other minimum resources was a factor in their efforts to implement curriculum reform. Addressing the poor conditions under which they work, they said, is in some ways more important than the quality of educators’ pedagogical practices. An educator in the NC, like her colleagues elsewhere, drew attention to problems related to language, curriculum, and assessment. She teaches in the medium of Tswana, but learning materials are in English. Two tests will soon be taken by her students: one conducted by the Education Department in September and the other by DDSP (HSRC) in October. It was not clear how the language issue will be addressed. While the tests have been translated into Tswana, in the past there was a problem with the quality of the translations. Moreover, both the Education Department and the HSRC are reluctant to give educators information about the tests. She feared that the tests would be unfair and not provide a true reflection of her learners’ abilities, about which, given the continuous assessment techniques she learned from DDSP, she now feels she now knows a great deal. 4. School Governance In DDSP, governance has been defined in terms of SGBs and SASA rather than the more general issue of community-school relationships. The SASA legislation had introduced a new and radical change, which made it important to assist schools in understanding its requirements and beginning to work with the communities in creating operational and effective SGBs. All the schools visited during the evaluation had SGBs. However, not all SGB were reported as having democratic elections; in one school the principal was said to have appointed the SGB chairperson; and, in another, SGB members were simply volunteers. Some members of most SGBs were female, but the chairperson was almost always male. A wide range of SGBs was encountered in this evaluation. In some, especially the SGBs of small and very rural schools, the parent members were often illiterate; though typically active and natural leaders of the community, this hampered their effectiveness. In others, well educated, skilled community members sat on the SGB and played an active role in school management. The membership of SGBs, especially in Limpopo and KZN, included individuals closely linked to the traditional power structures of the wider community. In most cases SGB members reported that they were pleased to be elected to the SGB and believed their selection had resulted from their position in the community, their personal skills or simply being well liked. A number commented on the sense they now had of having become a community leader. The bulk of the important decision-making in most schools is still in the hands of the SMT, with community members of the SGB and the wider community primarily confined to ratifying those decisions. Many factors in the community and school influenced the role played by SGBs. One large, peri￾urban school in the NC was being torn apart by ethnic divisions, an issue the SGB had been 31 unable to mediate. Community-school relationships were so varied that in two schools only a few kilometers apart in KZN, one community had stolen the doors off the school latrines, while parents were found observing classes and building a garden for school feeding at the other. In another school, qualified unemployed educators from the community were teaching on a voluntary basis. In some communities, school governance was perceived of as the role of educators and there was little understanding of why community members should become involved. The areas in which SGBs were most involved in support to the school were fund raising and the provision of labor for maintenance. Community SGB members listed what they thought their school roles were: solving problems between the community and school, making sure learners attended school, learner discipline, maintenance of the school, and providing protection for the school. Tensions between communities and schools that SGBs most often attempted to address dealt with fees, admission policies, and corporal punishment. SGBs were hardly ever involved in language policy or other curriculum decisions. Of the training offered by each grantee, the amount devoted to SGB training was always less than that directed to curriculum and school management. In the NC and KZN, governance training had been sub-contracted to small, local NGOs, so there was less control over the quality of the training offered. SGB training was almost always for the chairperson and the principal. Everyone – educators, SGB members, grantee staff, and PPDs – reported that the principals dominated the training workshops, a situation that was intensified when SGB and SMT training was combined. The chairperson attending the training often did not complete the entire training course. It is also questionable how much learning took place among community SGB members in the early stages of the program because some training was conducted in English and few materials were translated into local languages. Attempts were made to overcome this problem through approaches such as visual teaching guides and games and, by the second half of 2001, all training for SGB members was conducted in local languages. There were few reports of the community members of the SGBs attempting to disseminate what they had learned in these training sessions to other parents on the SGB or the wider community. For example, one SGB community member reported that “we don’t know what Section 21 is.” In the NC, a new governance trainer had begun carrying out SGB training at schools, often on Saturdays, an approach that was applauded in schools and communities where it had occurred. In all four provinces, almost everyone associated with the DDSP reported that the governance component of the project had encountered the most difficulties. Nevertheless, SGB members at many schools declared that the DDSP had helped them to understand the larger and more important roles they should be playing in school governance. Some of the difficulties encountered by the DDSP approach to governance may have grown out of assumptions about the homogeneity of communities, inadequate understanding of politics within communities, and community receptivity to taking on new responsibilities. It is difficult to focus on SGB roles according to the prescriptions of SASA in a context in which prior experience of community involvement in school governance, decision-making about schools, and democracy through SGBs has not existed. In addition, the type of training offered, which emphasized narrow technical skills taught through large workshops with documents in English, was poorly suited for community members. 32 5. Monitoring and Evaluation The complex monitoring mechanism appears to match the complexity of the overall DDSP design. The central component has been an indicator framework that has been described as “excellent” by some program participants, and as “burdensome and over-demanding” by others. The monitoring system, including site visits by national and provincial project staff, has focused on compliance with USAID grants and subcontract requirements, and assessing program delivery by service providers. Initially, 55 largely quantitative indicators of success focused on “measurable” targets were used. (The number has since been reduced to 45, but it remains too large.) Many participants, while recognizing the value of the quantitative data, felt that the quantitative focus and the excessive number of indicators did not provide adequate in-depth feedback about qualitative changes and processes at the different levels of the system. The framework was liable to be used as a checklist focusing on inputs or more easily measurable outputs. For example in monitoring training it was easier to capture numbers of people trained rather than the quality of training or how it may have changed trainees’ practices. The Indicator Framework and associated checklists have helped in developing some district officials’ ability to better monitor school level changes (e.g., in NC). It has helped districts and officials focus on key issues in schools and plan appropriate follow-up visits and interventions. It has created an appreciation for the need to have systems in place for effective and efficient management. In NC, for example, the indicator system is viewed as a monitoring and evaluation system but also as a mechanism to help district officials prioritize specific school improvement support needs. RTI-DDSP staff roles have included guiding and advising grantees and subcontractors, similarly advising/assisting provincial and district project managers, and monitoring results at the school level. RTI-DDSP also monitors grantee and subcontractor deliverables as agreed in the Grant Agreements and subcontracts, using a Project Tracking Matrix tool. The matrix is updated four times a year, sourcing information from grantee and subcontractor quarterly reports, training, and monitoring visit reports by RTI-DDSP staff. There is also tight fiscal monitoring of grant and subcontractor monthly and quarterly cash flows that provides clear reporting on expenditures. Despite some problems with the monitoring and evaluation mechanisms, they have generally worked well, not least at a program implementation level. They imposed a rigorous accountability, and improved reporting and grantee performance. These successes are balanced by their limited use so far in informing program improvement, particularly at the school level. 6. Assessment DDSP assistance to develop assessment systems and instruments for teaching and learning, educator development, and program evaluation was provided largely through a sub-contract from RTI to the HSRC for an Assessment Modeling Initiative (AMI). AMI was to contribute to DDSP and the “development and implementation of a fully functional national assessment system.” According to RTI, preliminary work on the project began in September 2000 and the sub-contract effectively began on April 1, 2001, though it was not formally executed until 33 August 23, 2001. Originally due to end December 31, 2002, the project will now conclude December 15, 2003. The project’s design required that a test instrument be applied in October 2001 to begin a series of annual measurements of achievement of Grade 3 students in the areas of literacy and numeracy. The instrument chosen, the Mahlahle Test, was one previously developed by one of RTI’s principal sub-contractors, JET Education Services (JET), the same institution which did the 2000 baseline study. Once the sub-contract with HSRC was executed, the administration of the 2001 test and subsequent assessment work became the responsibility of the Council. HSRC’s responsibilities included: development of a single district assessment model, with four operational applications, one for each province; dissemination of the lessons learned from this process to stimulate an informed discussion among relevant stakeholders in South Africa for the development of a national assessment system; generating information necessary for DDSP to further its aims to develop models of fully functional districts; correlating district-school-classroom factors with learner performance; and administering the annual test. The Grade 3 test is the single most important output indicator identified by USAID. A number of concerns about its validity were expressed to the team, mostly by educators. Assessment Resource Banks (ARBs), a multi-purpose set of materials for classroom use, were developed by HSRC to address the first of their responsibilities, district assessment models. At the same time, they provide educators, and students with practical tools for lesson plan preparation, classroom and homework exercises, and educator and student continuous self￾assessment linked to the content of the Mahlahle test and incorporating the principles of OBE. ARBs were produced for all three grades in the Foundation Phase (Grades 1-3) in both literacy and numeracy. Some translated versions were made available, though there reportedly were problems with the translations, as well as some of the graphics. The English version was piloted in October 2001; training was conducted in March 2002; and distribution began shortly thereafter. Though the ARBs were late in arriving, they were very well-received by educators and, according to educators, by students. It is apparent that, whatever their faults, they filled an important gap. There was an immediate demand for additional copies and supplemental teaching aids. HSRC informed the team that, by August 2003, double the anticipated number of copies had been printed and distributed, as well as a series of color “posters” to serve as visual aids, with HSRC sharing some of the cost. The ARBs are currently undergoing revision to reflect the first school year’s experience with them. In support of the AMI project, provincial assessment teams were formed to coordinate the roll￾out of the ARBs, develop training modules, and oversee performance. These teams were complemented by school assessment teams (senior educators and staff), which provided focal points for introduction of the materials. By 2002, the link between these two levels was being provided by a series of Assessment Resource Persons (ARPs), based at project offices to work 34 with the districts and schools to integrate the materials, and the principles which underlie them into their activities. Among other things, ARPs make school visits to assist educators, obtain their feedback, observe how the ARBs are being used, see how the students have performed, and to note how the educator has recorded the results (and uses them for preparing future lesson plans). Another reason for assigning the ARPs is to enhance prospects for sustainability of the assessment initiative, through closer involvement of the provinces, district and schools with AMI and, it is hoped, the incorporation of the ARPs in the districts, after the project ends. 7. Education Management Information The present national EMIS was started in 1995, and various processes and systems have since been developed. The NDOE is continuing with data collection, updating information, and maintaining the system. It is also developing policies that set standards, guidelines and definitions that the provinces should follow to prevent them from developing autonomous, independent systems. Initially, RTI negotiated a subcontract on EMIS with the Education Foundation (EF), with the purpose of supporting the implementation of school funding norms, national assessment, and training NDOE officials to use the system. DDSP provided an opportunity to shift the focus from the national level to district and school level needs. Here, the EF’s work has centered on training SMTs and District Managers. At the start of the EMIS project, building capacity at the grassroots did not receive much attention, and incomplete and inaccurate information were major headaches. Since then, much progress has been made in improving the capture and flow of information. There have been minor, logistical problems such as the fact that trainees have different levels of computer skills. Technical Assistants (TAs) in the field provide training and follow-up work. Challenges include the school environment, and the availability of electricity and computers. Additional problems are the shortage of staff, the lack of technical capacity, and the duplication of activities by the service provider, province, and national EMIS. EMIS seeks to feed school records, such as learner attendance, and financial and admission records, into the District, Provincial and National systems. It has been particularly helpful in assisting schools with budgeting and attaining Section 21 status. An HIV/AIDS component has recently been added. DDSP’s major achievement has been to develop a system at the school level where none had previously existed. The DDSP-EMIS system has now been completed. The NDOE and PDOEs need to undertake a phased integration and institutionalization of the system at the provincial, district, and school levels. Within DDSP and any successor project, there needs to be greater emphasis on the analysis of the voluminous data produced by the system and its use to strengthen educational outcomes, as was foreseen when the database was designed. Some provinces are beginning to move in this direction, e.g., in using TAs to work with schools and Districts on the use of their data. For instance, a school can judge the extent to which it is working towards the attainment of compulsory education for all in the community it serves through a review of admission, attendance and drop-out records. SMTs and Circuit Managers must ensure that their planning is informed by knowledge of their schools and the data collected and processed through EMIS. 35 Districts and Provinces, in turn, must spell out to the NDOE how their information analyses impact on existing and future policies. 8. Program Implementation During all phases of its data collection, the team looked at the issue of DDSP program implementation by all of the main institutional participants: USAID, RTI, the sub-contractors, the project grantees and partners, the provincial education departments, the districts, and the schools. What we found was, on the whole, a remarkably operational and generally effective structure, given the complexity of the project and the fact that it had been active on the ground for only 30 months. In that short period, a variety of systems and structures were installed, multiple technical and material resources were provided, extensive and wide-ranging training was carried out at both district and school levels, and schools were directly assisted through technical assistance, training, and other school support activities. The implementation process, understandably given the volume and rapid pace of the work done over the short period of two and a half years, was not flawless. From interviews with a wide range of DDSP participants, it is apparent that the keys to successful management of DDSP have been: strong and consistent support from senior provincial officials; attitudes and relationships reflecting the true application of partnership principles; adequately staffed and skilled district offices; enough organizational and staffing stability to ensure main tasks are carried out in a timely fashion; and effective coordination mechanisms. The team found that where all or most of these factors were present, work progressed well. Where they were absent, difficulties resulted. Regarding district office capabilities, review of the original project design (report of 1998 design team) suggests that existing capacity at the time was over-estimated. In reality, districts have not always been fully staffed or were unable to fill positions with the proper skills. In many cases, high-level personnel changes have occurred, with detrimental effects. Reorganizations in three of the four provinces, however justified, created significant difficulties in getting the project underway. In the course of the team’s work, a number of management issues were raised by project participants and other stakeholders. One is whether the project needed to be so complex, with such detailed monitoring and reporting systems, given the still highly underdeveloped nature of many of the areas in which it operates. There are good arguments on both sides. On the one side, is the limited absorptive capacity, especially of the schools, and the significant amount of time required to be spent on project reports and procedures. On the other, is the widely recognized need for greater structure, systems, and rigor in an environment where, for good and obvious reasons, there had never before been a chance for such to be developed properly. On this side of the ledger, as well, are critical needs for accountability and the building of sustainability. Another issue is whether the decision to recruit the principal service providers – the grantees and their partners – via a grant competition among NGOs rather than through normal contract tender 36 procedures, was the correct one. The grant mechanism tended to lead to the providers operating parallel to the regular educational structure, which went against the logic of a full partnership. It also produced four somewhat different programs in the four provinces. Again, reasonable arguments exist on both sides of the issue. Greater integration with departmental structures probably would have improved coordination, enhanced accountability, and created a stronger partnership. On the other hand, there is a perceived need for administrative flexibility to meet tight, demanding project requirements and the freedom to experiment with the different models. Empirically, when the time came to arrange for the implementation of EMIS, School Funding Norms, and Assessment components of the RTI scope of work, the decision was taken not to hold an award competition among NGOs, but rather obtain services for carrying out these activities through normal open bidding and subcontracts. Effective coordination of the large number of players within each district – district and circuit officials; staff of the regional office; grantee staff; staff of grantee partner service providers; RTI staff; and, of course, school officials, educators, and school board members – was unevenly achieved. While for the most part things worked well, some of the ultimate beneficiaries, the schools, reported that they occasionally were confused as to the different roles and identities of the many project-related people they encountered. Complicating the picture, given the lack of full ownership of the project by the provincial departments, de facto there was no one formally in overall charge. D. Research Triangle International Once the acknowledged delays encountered during the first 18 months of the contract were surmounted, RTI’s performance appears to have been highly professional and successful, a fact reflected, among other things, in the subsequent USAID performance reviews. The RTI-DDSP scope of work emphasizes grants management and technical assistance; it does not have full program execution authority or responsibility. Viewed in this way, it is clear that the main requirements of its contract have been met. The awarding and monitoring of the four NGO grants has been carried out thoroughly and professionally. Technical assistance to the grantees, subcontractors, provinces, and districts has been a major RTI function. A large, extensive training program also was carried out in support of program objectives. Initially, this was done primarily through centrally located workshops and other training events. Later, as experience was gained with the strengths and weaknesses of the basic training model, more training was done at the school level, often for clusters of “nodal” schools and more on-site follow-up support was provided. In the past 30 months, some 29,000 training units have been provided, with a unit being one trainee attending one training event. Importantly, RTI has been instrumental in helping maintain a project-wide focus on DDSP’s goals, in a situation where provincial autonomy and the different models being applied in the four provinces creates the risk that the overall program framework, as defined in the Request for Application (RFA), might be lost. 37 In addition to its DDSP portfolio, three other tasks were assigned to RTI by USAID: 1. Subcontracting, implementing, and monitoring of the Northern Cape Micro-Science Project (science kits); 2. Subcontracting, implementing, and monitoring of the Limpopo Furniture Procurement project for 30 flood affected schools; and 3. Subcontracting, implementing, and monitoring of the Limpopo Instruction Materials Procurement project for 30 flood-affected schools. 38 E. USAID Besides its normal responsibility for monitoring and evaluating the RTI contract and the entire DDSP, USAID/SA has directly provided other, important project inputs to DDSP, utilizing Indefinite Quantity Contracts (IQCs) and other available mechanisms, including the RTI contract. Included have been such things as the very successful box libraries, science kits, the provision of resource centers to clusters of DDSP schools, development of communications capabilities, the drilling of wells, and selected, technical assistance inputs in one province, Limpopo. No special problems or issues arose from the team’s review of these USAID activities. 39 CHAPTER IV. EVALUATION QUESTIONS/CONCLUSIONS A. Evaluation Questions 1. To what extent were planned objectives of the DDSP met? What contributed to objectives being met and what, if anything hindered progress? Reference each Province. Due to the delay in its implementation, DDSP has been in actual operation for only two and a half years (some components even less). Given this fact, the amount of work and progress that has been achieved in a relatively short time is remarkable. Everyone involved, including the provincial and district offices, RTI, the four grantees and their partners, the sub￾contractors, and above all the schools, is to be congratulated. In the team’s judgment, 30 months is too short a time to fully achieve the objectives of such an ambitious, integrated project as DDSP or reach definitive judgments as to its results. There is little doubt in the team’s minds, however, that important progress has been made in all provinces and that in particular, achievement of the central objective of “improved quality education delivery for grades 1-9 in the DDSP target areas,” despite the varying results in different provinces, is on track project-wide. At the school level, for example, significant management improvements have been made in a majority of the cases observed. A similar picture emerges of the impact of the project on teaching and learning, though the changes have come more slowly and their sustainability is more dependent on effective follow-up by school support personnel. The governance area, however, is a different story. In many instances, the SGBs are weak and are not performing the leadership, oversight, and community activation roles envisaged in the SASA. In the district offices, areas of relative strength are management and a more gradual development of school support capability. Contributing factors to success have been the energy and resourcefulness of the multiple project partners, the fact that the systems, training, and support that DDSP provided were badly needed and often helped fill a vacuum, and the fact that, despite their many problems, the leadership and staff of the schools generally cooperated to the best of their ability and continue to do so. A major hindrance to progress, factoring out the negative effects of the delay in implementation, was the lower than anticipated capacity levels of the majority of districts at the time the project started. Another was that the national educational transformation process, which naturally continued during the life of the project, notably in the EC, limited the ability of the districts to participate in intended ways. Other factors were changes in key personnel and the time it took to organize and coordinate the details of what is, by any measure, a complex undertaking. 40 2. What was the role and relative importance of implementers, e.g., the lead contractor, subcontractors, grantees, district offices, etc.? Consider such elements as integration, coordination, cost-effectiveness, and sustainability. Reference has been made to the structural problem of RTI managing and evaluating, but not directing, NGO grantees, with the latter and their partners playing the primary project execution roles. This arrangement created, de facto, a situation where it has been difficult to ensure that the overall project framework is consistently applied, i.e., that the project maintains the fully integrated character set forth in the project design. It also, particularly in the early stages, led to some confusion, lack of effective coordination, and occasional tensions between the project and the district offices in which it worked. Confusion at the school level was a corollary result. The effects of the arrangement on achievement of sustainability goals, though varying from province to province, probably have been adverse. It helped create a situation where the project, in most cases, never fully lost the image of being external, even when its work was well-appreciated. Today, virtually all of the parties believe that it would have been much better for the project to have been embedded, from the beginning, within the regular education structure. Many also believe, and this includes the team, that a standard sub￾contract, rather than grant approach, would have provided better results overall. 3. What evidence is there to indicate that integration of key components of the provincial programs is taking place at the school and district levels? – i.e., curriculum, management, governance, and district support. Is a holistic picture emerging of district offices and schools? A holistic picture of development at district and school levels is slowly emerging, but progress is slow due to a number of factors, summarized below. District Management Development and Support The district management development component of the DDSP has had significant but not optimal success, for reasons including: · On-going restructuring of districts and sub-regional units; · Inadequate staffing and other resources; · Inadequate knowledge or adaptation to local conditions for which the “district training” was being undertaken; · Inability to fully embed DDSP district office development activities within overall district development plans and activities; · Tensions arising from lack of clarity among district officials, project staff, and service providers regarding lines of authority and accountability; and, 41 · A focus on a narrow range of technical and administrative skills linked to district efficiency. School Management · DDSP school development was most successful when it supported the whole school goal through integrated holistic support. Whole school development was demonstrated best in the development of the school plan, where a collaborative process involving SMTs, SGBs, and educators was used. · SMTs are more democratic in their management, involving staff in many decisions and being more consistent in holding staff meetings. · SMT relationships with SGBs are more collaborative and interactive. · SMTs have understood the budgeting process and many have applied for Section 21 status. Preparation for Section 21 was inadequate, however, as many SMTs did not understand what was required of them, or what it would mean for their budget. · Training in educator performance appraisal was inadequate. It did not link the appraisal process to professional development. Most members of SMTs who did not attend curriculum workshops are not in a position to assess the performance of educators who are using OBE methodology. · Principals have not yet been adequately trained to be instructional leaders as well as “whole school” leaders. School Curriculum · Educators understand key elements of the new methodologies of teaching and learning and the basic ideas that inform OBE and Curriculum 2002. They are also well-disposed towards the reform initiatives. · Educators have changed their practices in specific ways. The area where the providers achieved most success is lesson planning. · The results in the classroom have been mixed and partial. Some educators continue to rely on rote learning. Others are experimenting with the new pedagogies and the training they received through DDSP. · The models and actual lessons used in training, in some instances were overly reliant on technical, top-down approaches. OBE training modules did not include adequate contextual information. · Learning materials developed through DDSP are used, but not by all educators. The materials are mainly in English. Learning materials do not focus enough on subject matter knowledge. · At some schools, improving the quality of teaching and learning depends on the provision of basic resources. School Governance 42 · The SGB component of the DDSP has been the least successful; to some extent this is due to the overly optimistic ambitious goals and assumptions regarding SGBs set forth in the SASA. · The DDSP, operating within its mandate, focused on strengthening SGBs to perform the roles according to the prescriptions of SASA, without addressing the related characteristics and needs of the surrounding communities, e.g., adult literacy or the frequent absence of effective coordination between the schools and local political and social institutions. · DDSP’s primary input was training; initially, the type of training offered, which emphasized skills taught through large workshops with documents in English, was poorly suited for community members, even when attempts were made to incorporate visual images and games; subsequent experiments with school-based SGB training appear to have been more successful. 43 Monitoring and Evaluation · The project monitoring and evaluation database provided a useful project management tool, although, in the team’s judgment, it is based on far too many indicators and places excessive strain on service providers. The tools allowed the contractor and the grantees to track change on a school level for each of the project management goals and to share experiences across the provinces. · Attempts to use this database as a development tool to guide school support and/or district approaches to schools, however, have encountered limitations because the database requires resources and capacity beyond that of most district offices. The database provides information on whether or not certain activities have been completed, but is of less value for assessing factors assisting or hindering change. · The concerns about the Grade 3 test were discussed in Chapter III. The team believes that one of the requirements of the follow-on project should be to assess the experience with the test and make changes, as necessary. 4. How satisfied are the beneficiaries (sample all levels) of the program? Consider the services provided to national, provincial, district, and school level beneficiaries. To the extent they were not satisfied, what is the exact nature of their dissatisfaction? Beneficiaries are, in general, satisfied with the training and support provided by DDSP. The districts (or sub-regional units) have not been able to provide the array of services that DDSP has, and so the attention schools, especially, have received was most welcome. At the NDOE level, admittedly less than full investment in the project meant that information about the project was conveyed primarily through written reports that were not read consistently. This reportedly has improved, as the project has created a stronger presence and identity. At the PDOE level, low, initial involvement of key provincial staff, due to restructuring, did not produce the easy flow of information envisioned by the project. Although forums were developed for the sharing of information, participation often did not generate the feedback loop envisioned. Embedding did not take place at this level, although gradually, the project seems to have attracted greater attention. Owing to reorganization and migration, few districts had the human resource capacity to attend to whole district development training offered by service providers. Embedding at this level was also problematic, but also began to move forward as the project progressed and staffing was more complete. DMs expressed a lack of satisfaction with the workshop delivery system of grantees, including the frequency, timing, and duration of each workshop, 44 the lack of follow-up interactions, the need for all workshops to be seen as an integrated INSET program, and the competitive spirit the project produced in providing services to schools. Other issues include: · The occasional conflicting demands on districts emanating from the provincial departments, as a result of a perceived need at that level to give priority to non￾DDSP problems and priorities; · The lack of computers to implement and utilize EMIS; and · The failure to choose all schools in a district for participation (making the job of districts more difficult and cascading a necessity for sustainability). At the school level, SMTs, SGBs, and educators were very satisfied with the content of the training they received, but had problems with the delivery system. Workshops were too intense and did not allow for absorption of all the new ideas presented, resulting in only partial ability to apply what was learned. There was insufficient time between workshops so educators could not comment from one workshop to the next on what worked and what did not. An insufficient number of materials were provided for use in the classroom. Workshops took too much educator time away from the classroom. Workshops should be provided at cluster schools. Other issues include: · The Foundation Phase received more attention that the Intermediate and Senior Phases; · Educators were not given credit for the training attended; · Educators did not understand continuous assessment very well; and · SMTs did not receive educator training and, thus, were not adequately prepared to conduct performance appraisals. Because the dissatisfaction registered was more of a service delivery fine-tuning nature than fundamental, the general satisfaction level can still be rated as high. 5. What modifications would it be feasible to recommend to increase the success and impact of the program during its remaining life and/or extension period? Please consult with key stakeholders. Due to the limited time for the implementation of the DDSP and the fluctuations occurring at district levels, the project generally began with a greater focus on school level improvement. In 2002, however, it began to focus more on district support and development. During the remaining extension period, the DDSP should continue this emphasis. Some of the particular elements that the team believes should receive priority at the district level are achieving a higher degree of project integration into district structures, higher level training for district managers, and greater emphasis on building school support teams and systems. At the school level, efforts should continue to go beyond “school visits” and focus more on site-based support, emphasizing practical training and demonstration sessions, preferably at the nodal or cluster level, for educators, school managers, and school governing bodies. 45 The team was informed by RTI that a process of evaluating and documenting the lessons learned and best practices developed by DDSP during the past two years is getting underway. It supports that initiative and expresses the hope that the activity will be carried out in close cooperation with the national and provincial education departments and that both external and internal experts will be utilized. The results will be important for whatever follow-on project is developed. One of the elements the team hopes will be included in the above-noted evaluation is HIV/AIDS. DDSP has not had a large focus on HIV/AIDS. The subject was included in the agreements signed between RTI and the grantees, and some useful work has been done. For example, the NC grantee developed a manual that is now used by all regions in that province. Additionally, the other provinces have taken the same manual and used it to their advantage. But, the nature and depth of the epidemic’s impact on the education system, including the DDSP districts and schools, is such as to raise the question whether the priority of HIV/AIDS activities should be raised from what it may have been when DDSP started. The team believes strongly that any follow-on project should include a greater emphasis on HIV/AIDS. In the interim, it recommends that the impact of DDSP’s HIV/AIDS efforts to date should be carefully evaluated, with a view to helping assess how this issue could be further addressed in a new project. For analytical purposes, the focus should be on seeking ways to (1) help the districts and schools mitigate and cope with the effects of the epidemic on the education system, and (2) utilize the schools as agents for assisting local governments and communities to help prevent HIV transmission and provide social support for those affected by the disease, either directly or indirectly, especially learners. 6. What are the pros and cons of replication of all or some of the DDSP beyond the project’s current time frame and scope including the extension period? Please consult with key stakeholders. From its fieldwork and other consultations with stakeholders, including the Stakeholders Workshop, the team is convinced that the task of helping South Africa develop an effective and sustainable model for district-based primary school improvement will require that any follow-on activity to DDSP include a strong focus on replication and sustainability. The kind of intensive support that DDSP has provided will be fully justified, and the results of it made sustainable, if the project’s best practices and lessons learned, as well as the experience of other, similar projects, are incorporated in a national replication model that has the support of the NDOE and the PDOEs and is embedded in South African education policies and budgets. The corollary is that future USAID technical assistance, training, and material support, provided via a follow-on project or other means, must support the national model. The challenge is to find a way to do this that maintains full government ownership but still allows for the degree of flexibility needed to make USAID assistance effective. As regards mechanisms for providing technical and training help, one possibility that deserves serious consideration is the development of partnerships with the education faculties 46 of the universities in the respective provinces specifically focused on these needs. In addition, efforts might be made to incorporate DDSP educator training best practices into the regular INSET program. Other steps which might be taken are to explore the possibility of increased private sector support, e.g., for learning materials and equipment, and to review other primary school development experiments in South Africa to determine how they have addressed long-term sustainability issues. In sum, the development of such a revised model, to be carried out in close cooperation with the national and provincial education departments should be a top priority of the proposed follow-on project. The approved model, then, could be tested in a number of additional districts and schools in the four DDSP provinces. The number of the districts and schools that would directly benefit would still be small in relation to the needs, but the impact on national policies, programs, and budgets would be great. Consideration should be given to a replication methodology that utilizes experts drawn from the most successful DDSP districts assisting the new districts and schools in implementing the new model. 7. What are the pros and cons of extending all or some of the DDSP project activities beyond the extension period? Please consult with key stakeholders. The team does not believe that further extension of the current DDSP, except as needed to ensure a smooth transition with a new follow-on project, is the best way to go at this stage. For reasons cited earlier, it is convinced that significant changes in governance, ownership, goals, priorities, and structure are needed if sustainability and replication goals are to be reached. B. Conclusions Regarding a Possible Follow-On Project The team believes that the proposed follow-on project should have three core objectives: (1) Consolidating the work begun under DDSP; (2) Working with the national and provincial departments on extracting and utilizing the experience and best practices of DDSP (and other key primary education improvement projects) to develop a national primary school improvement model, which will be sustainable under South African conditions and progressively replicated, as national resources permit; and (3) Helping the national and provincial departments, as needed, to strengthen their program evaluation capacities. In order to assure full control over project resources and delivery mechanisms, the new project’s management structure should be contract-based, at all levels. 47 The team’s recommendations for putting these conclusions into effect are presented in Chapter V. 48 CHAPTER V: RECOMMENDATIONS A. Follow-on Project The team recommends the prompt design and development of a three- to four-year follow-on project to replace DDSP. The project would have the core objectives set forth in Section IV (above). Its guiding principles and principal components would be: 1. Governance: Joint governance through a Project Steering Committee (PST), composed of national and provincial heads of department or designees, the Director of USAID or designee, the project director (South African), the prime contractor Chief of Party, and one or two outside persons, possibly a leading academic and/or private sector representative. Its functions would be to oversee the joint planning of the project in all its aspects, approve a policy framework for it, and thereafter to monitor its progress and act, as needed during the life of the project, to make whatever adjustments may be needed. A small staff would be needed to support the committee. A similar, provincial steering committee would be established in each participating province. 2. Ownership: Obtaining a strong buy-in from the national and provincial departments before the project begins is a top priority. For this to happen, the project must have a close fit with provincial priorities and needs. The project office and its components in the provinces and districts would be physically in and be accountable to the head of the education unit in which each is located: NDOE, PDOE, or District. 3. Outside Providers: Outside assistance provided through the project in accordance with the project design, would take the form of a prime contractor, to be selected by USAID, with the approval of the PST. The prime contractor would propose South African and other sub￾contractors to conduct special tasks in accordance with its scope of work, subject to approval by the PST. It is envisaged that project personnel would be located at all key levels: National, Provincial, and District. 4. Definition of District and School Support Teams: The word “district” signifies the unit in each provincial education structure in which the authority, budget and financial control, and responsibility for education service delivery resides. The term “school support team” refers to those subsidiary service delivery units, variously called circuits, areas, or education development offices that the districts employ to effectively reach the schools. The focus should be on functions, not terminology. 5. Priorities: Top priority would be given, in close cooperation with NDOE and the PDOEs, to development and testing of a model and methodology for replication and sustainability of DDSP, including a systematic effort to analyze and apply the project’s lessons and best practices. A second priority, closely related to the first, would be to consolidate the achievements of DDSP, particularly in such areas as leadership training at both district and school levels, professional development of educators, and further development of school support practices and mechanisms. A third priority would be to strengthen national, 49 provincial, and district capacities to coordinate, evaluate, analyze, and apply best practices from the full range of educational development initiatives sponsored by the government and assisted by all external donors, not just USAID. 6. DDSP Coverage: The focus of the consolidation work, with the possible exception of the ex￾Model C schools, would be on the existing DDSP provinces, districts, and schools. At the end of the consolidation phase, the original project districts and schools would “graduate.” 7. Replication Model: When an approved replication model and methodology become available, the first priority for extension to additional schools would be given to non-project schools in participating districts and then to new districts in the four provinces. The core of the proposed pilot replication effort would be the creation of Replication Support Teams (RSTs) in each province, drawn from experienced staff of the highest performing project districts and schools and other sources. RSTs would provide intensive planning and training assistance to districts, selected according to criteria developed under the model, followed by a period of follow-up support and mentoring as the model is implemented. In this latter role, RSTs might accompany district support teams to the schools, but they would not be service delivery agents. An RST could assist approximately three districts per year with replication of the model. 8. Learning, Disseminating, and Utilizing Results: The project design would include components for capturing the lessons learned and best practices of this and other primary school development projects in South Africa. With these best practices, a flexible but unified national model and plan for progressive replication throughout the country would be developed. It is proposed that this effort be led and coordinated by an appropriate department of the NDOE, in cooperation with the principal donors and under the overall direction of a steering committee, including national, provincial, and donor representatives. B. The Existing Project (DDSP) 1. The increased focus on district support and development should continue through the end of the project; in so doing, priority should be given to achieving a higher degree of integration of the project into district structures, providing higher level training for district managers, and putting greater emphasis on building strong school support teams and systems. 2. It is further recommended that support activities go beyond “school visits” to include more site-based, training and demonstration sessions, preferably at the nodal or cluster level. 3. The lessons learned and best practices of the past two and a half years should be carefully studied and documented, in close cooperation with the national and provincial education departments and using both internal and external analysts. 50 4. The experience with the Grade 3 test should be carefully reviewed, with a view to providing an input into the process of developing the assessment component of the follow-on project. (See Chapter VI., unresolved issues.) 5. Similarly, DDSP’s work on HIV/AIDS should be evaluated to assist carrying out the recommendation C.1 below. C. Other Recommendations 1. That a strong HIV/AIDS component be developed at all levels of the new project. (See Chapter IV and Annex F.) 2. That leadership training opportunities be developed and provided at provincial, district, and school levels. This higher level training, executive development cum leadership training, would build on the more skills-oriented training that has characterized DDSP so far. Included might be training modules in Effective School Leadership, Instructional Leadership, Financial Projections and Forecasting, Leading Culturally Diverse Institutions, Planning Human Resource Development Needs, Performance Assessment (school and district) and Whole School Systems Design. Financial management, human resource management, conflict resolution, and strategic planning. Providing this learning track for credit will enhance the professional status of school and district managers. 3. That increased in-service training opportunities be developed for educators, with special emphasis on subject matter training needs. 4. That greater emphasis be placed on developing instructional management capacities of top provincial and district officials and SMTs. 5. That greater use be made, where possible, of nodal schools and clusters as channels for providing school support and other purposes. 6. That more focus be placed on means of bridging the gap between educators’ theoretical understanding and their daily practices in the classroom, via classroom demonstrations by master educators, ideally as part of district school support teams, and provision of additional teaching aids and materials. 7. That more learning materials should be provided in the mother tongue, especially in the Foundation Phase, where the medium of instruction is not English. 8. That a needs assessment be undertakenof the qualifications, including subject matter knowledge of teachers in each district, followed by the development of interventions to address any gaps that are revealed. 51 9. That closer links and involvement with communities be developed, as means of creating greater understanding and support of the schools by the communities and using the schools to bring the communities closer together for cooperation on a range of common interests. 10. That consideration be given to greater use of low-cost incentives as a means of rewarding exceptional performance by circuits, school support teams, SMTs, SGBs, educators, and learners, creating a sense of competition among project participants. 11. That closer cooperation with university education faculties within each province be explored, as a possible source of training inputs to supplement those available from NGOs and others, particularly the training and mentoring of master educators and heads of department. Practice teaching arrangements are another possibility. 12. That increased efforts be made by the NDOE and PDOE to improve coordination with and among donors interested in primary and basic education improvement. Leadership must come from the GSA, with assistance from donors. 52 CHAPTER VI: UNRESOLVED ISSUES The one unresolved issue is the Grade 3 test. The team did not feel able to reach a conclusion on the matter, because of its lack of specialized knowledge of technical assessment matters and also of the time needed to look into it thoroughly. Nevertheless, in the course of its data collection, the team received enough expressions of concern about the test, mostly from educators, for that fact to qualify as a finding. In the team’s view, prior to the conclusion of the DDSP project, USAID and RTI should undertake or commission a careful review of the experience with the test, as an input for planning and development of the assessment component of the follow-on project... 53 CHAPTER VII. LESSONS LEARNED AND BEST PRACTICES With the close of every project, significant reflection on what has actually been learned in the course of the implementation of the project needs to take place. In the case of the DDSP, we believe that significant lessons that have been learned and should be taken into account in the design and delivery of new projects that seek to address basic education problems in South Africa. In addition, a number of noteworthy best practices have emerged from the DDSP experience to date, with no doubt more to follow, if the proposed follow-on project is approved. A. Lessons Learned · In the South African context, it is critical to have a strong buy-in from all relevant levels of government before a project is undertaken. In the case of primary education development, this means national, provincial, and district levels. Buy-in and eventual sustainability are best accomplished through joint project design, governance, and execution. · The focus on the district, as defined in Chapter V, is appropriate. The district is where education policies are implemented, or not, as the case may be. · To implement a coherent, integrated development project, contracts and sub-contracts with service providers, including NGOs, are preferable to grants. · The project model employed should be kept as simple and low-cost as possible, consistent with its core objectives. · Human capacity development is the key to success, but to be truly effective, requires strong follow-up and support systems to convert theoretical knowledge into practice. · A holistic approach to school development, especially governance, must include consideration of the communities, as well as the schools. The environments in which schools operate are critical for learning and the schools themselves, are unique resources for addressing such learning-related problems as parent literacy and parent and educator health. B. Best Practices to Date · Growing emphasis on leadership development, at both district and school levels, especially district managers, principals, and key educators. · Greater use of training systems with built-in follow-up mechanisms. 54 · Increasing focus on development of school support teams, with an appropriate range of skills and a vocation and systems for taking the services to the users. · Trend for school support to be provided to clusters of schools, rather than individual schools. · Provision to schools of basic learning materials, such as box libraries and assessment resource banks. 55 ANNEXES ANNEX PAGE A. List of Contacts .......……………………………….…2 B. DDSP Evaluation Team Work Plan………………...9 C. Methodology and Field Work Plan .………………..12 D. School Sample .....……………………………………25 E. Stakeholders' Workshop ……………………………29 F. Education and HIV/AIDS ..…………………………34 56 ANNEX A LIST OF CONTACTS EASTERN CAPE Provincial Name Designation Date Mr. B.T.M. Mfenyana Chief Director: Districts August 5, 2002 District Queenstown Mr. O.B Makhaza Mr. Nelson, Bula Mr. B. Mfenyana Mr. K. Jayiya District Director – Queenstown EDO for Circuit 6 – Wittlesea, EDO, Acting Director for Districts Coordinator of Special Projects August 5, 2002 Lady Frere Mr. Jojwana, Miss Njokwe Mrs. Balintulo District Manager Subject Advisor Education Development Officer August 6, 2002 RTI Mr. N. Godle Ex-PPD August 8, 2002 Grantee Mr. Roy Valentine Mr. Lenox Matshishi Mrs. Kholeka Madonondo Mrs. Khosi Khrani, Mrs. Vuyo Thompson Mrs. Ivey Mabaso Mrs. Nontando Jada Mr. David Project Director - READ Management Trainer, READ Project Manager, READ Curriculum Trainer READ Curriculum Trainer READ Curriculum Trainer, READ Curriculum Trainer, READ MCTP August 5, 2002 Schools Lady Frere JSS Mr. Sapiwa Machana Mrs. Ntsaliba Mrs. Malilu Mr. M. Ntlikonbini Mr. J.B. Mtebele Mr. M.W. Tokwe Mrs. N.F.S. Ntungo Principal Deputy Principal HOD Educator - Grade 7-9 Educator - Lang/Eng 7-9 Educator - Intermediate August 6, 2002 Sidakeni Primary Mr. Mantshi, Mrs. Gladys Ngquongwa Mr. Luvwyo Mndini Mrs. Anasala Rasmeni Principal, HOD Educator SGB Parent –secretary August 6, 2002 St. Theresa Primary School Mr. E.M. Lee Mrs. H.G. Doyle Mr. C.P. van Wyk Mr. David de Souza 9 Educators Principal Educator on SMT Deputy Principal SGB Parent Member August 7, 2002 57 EASTERN CAPE (Continued) School Name Designation Date Louis Rex Mr. Van Heerden Mrs. Maru Principal HOD, Junior Phase/Foundation Deputy Principal HOD Intermediate August 7, 2002 Kleinbooi Mrs. Mayekiso Mrs. Kyna Mrs. Boti 3 Educators Deputy Principal HOD Sciences HOD Humanities August 8, 2002 Chris Hani Primary Mr T. Maki Mr K. Lapi Mrs N. Buwa Mrs. Sojola Mrs. N. Madotheyi Mrs. Nhlongo Mrs. Maki Mrs. Makisa Mr. F.T. Keva Ms. Mthetha Mr. P.R. Sithonga Ms. By Mosina Mrs. Martins Principal Deputy HOD Educator Senior Phase Lead Educator Educator Foundation Phase Educator Foundation Phase Educator Senior Phase Educator Intermediate Educator Intermediate HOD HOD Chair, SGB August 8, 2002 58 LIMPOPO Provincial Name Designation Date Mrs. Miriam Segabutla Mr. Mukhavhudi Chief Dir for Reg Coord & Aux. Serv DDG August 12, 2002 District Polukwane Mr. Mametja Mrs. Maine Mr. Moses Letsoalo Mrs. Leduaba Acting District Manager Circuit Manager District Development Officer District Development Officer August 12, 2002 Hlanganane District Mr. M.T. Khosa Mr. Mogangi Mrs. Letsoalo Mr. Famanda Mr. Donald Mr. Michael District Manager District Officer District Officer District Development Officer District Development Officer Provincial Project Director August 13, 2002 RTI Ben Tladi PPD August 12, 2002 Grantee Mr. Zendile Kunene Mr. Thabo Mngope Mrs. Sarah Mukhawane Mr. Eugene Nzula Mrs. Seipati Machoge Mr. Josea Malake Mr. Juachim Mamabola Project Manager, MSTP Mngope, Business Services COUNT/MSTP MSTP MSTP ProTech Project Consultant August 12 and 16, 2002 August 16, 2002 Schools Pembunuka Primary School Mr. David Mr. Nndandulini Mr. Khosa Educator, Grade 5 Educator, Grade 3 Principal August 13, 2002 Nwa Mhandzi Mr. Dennis Salani Mr. D.J. Mahlangu Pastor Noel Ngoveni Hlupheka Deputy Principal Principal Chair, SGB August 13, 2002 Nkuzana Mr. M.J. Mkavele F.T. Mudaka 2 Educators Principal HOD August 14, 2002 Gija Primary School Mr. J.M. Makondo Ms. S. Mafanele 3 Educators Principal Senior Educator August 14, 2002 Leetetja Primary School Grade 5 Educator SGB Chairman August 15, 2002 Chokwe Primary School Mr. Matlopela Principal August 16, 2002 Phuti Makibela School Mrs. Poopedi 3 SMT Members Principal August 16, 2002 59 KWAZULU-NATAL Provincial Name Designation Date Dr. S. Mbokazi Mr. Z. Dlamini Mr. Gumede Dr. Khumalo Dr. Mhlongo Ms. Mbata Ms. Nxumalo Ms. C. Mpati Regional Director: Ulundi (& DDG) CES: EMD PPD/Director: Teacher Development August 12, 2002 August 12, 16, 2002 District Nkandla Mr. M.V. Mdletshe Mr. N. B. Mathenjwa Mrs. G.N. Mdlalose Mr. Mhlongo Mr. R. R. Sikhosana Mr. M.A. Zulu Mr. Mbuso Simemane District Manager CM - Godide CM -Chewezi CM – Ekhombo CM - Sibhudeni Sigananda PPC August 13, 2000 August 15, 2002 RTI See Provincial – C. Mpati Grantee/ Subcontractor Deva Govender Dolly Nxde Ndoba Ngubo Clement Mkwanazi Zine Bhongu Nomo Radebe Shabalala Dumisani Bongi Mkhize Zondi Jabu Nomsa Madukizda Project Director Foundation Phase Foundation Phase Natural Science Senior Phase Natural Science Intermediate Phase Intermediate Phase English Foundation Phase Language Foundation Phase Language Management & Governance Management & Governance August, 2002 Schools Manzanmyama Mr. M.Z. Zakhe Mrs. E.N. Sibiya Mrs. E. Mdunge Miss S.S. Sikhakhane Mr. Lindiwe Xulu Mrs. Phindile Zondi Ms. P. Malombo Mr. Sifiso Mtombeni Miss N.M. Gymede Mrs. B.M Mthombeni B.M. Mthosinbe S.D. Mdluli N.A. Shezi P.J.K. Ngcobo SGB Chairperson SGB Parent member Community Member Community Member Educator Grade 1 Educator Grade 4 Educator Grade 6 Educator Grade 7 Educator member SGB Educator member SGB Principal, member SMT HOD, member SMT HOD, member SMT Educator member of SMT August 13, 2000 60 KWAZULU-NATAL (Continued) School Name Designation Date Gcinukuthula Mr. S. Khanyile Miss. Jiyane Mr. Zwane Mr. Gabriel Jabulani Mrs. Alice Mbata Ms. Beatrice Sibisa Nozipho Khuboni Mni Ngcobo SGB Chairperson SGB Parent member SGB Parent member Educator Grade 7 Educator Foundation Phase Educator Intermediate Phase Educator member SGB&SMT Educator member SGB&SMT August 13, 2002 Ezimambeni Mr. Meshack Mpungose Mrs. C.C. Dlamini Mr. Hilson Langa Mrs. Janet Sibiya Mrs. C.C. Dlamiori Mr. N.M. Mpungose Mrs. Princess Nxumalo Mrs. Makhosazana Nene Mrs. Phiwsiwe Makhathini Educator Foundation Phase Educator Foundation Phase SGB Chairperson Community Member Educator Educator HOD, Member SGB&SMT Educator, Member SGB&SMT Acting Principal, Member SGB&SMT August 14, 2002 Sigananda Mr. A.S. Shezi Miss N. P. Duma Mr. N. C. Zulu Miss N.P Duma Mr. N.C. Zulu Mrs. G.M Ngcobo Mr. T.L. Gogo Miss T.G.- Bhengu Ms J. P Shezi S.V. Masondo Miss R.S. Mikhize SGB Parent – Secretary Foundation Phase Senior Phase Foundation Phase Senior Phase Foundation Phase Intermediate Phase Intermediate Phase Acting Principal Deputy Principal Educator member SMT August 14, 2002 Iwangu Mr. Mpungose Mrs. N. Dlamini Mr. L. Madida Mr. M. Nzuza Mr. B. Biyela Mr. T. Biyela E.H. Mthembu E.S. Mazibuko Principal HOD Foundation Phase HOD Int. & Senior Phase Chairperson Vice Chairperson Induna – appointed to SGB Educator Foundation Phase Educator Senior Phase August 15, 2002 Khomo Miss Z. Zwane Mrs. L. Cele Mr. Wiseman Ngonyoma F.E. Moume S.E.N. Mkhize M.A. Ziqubu C.M.C. Khansile B.V. Nzula L.D. Dumizane T.R. Khanyile Educator Grade 1 Educator Intermediate Phase Educator Senior Phase Chair, SGB Treasurer, SGB and Educator Secretary, SGB and Educator Educator, Member SMT Educator, Member SMT Educator, Member SMT Principal, Member SMT August 16, 2002 61 NORTHERN CAPE Provincial Name Designation Date Mr. Tex Moraladi Mr. Joe Mpuang Head of Department Provincial Project Coordinator August 9, 2002 District Mr. Sandile Beuzana District Manager, Frances Baard District August 5, 2002 Mr. T. Pharasi Colleen Cornelisseu Theres Ratikoane Chichi Sabana Basil Mothibi Ms G. Moredi Director, School Administration Assessment Resource Person ECD Coordinator Curriculum Coordinator (4-12) Community Development Officer Circuit Manager Barkly West August 5, 2002 August 5, 2002 August 5, 2002 August 5, 2002 RTI Mr. Sizwe Mbi Provincial Project Director August 5, 2002 Grantee Mr. Steve Harvey Mrs. Barbara Harvey Mr. George Mosimane Project Manager SSO and Training Coordinator Governance August 5, 2002 Schools D.L. Jansen Primary Mr. Basil Marsh Ms. Charlotte Jones Ms. Pella Bokala Mr. Christian Ms K Magwevana Ms T Billy Mr. J Humampe Principal Educator, Member of SMT Educator, Member of SMT Deputy Principal Educator, Member of SGB Educator, Member of SGB Educator, Member of SGB August 6, 2002 August 6, 2002 August 6, 2002 August 6, 2002 Priel Landgoed Primary Ms. Catharina L.E. Leroux Ms. Lachme Swartz Ms. Cathlween Philander Principal, Member of SGB&SMT Educator, Member of SMT Educator, Member of SMT August 6. 2002 August 6, 2002 August 6, 2002 Laerskool Andalusia Ms. Mercia Louw Educator, Member SGB August, 2002 Jankempdorp (New Ndwanya) Primary Ms FV Jantyies Mrs. N.M. Mathebula Mr. D Zwedala Educator Educator Educator August, 2002 Stillwater Intermediate School Mr. Lephoi Ms. S.M. April Ms. K.E Tau Principal, Member of SGB&SMT Educator, Member of SGB Educator, Memb er of SGB August, 2002 62 NATIONAL Organization Name Designation Date USAID Dirk Dijkerman Eileen Oldwine Don Foster-Gross Mathata Madibane Sharon Harpring Sibusiso Sithole Bunny Subedar Darlene v.d. Westhuizen Joann Lawrence Steffi Meyer Faroon Go olam Paula Bertolin Lessiah Msithini Kim Bolyard Mission Director Deputy Director Education Team Leader Deputy Education Team Leader Basic Education Team Leader Basic Education Specialist Basic Education Specialist Program Development Specialist Program Development Specialist Program Development Specialist Basic Education Specialist Regional Contracts Office Project Management Assistant Africa Bureau, USAID/Washington Meetings with USAID staff were held at various times in the course of the evaluation RTI Richard Cartier Brian Chinsamy Saeeda Anis Masenya Dikotla Luis Crouch Project Director Education Director Grants Manager Deputy Education Director Consultant Meetings with RTI staff were held at various times during the evaluation Telecon 8/24/02 NDOE Khetsi Lehoko Duncan Hindle Lulama Mbobo Firoz Patel Martin Prew Peter Ramatswana Kgobati Magome DDG, FET DDG, GET Director, EMIS Chief Director, Physical Planning Director, EMDG CES, EMDG National HIV/AIDS Advisor August 19, 2002 August 23, 2002 August 21, 2002 August 19, 2002 August 22, 2002 August 22, 2002 Sept. 5, 2002 Sub-Contractors Nick Taylor John Pampallis Michael Ogawa Lindani Mthetwa L. Mavimbela Anil Kanjee Hendrik de Kock Director, JET Director, CEPD Khulisa, Operations Director Prog. Manager, Ed. Foundation Director, Ed. Foundation AMI Project Director, HSRC AMI Project Coordinator, HSRC August 20, 2002 August 22, 2002 Grantees Pat Sullivan Cynthia Hugo Steve Blunden Director, MSTP Director, READ CEO, LINK August 23, 2002 August 26,2002 Donor Mokgapi Maleka Lusungu Kanchenche Education Adviser, DFID Deputy Program Manager, HRD August 20, 2002 August 20, 2002 63 ANNEX B DISTRICT DEVELOPMENT SUPPORT PROGRAM EVALUATION TEAM WORK PLAN TIMETABLE The District Development Support Program (DDSP) evaluation team’s work plan is divided into four phases. 1. A one week preparatory phase, consisting of: Briefings and discussion of evaluation goals, methods, and processes with USAID, the National Department of Education (NDOE), and the DDSP contactor, the Research Triangle Institute (RTI); collection and study of reports, analyses, budgets, statistics, and other project documents essential to the team’s work; development of a methodology and strategy for data collection; making internal team assignments and other concrete arrangements for executing the data collection plan, especially the field work phase; and developing a final report outline. 2. An approximately two and one half-week data collection phase. The majority of the time during this phase (two weeks) will be spent in the four provinces where DDSP is active; the remainder will be spent interviewing stakeholders and other sources of relevant information in Pretoria and Johannesburg and completing the team’s library of documents and other information; planning of a stakeholders workshop will begin after the completion of the provincial field work. 3. A data analysis, “brainstorming” of conclusions, and report drafting phase, comprising the latter part of the fourth week and the first part of the fifth; the stakeholders workshop will be conducted during this period. 4. A final phase involving the preparation and presentation of the team’s draft report, review of the report by the major stakeholders, making whatever revisions are required, and, finally, report production and distribution. This phase will take place during weeks five and week six. All team members except the team leader will end their participation in the project by August 31. The team leader will depart September 6. TEAM STRUCTURE AND ROLES The Aguirre team consists of five persons with a broad mix of skills and experience. The work plan is designed to utilize these capacities to the maximum extent. The team’s modus operandi will be participatory, and all team members will share in all major evaluation design and management issues, analysis of findings, development of recommendations, and in report design and writing. Several, if not all, members of the team will vet all team products. 64 INDIVIDUAL ROLES · Richard (Dick) Dye, Education Administration Specialist/Team Leader, will coordinate the activities of the evaluation team, serve as primary liaison with USAID and the contractor, develop the final design of the evaluation, oversee the development of evaluation instruments, monitor the provincial fieldwork, integrate the findings of different team members, and coordinate the preparation of final reports. In addition, he will be responsible for looking at DDSP in the context of other donor- assisted basic education improvement programs and examining cost-effectiveness, and budget and financial issues. At the report writing stage, he will take the lead on drafting the sections of the report dealing with recommendations for possible follow-on activities. · Nancy Horn, International Development Specialist, will take the lead on developing the team’s methodology and evaluation instruments, working with team member Everard Weber. She also will lead the sub-team which will assess the program in two of the four provinces, Eastern Cape and Limpopo, and will coordinate the preparation of the corresponding field visit reports. During the fieldwork, she will pay particular attention to grantee, district, and school management and whole district development issues, including related training. In addition, she will be responsible for leading the development and execution of the Stakeholders Workshop to be held in week five. · Joyce (Joy) Wolf, Evaluation Specialist, will work with team member Jordan Naidoo, in developing the team’s fieldwork plan and coordinating appointments and other arrangements. The two will also collaborate on determining the effectiveness of the formative evaluation to date and resolving questions relating to the degree to which improvements in school and student performance linked to DDSP interventions have occurred. In addition, Joy will lead the sub-team doing field work in Northern Cape and KwaZulu-Natal provinces and coordinate the preparation of the corresponding field visit reports. During the fieldwork, she will pay particular attention to governance, community participation, and EMIS issues and related training. · Jordan Naidoo and Everard Weber, Instructional Systems/Education Specialists, will work together to assess the appropriateness and effectiveness of individual learning￾related project interventions: Teacher training and subject matter expertise; the roles and use of master teachers; implementing the curriculum at the school level; student-oriented classroom teaching methods and strategies; student assessment, learning materials, facilities improvement, etc. Related to this, they will examine the quality of training of teachers and the effectiveness of monitoring and other follow-up to the training. Jordan will do field work in Eastern Cape and KwaZulu-Natal and Everard in the Northern Cape and Limpopo. In addition, they will draft the basic education and DDSP description sections of the report. FIELD RESEARCH METHODOLOGY Please see Annex C. 65 REPORTING TO USAID AND THE EVALUATION REFERENCE GROUP ON THE FIELD WORK After the completion of the fieldwork phase, the team will report to USAID and the ERG on what it was able to accomplish while in the provinces and will present for discussion some of the early issues that arose during the fieldwork. CONSULTING STAKEHOLDERS AND INFORMATION SOURCES IN GAUTENG PROVINCE In addition to the fieldwork in the four DDSP provinces, the team will be arranging meetings with an extensive list of stakeholders and other sources of relevant information in Gauteng Province (Pretoria and Johannesburg). Examples include key NDOE staff, current and past DDSP sub-contractors, and other donors. STAKEHOLDERS WORKSHOP A one-day workshop for approximately 30-40 people, plus observers, will be held during the fifth week of the evaluation. The proposed date is Wednesday, August 28, 2002. The workshop will bring together a representative group of stakeholders from the schools, districts, and provinces directly involved in DDSP. The purpose will be to give the evaluation team an opportunity to share key evaluation findings and recommendations with the stakeholders and receive their feedback. The workshop will be designed and organized by team member Nancy Horn, working with counterparts from USAID, the NDOE, and RTI, and will be facilitated by the evaluation team. REPORT PREPARATION, REVIEW, AND PRODUCTION The team’s report will be drafted during the fifth week of the evaluation. During the latter part of the week, the draft will be shared for comments with USAID and subsequently the Evaluation Reference Group. Team members scheduled to depart at the end of that week will redraft appropriate sections of the report before they leave. The team leader will remain for a further week to coordinate final revisions to the report and its production and distribution before his departure on September 6. 66 ANNEX C METHODOLOGY AND FIELD WORK PLAN INTRODUCTION In this Annex we present the overall research methodology employed for the evaluation and then delineate the specific instruments that were used in collecting data from the DOE at the provincial and district levels, the grantees, the schools and communities. While in Pretoria, the team held briefings and discussions with USAID, the NDOE, the DDSP contractor, RTI International, and other stakeholders (see Annex 1 for a full list of contacts). The team also collected secondary data, including studies and reports, analyses, budgets, statistics and other project documents. TEAM COMPOSITION AND SAMPLING METHODOLOGY The team was organized into two groups with two members in each group, and conducted data collection in the provinces. The composition of these sub-teams changed between the weeks: Nancy Horn and Jordan Naidoo was team one in the first week, Joy Wolf and Everard Weber team two; in the second week team one included Joy Wolf and Jordan Naidoo, and two, Nancy Horn and Everard Weber. The team leader, Dick Dye, worked with each team in all four provinces. During the first week, team one worked in Eastern Cape and team two in Northern Cape (the work week consisted of only four days due to a national holiday); during the second week, team one worked in KwaZulu Natal and team two in Limpopo. Due to the limited time allowed for data collection, five schools were visited in the Northern Cape, six the Eastern Cape, seven in KwaZulu Natal, and eight in Limpopo. The districts, schools, and communities within which data were collected were selected according to criteria designed to maximize the range of examples rather than a random sample. In the provinces in which there are multiple districts involved in the DDSP, two districts were selected; in provinces in which there is only one district, two circuits were selected. Schools within those districts were selected to represent a range of sizes, types and ethnic composition where relevant (e.g., at least one small and one large school and both primary and combined schools when available). Variation in terms of achievement levels based on the third grade assessment conducted by HSRC was used to select schools. Schools that scored both high and low on the 2000 test and schools that demonstrated either a substantial increase or decrease in test scores were selected, plus schools that were average in both initial scores and degree of change. The location of schools was also a factor: both schools that are relatively close to the district office and those farther away, plus schools that are located in peri-urban areas in contrast to more deeply rural locations were selected. The fieldwork schedule is outlined in the following table. 67 WEEK ONE: 8/5/02- 8/8/02 WEEK TWO: 8/12/02 – 8/16/02 Province Northern Cape Eastern Cape KwaZulu Natal Limpopo Team Joy & Everard Nancy & Jordan Joy & Jordan Nancy & Everard District or Circuit Barkly West Circuit, Kimberley District Jan Kempdorp Circuit, Kimberley District Queenstown District Lady Frere District Sigananda Circuit, Nkandla District Sibudheni Circuit, Nkandla District Polokwane District Hlanganani District Schools -Romance - Stillwater Combined -Breipaal -Laerskool Andalusia -Laerskool Hartvaal -Kleinbooi JSS -St. Theresa Primary - Louie Rex Primary -Chris Hani Primary -Lady Frere JSS -Sidakeni Primary -Manzamn￾yana -Ezmambeni -Geinuku￾thuda -Mathiya -Iwangu -Khomo -Sigananda -Phuti Makibelo -Kgantshi -Leetetja -Chokwe -Nkuzana -Nwa￾Mhandzi -Pembunuka -Gija 68 69 PRIMARY DATA COLLECTION METHODOLOGY In collecting primary data, three levels were addressed: Level 1 – Provincial Department of Education; Level 2 – District Department of Education, RTI and DOE Teams, and the Grantees; and Level 3 – Schools and Communities. For each level and with each stakeholder, the team devised a research instrument that was implemented in each province. The instruments were created in line with the Seven Key Evaluation Questions posed in the USAID Statement of Work, the Project Objectives and Indicators (see final three pages of this section), and the Project Tracking Matrices prepared by RTI for each province. SEVEN KEY EVALUATION QUESTIONS Question 1: · To what extent were planned objectives of the DDSP met? · What contributed to objectives being met and what, if anything, hindered progress? Reference each province. Question 2: · What was the role and relative importance of implementers, e.g., the lead contractor, subcontractors, grantees, district offices etc.? · Consider elements such as integration, coordination, cost-effectiveness and sustainability. Question 3: · What evidence is there to indicate that integration of key components of the provincial programs is taking place at the school and district levels (i.e., curriculum, management, governance, district support)? · Is a holistic picture emerging of District Offices and schools? Question 4: · How satisfied are beneficiaries (sample all levels) with the program? · Consider the services provided to national, provincial, district, and school level beneficiaries. To the extent that they were not satisfied, what is the exact nature of their dissatisfaction? Question 5: · What modification would be feasible to recommend to increase the success and impact of the program during its remaining life and/or extension period? · Present supporting evidence to substantiate any recommendation. Question 6: · What are the pros and cons of replication of all or some of the DDSP beyond the project’s current time frame and scope including the extension period? Please consult with key stakeholders. 70 Question 7: · What are the prose and cons of extending all or some of the DDSP project activities beyond the extension period? Please consult with key stakeholders. In each location, the team implemented the following instruments in both individual and focus group formats. In addition to the formal data collection instruments, we utilized the technique of participant observation to obtain information on the general setting and background to the research. While the number of questions we asked may appear high in certain cases, it is important to remember that some questions are derived specifically from the indicators. We have highlighted these questions on each instrument. Moreover, because data collection was all used qualitative techniques, it was critical that we triangulate information obtained from different sources. Hence, certain questions were the same posed across all interviewees and others were specific to the type of interviewee. LEVEL 1 Provincial Department of Education: We posed the following questions to the senior administrators, including the Chief Director, Head of Department, and the Education Management Development Coordinators either individually or in focus group format: 1. How does the DDSP fit into Provincial priorities? What types of support has DDSP provided to the province? 2. What is the nature of the relationship between the Provincial DOE and the National DOE? 3. What is the nature of the relationship between this province and its districts? 4. What is your perception of the impact of DDSP on the districts? On the schools? 5. How has DDSP support assisted you in providing support/training to districts to prepare them to take over greater responsibilities? (e.g., management, sustaining best practices, promoting the attainment of Section 21 status) 6. Within the framework of decentralization under DDSP, what difficulties have arisen in transferring different responsibilities to the district? To schools? (e.g., norms and standards, racial integration, promoting equity, hiring & firing, promotion of personnel) 7. What have you done within DDSP to facilitate the development of SGBs? 8. DDSP will be coming to an end soon. How do you think another project similar to DDSP could improve on the old one? LEVEL 2 District Department of Education: We posed the following questions of the senior administrators at the District Level, including the District Manager, Circuit Managers, Curriculum Unit leaders, and any other members of the District Support Team (DST) in a focus group form. 71 1. How does DDSP fit into district priorities? What types of support has DDSP provided to the district? 2. Is there a clear demarcation of roles and responsibilities in performing district operations? How does the DST work? 3. How would you characterize schools before DDSP began? How would you characterize schools now? Explain. 4. What types of support do you receive from the provincial DOE? 5. How has DDSP enhanced the teacher training (e.g., OBE, improving qualifications) you provide to educators? 6. How has DDSP enhanced the management training you have provided to schools? 7. What types of follow-on support does the district provide in each of the above two areas? 8. How has DDSP enhanced the work of school governing bodies (SGBs)? 9. DDSP will be coming to an end soon. How do you think another project similar to DDSP could improve on the old one? Grantees: We posed the following questions of grantees and their sub-contractors and/or consortium members in a focus group format. 1. What do you think about the implementation process for DDSP? (what has stayed the same, what has changed) 2. How would you characterize schools/districts before DDSP began? How would you characterize schools/districts now? Explain. 3. What specifically has each of you done to improve the quality of OBE at the district level? (e.g., curriculum/subject matter expertise) At the school level? (e.g., teaching methods, improving teacher qualifications, assessment, subject matter knowledge, utilization of resources) 4. What types of teacher training have you provided to educators? (e.g., methods and subject-matter knowledge; best practices, difficulties, frequency) 5. What specifically has each of you done to improve the quality of district management? School management? 6. What specifically has each of you done to enhance the effectiveness of SGBs? 7. What is the nature and frequency of the follow-up work you do with schools? (include reporting on changes observed as a result of training received) 8. What have you done to ensure the sustainability of activities after you leave? 10. DDSP will be coming to an end soon. How do you think another project similar to DDSP could improve on the old one? Provincial Project Directors and DOE Teams: We posed the following questions of provincial RTI employees and their DOE counterparts in individual interviews. 1. How would you characterize the relationship between provincial DOE and district DOE? What specifically has each of you done to improve relationships between provincial and district offices? 2. How would you characterize schools/districts before DDSP began? How would you characterize schools/districts now? Explain. 3. What specifically has each of you done to improve the quality of OBE at the district level? (e.g., curriculum/subject matter expertise) At the school level? (e.g., teaching 72 methods, improve teacher qualifications, assessment, subject matter knowledge, utilization of resources) 4. What types of teacher training have you provided to educators? (e.g., methods, subject￾matter knowledge, best practices) 5. What specifically has each of you done to improve the quality of district management? School management? 6. What specifically has each of you done to enhance the effectiveness of SGBs? 7. What is the nature and frequency of the follow-up work you do with schools? 8. What have you done to ensure the sustainability of activities after you leave? 9. DDSP will be coming to an end soon. How do you think another project similar to DDSP could improve on the old one? LEVEL 3 Senior Management Team (SMT) (Principal, Deputy, Heads of Department): We posed the following questions of the senior management team of each school in a focus group format. 1. Tell us something about the characteristics of this community. How do these characteristics influence teaching and learning at the school? 2. What is the scope of the SMT functions? Who is responsible for what? How is accountability demonstrated for each of these responsibilities? 3. How would you characterize schools/districts before DDSP began? How would you characterize schools/districts now? Explain. 4. How would you characterize your relationship with the district? Explain. 5. How would you characterize the role and relationship between this school and DDSP? [The following are 2 “probe” questions specifically related to the indicators.] Have you signed a “Subcontract” with the district ensuring meaningful participation in the DDSP towards contributing to improved school performance? How frequently and what do you report to the district about the school’s performance? 6. Has the school been prepared to apply for Section 21 status? What was done to prepare you? 7. How has your management style changed over the course of DDSP? 8. What is the relationship between the SMT and the SGB? [The following seven questions relate specifically to the indicators.] 9. How do you promote and cultivate the learning habits of your educators? What is the nature of labor relations of your staff? 10. How frequently do you appraise your educators’ performance? What process do you use? 11. Explain the process of implementing OBE in your school. (successes, challenges) 12. What type of records do you keep to track educator absenteeism and punctuality? 13. What type of records do you require your educator to keep on their students? 14. What plans do you have for the development of this school? 15. How have you created and used school/educator timetables? 16. How frequently do you hold staff meetings? What types of issues are discussed at these meetings? 73 17. DDSP will be coming to an end soon. How do you think another project similar to DDSP could improve on the old one? Educators: We posed the following questions to educations in a focus group format. 1. Tell us something about the characteristics of this community. How do these characteristics influence teaching and learning at the school? 2. How would you characterize this school before DDSP began? How would you characterize this school now? Explain. 3. Have you received training by the DDSP? Have you received DDSP training from another educator who was trained? How would you characterize the training you received? 4. How would you characterize your relationship with the district? How much in-service has the district and DDSP provided to increase your professional skills? Are you satisfied with what the district and the DDSP provide for your professional development? 5. How would you characterize your relationship with the SMT? Do you believe the school is run democratically? Tell us about labor relations at this school. 6. Are you satisfied with the activities of the DDSP at this school? 7. Explain what you understand about OBE? How have your teaching practices changed in light of this new curriculum approach? 8. How would you characterize your management style in the classroom? [The following five questions related specifically to the indicators.] 9. What learner-centered techniques do you use? How do you actively engage your learners in classroom activities? How do you develop critical thinking skills in your learners? 10. What impact has DDSP had in terms of your ability to generate lesson plans? To maintain a good learner marking book for homework and tests? 11. What types of teaching and learning materials have you developed? Did you work on these alone? With a team of teachers? 12. What workshops/learning activities have you participated in to increase your subject matter knowledge? 13. What types of learning and teaching materials have you used in your classroom? 14. What is your relationship to the parents of your students? To the SGB? 15. How frequently are staff meetings held? What types of issues are discussed at these meetings? 16. DDSP will be coming to an end soon. How do you think another project similar to DDSP could improve on the old one? Learners: Where possible, we conducted the following exercise/posed the following questions to small groups of learners selected randomly by the interviewer and who are willing to participate: Instructions for children up through grade 6: Provide sheets of paper and crayons to students. 74 · Tell them: It is the end of the school day. You have just finished your work at school. Draw a picture of yourself – of what you look like – at the end of the day. Be sure to pay special attention to drawing your face. · Debriefing: Have students come forward to explain their drawings. Probing Question: Why is this person smiling/frowning – happy/sad? Explain. Conceptual Question to be phrased appropriately for each situation: 1) What is the relationship of the facial expression to liking/disliking school? 2) What is the relationship of the facial expression to liking/disliking teachers? 3) What is the relationship of the facial expression to what goes on in the classroom (e.g., teaching methods, materials, discipline, rewards, explanations, etc.)? Instructions for children in grades 7 through 9 (Pose the following questions.) 1. Are you satisfied with the education you are receiving at this school? Explain. What would you change? 2. Are you satisfied with your teachers? Explain. What would you change? 3. Are you satisfied with the materials you use in your classroom? Explain. What would you change? 4. How do you think your parents should be involved in the school? 5. Do you feel you are being prepared for different issues you will face in life? School Governing Body (SGB): We posed the following questions to the members of the SGB (including parents, community leaders, teachers, principal (ex officio/secretary)) in both individual and focus group formats. 1. Tell us something about this community. How do these characteristics influence teaching and learning at the school? 2. Tell us about how the SGB was established. What is the structure of the SGB (roles)? What is the function of each member of the SGB (responsibilities)? What is the composition of the SGB (male/female)? 3. Who makes decisions? How are decisions made? Please give us an example of a decision the SGB made. May we see the minutes? 4. What kind of training did the SGB receive from the district? From DDSP? Have you been prepared to seek Section 21 status? 5. How do you set school fees? What other financial decisions does the SGB make? What is the budgeting process? Has your budget been audited? Does the school have a bank account? 6. What is your relationship to the SMT? To educators? (probe for any tensions) [The following six questions related specifically to the indicators and law.] 7. Have you developed the following: a. A constitution b. A mission statement of the school c. A code of conduct for learners of the school 8. What role do you play in teaching and learning in the school? 9. Do you administer and control the school’s property, buildings and grounds? 10. Do you encourage parents, learners, educators and other staff to render voluntary services to the school? 75 11. Do you identify, interview, discipline, and hire (and fire) school staff? What procedures did you follow when you hired the last staff member? How did you obtain the finances to support this position? 12. How have you set the language policy of the school? 13. What types of other policies have you set for the school? (e.g., integration, equity, school safety) 14. To whom are you accountable? How do you demonstrate your accountability? 15. What difference has DDSP made in the functioning of the SGB? Has the school become better or worse over the last few years? Explain. Community Leaders, Parents, Business Owners, etc.: In a few instances, we gathered a group of community members and posed the following questions: 1. Tell us something about this community. How do these characteristics influence the teaching and learning at this school? 2. What do you believe is the community’s role in influencing and/or monitoring the activities of the school? 3. Do you experience any difficulties when you want to see the principal or a teacher at the school? Explain. 4. What is your perception of the school and its practices? How do you think the school and its practices compare to other schools? 5. Are you satisfied with the school? Your child’s performance at school? Explain. 6. What do you know about the curriculum changes (OBE) that are taking place at the school? 7. What do you think about the educators and the teaching your child is receiving at the school? 8. What is your opinion of the school’s management? The SGB? 9. Has the school become better or worse over the last few years? Explain. 10. What would you do to improve the school and the children’s performance? Criteria for Classroom Observations: The team observed classrooms for the following: 1. Use of learner-centered teaching techniques 2. Learners actively and meaningfully engaged in learning activities 3. Use of prepared lesson plans containing identified lesson plans 4. Marked homework assignments 5. Systematic recording of learner performance 6. Continuous assessment 7. Understanding of subject matter 8. Development and use of own learning materials 9. Use of learning materials developed by others 10. Classroom management, including disciplinary/rewarding activities 76 GOAL, SUB-GOALS, OBJECTIVES INDICATORS GOAL: Improved Quality of Educational Delivery 1. Eliminated. Increased learner pass rate. for Grades 1-9 in the DDSP Target Area 2. Increased learner performance on Grade 3, exit point assessment. SUB-GOAL 1: Improved quality of curriculum practices 3. Increase in the mean index score for all educator-focused indicators below (except indicator 11). 4. Increase in the number of educators demonstrating the use of a variety of innovative learner-centered teaching techniques. 5. Increase in the number of classrooms where learners are “actively and meaningfully engaged” in learning activities. 6. Increase in the number of educators with prepared lesson plans containing identified outcomes. 7. Frequency of marked homework assignments. Objective 1.1: Improved teaching methods 8. Quality of marked homework assignments. 9. Increase in the number of educators that maintain a clear systematic recording of learner performance. Objective 1.2: Improved methods of assessment 10. Increase in the number of educators who practice continuous assessment. Objective 1.3: Improved educators knowledge in selected learning/subject areas 11. Increase in the number of educators who can exhibit an acceptable level of knowledge necessary to teach in the grades to which they are assigned. 12. Increase in the number of educators capable of developing their own teaching and learning materials (applicable for Zikhulise Project only). 13. Increase in the number of educators who use teacher-developed learning and teaching materials (applicable for Zikhulise Project only). Objective 1.4: Improved utilisation of resources 14. Decrease in the number of schools/classrooms found with learning materials locked up in storage and/or undistributed during school hours. SUB-GOAL 2: Improved quality of district/area and school management 15. Increase in the number of Section 21 schools of the South African Schools Act (SASA). 16. Decrease in annual educator absenteeism. 17. Increase in educator punctuality. 18. Increase in the number of schools maintaining systematic records of learners’ academic progress, resources and resource use, correspondences, and learner attendance records. 19. Increase in the number of schools that develop school development plans. 20. Increase in the number of schools that create and use timetables. 21. Increase in the number of schools that have minutes of regular staff meetings. 22. Eliminated. Increase in the percent of schools that have an agreed code of conduct for learners. 23. Increase in the number of schools that have safety and security plans in place. Objective 2.1: Improved school management by SMTs 24. Increase in the number of schools that have signed “Subcontracts” with the district ensuring meaningful participation in the DDSP towards contributing to improved school performance. Objective 2.2: Effective management of school curriculum by SMTs 25. Increase in the number of educator performance appraisal activities conducted by HOD or other school manager. 77 GOAL, SUB-GOALS, OBJECTIVES INDICATORS 26. Eliminated. Increased ratio of school funds spent for curricular development purposes compared to funds spent on maintenance/ physical infrastructure. 27. Increase in the frequency of support visits made by DO to schools. 28. The existence of DO job descriptions that delineate roles and responsibilities that further, support, and maintain whole district development. 29. Increase in the number of teacher training days provided and/or coordinated by DO to schools. 30. Increase in the “customer satisfaction” score on DO performance. Objective 2.3: More and better support provided to schools by District Offices 31. Increase in the number of school performance reports collected and maintained by the DO. Objective 2.4: More and better support provided to 32. Increase in the frequency of RO/PO visits to the DO. District Offices by regional/provincial offices 33. Increase in the “customer satisfaction” score on RO and PO performance. SUB-GOAL 3: Enhanced School Governance 34. Eliminated. Increase in the number of Section 21 schools. Objective 3.1: Democratically elected SGBs 35. Existence and evidence of democratically elected SGBs. 36. Increase in the percent of women serving on SGBs. 37. Increase in the number of SGBs/schools that fully meet the policy documentation requirements of SASA. 38. Increase in the number of SGBs/schools, which show evidence of applying SASA policies. 39. Percentage of schools that have audited or examined budgets. 40. Percentage of schools that maintain Bank accounts. Objective 3.2: Enhanced SGB performance 41. Percentage of schools with approved annual budgets. 42. Increase in the number of DO SGB-support visits. 43. Increase in the number of training days provided and/or coordinated by DO to SGBs (SGB training days). Objective 3.3: More and better support provided to SGBs by District Offices 44. Increase in the “customer satisfaction” score on DO performance. SUB-GOAL 4: Developed Theory and Best Practices for Whole School/District Development 45. Eliminated. Number of publications in which these models are described. Objective 4.1: Development of Effective Models of Whole District Development 46. Number of effective models of whole district development. 47. Approved course as a graduate level course at Wits University. 48. (Revised Indicator 48) Hire Technical Assistants for NDOE for 59 person months. Objective 4.2: Development of a Graduate-level course in Educational Economics and Finance aimed at supporting Whole District Development and Technical Assistance to NDOE on financial and policy matters and HIV/Aids co-ordination 49. Eliminated. Approved certificate programme of district officers involved in the DDSP. 50. An EMIS that supports implementation of the school funding norms. 51. An EMIS that supports the national assessment. 52. Number of DoE education officials trained to use EMIS for Norms & Standards for School Funding. Objective 4.3: Development of Education Management Education Management Information Systems (EMIS) aimed at supporting Whole District Development 53. A project Web site. Objective 4.4: Implementation of the school funding norms 54. Eliminated. Increased number of SASA Section 21 schools in the DDSP target areas. 78 GOAL, SUB-GOALS, OBJECTIVES INDICATORS Objective 4.5: Development of civic structures aimed at sustaining and/or furthering the objectives of Whole District Development 55. Number of institutionalised structures specifically designed to support ongoing educational transformation. Upon the team’s return from interviewing in the four provinces, it held interviews with RTI, USAID and Luis Crouch, a man involved in the original design of DDSP. The instruments used are as follows: QUESTIONS FOR USAID 1. Where and how well does DDSP fit into the Mission’s plans and priorities? Are there any significant differences in the Bureau’s and the Mission’s perspectives on the program? 2. How do you feel, generally, about the project’s progress to date, in relation to its goals? What, in your opinion have been its strong points and its less-strong points? 3. In your opinion, how well has the project’s design worked out? In retrospect, are there things that you think might have been designed differently? 4. Are you satisfied with the partnership arrangements with the national and provincial departments of education and their support of the project in policy and practical terms? How might these relationships be further strengthened in a follow-on project? 5. In your opinion, has the focus on empowering the districts to serve as a key agent for school development turned out well? Are there alternatives that should be considered? 6. The improvements in student performance on the grade 3 test, while encouraging, were less than hoped. What do you think explains those results? 7. From your perspective, how well have the project monitoring and evaluation mechanisms worked? 8. Are you comfortable with project cost performance to date? Are there components that you feel have been particularly cost-effective or cost-ineffective? 9. How does DDSP support the objectives of other IRs in SO 2 and other Mission SOs, and vice versa? 10. In your view, what are the principal lessons that have been learned so far from DDSP? 11. What are some of the changes you would like to see made in DDSP, should development of a follow-on project be undertaken? 12. In your opinion, what should follow DDSP? Are there other or additional ways to address South African basic education needs, building on the DDSP experience that you think the evaluation team should consider? 13. What are the possibilities for collaboration and cooperation with other donors in the development and support of a DDSP follow-on project? Do you think that might be a good idea? QUESTIONS FOR THE CONTRACTOR: RESEARCH TRIANGLE INSTITUTE 1. Could you please summarize the RTI contract, e.g., RTI’s scope of work, level of effort, and principal implementation and evaluation mechanisms? 79 2. Could you please elaborate on RTI’s grants management, monitoring, and technical assistance functions? 3. Could you please summarize the models employed by the four project grantees and indicate, as appropriate, any significant changes made in them over the course of the project? 4. Could you please describe the roles and purposes of the sub-contractors 5. How would you rate the DDSP project’s overall progress to date, in relation to its objectives? 6. Which components of the project, in your opinion, have been particularly successful and which less so? 7. Has DDSP generally operated in a cost-effective manner? Are there specific interventions that have been especially cost-effective? Are there others that raise concerns that the cost may not justify the results? 8. The results in terms of DDSP student performance shown in the 2001 grade 3 test, while encouraging, were not as good as hoped. What do you think underlies those results? Are there other measures of the project’s effectiveness that you feel should be considered in evaluating it? 9. In retrospect, what changes in project design might have improved results? 10. How would you describe the partnership arrangements between the national and provincial education departments and the project? 11. How well have the evaluation and monitoring mechanisms worked? 12. What, in your view, are the principal lessons learned to date from DDSP? 13. Do you have any suggestions for the evaluation team and USAID as to what should follow the current project? QUESTIONS FOR LUIS CROUCH 1. What was your role or roles in the National Department of Education? 2. What were the principal, specific activities that you were involved with? 3. Could you describe for me the role or roles you played in the DDSP? 4. What, in your opinion, were the Department’s major expectations of DDSP, i.e. what results did they most hope to get? 5. Do you believe that they were/are satisfied with it? 6. How would you characterize the relationships between the Department and DDSP? RTI? USAID? 7. What is your personal assessment of DDSP, and what do you think have been its strengths and weaknesses? 8. Were there features in the original design of the project that turned out to be problematic and needed to be changed? If so, what were they? 9. Do you think DDSP should be continued? Why? 10. What modifications in the project, if any, do you think should be considered, in the event it continues? 11. Should there continue to be a policy component in an extended DDSP or a follow-on project? If so, what are your thoughts as to it possible content? 12. Do you have any other comments or suggestions for the evaluation team? 80 ANNEX D SCHOOL SAMPLE EASTERN CAPE School Type Roll Grades 2001 Test Result/ Change from 2000 Fees REGION DISTRICT CIRCUIT Rural/Peri-Urban Distance to District Office Ex Primary 1113 1 - 7 48 +14.5 R25 NORTHERN QUEENSTOWN Rural/Small Town SINAKO TOWN/SHIP DORDRECHT Far DET Combined 778 R - 9 33 +4.3 R20 (FP) R30 (IP) R50 (SP) NORTHERN QUEENSTOWN Rural MACIBINI A/A Far TRANSKEI Combined 555 R - 9 41 +8.3 R35 (FP) R42 (IP) R55 (SP NORTHERN LADY FRERE Rural Small Town Close TRANSKEI Combined 1 - 9 59 0 R200 NORTHERN QUEENSTOWN QUEENSTOWN￾Urban Close HOR Primary 298 1 - 6 30 +4.6 R10 NORTHERN LADY FRERE Rural MKAPUSI A/A Far TRANSKEI Primary 651 1 - 7 60 -18.5 R250 NORTHERN QUEENSTOWN QUEENSTOWN￾Urban Close HOR 81 NORTHERN CAPE School Type Roll Grades 2001 Test Result/ Change from 2000 Fees REGION DISTRICT CIRCUIT Rural/Peri-Urban Distance to District Office Ex Primary 388 R - 7 86 -3.8 R1440 KIMBERLEY KIMBERLEY K2 JANKEMPDORP Peri-urban Far HOA Primary 611 31 +4.5 KIMBERLEY KIMBERLEY K6 Windsorten Rural Relatively Close HOR Primary 1129 1- 7 40 -0.9 R20 KIMBERLEY KIMBERLEY K2 JANKEMPDORP Per-urban Far DET Primary Farm 114 1 - 7 46 -11 R70 KIMBERLEY KIMBERLEY K6 Barkly West Rural Relatively Close Farm Combined Farm 90 1 - 8 32 +3.5 R10 KIMBERLEY KIMBERLEY K7 PHELINDABA LOC. Rural Relatively Close Farm 82 LIMPOPO School Type Roll Grades 2001 Test Result/ Change from 2000 Fees REGION DISTRICT CIRCUIT Rural/ Peri-Urban Distance to District Office Ex Primary 712 1- 7 36 -11.1 R40 CENTRAL POLOKWANE BAHLALOGA Rural Far Primary 523 R - 7 26 +13.3 R80 EASTERN HLANGANANI HLANGANANI CENTRAL Rural Relatively Close NEW EDUC DEPT Primary 65 1- 7 44 +4.7 R60 CENTRAL POLOKWANE KOLOTI Rural Far Primary 308 R- 7 36 +11.5 R50 CENTRAL POLOKWANE BAHLALOGA Rural Far Primary 717 1- 7 42 -16.9 R80 EASTERN HLANGANANI HLANGANANI CENTRAL Rural Far Primary 438 1 - 7 24 +10.6 R70 EASTERN HLANGANANI HLANGANANI CENTRAL Rural Far Primary 260 1- 7 19 +6.1 R50 EASTERN HLANGANANI HLANGANANI CENTRAL Rural Far Primary 602 R- 7 47 -13.9 R77 CENTRAL POLOKWANE BAHLALOGA Rural Relatively Close 83 KWAZULU NATAL NAME/ EMIS # School Type Roll Grades 2001 Test Result/ Change from 2000 Fees REGION DISTRICT CIRCUIT Rural/Peri-Urban Distance to District Office Ex-Dept. Ezimambeni P Primary 243 R - 7 35 +16.9 R20 ULUNDI NKANDLA SIGANANDA Rural Relatively Close KZ Gcinukuthula Sp Primary 326 1 – 7 27 +11.1 R20 ULUNDI NKANDLA SIGANANDA Rural Relatively Close KZ Primary 384 R – 7 54 -14.9 R30 ULUNDI NKANDLA SIBHUDENI Rural Far KZ Primary 436 R – 7 42 -6.4 R30 ULUNDI NKANDLA SIBHUDENI Rural Far KZ Manzamnyama Jp Primary 282 1 – 5 47 -27.1 R20 ULUNDI NKANDLA SIGANANDA Rural Relatively Close KZ Sigananda CP Primary 654 R - 7 36 - 0.55 R27 ULUNDI NKANDLA SIGANANDA Rural Close KZ 84 ANNEX E STAKEHOLDER’S WORKSHOP The Aguirre International Evaluation Team was contracted by USAID to present a workshop to a selected list of invitees. · To assess the lessons learned and best practices of DDSP; and · Based on the knowledge gained, to make suggestions to USAID regarding its future work in basic education Accordingly, the research team held a workshop for approximately 25 invitees at the Colosseum Conference Center in Pretoria on August 28, 2002, from 10:00 a.m. to 3:00 p.m. In the morning, the team presented its findings, and in the afternoon the team led a small group process that focused on answer the question: What should the future of USAID involvement in basic education in South Africa look like? Groups included a member each of the PDOE, RTI, Grantees, and the team. Results of the answer to the question were written on flip-chart paper and shared with all others present. The directions provided in the answers to these questions appear in various sections of the team’s report. The following background information on the workshop is provided: 1. Workshop agenda 2. List of Attendees 3. Results of Small Group Work DDSP WORKSHOP AGENDA DDSP Project Evaluation Feedback The Colosseum August 28, 2002, 10:00 a.m. – 3:00 p.m. TIME ACTIVITY 10:00 Welcome and Opening Remarks Sharon Harpring, Basic Education Team Leader, USAID Introduction to the Workshop Dick Dye, Team Leader 10:30 Research Findings Provinces – Nancy Horn Districts – Jordan Naidoo School Management Teams – Nancy Horn Curriculum – Everard Weber School Governing Boards – Dick Dye Challenges and Responses – Dick Dye 11:30 Discussion 12:15 Lunch Break 1:30 Future Directions – Small Group Work USAID will maintain its interest in basic education in South Africa. The main question we will explore in this session is: What should the future of USAID involvement in education in South Africa look like? 2:00 Report Back and Discussion 2:45 Closing Comments Sibusiso Sithole, Basic Education Project Specialist, USAID 3:00 Dismissal DDSP WORKSHOP ATTENDEES NAME DESIGNATION 1. National Department of Education a) Mr. Duncan Hindle Deputy Director-General, General Education b) Dr. Nomsa Mgijima Chief Director, Quality Assurance c) Ms. Lulama Pharasi Director, EMIS d) Ms. Carol Deliwe Director, Policy Support 2. Provincial Department of Education a) Mr. B.T.M. Mfenyana Acting Chief Director, EC b) Mr. S. Beuzana DM, Francis Baard District, NC c) Dr. Simon Mbokazi Regional Director, Ulundi d) Mrs. M. Segabutla Coordinator & Auxiliary, Limpopo 3. RTI a) Mr. Richard Cartier Chief of Party b) Ms. Saeeda Anis Grants Manager c) Mr. Brian Chinsamy Education Director d) Mr. Masenya Dikotla Deputy Education Director 4. Grantees a) MSTP Pat Sullivan, Director b) READ Educational Trust Evaluations c) LINK Steve Blunden, CEO 5. USAID a) Ms. Kim Bolyard Africa Bureau, USAID/Washington b) Dr. Sharon Harpring Basic Education Team Leader c) Dr. Faroon Goolam Basic Education Specialist d) M. Bunny Subedar Basic Education Specialist e) Dr. Sibusiso Sithole Basic Education Specialist f) Ms. Joanne Lawrence Program Development Specialist g) Ms. Steffi Meyer Program Development Specialist RESULTS OF SMALL GROUP WORK Workshop participants were charged with the task of answering the following question: What should the future of USAID involvement in basic education in South Africa look like? Four sets of responses were generated, one each of four small groups. GROUP 1 The district is the central position for education support. Out of the district comes support for the following. · Focus on community involvement (SGBs) · Flexibility in support needed · Whole district development (all schools in a district) · The department invites grantees to visit department workshops and work together · Link between service provider and district – locate main functions where the service is provided · At the district level assist districts in the implementation of their goals · HIV/AIDs policy implementation · Input of all role players relating to objectives of project GROUP 2 · The point of entry for support depends on province – suggestions made for province/region/district (note: challenge around where real power lies for delivery; note: challenges around structure0 · Focus on INSET for educators, using 80 hours, accreditation (maybe) in modular form – aim to improve content (note: must not be theoretical but practical, raises question of costs and how possible to reach schools in a meaningful way. Question around ways of delivering such information) · Mentorship (coaching) of District Managers to change practice? · Materials (many) in province – integration of these materials into other activities – building an integrated, but not necessarily homogeneous approach, to development · Empower/train leadership that is evident at discrete levels, i.e., province, district, region and school (note: “energized” individuals to change by example and motivation. Building confidence and feelings of importance through “nodal” and influential individuals) · Challenge – linkages of policy from DOE to province to schools. What animal is the district? Centralization vs. decentralization? · Take DDSP to other districts – replicable! · Consolidate and finish what is not yet finished · HR capacity building vs. infrastructure GROUP 3 · Obtain clear provincial buy-in · Undertake capacity building of the province as a whole (HR, Finance, etc.) · Build on strengths: research, dissemination, study tours · Definition of minimal resource base for districts · Focus on management of resources from donors to fill gaps · Issue of donor management · Issue of coordination of different role players · Discussion of RTI vs. QLP approaches to project management from a provincial perspective · Discussion of donor-led vs. externally managed projects · Provincial perspective: donor coordination · Thematic? HIV/AIDs GROUP 4 · National level engagement continual and consistently at a high level · Focus on achievement of provincial objectives · Capacity building at all levels – variety of strategies to suit specific level · Focus on broader community development and mobilization for school support · HIV/AIDs · Different modes of education delivery · EMIS improvement – use of nodal areas – assess · Institutionalization and policy · Development of models for sub regional, district and school development and support GENERAL The groups identified three additional items as a whole: · Depending on the money available, consider scaling up · Consider the focus of donor funds vs. those voted funds at the provincial level · Obtain lessons learned from other projects ANNEX F EDUCATION AND HIV/AIDS INTRODUCTION It is well known that South Africa is suffering through a severe HIV/AIDS epidemic, which is affecting all sectors of the society, not exempting education. Some of the most severe effects on the education system include: (1) loss of trained educators; school managers, and staff through illness or death; (2) learners who have lost their parents, may be caring for younger siblings, and who may be ill themselves, with consequent effects on attendance, dropout, and repetition rates; (3) new kinds of equity and equitable access problems; (4) potentially severe consequences for the quality of education; (5) disruption of long-range plans for education sector human resource development and learner enrollments; and (6) increased pressure on already over-stretched financial and other resources. The fact that the epidemic is affecting the institutions most responsible for educating the future leaders of society, and is having its greatest impact on the most vital sectors of any society – the very young and adult workers in the prime of their lives - makes it all the more tragic and difficult a challenge for the nation. But the effects of HIV/AIDS on the education sector are only part of the story. Another is the challenge to the sector to provide a wide range of educational interventions and work with other sectors, notably health and social services, to mitigate and, in time, make significant inroads on the problem. Improved curricula and teaching in the schools and communities on health and human sexuality, the facts of HIV/AIDS, and related gender and human rights issues are urgently needed, at all levels from early childhood to adult education. Some of the other urgent needs include: Improved educational opportunities for girls and women, especially in at risk sectors of society; a multi-sectoral effort to confront the problem of widespread abuse of girls and women (sometimes in the schools themselves), particularly by older men; and development of increased counseling capacity in the schools to deal with a whole range of new, HIV/AIDS-related problems. To address this range of needs effectively, there is a concomitant need to develop ways to more effectively use the schools, often the natural, and sometimes the only, center of community action, as agents for reaching and involving the communities in a joint effort to meet the challenges of HIV/AIDS, as well as other common problems. Partners in this effort must include the school governing boards (SGBs), local governments, traditional leaders, women’s organizations, and other community leaders. At the national and provincial levels, there is need, as well, for clear, strong, and persistent leadership, a range of supportive policies, and additional, targeted resources. The education sector in South Africa is well aware of and knows that it cannot stand aside from this challenge. Many feel that unless and until medical science comes up with dramatically more effective means of meeting this crisis medically, the best hope for dealing with it, at least in the short run, is through education. It is a view which the team shares, and though it was not part of the team’s scope of work to look at DDSP from that perspective, it decided to do so, at least in a modest way. THE EMERGING NATIONAL EDUCATION SECTOR HIV/AIDS PLAN OF ACTION A Conference on HIV/AIDS and the Education Sector was held in Gauteng from May 30 to June 1, 2002. Participating were: Educators and education officials from across the education sector and from all levels; young people and youth organizations; traditional and community leaders; representatives of non-government and community-based organizations; specialists working in the field of HIV/AIDS; representatives of international development agencies; professional associations of educators; and the university and college community. The Conference subsequently adopted an Education Sector HIV/AIDS Programme of Action. The National Department of Education is reportedly preparing to place education at the heart of the response to the HIV/AIDS crisis, with four main areas of focus: · Prevention: helping to prevent the spread of HIV · Social Support: working with others to provide a modicum of care and support for Learners and Educators affected by HIV and AIDS · Sustaining the Provision and Quality of Education: protecting the education sector’s capacity to provide adequate levels of quality education by stabilizing and protecting the teaching service and responding to new learning needs · Managing the Response to the Crisis: creating executive capacity and setting up structures within the sector appropriate to the extent of this crisis The emergence of these four areas as priorities for the education sector points the way for USAID and other donors in helping determine where their HIV/AIDS resources can best be applied. POTENTIAL ROLE FOR DDSP OR A FOLLOW-ON PROJECT The evaluation team has recommended (see Chapter V of the main report) that a follow-on project be developed to incorporate and carry the DDSP-initiated effort through to conclusion, as well as to take a series of steps to ensure the sustainability and replication of the program. Among its recommendations is one calling for the addition of strong HIV/AIDS components to the new project, in response to the manifest need for them and also the fact that if the epidemic is not somehow curtailed, long-range sustainability of this and other USAID education work may well be threatened. DDSP, in its two years of on-the-ground activity in four of the poorest provinces in South Africa, has helped the South African education authorities build human and institutional capacities that, with relatively modest additional support, could be reinforced and expanded to add HIV/AIDS to the project’s focus areas. Moreover, as DDSP is already a broadly integrated education district and schools development project, including HIV/AIDS within its framework would ensure that HIV/AIDS becomes another part – albeit a crucial one under today’s circumstances – of a holistic model designed to improve the quality of primary education for the country’s disadvantaged children, and not a separate, isolated activity. PRIORITIES In the team’s view, the HIV/AIDS components of the new project should focus on the first two of the afore-mentioned education sector action plan’s areas of focus: Prevention and Social Support. As conceived in the national action plan, they call for intervention and change at the level of the local district, school, and community, which is where the new project, like DDSP before it, will be. As such, they not only would be consistent with other, existing project components, but also would benefit from the new project’s proposed new measures to integrate educational assistance more fully into South Africa’s education system and develop and test innovative, low-cost replication and sustainability models. A partial list of possible activities would include: Assistance, through training and technical assistance (by the districts) to the establishment of new institutions of school-community cooperation, such as the proposed Community Life Skills Committees; training and other support to key district, school, and SGB members in carrying out their responsibilities in the HIV/AIDS area; continued provision of HIV/AIDS advisory assistance to the NDOE and, perhaps, consideration to providing similar, but more focused assistance to the four provincial DOEs; and provision of educational and public information materials. Pilot projects, of course, take time, and there is precious little time available. But, while the country of necessity is carrying out a series of emergency programs to deal with the crisis, someone needs to work on effective, viable, and sustainable longer-range ways to use education to confront the HIV/AIDS crisis and the underlying problems, which contribute to its severity in South Africa. In the team’s opinion that is the natural role for international donors and foundations to play, in a situation where all their available resources combined would scarcely make a dent in the face of the country’s massive needs, but where a measured application of money and wisdom could help point the way forward. USAID, through such measures as provision of senior technical advice in this area to NDOE, support of policy and technical conferences, and encouraging all its education IRs, as well as other sectors, to incorporate HIV/AIDS more into their programs, is already playing an active and important role. If it can and wishes to do more, the proposed follow-on to DDSP could provide an excellent vehicle.