Evaluation of Benin's Basic Education Reform Pedagogical Component Prepared for USAID/Benin Prepared by Charles J. Tesar, Team Leader William Lee Carrie Lewis Nzal Alimasi Dacia Van Antwerp Louis Berger / DevTech Team 2300 N Street, NW Washington, D.C. 20037 Contract No. AEP-I-811-00-00022-00 Final Report April 2003 Evaluation of Benin's Basic Education Reform Pedagogical Component USAID – Benin Louis Berger/DevTech i EXECUTIVE SUMMARY BACKGROUND Pedagogical reform in Benin’s primary schools was initiated more than ten years ago. The reform, known as les Nouveaux Programmes d’Etudes (NPE), began with the design and elaboration of a new program of studies that was tested under experimental conditions in selected schools. During the last four years, the new program of study has been extended to the first four grades of all schools (both public and private) in the country. From the outset, USAID/Benin has supported these activities. The first phase of USAID support known as the CLEF project was completed in 2001 and a new program known as EQUIPE began in 2003. USAID and the Ministry of Primary and Secondary Education have directed that an evaluation of the impact of the reform be conducted. The evaluation process began with the arrival of a team of US-based consultants provided by the Louis Berger/DevTech Group. Members of the group have extensive experience in educational reform in developing nations; and several have been actively involved in the design and implementation of the NPE here in Benin. The goals of the evaluation were the following: 1) assess the impact of the NPE on students, teachers, parents and school administrators in grades CI and CP; 2) evaluate past and current teacher training activities in light of instructional requirements of the NPE; 3) appraise the quality of student measurement and evaluation systems; and, 4) examine the role played by the network of documentary centers with respect to supporting the NPE. In addition, the team was asked to make recommendations to guide the next steps of the reform process, and to suggest strategies for periodic and regular evaluation of progress in the effort. The evaluation team concentrated research efforts on field activities: visiting schools, observing classes, interviewing parents, teachers and school administrators. More than 220 schools were selected for visit from a list provided by the DPP. They were selected on the basis of a stratified, random sampling procedure that assured broad representation among all public and private schools in Benin. Observation instruments and questionnaires were developed to guide the research with the cooperation of USAID and MEPS officials; school visits were conducted in February and March 2003. The team immediately encountered difficulties finding schools in operation. The teachers’ strike had effectively closed school operations throughout the country during the evaluation period, except for selected days (Mondays) and in private schools. Nevertheless, the team was able to visit some 211schools and observe activities in 141 classrooms. Evaluation of Benin's Basic Education Reform Pedagogical Component USAID – Benin Louis Berger/DevTech ii FINDINGS The team confirmed that the NPE program is widely known and appreciated in all schools visited and that materials are being used by teachers and students. We noted also that the great majority of teachers had been trained in methods of the NPE, and that they feel very favorable toward the program. MEPS administrators, including school directors are familiar with the new system, and view the program favorably. The team commends MEPS for disseminating components of the NPE. Findings in the classroom, however, were not so positive. Fewer than ten percent of children can speak, read or write French at the levels expected in NPE materials in CI and CP. We noted also that students in CE1 encounter similar difficulties in French, where about 20 percent operate at required levels of mastery. Since French is the only medium of instruction, and not just a separate school subject, the implications of this situation hold far-reaching, negative consequences for Benin’s children. Without the means to learn, children quickly abandon school or their parents seek out private education. Those who remain in school are promoted without regard to performance; hence upper grades are becoming filled with children unable to function at even the most basic level. Since more than 40 percent of primary school children are in CI and CP, this means that four of ten are at grave risk. Unless actions are taken immediately, it is likely that dropout and school abandonment will rise in the medium term. The cause of this situation has little to do with pedagogical reform or the NPE. These measures are impossible to implement in the environment that threatens primary education. Besides teacher strikes, which have plagued primary education for a number of years, but virtually crippled programming during the last two years, there are other systemic problems that impact on teacher and children performance. Financing for primary education services is in crisis. Not only are teachers striking for higher salaries, the educational infrastructure is suffering for lack of investments. The evaluation team estimated actual expenditures in the schools surveyed and found that expenditures per student are less than half what is allocated in the primary education budget. Experienced and better-trained teachers (APEs) are leaving the profession and being replaced by local-hires, who often come to the classroom with little more than primary school training. Classrooms are inadequate and crowded; an average of 62 children composed classes in CI and CP grades in schools surveyed. There are simply too many children for teachers to manage, and assuredly too many to provide the individual attention that CI and CP students require. Within MEPS, there is no clear organizational structure that is responsible for supporting and sustaining reforms. RECOMMENDATIONS Instability and systemic problems must first be addressed if pedagogical reforms are to advance. Unless schools are in operation, there is little justification or value for Evaluation of Benin's Basic Education Reform Pedagogical Component USAID – Benin Louis Berger/DevTech iii USAID to continue investments in the reform process. Even if teachers return to classes, USAID should condition further investments on the Government of Benin’s commitment to disburse expenditures budgeted for primary education. Similarly, there must be a concerted effort to expand the number of classrooms and hire trained teachers if reforms are to succeed. With respect to the NPE, the evaluation team recommends that USAID and MEPS stay the course that has been chosen, but make revisions in programming. Specific recommendations include 1) Initiate immediate retraining for teachers in CI and CP with the skills needed to accommodate large classes and to teach French. 2) Implement a MEPS-wide system of evaluation, diagnosis and feedback to better inform program planning. 3) Conduct a programmatic audit and reorganization of MEPS to clarify roles and responsibilities in support of the NPE; this should place special focus on the role of INFRE, CGNPE, DEP and the DPP. 4) Provide school directors with the skills to train and support new teachers on-site. 5). Establish a national network of demonstration/laboratory schools that can present teachers with a working model of the NPE. 6) Initiate public-private cooperation to support the private sector, which has made significant progress toward implementing pedagogical reforms. A more comprehensive and detailed list of findings and recommendations follows in the body of the evaluation report. There should be a sense of urgency in implementing these recommendations, since with each passing day, acquired skills and capacities installed through pedagogical reform are at risk of being lost or mitigated. And with each passing week, children are being permanently denied an opportunity to participate in or contribute to Benin’s development. The evaluation team expresses our appreciation to the staff of MEPS, in particular members of the Cellule pour la generalization de NPE, and to the dedicated teachers and school directors who are the heart of the organization. TESAR, Charles, Team Leader ALIMASI, Ntal LEE, William LEWIS, Carrie VAN ANTWERP, Dacia The Louis Berger/DevTech Team members. Evaluation of Benin's Basic Education Reform Pedagogical Component USAID – Benin Louis Berger/DevTech iv TABLE OF CONTENTS I. Introduction II. Methodology III. Context of Pedagogical Reform in Benin’s Primary Education System IV. Evaluation of the New Program of Study for CI and CP V. Findings and Recommendations for French and Mathematics VI. Teacher Training VII. Student Assessment and School Orientation VIII. Assess the Functioning of the Documentary Network Action Plan IX. Analysis of the CLEF Pedagogy Components of the Consultants’ Reports X. Periodic and Ongoing Evaluation Strategies for EQUIPE and MEPS Project Activities XI. Management and Use of Pedagogical Inputs ANNEXES Evaluation of Benin's Basic Education Reform Pedagogical Component USAID – Benin Louis Berger/DevTech v Abbreviations APE Association des Parents d’Elèves APE Agent Permanent de l’Etat BAC Baccalaureat, the Secondary school leaving exam BEPC First cycle of secondary school degree BET Basic Education Team, USAID/Benin’s Education Strategic Objective Team CAAP Centre d’Appui aux Activités Pédagogiques CEAP Certificat Elémentaire d’Aptitude Pédagogique CAP Certificat d’Apprentissage Professionnelle CD Centre de Documentation CDDIP Centre Départemental de Documentation et d’Information Pédagogiques CDDP Centre Départemental de Documentation Pédagogiques CDE Centre de Documentation d’Etablissement CDIP Centre de Documentation et d’Information Pédagogiques CEC Centre d’Etude Communautaire CI Cours d’Initiation CLAC Centre de Lecture et d’Animation Culturelle CLEF Children’s Learning and Equity Foundation CLP Centre de Lecture Publique CM Cours Moyen CNDIP Centre National de Documentation et d’Information Pédagogiques CNDP Centre National de Documentation Pédagogiques CP Conseillers Pédagogiques CP Cours Préparatoire CCS Chef de Circonscription Scolaire CS Circonscription Scolaire DAPS Direction de l’Analyse, de la Prévision et de la Synthèse DEP orDDEPS Direction Départementale des Enseignements Primaire et Secondaire DPP Direction du Planificqtion et de la Prospective DRH Direction des Resources Humaine ENI Ecole Normale des Instituteurs ENI Ecole Normale Intégrale GOB Government of Benin IFESH International Foundation for Education and Self-Help INFRE Information Nationale Formation et Recherche pour Education MEPS Ministère des Enseignements Primaire et Secondaire MESRS Ministère de l’Enseignement Supérieure de la Recherche Scientifique NPE Nouveau Programme d’Etude ONG Organisation Non-Gouvernementale RUP Responsable de l’Unité Pédagogique UP Unité Pédagogique UNB Université Nationale de Bénin Evaluation of Benin's Basic Education Reform Pedagogical Component USAID – Benin Louis Berger/DevTech 1 I. INTRODUCTION The Ministry of Primary and Secondary Education (MEPS) launched a program to reform the primary education of Benin in 1992. Implementation of educational reform has proceeded in the form of a New Program of Study (NPE), which comprises interventions in the development of a new curriculum; elaboration, printing and distribution of instructional materials; and teacher and administrator training initiatives. The United States Agency for International Development (USAID) has supported these initiatives with technical assistance and financial resources provided through the CLEF project, which has aided the Ministry’s Cellule de Généralization des Nouveaux Programmes d'Etudes (CGNPE) in the project execution. With the termination of the CLEF project and the initiation of a new operation, EQUIPE, USAID will continue to support MEPS to complete the introduction of the NPE to the upper two grades of Benin’s nearly 5,000 primary schools. The immediate goal of USAID support is to underwrite programming to extend the new program of study through CM1 and CM2 during the next two years, and to consolidate educational reform in all six grades of primary education. Background The educational reform processes began with the design of action plans, followed by the elaboration of curricular and training materials for the first two years of primary schooling (CI and CP). After designing these instructional programs, MEPS selected a cohort of 30 experimental schools to validate, revise and test the various curricular and training components of the reform. The following year, elements of the new program were prepared for CE1 for testing in the same group of experimental schools. During each succeeding year, the program was prepared and tested for the next grade level. As the materials and methods were tested and revised, the program was introduced to a larger group of 180 schools (impliquées) for further testing and revision. Beginning in 1999, the program for CI was replicated and extended (generalized) throughout the entire nation’s 4,500 or so primary schools. This was followed in Y2000 with the extension of CP training and materials to all primary schools. The new program of study was extended to CE1 and CE2 in 2001 and 2002, respectively; replication is planned for CM1 during the summer of 2003, and CM2 during the summer vacation of 2004. Four years have passed since the generalization of the new program of study to CI and three years have lapsed since the CP program was introduced throughout the country. USAID and the MEPS have determined to measure the impact of the program on student and school performance. In January 2003, the Louis Berger/DevTech consortium was selected by MEPS and USAID to conduct an intensive evaluation of reform effort. The purpose of the evaluation was to assess the outcomes of the reform process with respect to its impact on student performance; notably, improved skills in French and Mathematics, and acquisition of skills and attitudes in the Arts, Sciences, Civics and Physical Education. The overriding goal of the NPE is to improve children’s academic Evaluation of Benin's Basic Education Reform Pedagogical Component USAID – Benin Louis Berger/DevTech 2 abilities; life and social skills; to reduce grade repetition and school abandonment; and to improve children’s prospects for entering and completing secondary education. The Berger/DevTech evaluation team, comprising five U.S. citizens, began work in late January and completed the project in April 2003. The process began with extensive consultation with USAID and MEPS staff to determine the most appropriate approach to conduct the study. A field-based methodology was used to execute the evaluation. The approach included visits to some 200 schools, based on a sampling procedure that assured that all departments of Benin were included, along with experimental and private schools. Terms of reference for the evaluation included: 1) Assess the pertinence and quality of the overall curriculum for CI and CP; 2) Analyze the French and Mathematics programs; 3) Assess the quality, pertinence and timeliness of the teacher and administrator training program; 4) Assess the systems and methods used for student performance measurement; 5) Assess the effectiveness of a documentary network designed to support the reform effort. 6) Review the extent to which the reform process has integrated recommendations from the various consultants who provided technical assistance to the program during the last ten years. 7) Design an evaluation system to be implemented by MEPS; 8) Propose recommendations in all the above areas to be implemented by MEPS with support from EQUIPE. . Goal and Objectives The goal of the evaluation was to assess the progress made towards the implementation of reform measures in Benin Primary Education. The ultimate outcome of these measures, as expressed in USAID and MEPS programming, is to improve student performance and success, reduce retention and dropout, and equip children with skills to complete primary, as well as succeed in secondary education. The evaluation terms of reference also contained 11 specific objectives: i. Assess the first (CI) and second (CP) grades curricula (coherence with the reference documents, vertical and horizontal coherence); ii. Assess the conformity of the Math and French workbooks and textbooks, French wall charts, pedagogical guides with the approaches, strategies, and training contents as recommended by the new programs; iii. Assess the conformity of all the training documents and activities with the training plan on the one hand, the learning objectives, approaches and strategies as recommended by the new programs on the other; iv. Assess the conformity of the new system of student assessment and school orientation with regard to the learning objectives, approaches and strategies as recommended by the new programs; Evaluation of Benin's Basic Education Reform Pedagogical Component USAID – Benin Louis Berger/DevTech 3 v. Assess the functioning of the Documentary Network action plan; vi. Assess the recommendations of the Pedagogy component consultant reports funded under the CLEF project within the framework of activities directly related to the generalization of the new programs and their implementation; vii. Develop, in collaboration with “pilots”, MEPS and USAID technical staff, a periodical evaluation plan which will help assess the quality, relevance, progress achieved and the results obtained by all five components; viii. Assess the performance of first grade (CI) students on the basis of a representative sampling; ix. Assess the performance of first and second grade (CI-CP) teachers and school directors on the basis of a representative sampling, within the framework of the implementation of the new programs; x. Assess the management and use of all the school pedagogical inputs on the basis of a representative sampling; xi. Give recommendations for improving the implementation of the reform program. The report begins with a discussion of the methodology used in the study. Next is an overview of the social, political and economic conditions that affect educational reform in Benin. Following is a description of the specific findings, conclusions and recommendations for each of the subcomponents of the reform. These recommendations are prioritized according to the short, medium and long-term needs. In a subsequent section, we present a proposed plan for periodic and systematic evaluation of reform activities. Finally, a review of previous consultant activities and recommendations is presented. Support materials are included in the appendix. II. METHODOLOGY The evaluation team began work in Benin by soliciting information, perspectives, and opinions from MEPS and the USAID/Benin Basic Education Team (BET). Within MEPS, this included the pilotes who have been responsible for developing and extending the NPE, and who comprise the CGNPE. Members of the advisory team also included representatives of the MEPS’ Department of Primary Education (DEP) and the National Institute for Training, Evaluation and Research in Education (INFRE). Following meetings with members of the MEPS/USAID advisory team, the evaluation group met with key staff in MEPS, including the Minister, the departmental director of Primary Education (DEP) and staff of the Department of Planning (DPP). The evaluation team also met with a broad sample of MEPS personnel working at the department level. Interviews were conducted with all six of the Departmental Directors, eleven Chefs de Circonscription Scolaires, (C/CS) who oversee regional Evaluation of Benin's Basic Education Reform Pedagogical Component USAID – Benin Louis Berger/DevTech 4 activities, and more than 80 of their support staff, Conseillers Pédagogiques (CP). In addition, evaluation team members were able to observe field training underway for 80 newly designated conseillers pédagogiques, and several training activities for teachers. The first step in the evaluation process was a review of all teacher guides, student workbooks and manuals, and all project related reports. Information obtained from these documents provided an initial analysis of program inputs, particularly with respect to premises used to support material and training development. This data provided a benchmark against which to measure actual practices. The team was able to meet with and interview parents representing local school councils (Bureaux des Associations de Parents d’Elèves APE) in more than forty communities, and with several regional and the national federation of APEs. SCHOOL VISITS Sampling Since the overriding goal of the assessment program was to gauge the impact of reform efforts on children’s academic performance, field visits and classroom observations were the primary focus of the evaluation process. A representative sample of the nearly 5,000 primary schools in Benin was prepared from a list of public and private schools provided by MEPS Department of Planning (DPP). The school sampling process controlled for the following variables: 1) rural-urban; 2) public-private; 3) geographic considerations (schools in all twelve departments were included); and, 4) extent of participation in the reform program (experimental schools versus those in the generalized category). The original work design called for analysis of a sample of schools that would meet measures of statistical significance at the .005 levels. Given the limited knowledge of conditions in Benin schools, a random sample of some 200 schools (five percent of primary institutions), was selected for visit and analysis. Upon commencing school visits, it became immediately apparent that the teacher strike that began in October 2002 would cause significant logistical problems in conducting on-site visits. Since most teachers are on strike during midweek, it was necessary for the team to split into five groups to conduct the maximum number of visits on Mondays and Fridays. Using this approach, the target of 200 school visits was achieved (Annex). Instrumentation Two instruments were prepared to structure activities during school visits. A school questionnaire (annex 2.II was administered to each school director and whenever possible to APE members.] The school questionnaire probed the extent to which the NPE is implemented at the school level. The instrument also sought to elicit attitudes of school directors toward the program, along with perceptions held by teachers and parents. In addition, interviews with school directors sought to determine the degree to which curricular materials are delivered to schools. Finally, during the administration of the questionnaire, the evaluation team sought to gauge the role that directors have played in implementing initiatives of the NPE in their respective schools. The school questionnaire was administered to 186 school directors and 41 parent groups. The second and perhaps most important instrument used in school visits was a classroom observation checklist (Annex III). A major element of the NPE is the goal Evaluation of Benin's Basic Education Reform Pedagogical Component USAID – Benin Louis Berger/DevTech 5 of transforming teachers from dispensers of information, relying on the intermédiaire method of rote, frontal instruction, into learning facilitators. In this new role, teachers are expected to use interactive, multiple group, and objectives￾based teaching techniques. The purpose here is to accept each child at his/her own learning rate, with the expectancy that all children in class can reach a given instructional objective if given enough time and the proper learning approach. Fundamental to this approach is the need to understand and adhere to teaching guides and student instructional aids, the chief among which are student workbooks (cahiers d'activités) and manuals. The NPE generalization effort calls for each child to have a workbook both in French and Mathematics in CI and CP (provided free of charge with USAID financial support), and for teachers to follow a prescribed lesson program using a range of suggested learning activities. USAID and MEPS have underwritten annual training programs for in excess of 4,500 teachers every summer during the last four years. Given the complexity of the NPE, the teacher training programs have included 24 days of intense training during the school vacation every year since 1999. The classroom observation instrument was designed to measure the extent to which teachers had adopted NPE methodologies and to determine the extent of their training and the use of NPE materials. In addition, the observation instrument was used to gauge the performance of children in French and Mathematics, the two pillars of the reform. All members of the evaluation team carried out administration of the instrument. Whenever possible or feasible, they were assisted by MEPS officials, most notably by Conseillers Pédagogiques and Chefs de Circonscription Scolaire (C/CS). There was no notification given to selected schools in advance of the visit, to ensure that the team would encounter routine activities that were not pre-planned in anticipation of the team’s arrival. During the classroom visits, which focused on CI and CP classes, teachers were asked about the level of participation in summer training programs, the availability of student and teacher guides and workbooks, acceptance of the NPE and problems associated with implementation of the NPE. After a brief interview, teachers were asked to continue with the instructional program. During this phase, observers noted the general teaching style, the amount of student initiation and interaction, the organization of the room, and the availability and use of NPE materials and learning resources. Whenever possible, the evaluation team visited both CI and CP classes in each school. As school visits progressed, it became apparent that teacher strikes during the last two years have had an enormous negative impact on student performance in CI and CP. In many schools, teachers have held fewer than 20 days of class this school year. In most cases, schools have effectively been without teachers during this calendar year. The lack of student learning in CP this year augments the consequences of a prolonged strike and disruption of the last school year. Consequently, few if any children in CI or CP have received sustained, regular teaching during the past two years. In an effort to obtain a truer measurement of the impact of NPE on student performance, researchers visited classes in CE1, which are composed of students who began NPE after generalization and before disruptions of the strike. (We later learned that there were periodic and on-going teacher strikes since before the NPE was generalized.) Evaluation of Benin's Basic Education Reform Pedagogical Component USAID – Benin Louis Berger/DevTech 6 Although MEPS and evaluation team members acknowledged that the first years of generalizing the NPE program encountered many logistical and programmatic difficulties, it was decided that a rapid analysis of student reading and math skills in grades 3 and 4 (CE1 and CE2) would provide insight into the outcomes of the NPE methods. Beginning in the third week of school visits the classroom observation checklist was expanded to elicit the state of language acquisition among students in their third year. Whenever possible (in about 30 percent of schools visited) members of the evaluation team talked with teachers and students in CE1. A total of 150 classrooms were observed during the evaluation period. III. CONTEXT OF PEDAGOGICAL REFORM IN BENIN’S PRIMARY EDUCATION SYSTEM In Benin, primary education is provided by the departments in the Ministry of Primary and Secondary Education (MEPS), which is one of three separate public education ministries. Any effort to evaluate the quality and impact of educational reform measures launched by MEPS and supported by USAID must necessarily take into account the operational environment that has shaped the process. Following is a description of the gains achieved to date in implementing the reforms. In addition, we discuss the economic, social and political forces that impede implementation of pedagogical reforms, many of which are beyond the reach of USAID and MEPS. More detailed findings and recommendations are presented in the following sections that deal with each of the sub-components included in evaluation process POSITIVE ACCOMPLISHMENTS IN PEDAGOGICAL REFORM We begin our analysis of the state of pedagogical reform on a positive note: MEPS and USAID have made formidable strides in improving primary education in Benin, and under the most adverse of conditions. The evaluation team commends those involved with the reform process for accomplishing major changes in the primary education infrastructure and for efforts to implement the NPE. Several members of the evaluation team have been intimately involved with the reform process, having assisted in the initial design, or serving in consultative capacities during the last ten years. Before the reform, public school instruction was in a lamentable state. Children had no learning materials or aids; teachers lacked pedagogical skills and were often unpaid; schools were dilapidated, and lacked any support or direction; and the Ministry had little presence in the countryside. Today, school facilities are vastly improved; most classrooms, while crowded, provide an environment where learning is possible. More than 20,000 teachers and school directors have received initial and in-service training. Remarkably, (and in contrast to reforms in nations similar to Benin) teachers and directors have accepted the basic tenets of reform and speak highly of the NPE. For the first time, there is a national curriculum, established teaching methodologies, consistent school direction, and an administrative overlay that holds much promise for carrying educational reform forward. More importantly, USAID/MEPS instructional and didactic materials are available to children in all schools, even in the most remote rural communes. This reflects Evaluation of Benin's Basic Education Reform Pedagogical Component USAID – Benin Louis Berger/DevTech 7 the quality of logistical services provided by MEPS. Where properly used, teacher guides and children’s’ workbooks provide an exceptional foundation for better performance. During the last ten years, there is a marked improvement in the state of organization in schools; school directors have received training and improved supervision is apparent. With USAID assistance, MEPS has created an educational supervision and support structure that has been able to deliver reform measures to all schools in the country. A network of school inspectors regularly visits schools; and an information management system is used to document educational statistics that provide a relatively accurate description of the education system. The reform has assisted in the creation of a framework for local in-service training (unités pédagogiques), that is widely praised by teachers. This same framework offers broad possibilities for consolidating gains of the reform and institutionalizing a support system that could continue reforms after USAID participation has ceased. In short, there are dramatic improvements in MEPS capacity to deliver basic education and an infrastructure in place to carry reforms forward. Finally, it is worth noting that reform measures are achieving a significant level of success in private schools. A more detailed description of the factors that contribute to these developments is provided in another section of the report. IMPEDIMENTS TO THE REFORM PROCESS SYSTEMIC INSTABILITY Teacher Strikes Prolonged and intense teacher strikes during the last few years have had a devastating effect on Benin’s primary school system. Strikes disrupted classes in the 2001-2002 school year and required an extension of classes into the summer to avoid having the term declared an année blanche (year of no credit). The strikes began again with varying degrees of participation early in the 2002-2003 academic year and have continued through March 2003. In most of the nation’s schools, teachers classified as contractuels went on strike immediately after the school year began, in October 2002. Then the largest teacher group (APE / civil servants) walked out in January. Only locally hired teachers (communautaires) continued to show up on a regular schedule. Officially, schools were functioning on Mondays and Fridays, but for the most part, teachers took part in strike activities on Fridays. In the meantime, student attendance dropped precipitously and structured learning all but ceased. Teacher absence from schools effectively paralyzed school operations and had an especially deleterious impact on educational reforms. The evaluation team saw little evidence, in terms of student performance, of the impact of the new program of studies (NPE). The team attributes this largely to the fact that nearly a third of the teachers who participated in last summer’s USAID-sponsored training program went on strike at the outset of the school year and were joined by an even greater number of teachers at the end of the fall term. DISCUSSION It is outside the terms of reference of this evaluation to analyze causes of teacher strikes or solutions to the problem. It should be noted, however, that the net e ffect of these ongoing disruptions has been to render the public education system Evaluation of Benin's Basic Education Reform Pedagogical Component USAID – Benin Louis Berger/DevTech 8 dysfunctional. Gains made through the reform program are at high risk, as is the welfare of the children of Benin. Without proper teaching, children are not learning to read, write or speak French, the central language medium of Benin’s educational system. Inadequate Public Finance Teacher strikes are only one symptom of an educational system in deep crisis. Public financing is not sufficient to meet the basic needs for pr imary education. According to MEPS sources, the 2002-2003 annual budget for primary education is CFA 37.4 billion. Stated in US dollars this comes to about $40 per child. The budget also includes significant investments (approximately CFA 2.5 billion in support of pedagogical reform measures.) The evaluation team had difficulty verifying that funds budgeted for primary education are actually being spent in the schools visited. Using as a crude formula for calculating annual expenditures per student (dividing the average annual teacher salary (CFA 650,000 or about US$1,050) by the average number of students in class (62 in the 211 schools analyzed) we estimated direct instructional investments of CFA 10,400 ($17) per student.) To this figure (CFA 10,400 or $17) we add 30 percent for indirect costs, investments and administration, for an average cost per student of CFA 13,500 or about US $22 per annum for children in grades CI and CP. Extending the per pupil investment rate for CI and CP to all primary school grades, the Government of Benin invests less than US$ 27 million annually on the 1.138 million children in primary education. There is a significant difference between budgeted and actual expenditures. The World Bank’s recent report on Benin’s educational system Le système éducatif béninois, (World Bank, January 2002) noted MEPS has consistently under spent funds allotted for education, so this is an ongoing problem. GOB expenditures are further reduced when considering that a very large percentage of costs for the reform: teacher training, printed materials, instructional support, and school construction are borne by USAID and other international donors, or by parents. Due to these inadequate investments, there are not enough classrooms and teachers in the public school system. Students must study in classes that often exceed 100 pupils; in the schools analyzed in this study, the mean enrollment was 62 in CI classes. In many instances, the evaluation team noted that school directors must turn away incoming students in CI classes due to lack of space and/or teachers. Inadequate finance contributes to low poor salaries. These, in turn, are the principal cause of teacher discontent and a major deterrent to employing better￾qualified teachers. As a result, a growing percentage of teachers (les communautaires) are hired and paid by local parent associations using a combination of public and local funds. These teachers are poorly educated and motivated since parents cannot guarantee even a subsistence-level salary (the average wage for locally hired teachers is less than US$50 per month). Lack of public investment has increasingly forced parents to rely on private schools, which in many cases do not meet the minimum certification requirements of MEPS (clandestine schools). The Departmental Director of Atlantique/Littoral, for Evaluation of Benin's Basic Education Reform Pedagogical Component USAID – Benin Louis Berger/DevTech 9 example, estimates that as many as fifty percent of students in urban areas attend private and or clandestine schools. Elsewhere, especially in rural areas, the evaluation team noted a high incidence of students who have either never enrolled in school, or dropped out at an early age. DISCUSSION While it is outside the parameters of this study to evaluate public educational finance, we note the pattern of under-spending identified by the World Bank as an on-going problem, particularly as it affects implementation and sustainability of the NPE. Unless the GOB is willing to make a substantial and sustained commitment to the reform effort underway USAID should explore with MEPS alternate options to promote the education of children based on the NPE. One approach may be to explore an alliance with private education. As we noted above, private schools have demonstrated capacity to implement elements of the NPE and ARE succeeding in causing learning. A shift of support to the private sector has been an option regularly and effectively used by USAID missions to sustain primary education in the absence of public sector resources. Some success has been achieved using this method in nations comparable to Benin such as Haiti, Albania and Nicaragua. MEPS would also benefit from studying the public-private sector relationship in Cote d’Ivoire, which extends public resources to private schools based on enrollment. Another option is to shift investments to local parents in an effort to accelerate decentralization. This could take the form of financial and technical assistance toward the creation of a community-based, and local-government supported school system. The MEPS might continue to play a support role in such a local-private￾public partnership. If USAID prefers to continue supporting public sector programs, future investments should be conditional upon increased GOB financing for primary education. At a minimum, the national primary education budget must be significantly increased, not just in centralized financial accounts that may bear no reality to expenditures in schools, but in actual investments. Above all, MEPS must insure that schools are open, teachers are paid and at work, and the sector is in operation. Student Enrollment and Class Size The number of students enrolled in CI and CP classrooms makes it impossible to teach to each individual child. Educational research has documented that foreign language acquisition is one of the most difficult of instructional tasks, and that effective learning of a new language can rarely occur in classes with more than 15 students, and then only with the aid of a well trained instructor. Since the number of students in CI classes averages more than 60 pupils, students do not learn to speak, read or write in French. Therefore, they cannot complete the primary cycle and are denied access to secondary education. In the long-term, more classrooms and teachers are required to reduce class size to manageable proportions. In the short-term, more instructor support is required. Additional training programs are urgently needed to equip teachers with more language instruction skills, and to manage larger groups of children. There is an absolute necessity to provide more individualized contact with French-speaking aides or tutors in the early grades, so that each child can learn the language of instruction. Evaluation of Benin's Basic Education Reform Pedagogical Component USAID – Benin Louis Berger/DevTech 10 DISCUSSION Several methods exist to improve the language capability of children in the first grade. MEPS could for example use strategies for bilingual instruction that would provide students reading and writing skills in a language they already know. This methodology, advanced by the Education Sector of the World Bank, aims to bring pupils to a pre-established level of literacy competency before beginning instruction in the second language, which in the case of Benin is French. The method is used extensively in multilingual nations, most notably, in Nigeria, Chad and Ghana, and holds much promise for equipping children with fundamental language skills to succeed in upper grades of primary education. If the present French-language immersion policy is continued, teachers must be provided with assistance and teacher aides who can work with smaller groups. This could be accomplished by dedicating all teachers to working with incoming and/or French language-deficient students at the outset of each year until a universal level of competency is achieved. Such an approach would require all teachers, irrespective of grade assignment, to work with a target group of 15 students on an intensive basis at the beginning of each school year. They might only dedicate one or two hours per day in language instruction before returning to their regular assignments, with follow-up and remedial instruction for those who fail to master the prescribed level of expertise. Another approach would be to provide part-time teacher aides, such as older siblings, parents or upper-grade and secondary school students. These services could be supported by government financial grants provided to schools under the World Bank debt relief program. Responsibility, Leadership and Accountability The NPE advances the notion that teachers should assume the responsibility for causing learning among students. The tenet is based on the belief that most students can accomplish a given instructional objective if provided with enough time and the proper learning activity. For this practice to be applied at the classroom level, teachers must be held to standards of accountability for student success. By extension, school directors and administrators must be willing to support teachers and the learning environment toward the same goal. The evaluation team noted an absence of accountability and responsibility in the primary educational system. In many instance, teachers were unaware of student shortcomings, or resigned to the fact that 80 to 90 percent of students are not learning, much less actively participating in the learning program. School directors have adopted a similar attitude, placing blame for the situation on lack of funds, teacher strikes or poor parental support for learning. Upper level administrators take little notice of the state of instructional affairs. In those instances where problems are recognized, the familiar attitude of acceptance, resignation, or indifference persists. When probed for possible responses or solutions to remedy the situation, there was uniform response among teacher and administrator respondents that such problems are the cause of the strike, too large classes, or endemic to the system, and beyond the control of the respondent. Part of the predicament lies in the lack of any systematic or valid diagnostic or feedback system. The only systematic testing program in primary education is a Evaluation of Benin's Basic Education Reform Pedagogical Component USAID – Benin Louis Berger/DevTech 11 summative examination, the Certificat d'Etudes Primaires (CEP) that students take after the 6th year of instruction. The evaluation team noted that 27 of 100 students who enter primary schools typically pass the CEP in rural areas, while at the national level it is approximately 40 percent. Currently, the CEP is being criticized by secondary school administrators, who assert that the test does not adequately measure student skills. Central to this problem is a system-wide neglect of testing and measurement strategies. There is little indication that teachers acknowledge the plight of students who do not read, even though they are in the majority of the first four grades. We found no evidence of student testing and assessments during the school year. Most teachers and directors blamed the lack of testing on the teacher strike. Classroom observations confirmed that teachers tend to concentrate on the brightest students. Those who do not speak French are ignored or neglected. School directors manifest little concern, and rarely take remedial measures to correct the situation. During school visits, the team noted a high level of frustration among directors, but a marked absence of any proposed solutions. MEPS inspectors and pedagogical counselors visit schools but seem to leave little substantive methods, skills or expectations for higher student performance in place during their rounds. We found few indications that inspectors or supervisors had taken steps to deal with the levels of student failure, even among the worst performing schools. DISCUSSION The evaluation team appreciates that an organizational analysis of MEPS is called for as a task in the EQUIPE project, and that its recommendations might provide the basis for a more responsive primary educational program. The institutional audit would be of value to define roles, goals, and administrative structures that are currently unclear due to a dispersal of responsibilities for implementing educational reform, and for monitoring student performance in general. The objective of this audit would be to recast the ministry as a responsive, dynamic learning organization that is committed at all levels to teaching the children of Benin. One solution to this problem of accountability, leadership and evaluation may be to incorporate a performance-based management system. Such a system would integrate MEPS-wide, continuous diagnosis and feedback processess. The focus would be on causing better student performance, through better teaching, administration and Ministry support. Students, teachers and directors would be recognized and compensated for superior performance. Diminishing Corps Of Quality Teachers There are approximately 22,000 teachers in Benin, and a dwindling percentage has had any academic training in pedagogy. The number of qualified teachers is rapidly diminishing as they leave the system for other work or to retire. Few qualified personnel are available to replace them; as a consequence their posts are being filled by less qualified persons. There is only one pre-service teacher-training institute in Benin and its total enrollment is 80 students. Yet it is estimated that as many as 20,000 teachers are needed in the medium term, even more if classrooms expand to take on prodigious levels of student growth, and the student-pupil ratio is reduced to manageable Evaluation of Benin's Basic Education Reform Pedagogical Component USAID – Benin Louis Berger/DevTech 12 levels. In-service teacher training programs consist of those that were provided during the generalization process, and initiatives to use as teacher trainers through the unités pédagogiques. These efforts are insufficient to accommodate the growing numbers of children in school and demand for more teachers. Not enough teachers are being recruited and trained to support a system that seeks to provide universal enrollment and effective student learning. Nor are they sufficient to consolidate the limited gains made as the result of reform. DISCUSSION It is clearly in the interest of USAID and MEPS to establish an organized program to meet pre-service and in-service teacher training needs. Most nations rely on ministry-recognized training provided by external agencies, as public ministries have historically been ineffective as teacher training agencies. Ideally, the task of training new teachers would be assigned to a university, specialized professional development institute, or any other organization that articulates the interests of students, MEPS, teachers and parents. Whatever institutional vehicle is selected, training should be offered to teachers at the local level and should require a minimum of disruption in the teacher’s work or leisure schedule. In many nations, special university credit programs are offered at little or no cost by public universities, with university credit and professional advancement being the principal incentives for teachers to attend. With respect to training toward professional development, a systematic training program should be linked to levels of certification and competency, and serve as a principal criterion in a new salary structure for teachers. A redefinition of teacher promotion and salary increment criteria would provide an opportunity to eliminate the artificially imposed teacher categories of APE, contractuel and communautaire, and define a more rational basis for teacher compensation and assignment. Such a merit system could recognize individual teacher achievements toward adopting NPE methods, and assure that every teacher and school director would receive minimal training in reform pedagogical strategies. In the long term, the training program should be linked to specific competencies needed by schools, and respond to staffing needs in all sectors of education. CIVIL SOCIETY AND THE REFORM PROCESS Parental and community participation Educational research has identified the involvement and concern of parents, families and friends as the most important predictor of student success. This thesis holds special importance when children are trained in a language other than the one spoken at home. Parents oftentimes learn alongside their children, but above all they provide critical support for the internal discipline that children need to acquire another language. Despite the value of parents as tutors, counselors, co-learners and supporters, they have not been effectively utilized as an instructional resource in the NPE program. The main reason for this appears to be that many parents are uneducated and non￾French speakers. Instead, the NPE places highest priority on learning in school, to the extent that children are not allowed to take their workbooks home. Parents noted that they are invited to visit with teachers only in the event of a discipline problem, and that they are not oriented to the NPE or any other aspect of school Evaluation of Benin's Basic Education Reform Pedagogical Component USAID – Benin Louis Berger/DevTech 13 affairs, except when they are called upon to pay dues or help with some construction project. Teacher Unions Teacher organizations obviously possess the means to bring the primary education to a standstill. Although they serve a recognized role in civil society, their actions are often perceived as adversarial and detrimental to school operations and the implementation of change. In some countries, however, education ministries have learned to collaborate with teacher unions with surprisingly productive results. In Paraguay, for example, unions have taken on the task of training teachers through an apprenticeship program. Teacher cooperatives work with the education ministry to recruit new teachers, deliver training and certify skill levels. A liability in this instance has become a resource. DISCUSSION Although future USAID programming is expected to support decentralization of educational management and stimulate the involvement of civil society over the long-term, there are a number of issues that merit immediate action. Student workbooks and textbooks will soon be available in the market so that in future, a copy can be kept at home to help students. But there should also be an immediate policy to allow children to use books at home use this year as well. In addition, teachers should encourage parents and family members to visit schools on a regular basis, especially those who might be able to serve as community resources to work with children who are behind in their study. Though few parents in the rural areas speak French, other family members could assist with instructional tasks, especially older siblings and neighbors who attend secondary school. Because schools lack the audiovisual tools to promote language acquisition, parents may be able to expose children to French radio or TV programming. At the least, they can find print material for home use. Parents could also ensure that students complete their homework, receive assistance when the need arises, and help teachers with any chore that might make the classroom experience more productive. Role of School Directors in the Implementation of Reforms As we noted earlier, the NPE has consisted primarily of extending newly created curricula and didactic materials to teachers through a short term, summer training programs. During these training sessions, most school directors were exposed to the NPE; many have in fact served as trainers in the annual training. But the majority have only limited understanding of the tenets of the NPE, and even less knowledge about how to maintain levels of support that are required to consolidate the innovations. It is well documented that educational change is rarely successful without unswerving commitment from school directors. They oversee teachers’ daily activities, know all students, and have frequent interactions with parents. During school visits, the evaluation team found that very few directors are well versed in the application of NPE methods, evaluation and measurement techniques, learning theory, and supervision strategies. They are often unable to provide even the most basic counseling and supervision to teachers, students or parents. Evaluation of Benin's Basic Education Reform Pedagogical Component USAID – Benin Louis Berger/DevTech 14 Teacher strikes are but one of many actions over which directors have no control. Nor are they involved in hiring, assigning or firing teachers. They profess to have little say as to when or whether the Ministry provides handbooks, manuals or materials. During discussions with directors, the team noted that most feel a total lack of control over their school and do not fully understand their institutions’ mission or purpose. DISCUSSION Action is needed to render school directors more accountable for school investments, facilities, resources, personnel and, above all, for student success! Any reorganization of teacher salaries and terms of employment should place directors in a position of accountability and leadership, with respect to teacher certification, appraisal and personnel actions, and school performance. It is essential that school directors receive training in educational administration. This might be carried out through weekly training at demonstration or pilot schools. These same demonstration sites could also serve as teacher training facilities in support of the unités pédagogiques, the bimonthly program that brings teachers to local schools for professional upgrading. Whatever medium is used, directors should be exposed to concrete examples of dealing with school improvement, not theoretical discourse. They must learn hands-on how to deal with poor learning, and acquire the skills to do in-house staff development. Evaluation, Experimentation and Demonstration Although the first years of the reform involved almost four years of program design, testing, revision, and experimentation, this process has not carried over into the generalization phase of the NPE. The team noted that few efforts are made to measure the impact of the NPE on student, teacher and school performance, or to assess the satisfaction of parents. The lack of feedback and diagnostics reaches all levels within MEPS. There has been limited systematic assessment of the level of understanding, adaptation or application of the NPE since the generalization process began in 1999. Nor have there been any recent efforts to revise the program to accommodate new and less￾trained teachers. The evaluation team noted that MEPS officials were not completely informed of the state of learning in schools, or if they had been, were taking no measures to correct the problem. DISCUSSION USAID and the MEPS should initiate consistent and rigorous measures to study student and teacher performance. These should incorporate regular, on-sight inspections by MEPS officials, independent and objective student testing and systematic responses to those diagnostics. Programs need to be mobilized and flexible to respond to these findings. There is a need for continued experimentation, demonstration programs and model schools to make available the best practices of the NPE. Most teachers and directors have only learned of the system during summer training sessions, and have had limited opportunities to apply NPE strategies in class. The evaluation team also recommends the initiation of more experimentation with alternate language instructional approaches, such as bilingual instructions, peer teaching and the use of teacher aides to assist teachers. Evaluation of Benin's Basic Education Reform Pedagogical Component USAID – Benin Louis Berger/DevTech 15 One approach is to designate at least one school in each Department as a demonstration/laboratory school. Here the very best teaching methods, and administrative support and practices should be implemented. Demonstration schools might be at the site of newly redesigned documentary centers. These demonstration schools/documentation centers would provide programs to teachers, directors and parents in conjunction with demonstration school operations. Demonstration/laboratory schools could also support pre-service training: regional campuses where new teachers could work as interns. These schools could also enhance the training of unités pédadogiques, which currently pursues no systematic themes or program, but which is nevertheless well attended by teachers and directors. Model schools would be rich with materials, methods and teaching ideas, all of which could be exported to other schools with reproduction services provided through a redesigned network of documentary centers. IV. EVALUATION OF THE NEW PROGRAM OF STUDY: CURRICULA FOR COURS D’ INITIATION (CI) AND COURS PREPARATOIRE (CP). Terms of reference i. Assess the first (CI) and second (CP) grades curricula conformity with the reference documents, vertical and horizontal coherence); ix. Assess the performance of first and second grade (CI-CP) teachers and school directors on the basis of a representative sampling, within the framework of the implementation of the new programs; and xi. Give recommendations for improving the implementation of the reform program. Introduction This section of the report concerns the analysis of the curriculum and pedagogical inputs that support classroom instruction for the first two grades of primary education: CI and CP. Terms of reference for this activity required an assessment of the quality of materials designed, tested and distributed to the 4,500 primary schools in Benin with the support of USAID resources. Of particular interest are: 1) whether the methodology used in the curricular materials conforms to the instructional philosophy underlying the reform (conformity); 2) the extent to which the materials accommodate the entry level skills of students (horizontal coherence); and, 3) how well the program facilitates student performance in a manner that prepares them for the remaining years of primary school (vertical coherence). Finally, the team was to assess the performance of CI and CP teachers within the context of implementation of the new program. In addition to reviewing background and reference documents, the Berger/DevTech team examined the instructional assets that support the NPE. This includes: discussions with the principal architects of the program (international technical specialists and national consultants (pilots and concepteurs); an intense examination of teacher guides and manuals; and the review of student workbooks and manuals and supporting didactic materials. The team was further required to measure the impact of the NPE on student learning, through evaluation of students in a representative sampling of schools. As noted earlier, the evaluation team conducted in-school classroom observations in more than 140 schools throughout all 12 departments of Benin. The team members examined teaching practices, observed classroom activities, and conducted interviews with teachers, directors, administrators, children, and parents. Evaluation of Benin's Basic Education Reform Pedagogical Component USAID – Benin Louis Berger/DevTech 16 This sub-report describes first, the rationale and philosophy underlying the NPE and the processes used to create, test and revise the new curricula for CI and CP. Next we analyzed the principal methodological components of the program as designed. This is followed by a description of the findings and conclusions of school visits, classroom observations and field research. Finally, specific recommendations for immediate and future action are presented. Rationale and Philosophy of the New Program of Study (NPE) Ten years ago public primary education in Benin was best characterized as a disorganized chaotic collection of local schools operating without guidance, support standards or resources. Evaluations conducted by USAID and the Coopération Française in the early 90s noted that teachers were bereft of skills, classrooms had no instructional materials, and there was little evidence of governmental support. Schools were grossly under-capacity, poorly organized and managed; instruction relied exclusively on teachers writing lessons on a chalkboard and students copying those lessons into blank workbooks. This approach is referred to as intermédiaire, but in reality it was a system that relied on rote memorization, lesson repetition, and large-group (magistral) lecture￾drill using an entirely frontal approach. Under this primary education program, it was estimated that only about 40 percent of Benin’s children were enrolled in primary schools and less than one in ten ever completed the sixth grade and enrolled in secondary school. With USAID support, the Government of Benin began a decade-long process to reform primary education. The goal was to create a standardized program of study that relied on an individualized and competency-based approach that would provide teachers with the skills and materials to accept children at their unique entry-level and carry them to success in primary school. This process began in 1992 with the creation of working groups for curriculum development and teacher training. USAID technical and financial assistance was provided to MEPS to underwrite these activities. Committees composed of local teachers and administrators examined the newly developed curriculum materials (student handbooks and manuals, teacher guides, and audio-visual aids) for contextual validity. Once these products were approved by the review and revision committees, they were introduced to 30 experimental schools. Schools in the experimental group were selected from among the nations 4,500 public primary institutions. Teachers in the experimental schools were provided with extensive pre-service and in-service training in the methods required of the NPE. Experimental curricular materials were further revised by follow-up editors until the products met standards of sufficient quality that the NPE could be disseminated to the entirety of Benin’s schools. This generalization process began in CI in 1999 and continued for CP in Y2000. The generalization process necessitated publication and distribution of one workbook in French and one in Mathematics for each child and learning manuals for every two children. These printed materials were provided to children at no cost, a practice that has continued from 1999 to the 2002-2003 school year. For the 2003-2004 school year MEPS has contracted with private printing firms to print Evaluation of Benin's Basic Education Reform Pedagogical Component USAID – Benin Louis Berger/DevTech 17 and distribute student workbooks and manuals on a cost-basis through local bookstores and markets. To insure that teachers effectively use the NPE materials and resources, a nation￾wide training program was implemented. Teachers from all 4,500 schools (public as well as private) were trained in July and August before the schools opened. This began in 1999 for CI, and 2000 for CP. Additional training was provided to school directors, who in most instances also participated in the teacher training sessions. The USAID-supported NPE training (a first professional development exercise in pedagogy and instructional technology for most teachers) focused on application and practice of the new curricula. The NPE curriculum consists of six fields of study: 1) French, 2) Mathematics, 3) Social Education, 4) Art Education, 5) Science and Technology, and 6) Physical Education. In each subject, teachers received programs of study, teacher guides, and student handbooks and manuals. Characteristics Of The NPE The learning program for CI and CP is based on the following operational factors: French is the exclusive language of learning. Fully 90 percent of Benin’s children enter schools speaking a regional maternal language. National policy is that French to be used in all primary school activities. Verbal instructions are provided in French; all workbooks and manuals are in French, and all didactic support materials use French as the only language medium. The goal of entry-level activities is to accept children with no verbal or operational French skills and equip them with sufficient skills to read, write, and speak French at a level required in primary school. The NPE is highly structured, based on a pre-designed, common lesson plan for each day’s activities, all of which are linked to lessons in the student handbook and manual. Teachers are expected to follow guides and the calendar of instruction. They are also provided specific methods to employ in all six of the programs of study. Instruction and resources are designed to lead children through a sequence of increasingly complex lessons that require diverse and higher-order learning skills. The NPE is designed as an instructional format that shifts teaching from a frontal, lecture-repetition and large-group format towards a student-centered, interactive and student-centered approach. This is to be carried out through the use of small￾group and individualized learning methods, and the reliance on techniques such as role-playing, field activities, demonstration and independent study as the preferred means of causing learning. At the heart of the NPE is a change in teacher–student roles, and the elimination of the intermédiaire approach of rote memorization and repetition, to be replaced by the students’ assumption of the learning process. In this context, the teacher becomes a manager of the learning process rather than purveyor of information to be absorbed by students. Focus of the NPE is on cognitive skills acquisition, beginning from the first day of class. The program was designed with the premise that classroom activities are sufficient to stimulate French acquisition and learning. The teacher guides, however, make little mention of parent and community support, and require little homework or parental involvement as support elements. Evaluation of Benin's Basic Education Reform Pedagogical Component USAID – Benin Louis Berger/DevTech 18 School Visits The evaluation team was able to make classroom observations in 150 classrooms of the 211 schools visited. In many instances, the evaluation team was accompanied by a MEPS official who assisted in the observations. Since the teacher strike reduced the number of class days to two per week, and in some instances, to one day per week, the period for actual classroom observation was severely limited. Observers originally intended to visit CI and CP classes to evaluate teacher and student performance. However, since learning was so severely affected by the teacher strikes during the last two years, it was decided to extend the research to CE1. Most students in this grade entered school in 1999 and completed the first two years immediately after the NPE was introduced. It is assumed that in those two school years there were fewer disruptions in classroom instruction. Thus, observation of student performance in the third year might more accurately reflect the impact of the NPE. The principal focus of the evaluation process, however, was to measure the impact of the NPE on student performance and teacher behaviors in CI and CP. ANALYSIS Conformity and Coherence The evaluation team considered conformity as the extent to which curricular materials accommodate student and teacher needs. The central question posed was whether the NPE is appropriate to the needs of incoming students in CI, and for preparing students for CP. Coherence on the other hand speaks to the internal capacity of the program to bring children from their entry level as non-French speakers and provide them with a sequence of learning that helps them to attain the intended goals of the program. The NPE has been widely disseminated to primary schools in Benin. The program is well-known and supported among all levels of MEPS administration. Most teachers interviewed have heard of and accept the theory of the reform. The overwhelming majority of students has access to French and Mathematics workbooks and manuals. More than seven of ten teachers interviewed have attended summer training programs to introduce new materials of the NPE. For the most part, they rated the training experience very favorably. Most teachers hold the same favorable attitudes towards the methods and principles advanced by the NPE. Broad awareness and appreciation of the NPE was noted among school directors. Few if any were critical of the NPE curriculum or the training programs that introduced the system to teachers. The NPE is used with significant success in Benin’s private schools. In these institutions, the number of students in CI and CP averages 20 to 30 and rarely exceeds 32, while in public schools, the mean number of students is over 60, and often surpasses 90 to 100 students. Private schools also are characterized by parental assistance in the instructional process. In most cases, parents have photocopied workbooks and lesson plans and are required by teachers to work extensively with their children. The team noted that private school directors play a strong role in supporting teachers, and that teachers are very knowledgeable, as well as concerned about each individual student’s progress. Generally, private school students come from families with higher incomes, whose parents most likely speak, read or write French, and live principally in urban areas, where exposure to the French language Evaluation of Benin's Basic Education Reform Pedagogical Component USAID – Benin Louis Berger/DevTech 19 is much more pervasive than in rural villages. These factors all likely help explain the fact that the NPE is used with more success in these schools than in the public schools. Currently, private schools provide elementary education to between 10 to 25 percent (depending on the data used) of the nation’s students. Teachers of CI and CP are using curricular materials of the NPE. Most teachers attempt to follow lesson plans, use workbooks and manuals and consult teacher guides. Almost all classrooms visited in the evaluation survey have instructional posters (planches) and for the most part, it appeared they are used. Students in CI and CP had individual workbooks and shared textbooks, although the books are normally kept in the care of the instructor when not in use. School benches in most classrooms were organized in small groups of six to 10 students rather than in rows. Other instructional aids prescribed by the NPE, such as small stones, seeds, bottle caps and sticks were less evident except for a handful of classrooms. Although most teachers are trained and subscribe to the NPE, fewer than ten percent actually empl oy the system, as gauged by our classroom visits. Instead, classroom activities continue to center around teachers who still rely extensively on writing on the blackboard and lecturing, the same frontal methods used in the intermédiaire system. There was little interaction noted among students and little if any student initiated-learning or questioning. Neither was use of small group or independent study observed. Although students sat in small groups teachers did not seem to be aware of or to take into account the various skill levels of their students. The NPE is limited in meeting student needs, especially at the CI level and provides limited coherence with respect to causing student learning and success. The underlying reason is that students do not learn French, which as we note throughout this report, is the sole medium of instruction. Students’ inability to speak read or write French has a devastating impact on any learning that could occur. Student performance in CI and CP is in a state of failure. The evaluation team noted that less than 10 percent of students were able to participate in classes at the expected level. In CE1 we found similarly disturbing trends: here although students fared somewhat better, less than 20 percent of children were able to perform in French at expected levels Overall, the majority of CI and CP students do not understand the teacher, cannot read materials, and are unable to participate in class. As a result, they cannot complete lessons, engage in interactive (or even frontal) learning, and are marginalized for the remainder of their primary years. For the most part, their poor language skills reduce any prospects for continuing in secondary or tertiary education. Children arrive at school without benefit of pre-school or early childhood stimulation. The initial shock of entering the school environment and being taught in a foreign language, inside a large and alien classroom surrounded by up to 100 children is for many difficult to overcome. Teachers begin teaching directly with the RANV (Réponse Active Non Verbale) for at least two weeks before using the workbook and possess few skills to deal with the fear and discomfort that children Evaluation of Benin's Basic Education Reform Pedagogical Component USAID – Benin Louis Berger/DevTech 20 experience upon entering school. In this instance, the NPE is limited in addressing the affective needs of incoming students. Parents are largely excluded from the learning process. Student workbooks must remain in schools for fear they will be lost or damaged, despite the fact that they are consumed during the school year. In almost every instance during discussions with parents, there were complaints against the policy of not permitting children to use books for homework. Parent and community resources are seldom utilized as instructional resources or as teacher aides. This, despite classes that are so over-crowded that no effective teaching could ever occur unless children are provided with a lower student-teacher ratio. On a broader scale, we found few instances where teachers or school directors foster parent participation in other school needs, such as buildings and grounds maintenance or furniture repair. From the perspective of parents, schools remain the domain of the national government. Although every school visited has a parent association (APE), parents feel their main purpose is to sign checks from the GOB’s PPTE fund to pay locally hired teachers (communautaires). According to teachers and school directors interviewed, the NPE is difficult to implement. Teachers have had little or no experience with such a system, having been schooled in the traditional lecture-drill-frontal format. Their training in the NPE has largely been based on the same format: trainers lecturing large groups about interactive, student-centered and competency-base instruction rather than demonstrating this approach or having teachers practice it themselves. Consequently, most teachers have never seen a child-centered classroom using interactive methods. Their lack of exposure to the new program is further hindered because of limited academic training and pedagogical skills, particularly newly￾hired communautaires who have little more than primary education and a no background in pedagogy. Moreover, there is significant movement and reassignment of teachers, and a great number of teachers have left the grade for which they received training. Without exception, teachers ask for more training in the NPE system, and cite the need for better in-service support from directors and conseillers pédagogiques (CPs). Teachers and school directors have failed to assume a sense of responsibility for causing learning among all students. In almost every instance during school visits, when asked about their attitudes toward the NPE, teachers and directors invariably lauded the NPE as the basis for a great new program of learning. However, when team members asked that students demonstrate their ability to speak, read or write at the lesson level, and could not do so, teachers and directors were quick to assign blame for deficiencies on parents, poor teaching in earlier grades, or the strike. Rarely did teachers and directors acknowledge that they might be responsible for the situation. With respect to the conformity of the NPE to the needs of incoming students, we found that the system is effective when applied under proper teaching conditions. In private schools, the NPE is highly appreciated among teachers, and the team noted that most students were achieving at the intended level. In public schools, conformity is not so apparent. Here, large class sizes reduce teachers’ abilities to Evaluation of Benin's Basic Education Reform Pedagogical Component USAID – Benin Louis Berger/DevTech 21 implement the program effectively. The consequence is that students do not learn French at a level necessary to participate in the curriculum or to meet the demands of the next grade level. The team observed that the NPE is, for the most part, a coherent program. The logic, sequence and progression of learning are appropriate. Nevertheless, application of the system may be too complex for un-trained teachers, as well as for those who have only participated in the introductory NPE training. The coherence of the program is further limited by reliance on French, which if not acquired to the desired level, restricts student participation and performance. RECOMMENDATIONS The NPE curriculum materials, particularly teachers’ guides, should undergo continuous and rigorous evaluation and revision. They should also be simplified to respond to the needs and capabilities of teachers with minimal preparation and education. While its design provides a coherent and standardized program of lessons for some classrooms, (mostly in private schools) many improvements are needed. This process should begin with systematic diagnosis and feedback programs that include regular and unbiased testing of students and discussions with parents and teachers. These diagnostics should be ongoing, and the results incorporated in guides, workbooks and manuals for the following year. For the NPE to succeed, it must be under constant monitoring and review until all aspects of the program are understood and corrected applied. Even then, MEPS should continue this on-going feedback and revision process to ensure the program continues to be refined and improved. Teachers must have more training. This situation is described in detail in the following pages. Pre-service and in-service professional upgrading is essential to consolidate gains made by the NPE in the lower grades before they are forgotten or lost. The NPE should make provisions for teaching to large classes. An average of 60+ students per class is current and near-term reality and must be reckoned with. It is essential to reduce the pupil-teacher ratio, especially in CI and CP to manageable proportions. Elements of a new approach are discussed in the following section describing the French curriculum. It is recommended that schools utilize the many strategies for managing learning in classes with large enrollments, such as teacher aides, better use of human resources in the school and the community, in all grades and across all subjects. For its part, MEPS should extend a wider variety of didactic resources to students, in the form of supplementary workbooks (commercially available), more visual aids such as posters, flip charts, newspapers, and audio support such as radio and video programs. Teachers require skills to discover and create student support and didactic materials from local sources. Although the NPE offers recommendations on the use of locally-developed materials such as sticks, rocks, seeds, and corks, teachers have not acquired skills to extend the use of everyday products or the local environment to learning situations. They need to learn how to use field trips, simple laboratory exercises, local artisans, contests, games and songs. Demonstration and Application of NPE Methods: Every effort should be made to provide teachers with examples, models and methods of the NPE. There is a great need for more exposure to good teacher practices, and to ideas for using local Evaluation of Benin's Basic Education Reform Pedagogical Component USAID – Benin Louis Berger/DevTech 22 materials and resources. Teacher guides should contain ample concrete examples and case studies of successful classroom strategies. In addition, MEPS should consider the immediate creation of demonstration and laboratory schools, where these initiatives can be observed. Demonstration schools might be selected from among those used in the experimental phase of NPE development (or private schools), but at a minimum there should be one for each Department, and ideally one in each sub-prefecture. These schools can utilize materials and programs widely available from schools in Benin and West Africa. They might be used to host unités pédagogiques activities, be the site of centers of the documentary and support network, and serve as centers for dissemination and pre-service training. School directors need to play a more prominent role in supervising the NPE process. Experience in the implementation of innovations in primary schools has demonstrated the absolute necessity of strong and supportive leadership if sustainable changes are to occur. School directors must assume central responsibility for extending the NPE into all classrooms of the school. They must foster a learning environment through better teaching, more student involvement, and broader family participation. Directors must provide their institutions with a commitment that accepts no less than 100 percent learning among all children, and not just a few. There is a need to train school directors as instructional supervisors, and to expand their skills as catalysts for community, health and human development. They must be provided with an array of solutions, interventions and flexibility to support the mission of fostering student success through improved teacher performance. V. FINDINGS AND RECOMMENDATIONS FOR FRENCH AND MATHEMATICS Terms of Reference: i “Assess the conformity of the Math and French workbooks and textbooks, French wall charts, pedagogical guides with the approaches, strategies, and training contents as recommended by the new programs”. Vii: “Assess the performance in French and Math of CI and CP of students on the basis of a representative sampling” viii. Assess the performance of first grade (CI) students on the basis of a representative sampling; and xi. Give recommendations for improving the implementation of the reform program. OBSERVATIONS/FINDINGS The purpose of this aspect of the evaluation is to assess the nature and impact of pedagogical reforms in the teaching of French and Mathematics in CI and CP. The terms of reference required that the team investigate the conformity of the Math and French workbooks and material, French wall charts, and teacher guides and workbooks with the approaches, strategies, and training content as recommended by the NPE. Besides conducting a thorough review of the materials, the team was able to observe the materials and methods in their current application by visiting 150 CI and CP classes. This section begins with a description of the stated purposes, philosophy and design of the French and Mathematics programs, followed by findings of the school visits. Finally, recommendations are proposed for improving the implementation of the reform program. The goal is that the instruction, student materials and teacher guides conform to and accurately reflect the needs of children, while incorporating the approaches, strategies, and training content as recommended by the NPE. Evaluation of Benin's Basic Education Reform Pedagogical Component USAID – Benin Louis Berger/DevTech 23 The Goals For French And Mathematics In The NPE Goals for schools in Benin are set forth in the prefaces of programs of studies of the NPE for French and Mathematics at both the CI and CP levels. Attention is called to the rapidly changing and complex world in which we live and the necessity of teaching Beninese children to assume personal responsibility for their own actions, become capable of life-long learning, and believe in a democratic way of life. These ends are to be accomplished by competency or objectives-based instruction, an interdisciplinary approach that involves the student in his/her own learning, and in the resolution of real-life problems. The program calls for parents and other members of the community to be more actively involved in this educational process. Individual rates of learning are recognized and teachers are urged to use their initiative and creativity to assist the children in their learning. Centrality of French French, according to the Constitution of 1990, is the official language for the republic of Benin. Within the country, it is the language of political and social integration- the one language common to all Beninese. Externally it is a language providing access to a world culture and international commerce. French is also the language of instruction in schools. Without a solid foundation in French, students cannot succeed in primary school, since it is the vehicle for learning all other school subjects and the means to achieve the goals of creative thinking and problem solving. Without French, students will remain uneducated and functionally illiterate, and most assuredly face very limited opportunities. French is a foreign language for the vast proportion of Benin’s students who enter school speaking a regional, maternal language. Moreover, these children come from households and villages where exposure to French is most unlikely. Their only resource for learning French is in the school; therefore, the materials and methods used to extend French to children assume paramount importance. The French Curriculum French is the medium of instruction in public and most private primary schools. The NPE curriculum for French for CI and CP provides detailed programs of study, teacher guides, and guides to evaluation. To give some idea of the magnitude of this information available to teachers, documents for CI French alone contain more than 600 pages of detailed instructions for teachers. The introduction to the Teacher’s Guide for CI explains the structure of the program: how it relates to the pupil’s workbooks and manuals, and wall charts, and presents a table showing their inter-relationships. The eight stages of language learning are summarized as follows: 1. Oral communication; 2. Reading for meaning; 3. Strategies for reading comprehension; 4. Recognizing syllables; 5. Writing/drawing activities; 6. Fuller understanding of the text; 7. Expressing thoughts through writing; and, 8. Strengthening and enriching what has been learned. Evaluation of Benin's Basic Education Reform Pedagogical Component USAID – Benin Louis Berger/DevTech 24 The eight-step progression described above provides the structure for language learning in both CI and CP. The first two weeks in CI concentrate on non-verbal activities (RANV) that include drawing and listening to songs. The next three weeks are devoted to oral and pre-reading activities involving home, school and community objects and activities familiar to the child: the classroom, animals, food and clothing. The systematic study of French following the eight-step approach is to begin in week six. The next six weeks are to explore three themes based on the “expanding universe concept” in which the first vocabulary words are based on everyday events in the life of the child. Weeks six, seven and eight consist of vocabulary and phrases about family life: mother, father, and family. Lessons for weeks nine and 10 relate to school and class activities, with children learning words such as book and slate. In Week 11, time is set aside to review words and structures encountered in the first ten units. In week 12, the theme of a local festival completes Volume I. Volume II of CI in French adheres strictly to the eight-step progression linked to themes that last from one to three weeks and are to be completed at the pace of two units per week. In all cases, the lessons contain vocabulary words appropriate for the theme. Weekly themes are listed below: Week 13 My body Weeks 14-16 My activities Weeks 17-18 The market Weeks 19-20 My week Weeks 21-22 My friends, the animals Weeks 23-24 Those who work near me Week 25 The public square Volume II also comprises a table of strategies for reading, linked to the first six stages and an inventory of simple words to be written such as grand and petit. Additional lists contain key words and their sounds and a taxonomy of the different types of texts. In CP, the second year of French, the same eight-step progression is followed. Many of the same themes are utilized with review and additional vocabulary. As an example, the theme “My friends the Animals” appears in both CI and CP. In the former, simple words and sentences are introduced such as “here is the dog of Toni” while in the latter more meaning is attached with “the dog barks and scares burglars”. Other themes that re-appear are “the world around me”,” the school”,” and “what I do during the day.” New and more complex themes requiring more difficult words and structures are introduced: “making purchases”,” health education”, « different artisans and workers”, the “environment” and “rights and responsibilities”. The Math Curriculum The math curriculum has many of the same characteristics as the French curriculum. One finds a close coherence among the didactic materials: program of studies, the teachers’ guides (there are two in CI), the manual, and the student’s workbooks. The amount of material contained in the teachers’ guides is somewhat Evaluation of Benin's Basic Education Reform Pedagogical Component USAID – Benin Louis Berger/DevTech 25 less than in CI for French: 400 pages instead of 600, for a total of more than 1,000 pages of teaching instructions. The materials for CI mathematics contain very precise and detailed teaching instructions. The program is divided into nine sections (dossiers) each consisting of four to five modules alternating among the subjects of arithmetic, measuring and geometry and requiring approximately 120 hours of instruction. The modules specify the desired subject matter, and attitudes to be acquired, along with the preferred strategies and with cross references to pages in the manual and the workbook. Each module is allotted 3 hours and consists of a three-part pattern: the lesson begins with finding out what the learner already knows, followed by the new material with remedial and enrichment exercises and finally evaluation. Learning activities use familiar objects such as furniture, chairs, and table. Children are expected to comprehend and manipulate numbers up to 20. In the teachers’ guides for CP, the general orientation is, except for a few words, identical to the corresponding pages in CI. The instructions are detailed and precise, and in CP, the same instructional procedure is followed. Similarly, the program contains nine sections (dossiers), each with four to five modules, each consisting of four components that describe the subject matter, and attitudes, along with the preferred strategies and with cross references to pages in student manuals and workbooks. In each module, the same three part pattern is followed: the lesson is to begin with finding out what the learner already knows, followed by the new material with remedial and enrichment exercises and finally, evaluation. As with the CI program, the learning activities utilize objects and events familiar to the learner. The geometric figures are more complex and the numbers higher with examples up to 100 FRENCH AND MATHEMATICS Positive Effects Of The NPE Thé French and Mathematics programs has been disseminated throughout the schools of Benin. Didactic materials are in adequate supply in all schools visited during the study. Most teachers have participated in training and hold favorable attitudes toward the NPE. Most profess to have adapted the NPE program in their daily lesson plans. Teachers have been provided with programs of study, teachers’ guides, and wall charts, and student tracking forms that correspond to the NPE. Children in CI and CP all possess their individual workbooks and a manual to be shared with another child. There have been some logistical problems that have hampered distribution of student material; in many instances the workbooks arrived after the beginning of school. A few classes had not yet received workbooks, but these were an exception. For the most part, the schools are well equipped to implement the NPE. WEAKNESSES OF SCHOOLS French Instruction Despite the universal existence and use of NPE materials, and the training provided to teachers, children in Benin’s schools are not learning French in CI and CP in public schools. An overwhelming percentage of children analyzed in schools surveyed by the evaluation team are unable to speak, understand or write the French language. Even basic questions posed by the evaluation team, such as “What is your name?” went unanswered. Teachers in CI and CP confirmed that on Evaluation of Benin's Basic Education Reform Pedagogical Component USAID – Benin Louis Berger/DevTech 26 average, less than 10 percent of students have achieved French competencies to the level required of the NPE. Students in private schools fared better in French competencies but the quality of instruction here is in direct proportion to the amount spent per child. Children in private schools with higher per pupil funding perform better than those in less costly schools. In most instances this was related to smaller classes, and parental assistance and expectations for children’s’ success. As the evaluation study progressed, it became increasingly apparent that teachers, directors, pedagogical counselors, administrators or parents interviewed agreed with the findings of our public school visits: children are not learning French. This point was corroborated during meetings with a group of Pedagogical Counselors in training from all sections of the country, forming a rather complete cross section of knowledgeable informants. When asked why children were not learning French, not a single person contradicted the findings. The evidence for this finding was overwhelming and the consensus complete. The reasons for this deficiency in French are multiple. Teacher strikes, as noted earlier, have dealt a major blow to language instruction. But there are other problems contributing to the paucity of learning. This problem impacts all programs of study, since minimum competencies in French must be attained before attempting other school subjects. This requirement may appear to be self-evident, but it is not effectively met in the schools that were observed. It is further complicated by the approach espoused in the NPE: simultaneous learning of French with other subjects. Large Classes In an earlier section, we talked of the challenges to learning imposed by large classes, and that the mean class size in classrooms observed is 62. This number of students makes it impossible for a single teacher to give personal attention to individual students in language acquisition. If a child falls behind it is difficult to receive help from the teacher who is by necessity completely inundated with the work of dealing with 60 or more classmates. While instruction in Math may be feasible for large classes, individualized attention is essential for the acquisition of a foreign language. Teacher Preparation Few and fewer teachers in CI and CP have an academic background exceeding ten years of formal instruction, with many of the newer hires having only a primary school education. Most of these lesser-trained staff are recently hired, and in many instances have not participated in training programs designed to orient teachers in the use of NPE strategies and materials. Complexity Of Teacher And Student Guides Many teachers interviewed noted that the NPE guides for teachers are too complex and difficult to understand, and directors or Conseillers Pédagogiques lack sufficient understanding of the NPE to provide guidance in applying the system. In addition, a broad group of teachers noted that lessons prescribed in the NPE are difficult to comprehend and implement even to the best-trained teacher. USAID/MEPS have supported NPE pedagogical training, but little of it has dealt with the instructional theory and methods of teaching a foreign language. A Evaluation of Benin's Basic Education Reform Pedagogical Component USAID – Benin Louis Berger/DevTech 27 restricted academic and pedagogical background is perhaps less limiting in Mathematics than in French, but the problems remain rooted in French language deficiencies. Children cannot follow Math lessons or printed texts in French. The most common complaints of teachers about the French program, and to a lesser extent Mathematics, were that they are too complex and difficult for teachers, and the pace of material to be covered was too rapid for most children. In the meeting with pedagogical counselors mentioned earlier, one of the trainees reported, “finally after 4 months of training I am beginning to understand the Noveaux Programme d’Etudes.” This statement is both simple and eloquent as there is much to understand and even participants in this experienced and carefully selected group of conseiller pedagogique trainees, who must be school directors or eligible to become one, have difficulty comprehending the NPE. The evaluation team noted above that both CI and CP programs comprise more than 1000 pages of NPE guides, manuals and workbooks. Can one, then, really expect an untrained or under-trained (4-8 weeks) teacher even to begin to grasp the content let alone apply the principles in the classroom? Lack of a Language-Focused Learning Environment In Schools Schools do not provide enough aural or visual stimulation to support foreign language learning. As a result, teachers tend to fall back on the traditional intermédiaire method of repetition, chalkboard and large group rote processes. The evaluation team noted only a few classrooms with materials and language learning resources apart from that provided by the NPE. Teaching aids such as flash cards, newspaper props, magazine, posters (besides planches), and additional simple materials were usually lacking in most classrooms. Although small group learning is a central tenet of the NPE, children were seldom observed helping each other either individually or working together in groups according to skills level. Community And Parents Regional and maternal languages are spoken in most Beninese homes. Few children are raised in a French-speaking environment or have even been exposed to French media. This is especially the case in rural villages, where all social transactions are conducted in the maternal language; French media, such as newspapers, TV or radio are not accessible. As a result, children leave the French￾speaking environment of the school and have no means of using French in everyday situations, which is a critical strategy of language acquisition. In general, the evaluation team found that parents seem to approve of the child￾centered goals and active methods of the NPE. They are however frustrated in a number of ways. Most interviewed found it difficult to assist their children at home, for until recently the manuals were not in the bookstores and, as they had to be shared, could not be taken from the school. Many parents question the departure of the NPE from the intermédiaire system: especially since phonics is missing and writing begins with printing rather than cursive writing. And of course, most of all they are especially distressed about their children’s inability to speak, read, and write French. Lock Step Curriculum Evaluation of Benin's Basic Education Reform Pedagogical Component USAID – Benin Louis Berger/DevTech 28 Ample examples of the rationale for individualization can be found in the many programs of study and guides for the NPE. Typical of these is the statement in the teachers’ guide for CI Mathematics, “All children do not learn in the same way or at the same rate: they have the right of trial and error.” Despite this professed learning principle in French and Mathematics, teaching, for all intents and purposes, follows a lock-step curriculum where the entire class moves along together whether the material is mastered or not. What is missing is the awareness in both the French and Mathematics sequence that most children may not have mastered a criterion level of sufficiency to keep up with the program of study. Teachers seem compelled to follow the time sequence of the NPE, despite the fact that 80 to 90 percent of children cannot participate at these levels. The mind-set seems similar to traditional university instruction, in which teachers expect a certain percentage of students to fail. ANALYSIS Findings Specific to French Not enough time or resources are allotted for foreign language learning, and classes are too large for proper teaching. In most western countries, the program of study for non-native speaking children entering national language schools lasts on average two years. These programs are generally taught to groups of less than 20 children by well-trained teachers and supported by multimedia teaching aids that stimulate the children’s use of the new language in and outside the classroom. Findings Specific to Math With French the language of instruction, it is apparent that language deficiencies will impede learning in other subject areas. Even with language limitations, one of the more troubling findings was that the Mathematics lessons observed were uniformly restricted to the use of NPE materials, but with few departures from the frontal, intermédiaire method of teaching. There were very few additional visual aids, including sticks, corks, grains and other materials that are central to the NPE approach for teaching Mathematics. Instead, teachers relied on large group memorization of numbers, with haphazard lessons in arithmetic, measurement and geometry that confused children. In only a few instances did we observe examples of small-group work according to skill level. In reviewing student Math workbooks, the evaluation team noted that only a small number of CI classes have gone past the first few of the 20 lessons for the year. Even for these levels, we noted that children were rarely using workbooks regularly, and often had not progressed past the beginning pages. In the Math workbooks, we noted the lack of emphasis on counting and adding, with numbers learned only up to 20. Memorization seems to be avoided, yet mathematical proficiency requires a certain amount of it. Instructions in the workbooks are difficult to follow, requiring teacher time not available for children needing individual attention. Vocabulary used in student workbooks seem beyond the grasp of a 6 year-old child, and are more likely to impede rather than assist understanding. On the first page of the CI student workbook, for example, one finds instructions using the following terminology: forme l’ensemble, etiquette d’addition, etiquette de rang and etiquette numerique. On the next page, a difficult concept is introduced relating to space relationships, with the five -syllable word immediatement. Such concepts and Evaluation of Benin's Basic Education Reform Pedagogical Component USAID – Benin Louis Berger/DevTech 29 vocabulary are well beyond the grasp of a six-year old child during the first days of the first school year. In the Teachers’ Guides for both CI and CP, educational and pedantic terms are too complex to be understood by most classroom teachers. For instance, terms such as “cognitivist,” or “math based on a socio-constructivist” approach have very little meaning to most teachers. The idea that a six-year-old child can approach the mastery of Mathematics by “constructing his own knowledge” is a dubious premise. These same terms appear in the French program as well, and may serve more to confuse than clarify. Most teachers and parents find these approaches and vocabulary not only unfamiliar but also intimidating and misguided. RECOMMENDATIONS COMMON TO BOTH FRENCH AND MATH Follow The Goals Of Educational Reform Despite systemic problems, the goals and active methodology set forth in the NPE programs of study and teacher guides should continue to be the centerpiece and the inspiration for schools in Benin. A child-centered approach and setting can be highly motivating for young learners to have instruction based on people and events in their lives. The term “pertinent content” has been used to differentiate the new program from the previous one. These approaches are a significant departure from the intermédiaire system and provide a new context for schooling that should be encouraged and extended. Review And Improve The French Immersion Program The most obvious and glaring problem confronting the NPE, and primary education in general, is the weakness of programming for French language skills. Our findings merit repeating: children are not learning to read, write or speak French at the rate or level needed to participate in primary education. The e valuation team recommends that French instruction become the number one priority of pedagogical reform. Any number of strategies could be pursued separately, or in unison to improve children’s French abilities. Teacher guides, manuals and student workbooks should be reviewed to ensure they are appropriate to accommodate the entry-level skills of students, and can effectively be administered by teachers with minimal academic preparation. Here, we must recognize the difficulties of learning a foreign language, and provide additional learning resources to meet needs. We do not suggest that the present system need be subjected to lengthy or major revision; instead, more and varied didactic resources could be used to supplement activities. It is important that the French program assure that earlier, easier sequences are mastered before proceeding to more difficult ones and that children who lag behind the majority are not neglected. Explore Alternative Approaches There is a need for alternate approaches and experimentation. One such approach might require CI students to begin school with a preparatory term of several weeks. Children might attend such a program in shifts of small groups. MEPS should also consider testing a variety of bilingual training options. Within the CI program itself, language instruction should occupy more time as the time currently allotted is not sufficient to bring students to the functional levels needed, and should be expanded until minimum language competency is achieved. Evaluation of Benin's Basic Education Reform Pedagogical Component USAID – Benin Louis Berger/DevTech 30 Finally, strategies should be adopted to expand sensory exposure to French. This could come in the form of more printed material, audio recordings, flip-charts, pictures, games and songs: all of which would reinforce French language skills. Individualization In Large Classes All school and community teaching resources should be mobilized to teach French. Teachers from all grades should be assigned this task, as we have mentioned above. A corps of teacher aides, comprised of children in higher grades or in secondary school, members of the community, peer tutors and other community personnel can be trained to administer beginning French instruction. These initiatives could be supported with very simply designed training materials that would provide French teaching skills to those with the most basic academic skills. The net effect would be to reduce the student-teacher ratio to manageable proportions: ideally, 15 students per teacher. On any given lesson, children can be divided into three or more skill-groups, with enrichment for faster groups and additional assistance to slower groups. Children would not proceed to the next lesson until the current one is mastered. This approach is the easiest to implement in Mathematics, where students can easily be involved in peer instruction. From the current curriculum, teachers should create an individualized program of study for each child. Progress should be documented; verification of pupil mastery of competencies could be done by more advanced students. At designated intervals the teacher would check each student’s progress and make written notations on individual records, and institute remedial measures to bring the pupil to desired levels. Provision Of Student Learning Resources USAID/MEPS should continue to supply didactic materials to children until they are readily available in the bookstores throughout the country (Although the student workbook distribution program is now being handled by the private sector, the evaluation team did not find any workbooks for sale in March 2003. We urge a strong program of monitoring the distribution of these workbooks in the coming months to ensure they will be available to children in time for the next school year). Where financial needs necessitate, USAID/MEPS should continue to make student materials available without cost if families cannot afford them. Simplify And Compress NPE Documents Simplify material of the NPE program for the benefit of teachers, children and parents. Here we recommend a single volume of instruction for the teacher combining French and Math and including everything that is user-friendly, easily understood and applicable for both newer and experienced teachers. For those wanting additional information, references should be made to the original documents. As was mentioned in an earlier section because of the jargon and the sheer volume - more than 1,000 pages - the materials become almost incomprehensible to anyone using them. Teacher Preparation There is an immediate need to refocus programs to equip teachers with the fundamental competencies required for foreign language instruction. We have noted that French acquisition must not be treated as a separate lesson plan, which can be taught in the same manner as other subjects. Here active, child-centered and individualized methods are of supreme importance. Evaluation of Benin's Basic Education Reform Pedagogical Component USAID – Benin Louis Berger/DevTech 31 Community And The Parents Many MEPS personnel dismiss the potential contribution of parents in the learning process because of their minimal French skills or limited academic preparation. Indeed, we have documented and discussed the minimal reliance on parents and family in the schooling process. Nevertheless, the evaluation team views parents as an untapped human resource that should be used in the instructional process. Given that the maternal language dominates household and village life, the NPE should incorporate provisions to have parents work alongside their children with homework assignments. A parent may not be able to help children with French or other subjects, but they can certainly encourage their child to attend and work in school. Wherever possible they can bring external French media products and materials to the home to provide incentives for applied use of French. RECOMMENDATIONS SPECIFIC TO THE FRENCH PROGRAM Introduce Cursive Writing Earlier Cursive writing is not introduced until the beginning of CP. Discussing this recommendation with a group of educators in a debriefing session near the end of the consultancy, it was pointed out that for many of the children, CI would be the first time that they used a pencil so this extensive preparation of a full year with printing is necessary. The consultants are not fully convinced and at the very minimum the teacher should be encouraged to introduce cursive writing as early as possible. In fact, some teachers have taken the initiative of teaching cursive and block letters (script) together, spending a few minutes each morning on both. Doing so resolves the complaints about the difficulties in transferring from one writing system to another. This change would also eliminate one of the often-heard parental complaints about the NPE. While they accept the advantage of a curriculum based on every day events in the life of the child, parents fail to understand why a new writing system is taught, only to be changed the second year into one that they learned as a child. Supplement Whole Language Methods With Phonics The NPE relies extensively on a Whole Language method, in which the meaning of words is immediately introduced through pictures and actions to provoke the immediate interest of the learner. It is intended to replace the rote drill of sounds without a context for understanding. A major disadvantage is that once these supports are removed the learner has difficulty recognizing the words and understanding their meaning. In western nations, phonics (the oral repetition of sounds) is generally accepted as necessary in primary reading instruction. In France, for the teaching of French, phonics has always been stressed, even in the most radical and child-centered approaches such as the Ecole moderne pedagogie. Once the decision is made to include phonics, implementation is quite easy. In Annex I of the teacher guides for French in both CI and CP, words are divided into syllables. Teachers need only add oral repetition as a teaching strategy. Evaluation of Benin's Basic Education Reform Pedagogical Component USAID – Benin Louis Berger/DevTech 32 RECOMMENDATIONS SPECIFIC TO THE MATH PROGRAM Provide More Materials For Drill And Application Of Learned Concepts In Mathematics, children are very weak in understanding addition tables and the concepts needed to add and subtract, which cannot be done without prior acquisition of the base-ten system. Student workbooks need more drill, repetition and application. There are far too few pages devoted to application of learned concepts and use under diverse conditions. The program needs to introduce the base-ten system earlier and with more practice. VI. Teacher Training Terms of reference iii. Assess the conformity of all the training documents and activities with the training plan on the one hand, the learning objectives, approaches and strategies as recommended by the new programs on the other; ix. Assess the performance of first and second grade (CI-CP) teachers and school directors on the basis of a representative sampling, within the framework of the implementation of the new programs; and xi. Give recommendations for improving the implementation of the reform program. This portion of the report describes teacher-training activities that have supported the generalization of the NPE to the 4,500 schools in Benin. The first section describes teacher-training programs supported by USAID and instituted by the MEPS’ CGNPE. This is followed by a description of pre-service teacher training programs and resources. We then discuss the state of teacher competencies in schools visited by the evaluation team. Recommendations for future USAID and MEPS programs are then listed in order of importance. TEACHER TRAINING PROGRAMS Academic Preparation: Resources and programs The only teacher training college in Benin, located in Natitingou, had 76 students enrolled when we visited, despite a capacity for 200. The program of study leads to a Certificat Elementaire d’Aptitude Professionelle (CEAP.) All instruction is based on the NPE and four MEPS administrators trained in the NPE system now teach at the Training College. After receiving the CEAP, student teachers are assigned to a school for two years after which they undergo observations and examinations before receiving a teaching license. There seems, however, to be little incentive to attend a training college as the same exams and diplomas can be taken while teaching. Value added by going to a training college was not apparent to the team and we did not observe any impact on the teaching population. Initial (NPE) Training The pedagogical reform process has taken two principal tracks: material and curriculum development, and teacher training. Curriculum materials - student workbooks and manuals, and teacher guides and programs of study - form the core components of NPE training. Teacher and director training are based on a number of documents prepared for training purposes. They include: Guide de l’Apprenant (Teaching guide for teachers) Guide de Formateur (Training guide for trainers working with teachers and directors) Formation Initiale des Enseignants Contractuels – Guide (Guide for initial training of contractual teachers) Evaluation of Benin's Basic Education Reform Pedagogical Component USAID – Benin Louis Berger/DevTech 33 Formation Initiale des Enseignants Contractuels – Modules de Formation (Initial training of contractual teachers – Training modules) Guide de l’Apprenant – Nouveaux Directeurs (Teaching Guide for New Directors) Documents are developed annually for each new grade level undergoing generalisation. The documents include revisions from the previous year. Other printed materials used by teachers include the student’s manual and workbook. Initial NPE training began in 1999 for CI. Using a “cascade” approach; a group of 60 selected “encadreurers” who were involved in the elaboration of materials and programs of the NPE developed teacher-training programs. When these programs were sufficiently developed, the next stage of training targeted some 700 individuals (teacher trainers) in six department locations for two to three weeks. Finally, between August and September, the 700 teacher trainers trained about 4500 teachers and school directors at local training sites for four weeks. This approach continued each year with increasing grade levels. In 2000 only, CI teachers followed their students into CP and received training alongside new CI teachers. Consultation: 60 “encadreurs” Program designers one week Training: 700 « formateurs » (Pedagogic counselorss and some teachers and school directors) 2-3 weeks 4500 teachers and 4500 directors 4 weeks each In-service Training In-Service training benefits from a variety of pre-existing structures in the Beninese education system. The unité pédagogique (UP) is a subgroup of schools united to resolve common pedagogical issues. There may be 6-18 teachers in each UP. Regular UP meetings of teachers are led by a school director and CP, one of which will be the “responsable” or RUP. This is also where IFESH, a USAID contractor conducts its follow-up training in NPE practices. The UP normally meets twice a month in the afternoon. Due to the strikes during the evaluation period, however, most UPs had not met at all or had only conducted their first meeting of the year, which is administrative. Additional in-service training occurs during CP visits, or during recyclages, or follow-ups to initial NPE training. All in-service training now refers to NPE related activities. There are formal sessions normally lasting two weeks during the summer and less formal sessions that occur during the UP. The NPE training system focuses on delivery by using high performing teachers and school directors as pedagogic counselors (CP). A CP is a local resource person whose responsibility is to visit local schools twice a week to provide professional development support. Evaluation of Benin's Basic Education Reform Pedagogical Component USAID – Benin Louis Berger/DevTech 34 Benin is currently training a new group of CPs in the Inspectors’ training school in Porto Novo. These CPs are part of a new wave who will be released from their classroom responsibilities in order to provide ongoing staff development and teacher support in the field. In order to participate in this training one must be a director or eligible to become a director and succeed in a competition for this post. In 1995, a plan was prepared to set up a network of documentation centers across Benin. These centers act as local libraries, and provide students, parents and teachers the opportunity to read and conduct research. IFESH, which set up a number of documentation centers, uses these spaces as centers for training. A number of NGOs participate in assisting and training teachers. These organizations are useful partners in the creation of a greater understanding of the NPE. IFESH is completing its second phase of teacher training. Together both phases have covered all Departments in Benin. The purpose of the IFESH program is to increase the capability of inspectors, pedagogical advisors and heads of pedagogical units to ensure continuing education of teachers. Data is not collected on the number of teachers leaving the profession and for what reason. Interviews suggest, however, that a considerable number of trained teachers is leaving the system each year. There are 4,500-plus public and private primary schools in Benin. More than 22,000 teachers provide instruction for 1.15 million students. Yet, annually the system is in need of roughly 4,000 teachers. In 2001-02, there were 276,577 students in CI and 223,893 in CP. Together they represent 43% of the total primary school population. The Ministry of Primary and Secondary Education (MEPS) consists of at least 18 departments that have undergone several permutations in the last decade. The Ministry of University Education (MESRS) is now a separate institution and has appropriated the teacher training colleges, (Ecole Normales des Instituteurs, ENI) in Parakou and Lokossa as university branches. As a result, these former ENI no longer provide teacher training. At the same time, MEPS’ departments seem to have overlapping and sometimes ambiguous functions, and it is not clear that either INFRE or MESRS contribute effectively to teacher training or development. Evolution of the Ministry of Education - Benin 1990 MEN Ministère de l’Education Nationale (Overseeing Maternelle ‡ Université) 1996 MENRS Ministère de l’Education National et de la Recherche Scientifique 2001 MEPS METFP MESRS Ministère d l’Enseignement Primaire et Secondaire Ministère de l’Enseignement Technique et de la Formation Professionnelle (Includes nursing, engineering, agriculture etc.) Ministère de l’Enseignement Supérieur et de la Recherche Scientifique Evaluation of Benin's Basic Education Reform Pedagogical Component USAID – Benin Louis Berger/DevTech 35 Primary education consists of six grades, or three cycles. The exit exam for primary school is the Certificat d’Education Primaire (CEP.) Secondary school follows with seven years in two cycles. After the first cycle of “collège,” a student receives a diploma of “Brevet d’Etudes du Premier Cycle” (BEPC.) The second cycle terminates with the “Bacalaureat” or BAC. One is eligible to teach or prepare for further teaching licenses after the BEPC. In the case of communautaires, many have been hired with less than a BEPC. Primary School Cycle 1 Cycle 2 Cycle 3 CI Cours d’Initiatio n CP Cours Préparatoi re CE1 Cours Elémentaire CE2 Cours Elémentaire CM1 Cours Moyen CM2 Cours Moyen 6yo 7yo 8yo 9yo 10yo 11yo CEP Diploma Secondary School collège college 6th 5th 4th 3rd 2nd 1st « terminale » 12 yo 13 yo 14 yo 15 yo BEPC Diploma 16 yo 17 yo 18yo BAC Diploma Women in Teaching The percentage of female teachers has been declining steadily since 1999. Although equity in student participation has been rising, more effort is needed to provide models for young girls to promote recruitment of female instructors. Percentage of Female Teachers Year 1999 2000 2001 2002 2003 Percentage of female teachers 30.15 % 27.97 % 25.57 % 24.32 % N/A DPP/MEPS-SSGI 2002 Teachers Teachers are recruited in three ways: as community teachers, contractuals and civil servants (Agents Permanents de l’Etat or APE). Community Teachers are recruited and paid for by the community. The Association des Parent d’Eleves (APE) may elect to use the subsidy given to each public school to pay community teachers. In many cases, parents supplement this subsidy to cover the costs of additional teachers. These have often been the only teachers present during a strike. They are paid for the time that they are employed and they can be hired at any time during the school year. Contractuals work under contract with the state for the academic year. Their pay is for ten months. APE are civil servants. These teachers either attend the teacher training college or are recruited directly by the state. Their appointments are de facto guaranteed with pension. Teachers hired as APE may elect to undergo additional exams to become a Evaluation of Benin's Basic Education Reform Pedagogical Component USAID – Benin Louis Berger/DevTech 36 conseiller pédagogique or an Inspector at which point they will leave classroom instruction. Within the public system, APEs receive the highest salaries and are paid regularly. There are, however, salary arrears resulting from their recent reclassification. Contractuals receive a minimum of Cfa 35,000 per month (about $50), while community teachers earn as little as Cfa 15,000 to Cfa 20,000 per month (about $30). In fact, community teachers are often paid less than the official minimum starting salary unless alternative arrangements for in-kind payment are negotiated with the school director and the APE. In recent years, salary issues have been at the center of continuous strikes. In 2002-2003, the strike began informally in October by contractual teachers and became a formal strike in January when various teaching unions banded together. The strike has affected the Ecole Normale, the regularity of student attendance and parental confidence in the public system. It has also clearly affected in-service training for teachers, as UPs do not meet, CPs cannot visit teachers and NGOs have difficulty organizing training sessions. The strikes are therefore a serious handicap that have had and continue to have a very negative effect on the overall education system. Pathways to Teaching As yet, the ministry has not developed a systematic plan for in-service training. As a result, follow-up to initial NPE training of four weeks conducted during the summer holidays is limited to the UP and CP visits. A two-week “Recyclage” occurs only in the year following initial training. Training provided in the UP and via CPs were severely limited by the prolonged strike in 2002-03 and by previous strikes in the past three years. Instead, each Direction Départementale des Enseignements Primaire et Secondaire (DDEPS) was left to its own devices to promote greater understanding of the NPE. ANALYSIS Teacher Preparation Benin’s increasing reliance on teachers with 10 or fewer years of education seriously undermines its ability to meet current and future needs for a qualified Recruited by the Community… Less than BEPC ‡ recruited by parents ‡ Community Teacher (before 2004) Min. BEPC (from 2004 onward) ‡ recruited by parents ‡ Community Teacher By going to L’ENI… BEPC: Inst. Adj. Stag. ‡ 2 yrs.ENI ‡ CEAP : Inst Adj ‡ 2 yrs pratique ‡CAP : Inst. Ordinarie/Titulaire BAC: Inst Stag. ‡ 1 yr ENI ‡ 2 yrs pratique ‡ CAP : Inst. Ord. Without going to L’ENI and recruited by the state… BEPC: Inst. Adj. Stagaire ‡ 3yrs teaching ‡ CEAP: Inst. Adj. ‡ 3 yrs teaching ‡ CAP:Inst. Ordinaire/Titulaire BAC: Instituteur Stagaire ‡ 3 yrs teaching ‡ CAP: Instituteur Ordinaire/Titulaire Evaluation of Benin's Basic Education Reform Pedagogical Component USAID – Benin Louis Berger/DevTech 37 and dynamic cadre of teachers. There is a need for teachers who have the ability to implement the complex methods of the NPE. Teaching is a profession that requires more than a love for children, but an understanding of how children’s development affects their ability to acquire, retain and use information. No credible arguments have been put forward to convince the evaluation team that current levels of academic preparation and training for teachers in Benin is sufficient Improvement of Training Two issues surface with regard to current teacher training: (1) Training does not model and (2) Training encourages very limited participation. Essential to any new learning is the opportunity to practice a technique. This is true irrespective of educational level, but likely more crucial when presenting to those with limited education. Real learning occurs after a series of opportunities to practice. This is one of the most important shortcomings at all levels of training. Our observations of training delivered by persons considered the most qualified in Benin were evidence that a frontal, teacher-centered method, using little or no audio-visual aids, is still very much in use. In effect, the trainers of trainers are still using the old system of education that relies on the lecture approach. Evidence shows that the Pilote de Formation has attempted to respond to this enormous problem through the “cogestion sequence de classe.” It is not certain, however, that this limited exposure to participatory techniques can respond to the need for massive re-education in teaching methodology. There is no plan for systematic in-service training as a follow-up to initial NPE training. The existing network of UPs as a forum for in-service professional development has not functioned regularly due to strikes. Teachers and directors overwhelmingly laud the existence and assistance of IFESH and request that they carry on; yet the evaluation team saw no observable differences amongst teachers trained by IFESH. (IFESH works with CPs and inspectors to increase their capacity to support teachers, and with teachers to reinforce their capacities to apply the new teaching approaches.) A clear plan is needed to ensure that MEPS can recoup its investment in training. There is no system to train new teachers during the academic year even when vacancies go unfilled for months. Outside of initial training that occurs only during the year of generalisation, new teachers hired during the academic year do not have the opportunity to become well versed in the NPE. Community teachers hired in recent years did not receive CI or CP training, and collectively expressed the need and interest to learn how to implement the NPE program. While this task may fall to CPs during their periodic school visits and during UP sessions this task is not being accomplished. Training documents are not user friendly. The documents should encourage use without outside instruction. They emphasize writing exercises and discussions about teaching rather than teaching. This is also an issue of modeling. Training documents lack structure as reference documents, basic information on child development, and psychology and practical examples based on everyday situations related to the NPE. The documents tend to be too text rich. The teacher guides available ignore the fact that the majority of teachers have neither the level of education, nor the training Evaluation of Benin's Basic Education Reform Pedagogical Component USAID – Benin Louis Berger/DevTech 38 necessary to implement this rather complex program. To be useful, the guides should include illustrations and examples of lesson plans, classroom arrangement, sample tests and pages that are useful to teachers and possible for them to photocopy. The guides require a large number of lower-order thought activities -- finding, re￾stating and summarizing information. This is useful as an orientation to the program. The guides do not, however, provide enough balance by using application, evaluation and synthesis opportunities for teachers and directors to practice the NPE. Training does little to encourage teachers to use questioning techniques There is no evidence that the NPE, (which says it promotes autonomous learners,) teaches or describes how instructors can elicit questions from students or encourage autonomous learning. No one addresses the fact that asking questions – especially a child of an adult – is contrary to the home culture as well as to public life. Most parents and teachers want students to be able to compete in the global economy. Interviews with various stakeholders reveal that a student’s ability to show curiosity and search for answers based on that curiosity is a good thing. This spoken desire for curious students contradicts basic everyday interactions. Beninese social exchanges illustrate that questioning between individuals of unequal standing is uni-directional. There are no benchmarks for teacher training MEPS has no standards or criteria to identify skills that are required for an excellent trainer. Training participation is not used as a means to control for quality by requiring teachers to accumulate a certain number of training hours and to be able to demonstrate knowledge from such training. Organizational shortcomings The various MEPS’ departments involved in training do not coordinate their efforts. The process suffers from over-compartmentalization: each unit treats its mandate as separate and apart from work being done in other units. The effect of low salaries and training per diems Interviews suggested that a large percentage of teachers participate in training for the per diem rather than for the information offered. The resulting effect is that impedes their likelihood to fully participate in and implement training. The explanation given is that teachers’ low pay makes the per diems an attractive way to augment their salaries. Poor feedback and diagnostic system Deliberate and consistent information collection is the backbone to an effective training program. Currently, however, it is unclear how many participants actually attend training, who they are, where they are posted, or how many leave the profession and for what reasons. It was clear to the evaluation team that the office charged with program evaluation does not function in a systematic manner. Teachers receive little or no instruction on how to evaluate students or themselves. If we accept the fact that teacher strikes have disrupted a teacher’s ability to receive pedagogical support, we must also recognize the need to provide teachers with tools that can be used independently. The evaluation team found teachers Evaluation of Benin's Basic Education Reform Pedagogical Component USAID – Benin Louis Berger/DevTech 39 who were interested in their own professional development. Developing auto evaluations that can be used by teachers or between teachers and directors is a means to improve implementation of the NPE. Many directors responded that training is not fully effective because teacher assignments do not reflect the grade for which they were trained. Respondents noted that training goals and objectives are not clearly stated to convey that the methodologies contained in the NPE are crosscutting and applicable to all primary grades. Parents have been excluded in the training and generalisation of the NPE Parents need to have basic information when major changes are made that affect students. This basic information should include expected outcomes and ways in which they can support the change at home. The effect of excluding a major stakeholder and consumer of the program is to create a potential constituency of dissenters. No systematic plan exists to treat the issue of academic preparation A decision to reduce the hiring has shifted the burden of teacher recruitment and payment from the Ministry to the community. As many of these individuals are not qualified, training methods and content must be at a level that they can understand and utilize. With respect to CPs, it is questionable that they possess the skills and attitudes to convey the philosophy and methods of the NPE. The evaluation team held extensive discussion with a group of CPs currently in training in Porto Novo. They report being without guides or materials, and expressed difficulty in understanding the NPE, as well as discontent with the progress of their training. They also complained of the lack of funds to permit them to travel and visit to schools in their service districts. They receive no compensation for these out of pocket expenses, thereby reducing their salary and certainly reducing the will to perform their tasks. The MEPS relies on CPs as tools to promote and deepen the NPE in the field, yet the investment made to ensure this capacity appears wholly inadequate. RECOMMENDATIONS A teacher’s ability to implement any new program or style of teaching is heavily dependent on the quality and frequency of training. The extent to which this training is well-planned and details with specificity the goals of its implementation, will determine its effectiveness. A “cascade” approach is effective only when 1. training is highly structured and organized in both content and methodology, 2. it is followed by repeated in-service training and 3. trainers are motivated and capable. Initial training is nothing more than an introduction without regular and frequent opportunities for in-service training. The use of support materials – guides and visual aides – further improves the possibility to repeat with fidelity the initial training. The effectiveness of training should be verified by monitoring classroom instruction. Improved student success can be linked to a well-trained teacher who is in turn supported by a network of educators well versed in the nuances of the same program. Teacher Education Evaluation of Benin's Basic Education Reform Pedagogical Component USAID – Benin Louis Berger/DevTech 40 With respect to academic preparation, MEPS should specify minimum requirements for teachers. The BEPC diploma should serve as a minimum requirement for all new teachers (including community teachers). It is also desirable to increase this minimum to a BAC within the next few years. The BAC will at least provide a greater foundation in general knowledge necessary to teach in today’s world. It will also serve as an initial measure to control the quality of teachers entering the system. There is also a need to create local teacher training colleges and to establish a national criterion for the number of credit hours necessary to become a certified teacher, which can be based on the existing ENI program. Final certification could also be tied to the number of classroom teaching hours. We also recommend providing incentives to the private sector to create local teacher training colleges. Immediate need for CI and CP teacher training Consolidate gains already achieved in CI and CP before they are lost. There is an immediate need to re-train or provide initial training for teachers in these grades, where student performance is lacking. The evaluation team observed that due to attrition, there are a large number of teachers currently responsible for CI and CP who did not benefit from initial training. As previously stated, we know that the strike has disrupted UPs and NGO intervention. For this reason, it is imperative that CI and CP teachers receive (re)training and be able to reap the improvements made to training over the past three years. The evaluation team recommends that as of summer 2003, CI and CP teachers be re-trained in NPE methodologies with an emphasis on the methods of instruction for large groups, and teaching a second language. Immediately reformulate the CP curriculum at the Porto Novo training site In the six months remaining, CPs being trained in Porto Novo should receive full sets of the NPE materials including student books, training guides and intense instruction using demonstration and active participation methods. The new CP curriculum should include strategies for working with teachers of large groups and teachers of second language learners. The trainers must include highly trained individuals who have a profound understanding of the NPE. Initial and in-service training should be considered together The plan would be based on a cyclical system of Training ‡ Application ‡ Assessment ‡Re-Training ‡ Application ‡ Assessment. To exploit the totality of participants’ experience, require teachers to prepare experience-based questions during retraining and use as a forum for exchanging issues specific to their cycle. Improve the competencies of the teacher training staff The NPE advocates that teachers become facilitators of learning, while school directors should serve as trainers. NPE materials laud the goal of autonomous learners and the use of various learner-centered methods of instruction. During classroom observations, however, we noted that the practice of participatory learning is non-existent. For this reason, it is extremely important to ensure that teacher-trainers be experienced in applying the NPE system, as well as expert in training methods. It is also suggested that USAID/MEPS investigate using expatriate training specialists to bolster competencies, as the present situation is a reflection of limited capacities in MEPS. Evaluation of Benin's Basic Education Reform Pedagogical Component USAID – Benin Louis Berger/DevTech 41 MEPS should overhaul the pre-service training by retraining its original encadreurs. This is an opportunity to ascertain whether encadreurs are training in the same manner. The training specialists should assist in training all CPs. Finally, trainers must be able to model questioning techniques with Directors and Teachers. It is essential to reinforce the “cascade” approach with the partnership, assistance and guidance of said training specialists. The training specialist should focus on teaching methodologies (not content.) Training specialists should be able to demonstrate examples of teacher and student initiative and what methods and behaviours lead to greater participation. In any learning situation, the student/participant must have the opportunity to practice the concept within an appropriate context and time at least three times before it can be considered learned behavior. Training documents and teacher guides should place maximum reliance on participatory techniques and include questioning behavior into observation forms. Teaching by demonstration It is imperative that teachers be able to see first-hand how the NPE system is applied. As we discussed elsewhere in this report, demonstration/laboratory schools attached to documentary centers could serve to demonstrate and disseminate best practices. The documentary centers would then become a resource location for ideas and materials. Training is too long. The NPE contains an enormous amount of information with new strategies and techniques. One cannot expect that participants will retain all the information presented and retain at the same rate on day 1 as on day 24. To maximize retention of information, MEPS should consider two options for continued generalisation: 1. Reduce training to two weeks and move to a time immediately before the beginning of the school year so that teachers may immediately apply the program. Classroom visits to monitor teachers should begin shortly after teachers return to the classroom. 2. Two weeks training, followed by application of learned skills in demonstration classes for at least two weeks so that teachers and directors have the opportunity to practice the methodologies. Simplify training It is clear from information in the field that there exists a dissonance with regard to the expectation of training. Training should be limited to several key components. Examples of key components can include; how to use the guides, NPE methodologies, implementing the lesson plan and assessment and feedback of student performance. Trainers will then have clear and specific objectives that are continuously communicated to participants. To maximize retention of training information, MEPS should consider two options for continued generalisation: 1. Reduce training to two weeks and move to a time immediately before the beginning of the school year or 2. Two weeks training followed by two weeks of “cogestion sequence de classe” so that teachers and director have the opportunity to practice the methodologies. Evaluation of Benin's Basic Education Reform Pedagogical Component USAID – Benin Louis Berger/DevTech 42 During training and in all training documents, place less emphasis on the lesson plan as a document and more emphasis on the lesson plan as an outline for instruction. Train teachers to clearly communicate the purpose of the lesson and fulfill the “projet et retour.” Specify training outcomes with learning objectives Specify to participants what information will be covered during training and what information remains their responsibility. Distinguish crosscutting strategies from grade content during training. Specify what portions of the documents are most useful while teaching. Participants should begin training with a full understanding of how to use the guides and curricula once they are in the classroom. Organize and improve training documents Create distinctions in the guide between strategy and content. Include information on how teachers can evaluate the success of their own lesson. Include elements of evaluation: (pre/post/self) as a part of training and model its uses and behaviors. Teacher collaboration and exchange Share teaching skills and promote collegial exchanges by training APE/ Contractual and Community teachers together. Although each group enters the training process at varying levels of education, building on the experience of more knowledgeable teachers (1) builds confidence by taking their experiences into consideration and (2) assists less knowledgeable teachers with difficult classroom issues. Joint training for administrators Train inspectors and school directors together in educational management and educational leadership. Solutions to resolve teacher salaries MEPS should consider improving teacher’s living conditions by, for example, subsidizing teacher housing, establishing investment plans to increase pension or providing travel allowances. Internships Provide internship opportunities for university students to teach in schools. University students can be used to alleviate the teacher shortage. Mobilize university students to teach French in primary schools by developing a university class that provides NPE training, child development and pedagogy. The mutual benefit arises from university students being able to earn money and schools providing low performing students with additional opportunity to reach appropriate levels. Regional network for resource sharing USAID and MEPS should work with Central and West African education ministries of primary education to promote exchanges. The system could operate through virtual and actual visits and forums. The exchange can serve as a basis for sharing best practices in similar African environments and to make available instructional resources and materials to teachers. Media support for instruction and teacher training The NPE is considered as a curriculum developed and imposed from the outside. Create radio and TV campaign on the NPE to clarify what it is and what it aims to Evaluation of Benin's Basic Education Reform Pedagogical Component USAID – Benin Louis Berger/DevTech 43 achieve. Continue with the development of a video program on NPE to be distributed to video clubs across the country. Support such initiatives and provide for countrywide dissemination. Create a national day for parent open houses on the NPE, based on the initiative taken by several schools across the country. Sustainability An enormous amount of effort has been put into planning and implementing training for the NPE. There is near unanimous agreement that the NPE has many elements that are necessary to prepare a child in the 21st century. Changes to documents are made annually that reflect improvements based on identified needs. This shows an ability to be responsive to the needs in the field. On the other hand, many of the recommendations put forth in this report repeat those made previously. In order to recoup the investments in training made since 1999, MEPS and the Cellule must make immediate decision where to spend their efforts; expand to CM or retrain CI and CP. MEPS must consider that the long-term effectiveness lies in using a well-educated teaching force. Recognizing that the NPE is a Beninese program, educators must engage in a serious dialogue regarding the conflicts between the program and Beninese culture. Documents used for training require better organization and training must promote participation and model methodologies. The latter is perhaps one of the most difficult tasks, as it requires changes in behavior. It is however, the key to make training meaningful in the classroom VI. STUDENT ASSESSMENT AND SCHOOL ORIENTATION Terms of reference iv. Assess the conformity of the new system of student assessment and school orientation with regard to the learning objectives, approaches and strategies as recommended by the new programs; vii. Develop, in collaboration with “pilots”, MEPS and USAID technical staff, a periodical evaluation plan which will help assess the quality, relevance, progress achieved and the results obtained by all five components; viii. Assess the performance of first grade (CI) students on the basis of a representative sampling; and xi. Give recommendations for improving the implementation of the reform program. Introduction This section of the report describes student assessment and school orientation within the overall context of efforts to implement pedagogical reforms in primary education. The purpose of the evaluation was threefold, and includes: 1) Assessing the conformity of the new system of student assessment and school orientation with regard to the learning objectives, approaches and strategies as recommended by the NPE; 2) Assessing the performance in French and Math of first grade (CI) and second grade (CP) students on the basis of representative sampling; 3) Develop, in collaboration with the MEPS “pilots”, with MEPS and with USAID technical staff, a periodic evaluation plan to assess the quality, relevance, progress achieved in the area of student assessment and school orientation. It was decided that objective (vi) of the terms of reference, “assessing the recommendations of the Pedagogy component consultancy reports funded under the CLEF project within the framework of activities directly related to the Evaluation of Benin's Basic Education Reform Pedagogical Component USAID – Benin Louis Berger/DevTech 44 generalization of the new programs and their implementation,” would be a good starting point for the approach of this component. It is instructional to begin the report with expectations (what should have been done according to NPE documents and strategies, objectives of USAID and other reports from CLEF). Findings and recommendations follow this. As for the methodology for this component, we have reviewed the project documents and reports. This data we expanded on during conversations with MEPS, CLEF and CCGNPE consultants. While in the field conducting school visits, additional information was gathered from different stakeholders of Benin’s school system. After a midterm presentation on the status of our evaluation study to USAID and MEPS representatives, we made a number of adjustments to the data collection process. These changes were discussed during meetings with the CCGNPE Pilot of the “Plan d’Action” Student Assessment and School Orientation and the MEPS National Evaluation Team. Their recommendations were incorporated in the instruments used in school survey activity. The Evaluation Program as designed Based on the NPE’s objective to provide a more equitable education, we expected a continually increasing and equitable rate of school attendance. We expected to find an operational school equipped with the same resources, whether in rural or urban area; a school that offered the same opportunities to males as well as to females. In addition, we expected to find the school system’s goal of providing a quality education on an equitable basis preparing the children to play a productive role in society in operation. This implies the existence of an appropriate structure to achieve quality and equitable learning, and the implementation of measures and practices leading to the eradication, or at least the reduction, of school dropouts and classroom repetitions. Those were our modest expectations when we set out to assess this component “Student Assessment and School Orientation”. Based on a review of the teacher guides, Student Assessment and School Orientation documents, and other training reports in evaluation, we expected to find an existing methodology of evaluation with an effective feedback mechanism to students and to the system. We also expected (i) a definition of objectives and skills to be evaluated by the teacher, (ii) a written evaluation policy describing student control and monitoring strategies, (iii) corrective instruction and feed back instructions to students, parents and school administration. During our site visits, in addition to the questionnaires, we also used information sheets, our own questions and concerns emphasizing the focus of the study. From our perspective, student assessment and documentation programs have not been implemented. As a result, programs and investments proceed without being properly assessed. This has had enormous implications in the implementation of the NPE, because each successive step of the process is predicated on the success of earlier initiatives. However, as this evaluation study highlights, learning is not occurring, and many educators are not aware of the situation. There is significant divergence between the documents developed by the Cellule “Plan d’Action Evaluation et Orientation Scolaire” and practices in the field. Within the school system, there is a general tendency to interpret the concept of evaluation as testing. As a consequence, as far as evaluation practices are concerned only test results are considered, while daily and monthly class evaluations are ignored. Evaluation is not yet taking place. What takes place in the schools is measurement Evaluation of Benin's Basic Education Reform Pedagogical Component USAID – Benin Louis Berger/DevTech 45 (exams) to which a norm or a criterion is applied to make a decision. One might think they have evaluated, but without analysis, evaluation has not taken place. Review of Documents Evaluation documents made available to the trainer and the teacher are well presented, and the content informative. They summarize succinctly different types of evaluation, and offer, besides goals and objectives, reasons for evaluating. However, we noted the following points in the teacher’s guide to evaluation of learning in CI and CP: (i) Documents do not state in their goals or objectives the advantages and/or vision of evaluation or that evaluation could provide new information to inform the school system and raise questions regarding the curriculum. For example, the evaluation could help assess the relevance/coherence of the curriculum with the psychomotor development of children. (ii) The role parents should play in the system is absent from the subjects of the ‘Guide d’Evaluation’. Yet, parents are not only the first consumers of the evaluation products but also important catalysts of the learning process. (iii) The document describing orientations and objectives of the “Plan d’Action Réforme des Programmes d’Etudes” does not offer a detailed specification of the profile of the Beninese child in terms of skills, abilities, and attitudes measurable by an evaluation. In addition, there are several documents related to evaluation that have not been approved by the hierarchy or implemented in the field (classroom). That is the case concerning the definition of a national policy of school and professional orientation in the Primary school (Projet de Politique nationale d’orientation scolaire et professionnelle), of the student reference book (Cahier-temoin), and of the School orientation notebook (Livret scolaire). Why these documents have not been approved was unclear and some are probably out-of-date now. Observations and Interviews Class management Considering the unique conditions under which our study was conducted – during the teacher’s strike - there was little evidence that teachers or directors had any idea of the performance of each of the individuals in any one class. The team noted large disparities between the official number of students enrolled in classes and those actually present. The team visited classrooms where the teacher was not sure about the number of his students, especially since absenteeism had risen with the cancellation of classes. In those cases, the teacher could not say whether absent students had abandoned school or were simply away for the day of our visit. Testing and measurement In the vast majority of public schools, there has not been any testing this year due to the strike. In most cases, the teacher had no report or notes to indicate the total number of students who sat for tests the previous years. Class repetition, dropouts and system attrition According to the World Bank Report on the Benin School System (January 2002), Benin’s retention rate in primary school was at 34%; in terms of equity, the rate was much lower: 22% for rural girls. The absence in schools of internal reports with Evaluation of Benin's Basic Education Reform Pedagogical Component USAID – Benin Louis Berger/DevTech 46 reliable data does not allow for the corroboration of these figures. In some classes, dropout and repetition figures were written on the chalk or bulletin board with the number of students in the class, but this could not be observed everywhere. In a few cases, the figures marked on the board were not updated. The repetition rate (25%) is too high Teachers are convinced that the high repetition rates are a discouraging factor both for students and their parents. Many parents interviewed thought that their sacrifices were not worth it if children are frequently required to repeat the year. . Formative and summative evaluation strategies Although documents and school authorities affirm that the new curricula give priority to formative evaluation (Guide: Formation des Directeurs, 1998 p. 77), only summative (end of term/year) tests are administered in schools, and then only three times a year. We noted confusion between summative evaluation and normative evaluation. Several people see summative tests more as a means of selection to the extent that it is not used as a pedagogical tool. Lack of teacher understanding of techniques and value of evaluation With regard to evaluation, the situation in the classroom is alarming. Many teachers declared that they did not receive training relating to designing test items. In any case, one thing is clear, as a teacher from Mono summed it up “Nous ne savons pas faire une évaluation” (we don’t know how to evaluate,) a common statement when asked about the sad state of student performance. In another Department, a school director shared with indignation that “teachers are not equipped to evaluate, we don’t have the mastery of the evaluation tools; we cannot evaluate students. Nobody wants to train teachers to design the specification tables. Training for evaluation is kept as a secret reserved for a minority.” Diagnostic processes To date, during the four years since the commencement of the generalization process, there has not been any analysis of student performances at the central, dDepartmental, district or school level. MEPS officials observed that the expertise is available, but resources are lacking. Whatever the case, the system is unable to provide classroom information about student performance. Consequently, there is no basis for appraising the value or impact of USAID support. Restitution of evaluation Restitution in the field consists of sending test scores to parents after monthly and trimester tests. At the end of the term, parents’ meeting is organized, but the report given to them is limited to a report card with test scores from the summative exam. Teachers said that now and then, some parents whose children did not pass the test would approach the teacher to inquire about the failure. At that time, the teacher would give them explanations and suggestions. Most of the time they suggest hiring a tutor to help the child at home. Most school directors report that they usually met with teachers to discuss test results, but that none of them had minutes of or a memorandum from those meetings. As a result, reports we consulted at Circonscription offices only contained end of term or end of year test scores. There was no substantial information to inform administrators regarding the reasons for or state of student performance. Evaluation of Benin's Basic Education Reform Pedagogical Component USAID – Benin Louis Berger/DevTech 47 ANALYSIS The gap in evaluation between documents and the reality in school is the result of a number of factors that make evaluation unappreciated: · A broad, negative attitude towards research and documentation; · Few specific resources allocated to evaluation; · The poor quality of evaluation training at all levels of MEPS; · Lack of sensitivity toward parents · An instructional method that still gives priority to teaching inputs rather than student performance. Lack of understanding of the tenets of the NPE is pervasive The NPE calls for learning based on objectives, while the traditional intermédiaire teaching approach prevails universally in the schools visited. In other words, it is the teacher’s time and authority that regulates the learning process. Evaluation is conducted through tests of knowledge. Although tests of knowledge are also used in learning based on competencies, this approach particularly stresses evaluating the mastery of competencies. The NPE results from Benin’s goal to “educate active children, intellectually autonomous, able to react adequately to various life situations, able to innovate, create, take responsibilities and make decisions.” To achieve this goal, they chose learning by competencies. A system geared to competencies concentrates on the learner’s needs, and progress is assessed from the learner’s mastery of target knowledge and abilities. Pedagogical methods call for a learning process geared to mastery -- a principle based on the assumption that all learners can master the required knowledge. Participant progress in the program is individual, based on performance of specified competencies. Competencies to be mastered are not clearly described, checked and listed in advance. The criteria and conditions under which the performance can and will be evaluated should also be listed and made known in advance. Therefore, it is the curriculum that sets in motion individual development and the evaluation of each of the specified skills. As a result, evaluation by competencies must take into account the participant’s knowledge and behavior but the most important factor remains his/her performance of the specified actions as the primary source of evidence of their mastery. The philosophy of the NPE also means translating the profile of the Beninese child into specific measurable skills, developing and disseminating curricula relevant to that vision, well-trained teachers in classrooms, and an effective evaluation and monitoring of students’ performance. In the learning of the French language for instance, children’s performances should reflect the different phases of the process that the teacher is invited to follow, i.e. initial exposure, followed by analysis and acquisition. This is followed by reinforcement of the acquired skill and attitude. A next phase of learning is synthesis, and the reinvestment or usage of the acquisition followed by a guided application focused on autonomous replication. The evaluation of those competencies must also follow the same process, which includes preparation, production, correction or remediation based on learning levels. In Mathematics, the process goes from the discovery of the material to the use of the new concept acquired in various situations via manipulation of objects or materials to solve a given problem, and the representation of situations or Evaluation of Benin's Basic Education Reform Pedagogical Component USAID – Benin Louis Berger/DevTech 48 progressive departure from real manipulations to solving problems with drawings, diagrams, tables, sticks, etc, and via remediation after an evaluation. Language aptitude testing is non-existent Considering the entry-level maternal language skills of children one might expect a teaching method that will transition children into the use of French. The team found no instance where teachers began activities in maternal language. Far from denying the quality and benefit of French immersion, it is, however, appropriate to assess students comprehension level of what is said to them in French. Even when students don’t answer a question correctly, nobody can say for a fact whether it is because they don’t understand the question or they simply don’t know the answer to the question. For that, one needs to run a discrimination test in their mother tongues and the instructional language. Importance of training Trainees tend to teach in the same manner they have been trained, and absent the appropriate training technique, NPE users quickly fall back into the role of the traditional teacher. In the final analysis, while one may exhort use of a training method geared to competencies, if the teaching method is not grounded on competencies, one is less likely to see teaching and evaluation geared to competencies resulting from it. Repetition Patterns More than high rates of repetition rates, it is the mere acceptance of repetition by the system that is striking. If we agree with the assumption that except in special cases (too young, mental retardation, etc.,) every child is able to learn, the very high rate of repetition, particularly in CI is simply a reflection of a failed instructional system. It penalizes the child instead of readjusting the input (pedagogical action). If a valid measurement and testing system had been in place, the evaluation team would have found student levels in French so low that an appropriate repetition rate might well surpass 90 percent. Official repetition patterns have little meaning in a context where no measured or verifiable criterion is in place to judge who is to be retained or why. At a pedagogical level, although some teachers declare that making students repeat the year helps them learn faster the next year because they use them as group leaders, the great majority of teachers say that they hinder the normal progress of new students, mainly in CI. The danger is that a teacher who is less informed, less trained, worried by the amount of the curriculum to cover for the year or simply terrorized by the school director might simply focus attention on the more advanced students. Evaluation under NPE The evaluation system adopted by the NPE applies the following principles: · A commitment to provide adequate resources and pedagogical materials. Activities must support stated objectives. · There must be continuous interaction and feedback between participants. · Trainers should provide competency-based training. · Participants need to assimilate fully this new approach as it differs considerably from their past educational and training experiences. Evaluation of Benin's Basic Education Reform Pedagogical Component USAID – Benin Louis Berger/DevTech 49 · The trainer is most efficient when he/she plays the role of Facilitator, rather than being a source of information, and spends most of his/her time working with individual students or in small groups rather than lecturing. · Much of the training time is spent evaluating each student’s competence to perform specific activities. · Finally, and more importantly, students progress in the program at their own pace. Evaluating in a learner-centered learning process consists of more than pedagogical techniques that allow the teacher to manage the class in groups and motivate individual work. It also means individualized monitoring of the student performance, constant analysis of his/her capabilities, constructive feedback to the student as well as to his/her parents and the system. All of it, with the goal of correcting his/her weaknesses, reinforcing his/her strengths, and gearing the child towards areas/studies relevant to his/her competencies and motivation. RECOMMENDATIONS Implement individualized instruction and evaluate accordingly A student-centered learning process requires individualized monitoring of the student’s performance, constant analysis of his/her capabilities, constructive feedback to the student as well as to his/her parents and the system. There is a need to establish a sense of primacy that student’s performance (mastery of knowledge and competencies/attitudes) should be the final output and the priority of the learning process. It is imperative that radical change takes place in the school culture, which seems unable to grasp the place of parents in the educational process. The only constant factor in educational research is the impact on children’s learning when the school works closely with parents, no matter what their social status or level of instruction. The school system ought therefore to establish a permanent dialogue with parents, effective cooperation, and information sharing -- including about difficulties -- in order to individualize the learning process and successfully conduct the educational undertaking. Institute a regular and periodic diagnostic testing system of the children’s competencies in French and Math in CI and CP. We recommend in the short term, or immediately, the preparation and administration by the National Evaluation Team of a diagnostic test of knowledge in French and Math in the first two cycles of primary school. As a result of the teachers’ strikes taking place every year since the generalization of the new curricula, French and Mathematics levels of CI and CP students, even CE1 and CE2 students, are a cause of grave concern. The administration of this test could be supervised by the Departmental authorities, but we recommend the test administrator be different from the classroom teacher. The test should take into account the principles of evaluation of knowledge, and the objectives should be defined in advance. We would strongly recommend that in the cases where the results reveal a much lower level than the required competencies, the generalization of the NPE be refocused on CI and CP to ensure that students at these levels are succeeding. Development of a feedback/evaluation culture and the setting up of an evaluation/feedback process in MEPS Evaluation of Benin's Basic Education Reform Pedagogical Component USAID – Benin Louis Berger/DevTech 50 We recommend that MEPS institute a system-wide evaluation methodology as well as a sound monitoring system of school activities. This strategy will require a constant analysis of teacher and student capabilities in order to correct weaknesses and reinforce strengths and motivations. MEPS should require from evaluators written reports of their observations in order to give constructive feedback to students, to parents and to managers of the education system. Existing workbooks and reference books as well as copies of answers to tests are also useful feedback and evaluation tools to achieve individualized learning. Evaluation as a component of NPE training Evaluation strategies must be included in training in the NPE. We recommend that training workshops or refreshers for teachers include an evaluation component. In a learning approach geared to competencies, training participants should be prepared for an evaluation focused to competencies at the moment of the training. Test new models to involve parents In order to assist parents, particularly poor and unschooled parents, to support pedagogical action, we recommend that MEPS explore the strategy of Community Learning Centers (CLCs) as implemented in Mali. CLCs involve the collaboration of the entire local community (parents, administrative authorities, as well as the private sector and religious organizations) with the school. Together they provide classrooms with volunteer tutors (teachers, retired civil servants, advanced students and parents) to recreate a learning environment for children after their regular class hours. Evaluation personnel professional upgrading MEPS evaluation’s capacity should be strengthened by the availability of full-time and qualified personnel. Since evaluation is a non-stop activity the whole course of the year, we don’t think that an office of one person assisted by a team composed of people who work on call can efficiently do the job. If the NPE generalization plan is to be successful, it is necessary to provide the Evaluation component with the necessary financial and human resources. A national documentation network We also recommend that MEPS identify and provide for pedagogical documentation on training and evaluation geared to competencies. Located in the different “Centres Documentaires”, such documentation will revive the role and use of the Documentary Network as a pedagogical tool, while at the same time it encourages research by trainers and perhaps their trainees and students. Design specification tables We recommend that the Pilot ‘Plan d’Action Evaluation’ in collaboration with DDEPs ensure that inspectors, pedagogical advisors, pedagogical unit leaders master the process of designing and interpreting specification tables. This way they can complete their training and teach teachers as well. It is worth stressing that the heart of the specification table is in the statement of what one is looking for and its definition in measurable objectives. Homework and parental monitoring We suggest that in the mid term all the partners of the Beninese school (MEPS, NPE sponsor, parents, private sector) make additional e fforts and investments to allow every student to have workbooks and manuals. There is an urgent necessity for students to do homework and be evaluated at home. Evaluation of Benin's Basic Education Reform Pedagogical Component USAID – Benin Louis Berger/DevTech 51 Cumulative Folders We recommend that the Evaluation Team encourage schools to establish cumulative folders of their school results in order to share with the system information about the performance of different classes and their school as a whole. Provide incentives to schools, teachers, students and parents Schools should institute an incentive system encouraging excellent performance by publishing lists of schools, teachers and students with the best performance. They could establish at the Departmental level the institution of a Teacher of the Year Award that would give the winner a prize of excellence and recognition of his/her peers. At the school level, there could be established annual prizes of excellence for the best students. The only selection criteria should be the mastery of competencies/students performance. Decentralize evaluation expertise We recommend that in the framework of the current progressive decentralization process, each Department have its own expertise in evaluation, and provide training in modules relevant to learning geared to competencies in the classroom. The goal is to equip each teacher to evaluate his/her students, including preparing a specification table for each test, documenting students’ work and providing substantial analytical reports of daily, weekly, monthly and/or quarterly student performance to use in internal and external reviews. Strengthen and support the National Evaluation Team Once the unit is strengthened, members of the team should be assigned the responsibility of a DDEPS. Together with the Departmental team, the evaluator will be charged with monitoring the evaluation activity at the Departmental level, ensuring the liaison between the Department and MEPS central office. Longitudinal studies We recommend that in order to document the NPE a group of students be monitored in a longitudinal study throughout their school life. Documentary network It would be appropriate to use Documentary Centers as part of a constant feedback to the system. Results analysis, sample tests, cumulative sheets and other statistics and school experience could be stored for research and to inform the system. However, this requires making the Centers useable, which they currently are not. Improve Teacher Reference Books We suggest that the Pilots of ‘Plans d’Action’ Curriculum Reform, Teacher Training, and Student Assessment and School Orientation work in close collaboration to revise reference books. These resources should document measurable skills and attitudes necessary for training geared to competencies. Report Card between home and school We recommend that the ‘Cellule Pilote du Plan d’Action Evaluation et Orientation Scolaire’ improve the report book for the parents by adding measurable aptitudes to communicate to parents every month. Evaluation of Benin's Basic Education Reform Pedagogical Component USAID – Benin Louis Berger/DevTech 52 Octobre Novembre Décembre Janvier Février Mars Avril Mai Ténacité Vivacité Curiosité Capacités d’adaptation Attention Assiduité Visa des Parents We recommend a letter-grading system that identifies the positive and negative aspects of the student’s aptitudes and behavior. We suggest as an example E (Excellent) B (Good) P (Fair) M (Weak) MA (Very weak). The marks on these items can be used to reward the students for their good work or to encourage them to do better. Evaluation should be based on the following spiral model This approach is based on the three cycles of primary school. In an effort to address the high rates of repetition and provide the NPE with an evaluation process adapted to the approach based on competencies, we suggest the model of a spiral evaluation described and represented in the diagram below. This model can be effective only in the framework of the subdivision of the Primary level into three cycles. (CI-CP, CE1- CE2, CM1-CM2) and under two pre-requisites: (1) each teacher is well-equipped -- particularly in the conception of specification tables and the development of test items -- to prepare and administer tests and to analyze their results; (2) except in emergency cases no teacher can be separated from his students (transferred) before the cycle’s end (2 years). Entry Test CI CP CE1 CE2 CM1 CM2 Entry Test Entry Test Timely Knowledge Tests Timely Knowledge Tests Timely Knowledge Tests CEP Summative Normative Ev Summative Normative Ev v v v Evaluation of Benin's Basic Education Reform Pedagogical Component USAID – Benin Louis Berger/DevTech 53 The spiral above represents the synergy within the educational system and continuity within each cycle. The model is based on a rotational system within a cycle, which requires a teacher to take care of a group of learners for the whole cycle. For example, at the end of a school year, the CI takes his learners to CP and the teacher of CP makes a rotation to take a new generation in CI, which he will take to CP. At the beginning of the cycle, thanks to his/her own training, experience and knowledge of the required competencies to begin teaching, the teacher develops and administers an entry test. This is just a diagnostic test to assess the knowledge brought in by each learner on arrival. In CI classes, we suggest a discrimination test in local language to have an idea of the true cognitive capacity of the child. During the school year, in CI as well in CP, the teacher should administer student knowledge and aptitude tests on a daily and monthly basis, as well as a summative test at the end of the term. All these evaluations play a formative role. It is at the end of each cycle (CP, CE2, and CM2) that those summative tests can be an indicator for student promotion. Essential to this model are new practices to assess students’ performance and involve parents. It is of little value to monitor a student’s work for one year and thereafter judge him on the basis of a one-test performance. We recommend that the result of students’ work should reflect the process, and should be the total sum of all the results obtained during the year rated equally, i.e. they should be weighed on the same basis. Work done during the course of the year gives a truer image of the student’s work, not the results obtained on a general exam at the end of the year. Besides, we recommend that the grading system be varied to recognize different levels of competency mastery. With respect to large class management, it is unrealistic to ask a teacher to follow the individual pace of 62 students in foreign language instruction, and to institute NPE programs in CI and/or CP. We recommend that the conception team, trainers and evaluators of the NPE together with the technical staff of the EQUIPE project define for teachers and students, training and evaluation strategies for classrooms of this size. We recommend that MEPS adopt a merit system for teacher promotion based on students’ results. In conjunction with the introduction of performance incentives, promotion on merit would help improve school results considerably. At the request of the Minister of Primary and Secondary Education (MEPS), we recommend that CGNPE follow up with the Minister’s Cabinet request to explore collaboration with ENI – Natitingou. The idea is to add an evaluation component to the ENI’s Curricula training; and organize national or international workshops for professional exchanges on pedagogical practices (including evaluation). Periodic Evaluation Plan Following are a list of questions to be considered in constructing a periodic evaluation plan. 1. Frequency 1.1.How often are students tested? Evaluation of Benin's Basic Education Reform Pedagogical Component USAID – Benin Louis Berger/DevTech 54 1.2.Who does the testing? 1.3.Where does the test take place? In what conditions? 1.4.What’s done with the results? 2. Completion Rates 2.1. How many children failed last year in C.I./C.P.? 2.2. Why did they fail? 2.3. How did they do this year? 2.4. How many children were promoted last year? 2.5. How are they performing in C.P.? 3. Validity of Tests? 3.1. What do the tests measure? 3.1.1. Are they formative or summative? 3.2. When are tests administered? 3.3. Do they reflect what’s being learned in class? 3.4. Do they reflect the program? 3.5. Do they reflect what a C.I/C.P. student should be learning? 3.5.1. What should a child that age know? 3.5.2. Do they actually know it? 3.6. Do the tests fit the profile of tomorrow’s Beninese child? 4. Student report cards [Bulletins] 4.1. What communication (type and frequency) exists between the teacher and parents? 4.2. What does the reference book reveal about the pupil’s work? 5. Test Scores 5.1. Total 5.2.1. Sample at the national level 5.2.2. Departments 5.2.3. Schools 5.2.3.1.Urban vs. Rural 5.2.3.2.“Experimental” vs. “Non Experimental” 5.2.4. Gender 5.2.5. Usage by the school system 6. Students (Testing encompasses the test + score) 6.1. How does the testing/evaluation of student’s benefit? 6.1.1. The Students 6.1.1.1. Daily 6.1.1.2. Monthly 6.1.1.3. Trimester 6.1.1.4. Final 6.1.2. The School System 6.1.3. The Community 6.1.4. The Country 6.2. Parents + Community 6.2.1. Individual and social benefits 6.2.2. Community/Parents’ perception 6.2.3. How does it affect the child? 6.2.3.1. At home Evaluation of Benin's Basic Education Reform Pedagogical Component USAID – Benin Louis Berger/DevTech 55 6.2.3.2 At school (amongst peers) 7. School 7.1. Does the test tell the child/teacher/school system/parents the areas of strength and weakness of the child? 7.2. How is the information utilized? 7.3. How does evaluation inform the system the NPE? 7.4. How is the information shared? With whom? CLASSROOM OBSERVATION EVALUATION STRATEGIES Classroom Interaction Who does most of the talking? Teacher Students What’s the amount of student-student activity? Who responds to teacher’s questions? All Students Some Students Same Students After how many enunciations do students respond/react to question? How does teacher clarify the question? Repetition Translation VIII. ASSESS THE FUNCTIONING OF THE DOCUMENTARY NETWORK ACTION PLAN Terms of reference v. Assess the functioning of the Documentary Network action plan; vii. Develop, in collaboration with “pilots”, MEPS and USAID technical staff, a periodical evaluation plan which will help assess the quality, relevance, progress achieved and the results obtained by all five components; and xi. Give recommendations for improving the implementation of the reform program. OBSERVATIONS/FINDINGS This section of the evaluation report describes a national network of documentation centers. Along with curriculum development and teacher training the network of documentary centers is one of the principal pillars of the new program of study (NPE). The National Conference (Etats Généraux) of 1990 was the foundation for an educational renewal for Benin. One of the outcomes of the conference was a commitment to primary educational reform. In 1992 USAID/Bénin began supporting a series of interventions, primarily through the CLEF project, to implement reforms through a series of experimental schools, with the intent to generalize improvements to the more than 4,500 primary schools in Benin. This project was termed the new educational program or NPE. One of the key features of this process was the creation of local resource centers that would lend support to local schools, directors, teachers, parents and students toward the adaptation and application of the curricular and instructional elements Evaluation of Benin's Basic Education Reform Pedagogical Component USAID – Benin Louis Berger/DevTech 56 of the NPE. The network of documentation centers was designed as a mixture of library, reference center, professional development program, and as a supporter of local teacher skills upgrading activities, such as bimonthly teacher workshops. The underlying rationale for establishing the system was to help fortify teacher advancements and to promote acceptance of the NPE. The network was meant to extend in-service teacher aid beyond the one-time only teacher training workshops held each summer. In addition, the network was designed to be a depository of instructional and didactic materials that would compliment those aids that are provided to teachers for classroom use. Finally, the centers were designed for use by school directors, students and parents, and to serve as conventional regional libraries. Such libraries would ideally have access to distance learning resources, through, whenever possible, internet access, and to bring audio-visual assistance to teachers in their service area. The evaluation team visited all Departments of the country to see schools in operation. Team members also made a special effort to visit documentary centers. During classroom visits, we asked teachers and directors the extent of their knowledge and use of the centers of documentation. Less than one-half of one percent (.005) responded positively and only half of those who knew about the centers said that they had used one this year. The team also visited eight centers to appraise the quality of holdings, resources and programs. With few exceptions, the documentation centers were disfunctional. Printed materials and text collections are old and outdated. They do not respond to the needs of their users. In the centers comprising the former IFESH network of nearly twenty libraries, most of the documents are in English. In many cases the number of holdings in English outnumbered those in French. Books and articles on instructional methods or primary teaching principles are lacking, even copies of the texts and workbooks of the NPE were not present in most centers. In the Center of Documentation and Educational Information centers (CDIP) which has seven (7) documentary centers only four (4) are functional. In the Appendix is an instrument to be used in evaluating the functioning of the centers. ANALYSIS There have been significant resources devoted to the MEPS documentary network. In addition to USAID support of the network through CLEF and IFESH, other donors include the Cooperation Francaise and the Contintenal Bank of Bénin. According to the definitve reports on the installation and equipment for the Documentary Centers in the Unités Pédagogiques in Bénin published in 2003, there are 65 centers, located in all twelve of Bénin’s Departments and most subprefectures. Typically, these centers are located near the center of town or the offices of the Departmental Directors of Primary and Secondary Education. Most are staffed with full-time personnel and have a number of holdings and in several cases, computers. The Centers are administered and supported by a variety of agencies. Some (CDDIP) are within the INFRE system, some (CLAC) are under the Ministry of Culture, others (CLP) have the support of the French Ministry of Culture. There are some (CDE) in colleges. Other IFESH centers are staffed by USAID/IFESH. Evaluation of Benin's Basic Education Reform Pedagogical Component USAID – Benin Louis Berger/DevTech 57 The INFRE system has been the subject of a number of consultant studies that have offered excellent recommendations. A 1995 national conference highlighted the shortcomings of the network, calling for more coherent programming, better management, and additional resources. However, few if any of these recommendations have been implemented As a result the Documentation center network has been severely hampered by disorganization, lack of purpose and mission, and most of all by limited resources, programs resulting in lack of demand or use. There are only limited prospects for the centers in the near-term however, as MEPS has only committed (see Projet de Budget Années 2003-2005), some US $20,000 (12 million CFA) for training of personnel and center operational costs. This comes to less than $370 per center, which might meet the costs of utilities, if they are sparingly used. It seems that the centers are mainly used by teachers and administrators who seek information or resources to prepare for examinations required to become school directors or inspectors. Use is very limited in the centers scrutinized; in most cases, less than 100 persons per month visited them, and this often is to meet to submit reports to regional inspectors. At one center, we noted a meeting of some 30 school directors whose purpose was to prepare and submit trimester reports to the office of the Departmental Director of Primary and Secondary Education. One teacher stressed the fact that when the Teachers are free the centers are closed. reasons explain the present condition of the documentary network. The present poor state of the documentary network is the result of under-funding which means they are unable to meet their objectives. To be certain, funding for the centers is so little that the fact they still exist is testimony to the dedication of the staff in each center. Other factors contribute to the problem as well that do not necessarily result from poor funding. One major factor is the lack of clear mandate and coherent management. There are simply too many disjointed and redundant elements of the program to call it a network. A more integrated management scheme might go far to equalizing holdings and programs, and offer the possibility for the modest accomplishment of goals. Documentation centers cannot be all things to all people. Without books they serve little value as libraries. Absence of distance learning resources, such as computers, copiers or faxes renders them useless as dissemination centers. Without imaginative programming that would genuinely help teachers or students, these potential clients will continue to ignore the centers. Thus, putting into operation in local centers clearly established programs for teachers will go far toward presenting the centers as genuine resource centers for the implementation of the NPE. An instrument to evaluate the use of Documentary Centers is included in the Appendix. RECOMMENDATIONS 1. USAID, MEPS and other interested parties should make a determination as to whether to continue the network of centers for documentation. Given the scant attention that the network has received during the last 10 years, and the current budgetary alignment, it is highly unlikely that resources will be forthcoming. Unless the project is supported with a minimal operational budget Evaluation of Benin's Basic Education Reform Pedagogical Component USAID – Benin Louis Berger/DevTech 58 by the GOB, there is no need for USAID to continue with the program. While the evaluation team recognizes the inherent value of an operational network of resource centers, we cannot recommend that the program continue. The limited resources that have been allocated to the network would be better spent on more productive investments 2. If the decision is made to continue support of the network, MEPS must overhaul the system to scale back operations to a functional level. There are too many facilities with too few resources to continue under the present arrangement. One option would be to support only 12 centers, one per Department. This would permit each center to receive adequate funding to carry out some services. 3. The mission, role and scope of the network must be redefined. Since the costs associated with creating and maintaining a functioning library well surpass the funds available, the network should focus on its original goal: support and promotion of the NPE. Programming would be in support of staff development efforts conducted during bimonthly UP meetings, or as an adjunct to a new type of demonstration/laboratory school programs. Here the network would provide instructional materials, teacher training, in-service programs and other support for teachers wishing to participate in demonstration school activities. 4. Programs offered by the network must become more proactive, extending innovations to all schools in the service area. In this instance, we commend the staff of the documentary center of Natitingou for extending mobile library service to local schools. Such efforts can be easily replicated in other Departments. These programs could be invaluable to promoting the methods and materials that are needed to implement the NPE. Our observations of classrooms noted that teachers uniformly lack competencies, skills and the imagination to seek out or create local materials in support of instruction. They depend almost entirely on the flip-charts (planches) that were provided by USAID. For the most part, they have never seen the materials mentioned in teacher guides, nor do they know how to create them. Network centers could go far toward extending learning resources to even the most remote school if they were perceived as valuable by teachers and directors. 5. Learning resource centers. The notion that centers serve a role in distance education is neither technologically nor financially viable at present. Lack of telephone lines and the cost of line use negate the installation of computers to tap the Internet. And the costs of computers pose a significant obstacle to installing several in each center in a reduced network. They can, however, provide a service for the distribution of didactic materials that complement the NPE. Teachers need assistance in finding materials that can be used in class. They also need examples of audio-visual aids such as cassette tapes for French instruction, and an opportunity to see video-based training and demonstration materials. All these services could be offered by the network centers. Use of Center Resources 0 5000 10000 15000 20000 25000 1986 1987 1988 1989 1990 1991 1992 1993 1994 Years Number of Times Used Inscrits Emprunts Passages Evaluation of Benin's Basic Education Reform Pedagogical Component USAID – Benin Louis Berger/DevTech 59 IX. Analysis of the CLEF Pedagogy Components of the Consultants’ Reports Terms of reference vi. Assess the recommendations of the Pedagogy component consultant reports funded under the CLEF project within the framework of activities directly related to the generalization of the new programs and their implementation The evaluation team was required to assess reports and studies prepared by consultants during the ten years of CLEF activities. The team assessed and selected the most pertinent parts of the pedagogy consultants’ reports related directly to the generalization of the new programs and their implementation and have included a summary of them in this section. Continuing Primary School Teacher Training Policy in the Republic of Benin, CLEF, September 1994. Commentary In this report, evaluation was listed as an essential ingredient of primary teacher preparation. Teaching strategies need to be strengthened. Giving the teachers actual practice in the classroom will solidify their performance. Two other things have to be developed in teachers. Besides diagnostic skills the report noted the need to instill elements of teacher creativity and a sense of responsibility in the training system. Analysis : There is no evidence that these suggestions were incorporated into the initial NPE training program or mentioned in teacher training guides. In-Service Teacher Training: Profiles of Primary School Cadres and Assessment of Teacher Training Needs, Bernard Gagne, Creative Associates, for CLEF, November 1994 Gagne notes that a child-directed learning approach has not yet fully penetrated the NPE program planning. Classroom interaction has yet to be emphasized. Teachers need to learn basic pedagogy, child development and psychology skills. Specific approaches to teaching large classes are also needed. The level of professional preparation should be raised to the BAC in the long term. Specific suggestions are made to help ensure the successful implementation of an in-service training program. Another suggestion is that 5 or 6 successful teachers be selected to prepare training modules. The provision of audio-visual materials for training would help. Establishment of an action plan team to oversee content, methodology, follow-up, and evaluation of training would go far towards the improvement of in￾service teacher training. Analysis: The Berger/Devtech team observed the same classroom and training situation nine years after this report. The NPE process has, however, integrated teacher input into the preparation of training modules, a practice that continues today. The Documentary Problem in Primary and Secondary Schools, INFOSEC, Cotonou, 21 – 23 November 1995. Commentary This is a report of the National Seminar on Libraries in the Primary and Secondary Schools of Bénin. The documentary network of INFRE, its history, mission, Evaluation of Benin's Basic Education Reform Pedagogical Component USAID – Benin Louis Berger/DevTech 60 activities, center policies, decentralization, and the different means it uses were outlined. Analysis of the documentary situation in primary and secondary schools mentioned five needs. 1. The need for education and documentation to come together. 2. The dependence of the documentation sector on private initiative. 3. The need for the complete outfitting of the documentary centers. 4. Often policies are made that do not result in concrete action. 5. National resources are not sufficient to maintain or improve the centers or the network. Recommendations included that centers require a structure in an adequate location with qualified personnel, equipment and collections. They need to be marketed and their collections need updating. Also, the study asserts that at least 5 percent of the MEPS budget be appropriated for upgrading the centers. Analysis: None of these recommendations have been implemented. An Evaluation of a Sampling of CP and CM1 students in French and Mathematics, Statistical Analysis, the report of a group of twelve inspectors and directors, January 1996, National Education Ministry of Benin. Commentary This is a very ambitious study, and the first ever of sampling research in Benin. Work began in 1994 with development of a sampling instrument that was administered to 130 schools in 1995. This report should be read to give a perspective on the recurring problems in schooling in Benin. Overall the results in both French and Math were deemed to be very disappointing in both CP and CM1. Among the other findings were a general lack of infrastructure and an indictment of the educational system that included unmotivated teachers, outdated teaching methods, little help from inspectors and the use of obsolete teaching methods.. Among the more pertinent suggestions were: to improve pedagogical support for the teachers, that the Inspectors give priority to the improvement of teaching, that teachers need to keep up-to-date on professional developments, and that parents be included in their children’s learning efforts. Analysis : The evaluation team observed that the same conditions apply to primary education today, and that the recommendations were not implemented. The Mid-course Evaluation of the Program and Project of the Children’s Learning and Equity Foundation’s July 1997 Education Development Center Commentary This document provides a perspective on the recurring problems of primary education. Conclusions and recommendations include : the need to be more open and direct financing of education. Technical assistance within CLEF needs to be consolidated. There needs to be a financial audit of MEPS. It was recommended that USAID withdraw from operational responsibility. A redefinition of a basic quality school needs to be done as the current definition is not reflected in the schools. There need to be inexpensive alternatives to costly textbooks. Reform is heavily dependent on foreign aid. It is a fact that music, dance, and other non-academic activities support learning and help create literacy. Teachers strikes affect the reform effort. Teachers need more training. Analysis: These recommendations that were offered six years ago almost mirror those provided in this evaluation. Little has been done to put them in place. The Textbook Production Process in Benin, Marie Venier, Creative Associates, September 1997. Evaluation of Benin's Basic Education Reform Pedagogical Component USAID – Benin Louis Berger/DevTech 61 Commentary This thin volume surveys the textbook production capabilities in Benin. Although the local printers were relatively inexperienced they were deemed to have the competence to produce the textbooks, and should be used for text and materials production over MEPS facilities. Analysis: The MEPS has begun using private sector publishers as recommended. Curriculum Development: Theoretical Section Of The Six Curriculums, Denis Chabot, Curriculum Associates International (CEA,) October 1997. Commentary This is the report of a two-week workshop held in Cotonou in 1997. The document describes the six working groups whose work would eventually result in preparing the NPE. He also expresses a certain frustration. In the Introduction, he urged that training, « …apprendre en faisant, comme je me plais à répéater. La situation dénoncée est courante. On dit souvent comment faire ; on ne le fait pas. » This translates to “learn by doing,” with the observation that “we know how to do it, but we do not apply it in teacher training.” Analysis: The author sensed the risk involved in training by lecture, which is still the principal means of providing training to teachers, and they in turn, to students. This is the same observation made by members of the Berger/DevTech team evaluation team. Preparation of “Formateurs” and Teachers in the Application of the New Programs, CLEF, Feb 1997, Daniele Schaeffer Campbell, Project No. 680-0208, Contract No. 624-0208-I-00-4007-00 Commentary The author noted four impediments to teacher training. There is an insufficient amount of material. Class sizes are too large. Teachers need training in evaluation methods. More training is needed for both teachers and directors in methodology, use of materials and as indicated above, evaluation. Based on the author’s observations are eight recommendations. Design teams must be given school manuals and office space, equipment (including computers and printers) and supplies in order to do their work. A permanent team must be responsible for the follow-up of the reform. An effective follow-up study should include classroom visits. This follow-up must give priority to the new methods and use of pedagogic materials. Organize a training workshop for teachers of CI and CP during the February break as a way to solidify their skills in interactive lessons, in French and in second language instruction. Create a documentation center for the national and departmental team members to use as a resource. Organize a training workshop for departmental trainers centered on the practical application of interactive approaches and pedagogic support. Collect and analyze the results from the NPE’s first year of application in the 106 schools; make the necessary changes. Analysis: These recommendations were not acted upon. Evaluation of Benin's Basic Education Reform Pedagogical Component USAID – Benin Louis Berger/DevTech 62 The Policies of Ongoing Primary School Teacher Training in Benin, CLEF Program and Project, Prepared by EDC, July 1997 Commentary This report states that only part of funds provided by USAID to the GOB is at the disposal of MENRS. Of this amount, only a portion has gone to the classrooms. The evaluation team recommended that the national budget support not be mingled with the money that should be applied to the primary teaching reforms. The Team has found that the institutional reenforcement for the Finance Ministry and for MENRS has been successful and that the procedural reforms continue without outside help. It was recommended that the teachers and classroom activities take precedence in the use of all resources for the program and for project CLEF for the two years that remain. Recommendations also include that USAID totally disengage itself from the administration of technical assistance and supervise the CLEF project. Activities should concentrate on the classrooms, the application of pedagogy and on-going training and development adapted to the communautaires. The personnel crisis in primary education is affected by poor teacher qualifications, meager salaries and the lack of educational resources. While this report was prepared, schools were closed as the teachers were on strike .The Documentation Network centers need to be furnished with more materials and placed in areas accessible to the greatest number of teachers possible. Analysis: Since these same recommendations are restated by the Berger team, they were obviously not implemented. An Identification of Needs Study in the Formation of Technical Aptitudes, 1999. Commentary This is useful for French education vocabulary and organization of the report. Otherwise it refers to technical education and unrelated to primary schooling. It contains a description of other international donor activities. ADDA, Evariste. “Needs Assessment for the School Districts in Benin,” USAID/Benin, November 2000, Cotonou. Commentary This study entailed an analysis of the physical and logistical needs of Benin schools. The purpose of the evaluation was to find out what problems to anticipate and the best procedures to implement a decentralized system of educational administration. The research also noted the poor working conditions for teachers and students in schools, including the lack of teachers, dilapidation of existing facilities, and lack or inadequacy of furniture in all schools visited. Recommendations include: a school-based budgeting system overseeing personnel matters at the Department level, with maximum input and allocations made by the schools and Bureuax des Associations de Parents d’Eleves (APE); implementation of a merit system of personnel, hiring, appraisal and management, with clearly prescribed criteria for the selection and compensation of personnel. The report recommended the immediate implementation of a policy to release workbooks and manuals for home use, in order to bring pupils’ parents into the implementation of the NPE. Evaluation of Benin's Basic Education Reform Pedagogical Component USAID – Benin Louis Berger/DevTech 63 Analysis: The Berger/DevTech team found no indications that either CLEF or MEPS have implemented any of Adda’s recommendations. All the deficiencies noted still persist. Information and Teaching Resources in the Service of Education, Editions INFRE, 2000, Edited with the help of Coopération Française. Commentary This is a report of the studies of the Documentary Network of INFRE. It presents an analysis of the documentary network of INFRE and presents a proposal for a new bureaucratic structure. Included is information on what a documentation center should be and the kind of equipment it should have. It began by stating what it is at present - the actual state of the documentary network at l’INFRE. The director was at a disadvantage because of the network’s position in the hierarchy. Material, personnel and financing means were lacking. In addition the network is not effective. The two things that damage the implementation of the documentary network are the lack of autonomy and the lack of means. The new network structure and its functions was described. Analysis : The same situation holds true today . None of the recommedations have been implemented. The Definitive Report on the Project to Begin and Equip Documentation Centers in the Unités Pédagogiques au Bénin. 2003, Published by the Continental Bank-Bénin. Commentary These documents present the results of two studies indicating where Educational Documentary Centers presently exist, where they should be placed and how they should be organized, planned and function. The present centers are shown in the graph below. The abbreviations are defined here. IFESH – International Foundation for Education and Self-Help CDDIP-Centre éDepartemental de Documentation et d’Information éaquePedogogic (INFRE) CDE - Centre de Documentation d’Establishment CLAC – Centre de Lecture et d’Animation Culturelle CLP - Centre de Lecture Publique Bib Depts – Bibliothèques Départementales Evaluation of Benin's Basic Education Reform Pedagogical Component USAID – Benin Louis Berger/DevTech 64 The reports also list the works to be acquired and the necessary equipment. The report urges that a pilot phase with model programs be instituted. If evaluations are positive, more Centers will be established. The authors outline how documents will be collected, accessioned and administered, with the combined management of the parents associations and teachers. The study further recommended that all centers be staffed by trained librarians. They recommended that a definitive role for the Documentary Network as a center for applied learning. They also recommended that the leaders of the country take control of the present vegetative state of the “reseau documentaire” and create conditions favorable to its improvement. SUMMARY ANALYSIS OF CONSULTANT FINDINGS The evaluation team analyzed and assessed the recommendations of consultant reports that were directly related to the generalization of the new programs and their implementation. Their recommendations all of which would support in some manner the reform efforts fell into five general categories: Finances, Evaluation, Teacher training and instruction, documentary network, and parents and community. Finances Five recommendations were made by consultants regarding finances relating to the NPE and at least three were implemented: USAID funds are no longer mingled with those of GOB, student manuals and workbooks have become inexpensive alternatives for expensive textbooks and they are being printed and distributed by the private sector. However, there appear to be few cooperative endeavors with other donors and there is still an unhealthy reliance on foreign aid that raises questions about the sustainability of innovative, reform programs. Evaluation An evaluation plan was recommended and designed for the reform program prior to it being generalized. However, this evaluation was never carried out. Teacher Training and Instruction With respect to the design of NPE materials and their introduction to teachers, consultants have stressed the need to learn by doing. That is, the initial training should have provided ample opportunity for teachers to practice application of principles as they were acquired. This has not been a feature of training. Recommendations included developing teaching strategies and promoting creative Les Centres de Documentation 0 2 4 6 8 10 Atlantique Littoral Ouémé Plateau Mono Couffo Zou Collines Borgou Alibori Atacora Donga Departements Nombre des Centres IFESH CDDIP CDE CLAC CLP Bib Depts Evaluation of Benin's Basic Education Reform Pedagogical Component USAID – Benin Louis Berger/DevTech 65 decision making by teachers. Although this is mentioned in NPE documents, these are not characteristic of teachers observed by the Berger team. The NPE is a curriculum based on objectives in the CI and CP and competencies in CE1 – CM2 but follow-up, feedback or re-assessment on these objectives and competencies have not been implemented. An action plan to oversee training produced both content and methodology yet, in the classroom, strategies such as teaching French as a second language and working with large groups has been woefully absent. In general, the initial teacher training as a means for generalisation has been regular but inadequate. As for the recommendation that teachers take the initiative to update their own professional development, basic practices such as visiting a local documentation center were lacking. Inspectors, as well, do not make teaching improvements a priority. The NPE documents espouse a child centered learning approach and classroom interaction. In practice, however, the evaluation team did not see evidence of either. Though pedagogical concepts are described in the manuals these concepts are not modeled for teachers. Music and dance are less evident today than in the years past and there is no attempt to link them to literacy. Health education is the domain of MCDI and has been incorporated into the curriculum. As for books and materials, many schools still suffer from the late distribution of and insufficient number of books. Although manuals, workbooks and wall charts have been developed and distributed to classrooms, teacher created materials were not evident nor were the Math and French kits. Documentary Network With respect to the documentary network, the four reports reviewed indicated that the state of the network was in neglect. Numerous recommendations are given for improvement, expansion and focus of the centers and while plans have been made none has been implemented, even those requiring simple actions such as providing books. Parents and Community A number of recommendations were made to encourage more parent and community involvement in school affairs. Unfortunately, these have not yet been incorporated into implementation of the NPE. In summary, the Berger team noted that with few exceptions, recommendations rendered by consultants have not been incorporated in reform activities. X. PERIODIC AND ONGOING EVALUATION STRATEGIES FOR EQUIPE AND MEPS PROJECT ACTIVITIES Terms of reference vii. Develop, in collaboration with “pilots”, MEPS and USAID technical staff, a periodical evaluation plan which will help assess the quality, relevance, progress achieved and the results obtained by all five components; The Berger/DevTech pedagogical reform evaluation team was also required to propose actions that should be incorporated into the EQUIPE program. This project will continue the generalization of NPE and other educational reforms. It will entail the generalization of NPE to CM1 and CM2, the initiation of programs in support of girls’ education, provision of technical assistance to improve health education, and support for the decentralization of educational management. In addition, the Evaluation of Benin's Basic Education Reform Pedagogical Component USAID – Benin Louis Berger/DevTech 66 EQUIPE project is expected to consolidate gains obtained from initiatives launched by CLEF. In other sections of this report, we cite deficiencies in testing, diagnostic, or evaluation. Rather than restate these findings, the reader is urged to refer to the body of the evaluation report. In this section, we propose a number of steps to be taken, primarily by USAID and the EQUIPE operation, beginning with those tasks that merit immediate implementation, and continuing with medium and long-term recommendations. Components of the evaluation system Student testing In collaboration with MEPS, the USAID contractor (Creative Associates and partners) should design a comprehensive system of testing and measurement of student performance. This will necessitate the elaboration of tests for competency in French and Mathematics at all grade levels. These tests should be relatively simple, and take no more than 20 minutes to administer to students. We do not advise that the tests be administered throughout all Benin schools. Nor should the tests be administered every year for every grade. This process is expensive and cumbersome, and subject to any number of threats to the validity of tests (such as teaching to tests, student copying\cheating, or school manipulation of scores.) Instead, the tests should be administered to a small sample (no more than 50, selected at random) of schools by an independent, unannounced team of evaluators. Results of the tests should be tabulated with an easily designed software program that will provide immediate results, and with little need for data manipulation or analysis. Results of the first test should be used to determine baseline indicators for the EQUIPE program, and for MEPS to use in developing programs to extend and consolidate the NPE. Teacher Evaluations After the development of a simple, easy-to-administer teacher evaluation instrument, appraisal of teacher competencies and practices should be initiated on a broad scale. The instruments would include classroom observation checklists, teacher interviews, student observation and questions, and interviews with directors. The instruments would be administered in the same schools chosen for student testing. However, we urge limiting the study to a small number of schools once a year. The results would serve to guide investments and interventions to better teacher performance. Director evaluations Acceptance of pedagogical innovations among teachers and students is directly related to the quality of supervision and support for change provided by school directors. We have noted throughout the report that theirs is a paramount role in implementing reform measures. A program of director evaluation should be instituted that incorporates appraisals from parents, student, teachers, and CPs. Conseillers Pédagogiques The evaluation of administrative support must be extended to CPs as well. Their role is perceived to be of utmost importance in the support of teacher training and adaptation of educational reforms. They should be evaluated regularly by school Evaluation of Benin's Basic Education Reform Pedagogical Component USAID – Benin Louis Berger/DevTech 67 directors and teachers, again using the same small sample of schools as the source of information. Assessment of the impact of USAID support Ministry support for innovations and progress towards providing institutional measures to continue the NPE after the life-of-project. If the NPE and other elements of pedagogical reform are to be sustainable, MEPS actions to institutionalize programming must be periodically appraised. The Berger/DevTech team noted that the NPE is largely a product of USAID investments, and that little has been accomplished within the Ministry to supplement and support these changes. The issue is transferring the responsibilities of the lCellule, which is a temporary entity, within the Ministry. The absence of any implementing department or sub-agency charged with delivering, improving or documenting the initiatives of the NPE is demonstrative of Ministerial ambiguity. It is recommended that USAID establish criteria and conditions for the role and participation of MEPS, and that these be reviewed annually. Teacher training activities Under the current plan, blocks of teachers from successive grades are trained in the implementation of NPE. There is, however, no method in place to evaluate the quality of those training programs. And, as stated above, members of the evaluation team noted the tendancy to repeat frontal teaching methods rather than modelling the active methods on which the NPE is based. There is clearly need to appraise the quality of training, not just the perspectives of teachers who participate, but the extent and nature of changes in teacher behavior in classrooms. The question to pose is: Is the training equipping and motivating teachers to implement the NPE, and if so, how? This will require immediate follow-up to training programs, using diagnostic instruments that permit outside evaluators to assess classroom interaction and teacher comportment. These measures should include school directors, so that they can continue the process on an ongoing and daily basis. On a broader scale, teachers who have received training need to be assessed to identify and determine the skills they acquired and actually apply. This exercise will no doubt reveal the necessity of reconfiguring the training schedule to provide training in all grades for newly hired and incoming teachers who have no skills in applying principles and strategies of the NPE. Instructional Materials The evaluation team is concerned about the lack of revision of curricular materials that may be of limited quality, or of questionable pertinence. The EQUIPE project should take immediate steps to assist MEPS in performing regular assessment of the content, pertinence and coherence of NPE materials. This should be done by a team of experts who talk with teachers and children to determine elements of the program that are inconsistent with materials that have been designed and reproduced at great expense. Results of these evaluations need to be received by a text and materials revision committee, which can take appropriate steps to improve instructional support. MEPS Organizational Improvement The evaluation team has recommended an in-depth analysis and audit of MEPS mission, programs, functions and personnel needs. The results of this analysis Evaluation of Benin's Basic Education Reform Pedagogical Component USAID – Benin Louis Berger/DevTech 68 must be utilized to ensure that appropriate measures are executed and suggested changes are institutionalized. XI. MANAGEMENT AND USE OF PEDAGOGICAL INPUTS Terms of reference: x. Assess the management and use of all of the pedagogical inputs on the basis of a representative sampling. xi. Give recommendations for improving the implementation of the reform program. INTRODUCTION This section assesses the management and use of all pedagogical inputs using a representative sample and by briefly summarizing information contained in this report. Pedagogical inputs consist of four components: Didactic materials Training Evaluation programs Documentary center network The team evaluated the extent and use of pedagogical inputs, and the findings are presented in this report. Instead, we present a brief summary of the evaluation process and conclusions as regards the extent and management of pedagogical inputs. The evaluation effort itself was based on a representative sampling of schools in Benin from which more than 200 schools were visited. A description of the sampling procedure is available in the section on methodology. FINDINGS Didactic materials and training efforts Considerable time and expense were invested to design, write and publish a vast array of pedagogical materials: programs of study, teachers’ guides, wall charts, and teacher and student manuals. These abundant resources contrast with almost the complete lack of didactic materials available under the previous, intermediate program. Distribution of these materials to schools throughout Benin has proceeded in an orderly and timely fashion and usually arrived in most schools within a few weeks of school opening. Only in very rare exceptions did the evaluation team visit schools lacking these didactic materials. Training efforts Training activities in support of the NPE have been prodigious, involving all of the teachers in CI and CP except for the recent hires. In most instances, this has been the only pedagogical exposure received by Benin primary school teachers and they welcomed the opportunity to participate. Given the complexity of the NPE and the minimal pedagogical skills of teachers, the length of training was likely too short-a total of 4-8 weeks- and of the wrong kind. Instead of demonstrating the teaching strategies, they were talked about. Modeling the desired behavior and demonstrating the skills necessary for teaching classes of 60 or more students were missing and their inclusion is one of the priority recommendations of this evaluation. Evaluation of Benin's Basic Education Reform Pedagogical Component USAID – Benin Louis Berger/DevTech 69 The evaluation team noted the availability of didactic materials in almost all of the classrooms. The use of student workbooks was spotty, which could be due to the to the labor disturbances or the inability of teachers to monitor the activity of every student in classes of 100 and more. While teachers professed to approve of the NPE approach, it is doubtful that the pedantic terminology and explanations contained in the programs and guides is understood by many of them. There is also the sheer volume of instructions to be read - more than 1000 pages in CI alone! It was difficult for the evaluation team to assess the impact of the training because of disruptions caused by the strike situation. As most APE and contractual teachers were on strike, about half the classrooms observed were staffed by communautaires. Irrespective of their academic background and experience, very few teachers are applying the strategies of child or competency-based learning in classrooms. Though most of the teachers used the new learning materials provided and said they found them useful, they were unable to demonstrate the teaching strategies and methods that are an integral part of the NPE. The most logical explanation for the almost exclusive use of frontal teaching is that teachers continue to teach as they have learned and been trained themselves. . This suggests that the NPE training consists mainly of lectures with the trainees passively ingesting the new information. The evaluation teams’ observations of various training activities held by MEPS corroborated use of these methods by CCGNPE trainers. We attended seven separate training sessions for Conseillers Pédagogiques and teachers, and not in one instance was any method other than frontal instruction used. Under these conditions it is little wonder that little child￾centered, active learning was observed. RECOMMENDATIONS We refer the reader to the sections on Evaluation and the Documentary Center Network for an exhaustive analysis of the contributions of those programs to the pedagogical reform. Similarly, it would be redundant to discuss recommendations for training and didactic materials that were included earlier in the report. The principal recommendation we can suggest with respect to the management and use of pedagogical inputs is that teachers have an opportunity to see these in use: demonstrated to them first hand, in actual operating schools. We emphasize the urgency with which teachers have exposure to better teaching methods and materials. This can be easily accomplished with the creation of a demonstration/laboratory school network in each Department that will serve as a model to be followed. In all future training, this should be the cornerstone of the training strategy, providing pre-service training to new teachers and in-service support for current teachers. The demonstration model would emphasize small groups of teachers who could learn by doing, rather than learn by watching and listening. In short, this would incorporate the same teaching strategies as espoused for primary school classrooms under the NPE. Evaluation of Benin's Basic Education Reform Pedagogical Component USAID – Benin Louis Berger/DevTech 70 ANNEXES I. Schools Visited II. School Questionnaire and Analysis III. Classroom Observation Instrument and Analysis Evaluation of Benin's Basic Education Reform Pedagogical Component USAID – Benin Louis Berger/DevTech 71 Annex I Visits to Benin Schools Below are listed the names of over 200 schools which the Evaluation Team visited during February and March of this year, 2003. Of the schools visited 7% were Experimental Schools, 74% Other Public Schools, and 19% were Private Schools. Fifty-three percent were Rural and 47% were Urban. Schools were visited in each Department: Atacora 15, Donga 7, Atlantique 34, Littoral 20, Borgou 19, Alibori 2, Mono 14, Couffo 7, Oueme 25, Plateau 43, Zou 13, and Collines 12, In Savalou, Collines, 8 School Directors were interviewed and in Sakete, Plateau, 30 School Directors were interviewed. Département Sous-Prefecture R/U Ecoles Ecoles Ecoles Experimental Publique Privée Atacora Kouande R Birni Centre/A Kouande U Becket-Bourame Natitingou U Yokossi/A Natitingou U Yokossi/B Natitingou U Assemblée de Dieu/A Natitingou U Assemblée de Dieu/B Natitingou U Dassagate/A Natitingou U Dassagate/B Natitingou U Ourbouga Natitingou U Kotopoungo/A Natitingou R Kofounpounga/B Natitingou R Gorgoba Natitingou R Kouboro Tanguieta U Tchoutchoubou/A Tanguieta U Godje/B Donga Bassila R Kokohou Djougou R Bellefoungou/A Djougou R Bellefoungou/B Djougou U Djougou Centre/A Djougou U Djougou Centre/D Duougou R Korokou Duougou R Pelebina Atlantique Abomey Calavi R Cococodji/A Abomey Calavi R Cococodji/B Abomey Calavi R Cocotomey/A Abomey Calavi R Cocotomey/B Abomey Calavi R Cocotomey/C Abomey Calavi R Avissogbe Abomey Calavi R Atrokpocodji Abomey Calavi R St:Dominiaue-Atrokpocodji Abomey Calavi U Azognon Abomey Calavi U Abomey Calavi Centre/A Abomey Calavi U Abomey Calavi Centre/B Abomey Calavi U Abomey Calavi Centre/C Abomey Calavi U SOS/A Abomey Calavi U SOS/B Abomey Calavi R Adjagbo Abomey Calavi R Agongbe Abomey Calavi R Drabo Abomey Calavi R Togba Abomey Calavi R Docome Evaluation of Benin's Basic Education Reform Pedagogical Component USAID – Benin Louis Berger/DevTech 72 Allada U Allada Centre/A Allada U Allada Centre/B Allada U Allada Centre/C Allada U Agoube Allada R Agbanou/A Allada R Ahouannonzoun/A Allada R Adjidonou/A (Ex Niaouli/A) Allada R Adjidonou/B (Ex Niaouli/B) Allada R Adjohoun Allada R Ahota Allada R Adimale Toffo R Houègbo Aliho Toffo R Houègbo-Gare/A Toffo R Sehoue-Agbozounkpa/A Toffo R Sehoue-Agbozounkpa/B Littoral Cotonou U Abere Cotonou U Avotrou/A Cotonou U Avotrou/B Cotonou U Tanto Cotonou U Amakene-Esperence Cotonou U Francs Bourgeois Cotonou U Hinde/A Cotonou U Anihuvi Cotonou U Les Destinées Cotonou U Akogbato Cotonou U Cocotiers Cotonou U Ecole Primaire Protestant Cotonou U Fidjrosse/A Cotonou U Fidjrosse/B Cotonou U Fidjrosse/C Cotonou U Mawukpehu Cotonou U Fifadji/D Cotonou U Menontin Nord/A Cotonou U Menontin Nord/B Cotonou U Gbegamey 1/A (Ex Rond Point C) Borgou Ndali U Treboun/A Ndali U Treboun/B Ndali U N'Dali Parakou U Hibiscus de Parakou Parakou U La Source Parakou U Al-Houda/A Parakou U Al-Houda/B Parakou U L'Entente Parakou U Les Merveilles/A Parakou U Les Merveilles/B Parakou U Mon Avenir Parakou U Enfant Epanoui Parakou Evaluation of Benin's Basic Education Reform Pedagogical Component USAID – Benin Louis Berger/DevTech 73 Parakou U Titirou/A Parakou U Ebenezer Perere U Tissarou/B Tchaourou R Badekparou Tchaourou R Tchaourou-Centre/A Tchaourou R Tchaourou-Centre/B Tchaourou R Tchaourou Gare Alibori Gogounou R Gogounou/A Kandi U Kandi/A Mono Lokossa U Lokossa/C Lokossa U Enfant Epanoui Lokossa Athieme U Monoto/A Come U Come Gare Come U Come-Quartier/A Come R Guezin-Kpota/A Come R Guezin-Kpota/B Grand-Popo R La Source, Le Verbe Grand-Popo R Agoue/A Grand-Popo R Nicoue Condji/A Grand-Popo U Grand Popo D Houeyogbe R Houeyegbe/A Houeyogbe R Houeyegbe/B Lokossa R Adrogbo/B Couffo Aplahoue U Aplahoué/B Aplahoue U Kaiteme Aplahoue U Avegodo/A Aplahoue U Dogbo Ayomi Zohoudji/C Dogbo R Zohoudji/C Djakotomey U Djokotomey/A Djakotomey U Djakotomey/B Ouémé Porto Novo U Sadognon Porto Novo U Protestante Adjegounle/A Porto Novo U Protestante Adjegounle/B Porto Novo U Agbokou Kpakpassa/A Porto Novo U Agbokou Kpakpassa/B Porto Novo U Gbokou-1/A Porto Novo U Gbokou-3 Porto Novo U Lokpodji Porto Novo U L'elite Porto Novo U Ouenlinda/A Porto Novo U Descartes/A Adjohoun U Adjohoun Centre/A Adjohoun U Adjohoun Centre/B Adjohoun R Azowlisse Kpodedji/A Adjohoun R Azowlisse Kpodedji/B Evaluation of Benin's Basic Education Reform Pedagogical Component USAID – Benin Louis Berger/DevTech 74 Akpro-Misserete U Abogome/A Akpro-Misserete U Abogome/B Akpro-Misserete U Akpro-Gbedji/A Akpro-Misserete U Akpro-Gbedji/B Akpro-Misserete U Le Flambeau Akpro-Misserete R Gomè Doko Akpro-Misserete R Gbakpo-Sedje Akpro-Misserete R Tchian Amouloko Houegbo/A Akpro-Misserete R Katagon/B Seme-Podji R Houinta Plateau Adja-Ouere R Oke-Ola/A Adja-Ouere R Oke-Ola/B Adja-Ouere R Oke-Ola/C Ifanghi U La Concorde Pobe R Ahoyeye Pobe R E Prim Pub Pobe/D Pobe R E.P.Ogouake(Ike-Bol) Sakete U Gbokou-Dagbao Sakete R Ita-Djebou/A Sakete R Ita-Djebou/B Sakete R Ikpeguile/A Sakete R ComSchSMarieAn Sakete R Takon Centre Sakete R 30 school directors Zou Abomey U EUC/E (Abomey/E) Abomey U Enfant Epanoui Abomey U Hountondji/A Abomey U Oukanme Agbangnizoun R Sainte Marie-Claire Bohicon R Lissezoun/A Bohicon R Lissezoun/B Bohicon U Zakpo/A Bohicon U Adamé Ahito/A Bohicon U Le Primat Bohicon R Ecole Chretienne de Zounkpa Bohicon R Primat Bohican Zakpota U Kemondji Collines Glazoue R Thio/B Glazoue R Hoco Ouesse R Kokoro Ouesse R Toui-Atakpa Savalou - Directors R 8 schools Evaluation of Benin's Basic Education Reform Pedagogical Component USAID – Benin Louis Berger/DevTech 75 Annex II Questionnaire for School Visits Nom de l’Ecole ____________________________Departement _______________________ Alt R Ex PR Effectif Total F ____ G ____ CI _____ CP ______ Perspectives du/la Directeur/Directrice (interview) (Favorable=5. Non favorable=1) Connaissance du NPE 0 1 2 3 4 5 Attitude envers le NPE 0 1 2 3 4 5 Français 0 1 2 3 4 5 Math 0 1 2 3 4 5 EST 0 1 2 3 4 5 Ed Phys 0 1 2 3 4 5 Ed Sociale 0 1 2 3 4 5 Ed Artistique 0 1 2 3 4 5 __________________________________________________________________________________ ________________________________________________________________________________ Materiels Quantite des materiels didactiques Français I/S Cahiers 0 1 2 3 4 5 I/S Manuels 0 1 2 3 4 5 Math I/S Cahiers 0 1 2 3 4 5 I/S Manuels 0 1 2 3 4 5 Qualite des materiels didactiques Francais 0 1 2 3 4 5 Math 0 1 2 3 4 5 ___________________________________________________________________________ ___________________________________________________________________________ ____________ Formation Formation (Dir) BEPC BAC ENI CEAP CAP Autre ______ Quantite de la formation initiale sur le NPE (semaines) 0 1 2 3 4 5 Suivi de la formation initiale 0 1 2 3 4 5 Qualite de la formation sur le NPE 0 1 2 3 4 5 ___________________________________________________________________________ ___________________________________________________________________________ ____________ Soutien pedagogique: UP 0 1 2 3 4 5 CS/CP 0 1 2 3 4 5 Syndicat et autre 0 1 2 3 4 5 Autre Orgs 0 1 2 3 4 5 Evaluation des éleves Avez-vous fait une evaluation sommative cette année ? CI: Oui / Non CP: Oui / Non Combien d’évaluations mensuelles avez-vous fait cette année ? 0 1 2 3 4 5 Evaluation of Benin's Basic Education Reform Pedagogical Component USAID – Benin Louis Berger/DevTech 76 Qui la compose? Enseignant(e)_____Circonscription____ Dept.______ Travaillez-vous avec l’enseignant(e) et ses pairs sur les items? Oui Non Rapports aux Parents (combien de fois?) 0 1 2 3 4 5 Rapports à la CS (combien de fois?) 0 1 2 3 4 5 Comment l’ecole reagit-elle aux evaluations? Analyse de l’enseignant avec le Directeur0 1 2 3 4 5 Analyse de l’enseignant avec les Parents 0 1 2 3 4 5 Comment les enseignant(e)s acceptent-ils(elles) le NPE? 0 1 2 3 4 5 Documentation Systeme de transmission de donnees: Quand? Chaque mois _______ Chaque trimestre ______ De temps en temps________ A qui? DDPS _____ CS _______________ Soutien a l’apprentissage Bibliotheque: Oui /Non Ordinateur Oui / Non Machine a ecrire Oui/ Non Existe-t-il un Centre Documentaire dans la circonscription ? Oui/Non Combien de fois l’avez-vous visite ? 0 1 2 3 4 5 ________________________________________________________________________ ________________________________________________________________________ ________________________________________________________________________ Impact de la grève sur: CI 0 1 2 3 4 5 CP 0 1 2 3 4 5 ___________________________________________________________________________ ___________________________________________________________________________ ___________________________________________________________________________ Perspective des/sur les parents Connaissance du NPE 0 1 2 3 4 5 Attitude envers le NPE 0 1 2 3 4 5 Soutien des parents a la maison 0 1 2 3 4 5 Soutien des Assoc. des Parents d’El.0 1 2 3 4 5 Subventions scolaires Quel est le montant annuel de la subvention scolaire?_______ Evaluation of Benin's Basic Education Reform Pedagogical Component USAID – Benin Louis Berger/DevTech 77 Analysis of the Responses to the School Visit Questionnaire We were able to visit 211 schools. The team travelled in pairs most of the time ; one person interviewed the school director and the other observed CI and/or CP classes. Not every respondent answered all questions, as there was irregular attendance of teachers and directors, and many schools were closed because of the teacher strike. Consequently, the number of answers to any one question will vary. Of the schools visited 7% were experimental Schools, 74% were other public schools and 19% were private schools. Fifty-three percent were rural schools and 47% were urban schools. The evaluation team visited schools in each of the Departments. Originally the team had chosen some schools to visit in case we were unable to visit schools selected by our sampling technique. The team was able to visit most schools chosen in the original sample but when we found any of these schools were closed, schools with similar profiles were selected in order to complete our sample. From the responses to the Questionnaire we noted that the vast majority of school directors and teachers knew of the NPE and liked it. Their attitude towards French was low (only about 2.3 out of 5,) but for the other areas of CI and CP study, Math, EST, Physical Education, Social Education and Artistic Education, it was high. In many cases there were enough workbooks and manuals. Respondents found the quality of the French teaching materials low, but the quality of the Math, high. Just under half of the Directors interviewed held the BEPC. Less than 20% had university training and a little under 10% had only their BAC. About 15% had the CEAP. Most had been initially formed in the NPE and a few had some further training. Directors were satisfied with the training for the NPE (4.4 out of 5.) Their response on support from the various areas was mixed. They gave high marks to the Unite Pedagogique, very low to the Circonscription and Conseillers Pedagogique. Even lower was the mark given to Syndicats. Practically no student evaluations have been administered this year. Only three public schools in the country have libraries. One school used to have a library but it was dismantled, as they needed the room for a classroom. Only 12 Directors knew if there was a Documentary Center in their Circonscription. Of those only four had used it. Practically all mentioned the impact that the strike had had on their school. Parents have a good knowledge and attitude towards the NPE. The APEs were rated highly for support of the schools. The average school budget money from the MEPS subsidy was CFA 682,491. Evaluation of Benin's Basic Education Reform Pedagogical Component USAID – Benin Louis Berger/DevTech 78 Annex III Classroom Observation Grill INFORMATION SUR L’ECOLE _____________ P/UDepartement________ Type de classe CI ___CP ___ Effectif Total de la classe F____G____ INFORMATION SUR L’ENSEIGNANT(E) Categorie de l’enseignant(e) APE ____ Cont. ____ Commun. ____ Annees d’experience ____ Sexe __ Formation Academique: Moins que BEPC__BEPC___BAC___ENI___CAP___ Autres____ Connaissance du NPE 0 1 2 3 4 5 Attitude envers le NPE 0 1 2 3 4 5 __________________________________________________________________________________ ______________________________________________________________ NPE Materiels : De l’enseignant:PdE O_ N_ Guide O_ N_ Evaluation O_ N_ Planches O_ N_ De l’éleve : Manuels O_ N_ Cahiers O_ N_ Qualite des materiels du NPE 0 1 2 3 4 5 NPE Formation : Quantite de la formation NPE (semaines)0 1 2 3 4 5 Qualite de la formation NPE 0 1 2 3 4 5 Soutien pedagogique: Directeur 0 1 2 3 4 5 Conseillers Pedagogiques0 1 2 3 4 5 Inspecteurs 0 1 2 3 4 5 Unite Pedagogique 0 1 2 3 4 5 Parents 0 1 2 3 4 5 EVALUATION DES ELEVES Avez-vous fait une evaluation cette année? _____________ Combien d’enfants ont participe a l’evaluation? ____/____ Nombre d’enfants aui ont eu la moyenne. F_____G_____ Que faites-vous des copies d’evaluation des enfants ? Bulletins aux Parents ___________ Discussion avec le/la Directeur/Directrice ______________ Qu’est-ce que les parents ont fait après reception du rendement des éleves ? Rien ________ Ils ont contacte l’école. O ___ N ___ Nom d’observateur/trice __________________ Date __________________ Evaluation of Benin's Basic Education Reform Pedagogical Component USAID – Benin Louis Berger/DevTech 79 ORGANISATION DE CLASSE Organisation physique de la salle0 1 2 3 4 5 Gestion d’apprentissage: 0 1 2 3 4 5 Presence de materiels NPE 0 1 2 3 4 5 Usage effectif des: cahiers 0 1 2 3 4 5 manuels 0 1 2 3 4 5 planches 0 1 2 3 4 5 outils didactiques 0 1 2 3 4 5 Commentaires : __________________________________________________________________________________ __________________________________________________________________________________ ____________________________________________________ LEÇON OBSERVEE Strategies d’instruction Magistrale 0 1 2 3 4 5 Travail collectif 0 1 2 3 4 5 Travail de groupe 0 1 2 3 4 5 Travail individuel 0 1 2 3 4 5 Grandeur pour les activites? 0 1 2 3 4 5 Niveau de participation des éleves0 1 2 3 4 5 Niveau d’activites éleves-éleves 0 1 2 3 4 5 Participation equitable des filles et garçons 0 1 2 3 4 5 N/A Usage de la langue maternelle 0 1 2 3 4 5 Les éleves posent-ils des questions?0 1 2 3 4 5 PERSPECTIVES: Qualite generale d’apprentissage 0 1 2 3 4 5 Commentaires____________________________________________________________________ __________________________________________________________________________________ __________________________________________________________________________________ CE1 Capacité des enfants en Français 0 1 2 3 4 5 Math 0 1 2 3 4 5 CENTRE DE DOCUMENTATION Existe-t-il un centre de documentation dans la circonscription ?O ____ N ____ Combien de fois l’avez-vous utilisé cette année ? 0 1 2 3 4 5 Nom d’observateur/trice __________________ Date __________________ Evaluation of Benin's Basic Education Reform Pedagogical Component USAID – Benin Louis Berger/DevTech 80 Analysis of the Responses to the Observation Grill Although the team visited 211 schools, many were partially closed or only open on Monday and Friday due to the ongoing teachers’ strike. As a result of this situation, some,students, teachers and directors were absent. The team was able to observe 141 classes in session. In classrooms observed, about 20% of the teachers were contractuels, about 40% of the others were APEs, and another 40% communautaires. Classroom size averaged 62 students, with a maximum of 120 and minimum of 20 in CI and CP classes observed. The average number of years that the teachers had been teaching was a little over 7 years. About 50% of the teachers had their BEPC, about 25% had the CAP, 10% had been to the University (ENI) and about the same percentage had less than the BEPC or another certificate. One communautaire we spoke with had a certificate in computers. The knowledge and attitude of the teachers towards the NPE was generally positive reflecting a score of about 3.8 out of a possible 5. Most of the teachers were well supplied with materials. However there were many workbooks missing for the children. Teachers felt their training in the NPE was better than expected. Most of the teachers had at least for weeks of initial training in the NPE and about 25% of that number had follow-up training. Following are the results of their measurement of the professional support they received from various sources. Responses range along a scale of 0 to 5, with 0 being the most unfavorable: Support from: Director, 3.1, from Conseillers Pedagogiques, 1.8, from Inspectors, 2, from Unite Pedqgogiques 3.5, and from the Parents, .01. Hardly any teacher had given an evaluation this year. In most classes the student desks and tables were arranged in groups of 5-8 in spite of the crowding. Materials from the NPE were present in about 90% of the classrooms. Classroom management was rated poor. The teachers seemed to be using the texts and workbooks. There was a great difference among the classes in the progress made in both the French and Math workbooks. Most of the teaching was from the front of the class and very few children responded to the questions asked by the teacher. Unfortunately the same children seemed to be called on. No one on the evaluation team observed a child ask a question. In only one class did the teacher have the class divide into groups to work on their own. Girls were called on and participated at the same rate as boys. In all of the classes visited French was the language of instruction. Only 30 respondents knew of a documentation center in their Circonscription; of those asked, only eight had visited a center this year.