TEACHER MOTIVATION AND TRAINING (TMT) PROJECT, BENIN 2009-2013 FINAL EVLUATION REPORT July 1, 2013 By Amitabh Dabla, PhD 2 Table of Contents List of Tables and Graphs 3 Acronyms 4 1 EXECUTIVE SUMMARY 5 2 INTRODUCTION 10 2.1 TMT Project Summary 11 3 METHODOLOGY 15 3.1 Evaluation Design 15 3.2 Methodological Limitations 18 4 PERFORMANCE INDICATORS 20 5 IMPROVING THE QUALITY OF PRE-SERVICE TEACHERS 23 5.1 ENI Instructors 23 5.2 ENI Students 27 5.3 ENI Resource Centers 28 5.4 Challenges 29 6 IMPROVING TEACHERS PERFORMANCE 31 6.1 MAP and Non-MAP Project Schools & Frequency of Trainings 31 6.2 TMT Impact in Schools 33 6.3 Challenges 36 7 CONCLUDING DISCUSSION 38 7.1 Achievements of the TMT Project 38 7.2 Issues Affecting the Primary Education Sector in Benin 40 7.3 Sustainability of the Achievements of the TMT Project 41 ANNEX A: OSEP Tool 44 ANNEX B: Documents Reviewed 46 ANNEX C: List of ENIs and Schools Visited 48 ANNEX D: Final Evaluation Indicators 50 ANNEX E: Evaluation Instruments 54 3 List of Tables and Graphs Table 1: Student Usage of Resource Centers in the 5 ENIs 21 Table 2: Number of School Directors and MEMP Officials Trained 22 Table 3: MAP Participants and Winning Teachers & Schools 31 Graph 1: ENI Instructors Years of Teaching Experience 23 Graph 2: Impact of ENI Instructors Trainings 24 Graph 3: Impact of Instructors Manual 24 Graph 4: Impact of TMT Project 26 Graph 5: ENI Instructors Satisfaction with the TMT project 26 Graph 6: ENI Instructors Use & Impact of Resource Centers 29 Evaluator Contact Information: Dr. Amitabh Dabla, 1803 Biltmore Street, NW, #803, Washington DC 20009, USA. Phone: 2022481339; cell (5123512576). Email: amit_dabla@yahoo.com; amitd84@hotmail.com 4 Acronyms CoP Chief of Party CP Conseiller Pedagogique (Pedagogical Advisor / Staff Developer) CCS Chef de Circonscription Scolaire (School Administrator/ Pedagogical Advisor) DIP Direction de l’Inspection Pédagogique (Directorate of Pedagogical Inspection) EDC Education Development Center EFA Education for All ENI École Normal des Instituteurs (Teacher Training College) FGD Focus Group Discussion GDP Gross Domestic Product GNI Gross National Income M&E Monitoring and Evaluation MAP Merit Awards Program MEMP Ministère des Enseignements Maternel et Primaire (Ministry of Pre-school and Primary Education) MGDs Millennium Development Goals OSEP Outil de Suivi et d’Évaluation Pédagogique (Standards-based Classroom Observation Protocol — SCOPE) PMP Performance Monitoring Plan RC Resource Center SOW Scope of Work TMT Teacher Motivation and Training USAID United States Agency for International Development UP Unité Pedagogique (Pedagogic Unit) 5 1 EXECUTIVE SUMMARY Education Development Center (EDC) received a $6.3 million award from the United States Agency for International Development (USAID) to implement the Teacher Motivation and Training (TMT) project in Benin from 2009 to 2013.1 The project had two main result areas: (1) Improving the quality of pre-service teacher training in five public École Normal des Instituteurs (ENIs) (teacher training colleges). (2) Improving teacher performance in primary schools through the training of officials from the Ministère des Enseignements Maternel et Primaire (MEMP) including Conseillers Pedagogiques (CPs) and Chefs de Circonscription Scolaire (CCs) and primary school directors. The project also included a pilot program, the Merit Awards Program (MAP), which aimed to improve performance via a motivation initiative whereby teachers demonstrating the most improvement in teaching performance & best performing schools received prizes and awards. A fundamental component of the TMT project is OSEP, which is a pedagogical tool consisting of 16 criteria to observe and evaluate teachers & which supports improvements in the teachers teaching abilities and classroom performance. All the activities and trainings carried out by the TMT project in the ENIs and for the primary schools (including selecting winners for the MAP) incorporated OSEP. Methodology The overall design of the final evaluation of the TMT project was based on a qualitative approach and consisted of the following methodology,  Document and data review focused on education in Benin and the TMT project  Key information interviews with TMT staff, representatives from USAID-Benin, MEMP officials, ENI directors, ENI resource center (RC) managers, school directors and EDC officials  Focus group discussions (FGDs) with ENI instructors and students & teachers in schools  A survey questionnaire for ENI instructors Extensive data was collected; however, the main limitation affecting the final evaluation was that primary schools had exams during the field visits thereby making it difficult to meet a greater number of teachers. Further, students in two ENIs and three ENI directors were also not available for FGDs and interviews. Performance Indicators The TMT project had to meet certain agreed upon results/targets based on performance indicators, which were specified in the contract award. The project realized all these results/targets including,  All four baseline/feasibility studies were conducted pertaining to teacher motivation and performance in schools and ENI instructors performance  All 117 (108 male, 9 female) permanent and temporary instructors at the 5 ENIs were trained  300 copies of the ENI Instructors Manual were provided to the 5 ENIS and MEMP  Ten MEMP officials were trained to become the manager and assistant manager of each RC in all the 5 ENIs 1 The award was reduced to $5.9 million in 2012. 6  Each RC has a functioning cost recovery system as ENI students pays a onetime fee for the upkeep of the RC & the salaries of the RC manager and assistant manager are paid by MEMP  Students make regular and extensive use the RCs in all of the 5 ENIs  Instructors in all the 5 ENIs performed better on the 16 OSEP criteria while teaching at the end of the project compared to the baseline (based on the ‘ENI Instructors Performance Improvement’ study report released by the TMT project)  3112 school directors (of the approximately 6000 primary schools in the country), 70 CCs, 233 CPs (consisting of all the CPs in the country) and 40 other MEMP officials were trained in the first round of trainings. 855 school directors, 73 CCs and 203 CPs were trained in the second round of training (all those trained in the second round had also been trained in the first round) Improving Quality of Pre-Service Teachers The unanimous consensus of the ENI instructors about the TMT project was that it was the first time that they had received this type of training and materials support in their professional careers. The project not only improved, but also changed their method of teaching thereby allowing them to go academically beyond what they previously did as instructors. The ENI instructors were highly satisfied with the three weeks trainings they received and the Instructors Manual developed by the project; agreeing that these inputs improved their skills and knowledge, helped them to introduce learner centered instructional methods in their teaching and helped them to better teach and prepare the students. Other impacts of the project on the ENI instructors include,  Raised their level of pedagogical knowledge and competence  Allowed them to better understand constructivism and to use it in their pedagogy  Allowed them to design class modules based on learned centered methods and templates The instructors also felt that OSEP had reinforced their own learning on how to teach & on how to observe and counsel students, as they used it to evaluate their students when they were doing internships in schools. ENI students benefitted from the project in the following manner,  All the knowledge/skills the instructors obtained from the project is passed on to the students, thereby permitting them to develop an exit profile to be able to better teach in schools  The students are better prepared to master the subjects they will teach in schools and have better knowledge (and strategies) to communicate with the students  Learner centered instruction has been incorporated by the instructors, which has allowed the students to be directly involved in their own learning All the RCs in the five ENIs are operational and are being used for documentation and Internet research by the students. The impact of the RCs includes,  Students have greater access to knowledge and information via Internet and documentation research that they did not have before  Students are becoming used to using information and communication technologies (ICTs) for their learning, which is beneficial for their future learning and teaching  What students pay for the Internet and photocopying in the RC is much less than what they would pay outside the ENIs 7 The TMT project had a very positive impact on the ENI instructors and students and the RCs are providing valuable resources, yet certain challenges existed. These include,  Mastering the theory and practice of knowledge-building pedagogies within the span of three week-long training sessions and with the support of a 200-page manual is difficult, and certain concepts in the instructors manual and training were dense for the instructors.  Training requires follow-up and reinforcing and many instructors expressed the need to build further skills and knowledge.  A blog was created for ENI instructors so that they could pose questions to each other, share resources and build an online community. However, the instructors citing their lack of knowledge of using computers are not taking advantage of this networking resource.  There are only six laptops available for students use in each RC, thus each student’s use of a laptop is limited to 30 minutes. Improving Teacher Performance Out of the 3,112 school directors that were trained by the project, 588 directors and their schools were selected to take part in MAP (over its 2 editions) & each year 120 teachers and 24 schools won awards. School directors coached, counseled and advised their teachers to incorporate and perform better in the OSEP 16 criteria to win MAP awards. Hence, for a teacher in a MAP school to improve his/her performance, effective teaching/training of the teacher by the school director on OSEP was required and critical. Teachers in MAP schools were also motivated to learn, improve and to win prizes; so they were receptive to the school director when coached/counseled on OSEP. Further, CPs also observed and evaluated the teachers using OSEP (as part of the selection criteria for choosing winners) and thus also gave them feedback and advice on how to improve on the 16 criteria. On the other hand, 2,524 school directors (and their schools) did not participate in the pilot program, the MAP. The directors of these schools were expected to observe their teachers using OSEP and to counsel and advice the teachers on improving on each of the 16 criteria. The key for improving teaching performance in the non-MAP schools thus was, again, the school director who had to be pro-active and spend time and effort to observe, counsel and advice the teachers. Another critical issue affecting the teachers’ performance was how many round of trainings each school director received; as 855 school directors received two rounds of trainings from the project providing them with a better platform to incorporate OSEP in their schools. Consequently, the final evaluation data demonstrates that school directors who either (i) did not participate in MAP or (ii) did not receive a second round of training were the least capable of providing OSEP based counsel and support to their teachers. Therefore, those teachers who were in (i) MAP schools or (ii) in non-MAP schools where the school director had undergone two rounds of trainings were more proficient in the use of OSEP and thereby improved their performance; provided that the school director was pro-active and spent time observing and counseling the teachers. In these cases the project’s impact in the schools included,  All school directors and teachers teaching methods and strategies improved  More tools and information were provided to school directors on how to observe, evaluate, manage and supervise teachers  School directors learnt how to better counsel and advice the teachers in classroom practices  School directors and teachers got more knowledge about pedagogy 8  Teachers were better equipped to manage the planning and sequence of their classes, to ask questions and to enable both boys and girls to participate equally in classroom discussions The TMT project had a positive impact on teachers’ performance in the primary schools, yet certain challenges existed. These include,  Directors who did not receive two rounds of training had difficulties to successfully impart OSEP in their schools, even if they were pro-active and willing to do so.  Lack of materials and resources, class size and lack of proper buildings in schools affect the application of all the OSEP criteria in the classroom  Pro-active MAP school directors worked hard to train their teachers in OSEP yet the 1st edition of MAP did not provide any prizes for them; leading many of them to resent the set up of MAP, even thought the 2nd edition of MAP had prizes for school directors.  Non-MAP schools did not have anyone associated with the TMT project come to their schools to see how the trained school directors were using OSPE, how they were advising and counselling their teachers etc. TMT Project’s Sustainability In addressing several objectives, including improving pre-service teacher’s training and teachers’ performance, the TMT project met a range of acutely felt needs in the primary education sector in Benin. However, certain contextual issues exist that pose a challenge to the project’ sustainability. These include,  Some trained ENI instructors are retiring while other instructors have left the ENIs. A key issue is who will train the new instructors who come to teach at the ENIs.  Current fees provided by the ENI students are adequate for the functioning of the RCs, but no mechanisms are in place whereby additional funds will become available (in the future) to pay for the renewal of journal/magazine subscriptions & for buying additional computers/books.  Personnel changes in schools have a range of negative impacts on the durability of learning and changed practice inspired by the project.  ENI instructors and school directors need re-fresher trainings so that they do not loose what they have learnt.  The education system in Benin is very politicized and teachers’ strikes are common, leading to great disturbances in teachers’ performances and students learning.  MEMP has not hired any new teachers for primary schools in the past two years and it is not clear if any new teachers will be hired in the next academic year.  Many pro-active MEMP officials who supported the project have been transferred These issues are very challenging, nonetheless, data gathered for the final evaluation demonstrates that the achievements of the project are significantly sustainable based on two overlapping and synergetic domains. At the level of individuals, the following factors bode well for the sustainability of the TMT project.  All the CPs in the country along with a larger majority of CCs have been trained and these officials will continue to act as master trainers/change agents to further spread OSEP in schools.  What the trained ENI instructors have gained from the trainings and manual they will keep on applying and transmitting to the students, thereby consistently improving the student’s exit profiles to teach in schools. 9  Pro-active school directors will continue to use the new skills they have gained to supervise their teachers & pro-active teachers will continue to use their new knowledge (based on the OSEP criteria) to better teach students.  Certain pro-active MEMP officials remain who support the benefits the project has brought At the level of the primary education system, the following factors bode well for the sustainability of the TMT project.  OSEP has been ingrained in the primary education system via the trained CCs, CPs, ENIs and school directors  In MEMP the ‘Harmonization of Supervisory Instruments and Pedagogic Control’ based on the OSEP criteria is underway. When this harmonization is complete, all the CPs in the country will use an instrument based on OSEP to evaluate teachers in schools.  The Benin 2013-15 Education Sector Plan calls for the improvement of the quality of teaching and better evaluation of teachers. As part of this plan, studies for the actualization of an improved primary curriculum are planned. The department of Direction de l’Inspection Pédagogique at MEMP is partly responsible for these studies, and officials in this department plan to include the learning from OSEP and the benefits it brings to teaching and learning to inform the studies. Reflecting on the TMT project in this scenario, the final evaluation has clearly demonstrated that with its limited budget and funding cuts, the project clearly addressed critical needs of the primary education system in Benin. While the project faced limitations, it was able to make a positive impact on improving pre-service teachers’ trainings and teachers’ performance in schools. The factors affecting the primary education sector in Benin are complex and the final evaluation acknowledges that more work needs to be done for improving the quality of the primary school teachers in Benin. Nonetheless, supporting factors at both the individual and systemic level exist that bode well for the durability of the TMT project’s positive impacts for the immediate future. 10 2 INTRODUCTION Education Development Center (EDC) received an award from the United States Agency for International Development (USAID) to implement the Teacher Motivation and Training (TMT) project in Benin from 2009 to 2013. The project aimed to improve teaching through: training, instruction, supervision and motivation at the primary education level. This report encapsulates the results of the final evaluation of the TMT project. *** Benin is a small country with a population of 9.1 million (2011) situated on the Western coast of Africa. GNI per capita (US$) in 2011 was 780, while the GDP (US$ billions) rose from 2.0 in 1991 to 7.3 in 2011. The national economy relies heavily on the agriculture sector, which accounts for about 32 percent of GDP and is the source of livelihood for nearly 70 percent of the country’s workforce. Benin has made progress in improving access to basic services but achieving some of the Millennium Development Goals (MDGs) remains a significant challenge. The nation is on track to meet the MDGs for access to potable water in rural areas, eradicating hunger and reduced HIV/AIDS prevalence. On the other hand, in the health sector child and maternal mortality remain high. In the education sector, the universal primary education goal and the completion rate goal for boys are likely to be attained, but the 2005 goal of parity in primary and secondary education has not been met and will likely not be reached by 2015. Furthermore, improving the quality of education and the management of this sector remain key challenges.2 Over the past four decades, significant changes have occurred in the education system, but the current reality is that many primary schools in Benin fail to provide students with a quality education and adequate language or Math skills. A combination of factors that include increasing primary-grade enrollments resulting from Benin’s response to the challenge of Education for All (EFA), and lack of both trained teachers and capacity for teacher training are responsible for this challenge. While primary completion rates have risen (from 21 percent in 1991 to 65 percent in 2005), girls remain underrepresented in school (40.6 percent of enrollment in primary grades, 31.2 percent of enrollment in secondary grades) and increases in literacy rates have been minimal (from 40 percent in 1990 to 45 percent in 2006).3 These figures suggest that the efficiency of the system has increased, without an augmentation in its effectiveness. In addition, the proliferation of teachers’ strikes that can cut weeks of instructional time out of a given academic year greatly affect the value of school completion rates of students. 2 See: http://www.worldbank.org/en/country/benin; http://www.unicef.org/infobycountry/benin_statistics.html#103 3 See: UNESCO Institute for Statistics 11 The performance monitoring plan (PMP) of the TMT project links primary students’ poor levels of learning to teachers’ lack of professionalism while the 2008 EFA monitoring report states that Benin is likely to reach universal primary education by 2015, but it will do so at a tremendous cost of quality. Further, low morale and motivation are important factors in the poor quality of Beninese education; and a lack of adequate supervision and guidance as well as inferior working conditions have led to a decline in professionalism among teachers. In the 1990s, the government of Benin undertook an extensive curriculum-reform project with support and guidance from USAID. This process resulted in an increased curricular focus on competency-based learning, undergirded by constructivist approaches emphasizing active learning (‘learning by doing’), collaborative methods and the linkage of content to students’ social and cultural milieu. However, this reform has been impacted by under-trained and non-professional teachers, increasing class sizes and lack of suitable classrooms and textbooks. Further, in 2006, five Écoles Normales des Instituteurs (ENIs)4 were opened focusing on the education of primary teachers.5 Each ENI is made up of permanent and temporary faculty whose experience is generally extensive, but much of it has taken place in schools that lack resources and where overall levels of expertise, professionalism and teacher motivation are low.6 The primary education system in Benin has resource shortages and teachers that lack training and professionalism, and it is within this context that the TMT project was implemented. 2.1 TMT Project Summary The TMT project has two main result areas (1) Improving the Quality of Pre-Service Teacher Training Through:  Developing an instructor manual based on modern teaching principles for ENI instructors  Providing intensive technical training to instructors in the 5 ENIs  Setting up resource centers (RC) in the 5 ENIs (2) Improving Teacher Performance Through:  Conducting a baseline study of teacher performance  Conducting annual assessments of teacher improvement, impact on students and gains in quality and frequency of supervision7  Training of school directors & supervisors in the Ministère des Enseignements Maternel et Primaire (MEMP) (Ministry of Pre-school and Primary Education) in the use of OSEP (a classroom observation tool) The project also included a pilot program, the Merit Awards Program (MAP), which aimed to improve performance via a teacher and school motivation initiative whereby participants demonstrating the most improvement in teacher performance and best performing schools received awards. 4 ENIs are Teacher Training Colleges 5 Prior to that time, in the late 1980s, Benin’s teacher training colleges were closed due to political reasons. 6 ENI instructors are technically not professors, as they do not have Doctorate degrees or very extensive academic credentials. 7 Please note that this objective was related to the MAP component of the project and no formal annual assessments of teachers were conducted by the project. 12 Outil de Suivi et d’Évaluation Pédagogique (OSEP) A fundamental component of the TMT project is OSEP, which is a pedagogical tool to observe and evaluate teachers & which simultaneously serves to support improvements in the teachers teaching abilities and classroom performance. OSEP centers around 16 criteria, which support observation and evaluation of teachers’ pedagogical activities in the classroom, enabling evaluators to identify needs for skill development and to provide feedback to teachers. OSEP was adapted from a tool developed by EDC for use in Egypt, the Standards-based Classroom Observation Protocol for Egypt (SCOPE). The 16 criteria of OSEP address areas of classroom practice ranging from classroom management to pedagogical practices, and include issues such as:  Management of time and learning resources  Creating a learner-centered classroom environment  Ensuring equal opportunities for learning for all students, including girls  Asking open (rather than closed) questions The 16 OSEP criteria are reflected in a grid (or “grille”) that is used to record a teacher’s performance in relation to each criterion. To help achieve objectivity and consistency, scores for each of the criteria are linked to five descriptions of classroom practices, with “1” indicating least-conforming practice and “5” indicating best practice. The criteria award higher scores to teachers who structure learning around active, learner-centered, collaborative and broadly constructivist approaches. Teachers receive higher scores when they engage students in small-group work; pose open questions; link material to students’ lives and experiences outside the classroom and; provide opportunities for students to solve problems. Teachers who pose questions that are to be answered in unison by students, or who emphasize mastery of content rather than analyses of relationships within that content, receive lower scores. For example, in relation to the first criterion, which addresses classroom management, a teacher would receive a score of 1 (“Instructional time is poorly planned and managed”) when more than half of class time is lost as a result of poor distribution of materials, poor transitions between activities or similar factors. A score of 5 (“Instructional time is well planned and is managed effectively”) would be awarded for classroom management in which practically no time is lost to inefficiency or lack of preparation. See Annex A for the OSEP tool. ENIs Courses taught at the ENIs address the subjects (Math, French, Science etc) that teachers teach in primary schools. Augmenting these courses are other courses intended to undergird the students understanding of other key areas including child development, pedagogical theory, evaluation of learning and artistic education etc. Students at the ENIs participate in a two-year program. During the first year, students (èléves maitres) attend courses at an ENI and are sent to nearby primary schools for three practical sessions lasting two to three weeks. Second-year students do not attend classes at the ENI and are posted to schools as student teachers (stagiaires). The Instructors Manual developed for the project ranged across conceptual and practical materials to help ENI instructors address gaps in their knowledge of teaching. The eight modules of the manual include:  An exit profile for student teachers  A vision of Beninese schools 13  The OSEP teaching evaluation tool  Constructivism  Critical thinking and problem-solving skills  Learner-centered pedagogy  Lesson planning  Conclusion Each module provides critical information accompanied by exercises and activities. Instructors can engage in these activities and can also use or adapt these activities in the courses that they teach. In engaging the ENI instructors in small-group work, problem solving and other activities, the manual supports modeling of learner-centered pedagogies. The manual was also designed to support the training provided to ENI instructors. Permanent and temporary instructors in all of the five ENIs participated in three training sessions centered around the contents of the manual. One hundred seventeen (117) instructors participated in the trainings, which were conducted by non-Beninese education experts and Beninese Master Trainers. The three training sessions had different areas of focus, and special attention was paid to the OSEP criteria. The TMT project also carried out a needs assessment in 2010 in the 5 ENIs to determine what resources and materials were available and what were needed for students and instructors. Based on this exercise a plan was devised to situate a RC in each ENI and to train two MEMP officials who would serve as the manager and assistant manager of the RC. The RC personnel were selected in 2010 but were only trained in 20128 in various subjects including,  Management of RC  Documentation  IT/computers  PH-mybiblio and management software Each RC was provided with seven computers (6 laptops and 1 pc), reference books, academic textbooks, novels, journal/magazine subscriptions and Internet connectivity.9 Schools Schools in Benin are staffed by small faculty teams and school directors continue to teach classes and fill leadership, administrative and educational roles. Faculties in the schools visited for the final evaluation ranged from three to six teachers, typically providing instruction to classes with 30+ students. Further, primary education in Benin relies on a well-defined organizational structure for teacher observation and evaluation. Teachers are evaluated routinely by their school directors, with additional evaluations performed by Conseillers Pedagogiques (CPs), who are MEMP district officials who inspect and evaluate schools and provide pedagogical support. As part of the turn-key trainings10 for school directors, school superintendents (CCs)11 and CPs on OSEP were carried out over the duration of the project and can be broadly divided into three components, 8 The delay was due to MEMP not officially approving the RC officials for more than a year 9 Journal/magazine subscriptions and Internet connectivity payments were for the duration of the TMT project only. 10 The project essentially had one level of turnkey training, where one core cadre was trained and they in turn trained all the other groups of people who received training. 14  First round of OSEP trainings for schools directors, CCs and CPs  Second round of trainings for school directors, CCs and CPs in supervision and evaluation of teachers (including reinforcement of OSEP concepts and strategies)12  Training for school directors and CPs participating in MAP (as part of the first round of OSEP trainings) Trained school directors were expected to observe and evaluate the teachers in their schools using OSEP and then to counsel them on improving in the 16 criteria. CCs and CPs were to become familiar with OSEP and CPs, when visiting schools, were to use it to further evaluate and counsel teachers. The schools in MAP were selected randomly by the project and the program consisted of awards for best performing teachers and best performing schools. Teachers in MAP schools were evaluated several times in relation to OSEP. First, all teachers were evaluated by their school directors, who sent their evaluations to the TMT office. After all the teachers evaluation forms were received, they were examined and the top performers were selected to be evaluated again by a CP (using OSEP), who filed a second￾round report. In the third and final stage, all teachers nominated for the final MAP award (based on the CPs second round evaluations) underwent a final round of external evaluation, conducted by TMT master trainers. Thus, teachers nominated for a MAP award were evaluated three times. Teachers nominated as individuals (without their schools being nominated for a MAP award) were evaluated two times with no second-round evaluations by CPs. MAP existed for two editions (over two academic school years: 2010-11 & 2011-12) as part of the project and winning teachers and schools were selected across the country; who were then honored at an annual Merit Awards ceremony. The top ten winning teachers at the national level received laptop computers and other winning teachers received cameras, books etc & prizes for schools included bookshelves, French dictionaries and grammar books, soccer goal posts, materials for school upkeep etc. *** This report is structured in the following manner. Chapter 3 provides a detailed overview of the methodology employed in the final evaluation along with its limitations. Chapter 4 enumerates the results/targets achieved by the project based on the performance indicators stipulated in the contract with USAID. Chapter 5 discusses the impact of the project on the quality of pre-service teachers. Chapter 6 discusses the impact of the project on teacher performance. Chapter 7, the concluding chapter, summarizes the final evaluation results and includes a discussion on the sustainability of the impact of the TMT project in Benin. 11 Each CCs is responsible for a certain number of schools and provides pedagogical advice to school directors and teachers through the Unité Pedagogique (Pedagogic Unit), which are held twice a month in each school. Administratively one CCs oversees numerous CPs in a given area. 12 Only those school directors, CCs and CPs who had undergone the first round of training were eligible for the second round of training. Further, MEMP district officials were responsible for selecting which of the previously trained personnel would attend the second round of training. 15 3 METHODOLOGY The purpose of the final evaluation was to assess the activities of the TMT project over its entire duration as well as the project’s performance in improving the quality of pre-service teacher training and teacher performance. As per the scope of work (SoW) the objectives of the final evaluation were to,  Assess the project’s contribution to improving the quality of pre-service training and teacher performance, based on previously agreed results/targets as specified in the award.  Describe the extent to which the project incorporated findings and recommendations from the mid-term evaluation and if possible determine to what degree these recommendations contributed to the overall performance of the project.  Offer concrete recommendations for future teacher-related programming in Benin. Other complementary objectives of the final evaluation were to,  Determine the extent to which goals and objectives of the project were met  Evaluate—and identify signs of—the sustainability of project activities completed  Evaluate the change in instructor quality/skills in the five ENIs including the impact on student teachers  Evaluate the use and utility of the resource centers attached to each ENI  Evaluate the degree of satisfaction of the diverse beneficiaries including instructors, student teachers and the administration at the five ENIs; school directors and teachers in MAP and non MAP schools; MEMP officials and; USAID 3.1 Evaluation Design The overall design of the final evaluation of TMT was based on a qualitative approach due to the timeframe for conducting the field research in Benin (between May 15-June 5, 2013). The qualitative approach is inherently more flexible and responsive to new information about the experiences and opinions of respondents, and the methods employed in the final evaluation have ensured that all relevant information about the project’s implementation and outcomes were captured. The final evaluation consisted of the following methodology, Document and Data Review Document and information-resource review focused on the development of background information on education in Benin and on detailed understanding of the TMT project. For a list of resources consulted, refer to Annex B. Key Information Interviews Key information interviews were conducted with the following:  TMT management and program staff in Benin (including the Chief of Party, M&E Officer, Training Specialist, MAP Program Officer, IT Manager) 16  Representatives from USAID-Benin (including the Education Officer and Ex-Head of Education and Social Service Programs)  Adjoint de l’Inspection Pedagogique, Conseiller Technique a l’Enseignment Primaire, Membre du Cabinet, MEMP  2 ENI Directors and 1 Chief of Educational Services & Documentation (2nd highest ranking official in the ENI)  5 ENI RC managers and 3 assistant managers  21 school directors  EDC TMT project Program Director in the United States Focus groups discussions (FGDs) FGDs were held with the following:  35 ENI instructors  24 ENI students  19 teachers in schools To review a list of the schools and ENIs visited see Annex C. Survey A multiple choice survey questionnaire was administered to 35 ENI instructors during the evaluator’s visits to the five ENIs. Checklist A checklist was administered in the RC of all 5 ENIs. *** The sample of TMT project schools was structured to ensure adequate representation of:  Rural and urban schools  Schools from the North and the South of the country  MAP participating and non-MAP participating project schools In response to logistical and other considerations, only schools participating in the TMT project were visited in the course of the final evaluation; this evaluation does not include an assessment of or comparison with teachers at non-TMT project schools. Further, the resulting sample of schools visited should be considered a quota sample; the sample was not formally randomized but was instead structured to ensure adequate representation of schools according to the above criteria. Evaluation Models The TMT final evaluation was designed as a summative evaluation, and as such assesses the impact of the project in relation to the experiences of intended beneficiaries, while also identifying barriers to impact, issues of sustainability & generating success stories and recommendations. The evaluation design also includes the evaluative model of Most Significant Change (MSC). MSC emphasizes an open and qualitative process for data collection. Field-level respondents (ENI instructors, school directors and 17 teachers) were asked to describe the most significant change resulting from the project intervention; these changes were then assessed by project implementers and other stakeholders to best exemplify the impact/outcome of the project. Indicators Indicators used in the design of the final evaluation fall into three categories:  Performance indicators drawn from the PMP include items such as the number of school directors trained, the number of baseline studies conducted etc.  Impact indicators are linked to specific project objectives and were developed to enable assessment of the effects of TMT on pre-service teacher training and on teacher performance.  Indicators emanating from the recommendations of the mid-term evaluation include post training support provided to school directors, networks established for ENI instructors for knowledge sharing etc. Please note, however, that recommendations made in the mid-term evaluation report were not compulsory requirements that the project had to incorporate and implement. Further, deliverables based on these recommendations were not part of EDC’s contract with USAID. For a complete list of indicators, see Annex D. Interview, FGD and Survey protocols The following instruments were developed for this final evaluation:  Interview protocol, ENI director  FGD protocol, ENI instructors  FGD protocol, ENI students  Interview protocol, school directors  FGD protocol, school teachers  Interview protocol, MEMP personnel  Interview protocol, USAID and EDC personnel  Survey protocol, ENI instructors Examples of these instruments and the ENI instructor’s survey are included in Annex E. Analysis Data collected from the field visits in Benin was analyzed qualitatively and limited levels of descriptive statistics were employed. Given the sample size and qualitative design of the evaluation, descriptive statics were used primarily to enable quick understanding of the overall contours of the project (as portrayed by ENI instructors’ survey results and data provided by EDC for the performance indicators). Further, opinions are characterized as such and limitations in data collection, if any, are noted. Points or concerns that emerged in the course of the evaluation were incorporated to support assessment of these points’ validity and importance. This evidence-based and participatory approach to the evaluation thus ensured that all voices (especially those of the targeted instructors, teachers, school directors etc) were heard and that the conclusions reached are grounded in the data gathered. 18 3.2 Methodological Limitations The main limitation affecting the final evaluation was the time-period when the fieldwork was carried out. The evaluator visited Benin from May 15 to June 5, 2013; however, primary school students had exams beginning on June 1. Due to this reason, it was not possible to visit any schools after May 31 and in all the schools that were visited, it was difficult to meet and host FGDs with a large number of teachers as they were occupied with the upcoming exams.13 Thus, several factors including the upcoming exams, time constraints, travel required etc limited the number of schools that could be visited during three weeks in Benin. While the sample for the final evaluation is appropriately diverse—representing MAP and non MAP project schools and teachers, rural and urban schools, and schools in most parts of the country—the schools visited represent a far smaller percentage of the 3,112 government primary schools that were targeted by the TMT project.14 In general, however, the approach outlined for the final evaluation minimizes the importance of sample size, as it is less critical than the quality of the sample (in terms of representation) and the opportunity that this sample provides to conduct an open-ended investigation. The evaluator visited all 5 ENIs and the project informed each ENI about the evaluation visits so that the ENI director, ENI instructors and ENI students would be available for interviews and FGDs. However, three of the five ENI directors were not present during the evaluator’s visit. The evaluator was also not able to talk to any 2nd year ENI students as they were interning in schools, while in two ENIs no 1st year students were present during the evaluator’s visit.15 Further, all the trained ENI instructors were not present during the evaluator’s visits as either (i) they were not scheduled to teach classes on that day or (ii) they had retired or left the ENI.16 It should also be noted that the evaluator had planned to speak with/interview up to 2-4 senior MEMP officials, but after making concerted attempts only one MEMP official made himself available for an interview. Nonetheless, to the extent possible, the design of the final evaluation mitigates the limiting factors. Key mitigating steps include the following:  Multiple information sources. At ENIs and schools, questions to primary respondents (ENI instructors, school directors etc) were supported by similar questions asked of ENI students and schoolteachers, thereby enabling the information provided by instructor and directors to be crosschecked.  Follow up questioning. Rather than asking respondents a question and moving on to another question, the evaluator posed follow up questions (as far as possible). In this manner more detailed, holistic and contextual data was obtained. 13 In two of the schools that the evaluator visited, there were no teachers present as they had decided not to attend schools before the exams. This example points to a bigger issue in the primary education system in Benin, where teacher absenteeism is an ongoing and serious challenge. 14 There are approximately 6,000 government run public primary schools in Benin. 15 In ENI Kandi, the1st year students were doing their two-week practical session in local schools. In ENI Djougou, the interview with the directors and instructors had to be rescheduled from the morning to the evening, as the instructors were not available in the morning. However, when the interviews and FGDs were carried out in the evening, no 1st year ENI students were available on the ENI premises. 16 The final evaluation does not have any concrete data on how many instructors had retired or left the ENIs. 19  Open and in-depth interviews and focus groups. The evaluation design focused on qualitative investigation, remaining open to comments, opinions and experiences of all types to mitigate limitations posed by the small sample size. By capturing any and all forms of inputs on the part of the project participants, and by being designed to be adaptive, the evaluation could identify, pursue and assess success factors, field-level challenges and other important information as they arose. 20 4 PERFORMANCE INDICATORS This chapter provides an assessment of the TMT project’s contribution to improving the quality of pre￾service training and teacher performance, based on previously agreed results/targets as specified in the contract award. These previously agreed results/targets were enumerated through a number of performance indicators in the PMP.17 Data collected from EDC records demonstrate that all the results/targets pertaining to the performance indicators were achieved by the TMT project. These performance indicators include,  Number of baseline or feasibility studies conducted Four studies were completed including, 1. Teacher motivation baseline study 2. Teacher performance baseline study 3. ENI instructors performance baseline study 4. ENI instructors performance improvement study  Number of educators/instructors in ENIs trained with USG support All 117 (108 male, 9 female) permanent and temporary instructors at the 5 ENIs were trained in a three week training program.18  Number of textbooks and other teaching and learning materials provided to the ENIs with USG assistance 300 copies of the Instructors Manual were provided to the 5 ENIS and MEMP. All the 117 trained ENI instructors received a copy of the instructors manual.  Number of ENI resource personnel trained in library and center management Two MEMP officials were trained in library and center management for each RC in all the 5 ENIs. These two officials were then hired as the manager and assistant manager of each RC. An additional official from the ENI in Aldada (that trains students to become pre-school teachers) was also trained as part of this exercise.  Number of ENI library/resource centers with functioning cost-recovery systems All the 5 ENIs have functioning cost-recovery systems for their RCs as each student pays a onetime fee of CFA 5,00019 specifically for the RC (when joining the ENI for the 2 years of education/training). Further, the salaries of the RC manager and assistant manager are paid by MEMP. 17 Due to the cutbacks in the project funding in 2012 certain performance indicators were removed. See Annex D for more information on the performance indicators that were removed. 18 This number refers to all the instructors who were teaching at the ENIs at the time the trainings were carried out in 2011. 19 $1 is approximately CFA 500. 21  Number of students using the ENI resource centers Table 1 below provides data on the number of students using the RCs in the 5 ENIs in 2013.20 RC officials did not gather data for the number of instructors using the RCs. Table 1: Student Usage of Resource Centers in the 5 ENIs All visits refer to the number of students that came to the RC and signed up to use the laptops or to read the books/journals every day (these numbers were then tallied for each month). New visits refer to the number of students who had never been to the RC before and came to the RC for the first time over the course of each month. Many 1st year students who were not using the RC started to do so over time and in many of the ENIs 2nd year students also visited the RC, hence there was always a steady stream of new students visiting the RC. It should also be noted that there are only 6 laptops in each RC therefore students have to wait a long time to use this resource, and this issue disinclines some students to visit the RC. 20 These are the latest numbers provided by the ENIs to the project as of writing the final evaluation report. Month (2013) ENI Abomey (m/f) ENI Kandi (m/f) ENI Djougou (m/f) ENI Porto Novo (m/f) ENI Dogbo (m/f) All Visits January 229 (140/89) 52 (47/5) 104 (86/18) 86 (52/34) 77 (54/23) February 200 (126/74) 21 (20/1) 75 (71/4) 82 (53/29) 6 (2/4) March 303 (211/92) 576 (482/94) 263 (252/11) 86 (52/34) 409 (266/183) April 161 (103/58) 57 (55/2) 140 (137/3) 51 (30/21) 21 (12/9) May 219 (149/70) 43 (42/1) 79 (76/3) 56 (24/32) 146 (43/103) June 119 (64/55) n/a 49 (46/3) n/a n/a Total 1231 (793/438) 749 (646/103) 710 (668/42) 361 (211/150) 659 (377/282) New Visits January 95 (56/39) 44 (39/5) 83 (65/18) 43 (27/16) 23 (10/13) February 41 (26/15) 10 (9/1) 25 (22/3) 23 (14/9) 0 March 117 (50/67) 148 (100/48) 136 (111/25) 43 (27/16) 444 (261/183) April 44 (21/23) 8 (6/2) 7 (6/1) 51 (30/21) 21 (12/9) May 16 (11/5) 2 (1/1) 1 (0/1) 28 (22/6) 146 (43/103) June 8 (7/1) n/a 3 (2/1) n/a n/a Total 321 (171/150) 212 (155/57) 255 (206/49) 188 (120/68) 634 (326/308) 22  Percentage/number of ENI instructors with improved teaching skills Based on the ‘ENI Instructors Performance Improvement’ study report (Rapport de l’Etude sur la Performance des Formateurs d’ENI au Benin) released by the TMT project in May 2013, instructors in all the 5 ENIs performed better at the end of the project compared to the observation results made during the baseline on ENI instructors performance.21 However, the report also states that further efforts are still required for improving the ENI instructors performances.22  Number of school and MEMP administrators and officials trained Table 2 below provides an overview of the number of officials and administrators that were trained as part of the TMT project. All the CPs in the country were trained and the project as well as nearly half of the entire primary school directors in Benin. Table 2: Number of School Directors and MEMP Officials Trained School Directors (m/f) CCs (m/f) CPs (m/f) MEMP Officials (m/f) Total (m/f) First Round of Trainings 3112 (2471/ 641) 70 (69/1) 233 (213/20) 40 (32/8) 3455 (2785/670) Second Round of Trainings23 855 (668/187) 73 (69/4) 203 (190/13) - 1131 (927/204) Total (m/f) 3967 (3139/828) 143 (138/5) 436 (303/33) 40 (32/8) 4586 (3712/874) Challenges The TMT project achieved all its targeted results based on the performance indicators that were included in the contract award. However, there are some challenges with these results.  300 copies of the Instructors Manual were provided to the 5 ENIs and to MEMP and all the trained instructors received copies of the manual. However, some temporary instructors did not receive a final copy of the manual.24 Each ENI, however, was provided enough copies of the final manual to give to each trained permanent and temporary instructor, and the reason for some temporary instructors not receiving the final copy is due to a problem with the internal distribution system in the ENIs.  Ten officials (five of whom became RC managers and five who became assistant RC managers) were trained by the project and MEMP pays the salaries of these officials in each of the 5 ENIs. However, two of the RC assistant managers have left and no replacements have been hired by MEMP. 21 Observations for both studies were based on the OSEP criteria. 22 The final evaluation did not carry out observations to measure the improvements in the ENI instructors performance. More information about the improvements in the instructors performance can be obtained by requesting a copy of this report from EDC. 23 All those trained in the second round had previously been trained in the first round. 24 Initially a draft copy of the manual was provided to all the ENI instructors before a final copy was published and provided to them. All the permanent ENI instructors have received the final copy of the manual. 23 5 IMPROVING THE QUALITY OF PRE-SERVICE TEACHERS This chapter provides an analysis of the impact of the activities of the TMT project on improving the quality of pre-service teacher at the level of the ENI instructors, students and the resource centers. 5.1 ENI Instructors Thirty-five trained ENI instructors provided their inputs to the final evaluation via FGDs and a multiple￾choice survey. Twenty of the instructors were permanent while 15 were temporary; and 30 of the instructors were male while five were female. The teaching experience of the instructors varied, with the average being approximately 6 years (see graph 1 below). Graph 1: ENI Instructors Years of Teaching Experience The ENI instructors were highly satisfied and greatly valued the three weeks trainings. As Graph 2, below, demonstrates the vast majority of instructors agreed that the trainings improved their skills and knowledge, helped them to introduce learner centered instructional methods in their teaching and helped them to better teach and prepare the students.25 These sentiments were accentuated in the FGDs with many instructors commenting that they had never received such type of beneficial training in all their professional careers. However, some instructors felt that too much material was covered over the three weeks training period and that they would have benefitted from an additional round of re-training six months to one year after their initial trainings. 25 The words ‘ENI students’ and ‘pre-service teachers’ are used interchangeably in this report. 0 2 4 6 8 10 12 14 Two Years Three Years Four Years Five Years Six Years Seven Years Twenty Four Years Twenty Nine Years 24 Graph 2: Impact of ENI Instructors Trainings The ENI instructors were also highly satisfied with the Instructors Manual with the vast majority of respondents saying the manual helped improve their skills, helped them better teach and prepare their students and helped them incorporate learner centered instruction (see graph 3 below). In the FGDs, the instructors also commented that they refer to the manual regularly as it helps them plan their class modules. Further, the instructors said they regularly use the 16 OSEP criteria when they teach their students & employ this tool to observe and evaluate their students (when they are doing internships in schools). Some instructors, however, did point out that many of the concepts in the manual were quite dense and it took them some time before they could fully master all the manual’s contents. Graph 3: Impact of Instructors Manual 0 5 10 15 20 25 30 35 Helped me to improve my skills and knowledge Helped me to incorporate learner￾centered instructional methods into my teaching Helped me to better teach and prepare the pre-service teachers I am satisfied with the trainings Very much Somewhat Not so much Not at all 0 10 20 30 40 Helped me to improve my skills and knowledge Helped me to incorporate learner-centered instructional methods into my teaching Helped me to better teach and prepare the pre-service teachers I refer to the Instructors Manual on a regularl basis I make use of OSEP in teaching and evaluating the pre-service teachers I am satisfied with the Instructors Manual Very much Somewhat Not so much Not at all 25 The unanimous consensus of the final evaluation data was that this was the first time that the ENI instructors had received this type of training and materials support in their professional careers; and this support had helped them to improve their skills, quality and teaching abilities substantially. The other key impacts of the project on the level of the ENI instructors include,  Raised their level of pedagogical knowledge and competence  Allowed them to strategize to use student centered learning  Allowed them to better understand constructivism and to use it in their pedagogy  Permitted them to better manage and evaluate their students  Give them an opportunity to think more deeply about the knowledge they are passing on to their students  Grounded them in classroom practices  Allowed them to design class modules based on learned centered methods and templates  Allowed them to improve their teaching behavior The instructors also felt that OSEP had reinforced their own learning on how to teach & on how to observe, evaluate and counsel students on the 16 criteria. They also used it to evaluate their students (when they were doing internships in school) and this provided them another opportunity to advice the students on how to become better teachers. The instructors are thus drawing on the manual and the OSEP criteria to model active-learning approaches in their courses and to use active-learning principles as standards for evaluating the work of student teachers. Applying the most significant change (MSC) method (as listed in the methodology), the following phrases provide an insight on what aspects of the gains from the project the ENI instructors considered as being most significant,  Practice of class teaching  Planning of classes  Strategies of teaching  Fundamental use of strategy  OSEP criteria for evaluation  Approach to resolution of problems  How to approach pedagogic practice The goal of the TMT project was to improve the quality and competence of ENI instructors and hence the level of teaching in the ENIs. Without a doubt this goal was attained. As Graph 4 demonstrates below, the vast majority of the instructors felt that the project had an impact on better preparing the students, on student centered learning and on their teaching and skills. Further, instructors who provided feedback to the final evaluation agreed that the project had not only improved, but also changed their method of teaching thereby allowing them to go academically beyond what they previously did as instructors. “This was the first project that came to specially aid the ENI instructors. The manual was a first for us…the trainings were a first for us. So this was a pioneering project.” ENI Instructor, Kandi 26 Graph 4: Impact of TMT project With all the benefits attributed to the project it should noted that the instructors did raise concerns about the advantages of the project being sustainable and its long-term impact on the teaching system in Benin. Nonetheless, it is a fact that the vast majority of the instructors (97%) were satisfied or very satisfied with the TMT project (see graph 5 below). Graph 5: ENI Instructors Satisfaction with the TMT Project It should also be pointed out that the final evaluation was not able to get a lot of feedback from ENI directors as three of the five ENI directors had been recently appointed and the evaluator was only able to talk to two directors (one of whom was new) along with one other senior ENI official. Nonetheless, the ENI Djougou director (who had been with the ENI since the start of the project) was full of praise for the project’s positive impact on the instructors and teachers. He also stated that main impact of TMT on his position was that he had learnt to use computers and email and this had fundamentally helped him to do his work for the ENI. 0 10 20 30 40 Has made an impact on my knowledge and teaching skills Helped me to incorporate learner-centered instructional methods into my teaching Made an impact on better preparing the pre-service teachers in the ENI Made an impact on the teaching system in Benin The benefits from the TMT project are going to be sustainable in Benin Very much Somewhat Not so much Not at all 0 0 1 25 9 0 5 10 15 20 25 30 Not Satisfied Less Satisfied Don’t know Satisfied Very Satisfied 27 5.2 ENI Students Over the course of the evaluation, ENI instructors were asked what impact the TMT project had on the ENI students. Their feedback demonstrates that the project provided various benefits to the students. These include,  All the knowledge/skills the instructors obtained from the project is being passed on to the students, thereby permitting them to develop an exit profile to be able to better teach in schools  The students are better prepared to master the subjects they will teach in schools and have better knowledge (and strategies) to communicate with the students. The OSEP criteria have made a big impact in this area  Learner centered instruction has been incorporated by the instructors, which has allowed the students to be directly involved in their own learning  Students are now receiving instruction that is better informed in relation to Benin’s competency￾based curriculum (with its emphasis on learner-centered instruction)  The RC has a big impact on the research and learning of the students as they are now exposed to new information and knowledge (discussed in section 5.3 below) First year ENI students in the FGDs also supported their instructors claims regarding student centered learning, use of methods/activities from instructors manual, small-group work, problem solving activities etc in their classes & also expressed their strong approval and enjoyment of these techniques. The project had an impact on the students as when the instructors’ teaching improved, the student’s leaning improved. The improved teaching methods that the instructors employed along with the use of the OSEP criteria demonstrated to the students how they could become better teachers. Therefore, the instructors passed on their new abilities to the students. Many instructors also concurred that the new pedagogies inculcated in the instructors had better prepared the students for teaching in schools, more so that the previous cohorts who attended the ENIs before the instructors were targeted by the project. Some schools directors also mentioned that the new ENI students who were coming to do internships in their schools in the past 2 years seemed to be more prepared than the ENI students who had done internships in their schools 4-6 years ago.26 As part of improving pre-service teacher training, the goal of the TMT project was to reinforce the capacity of the ENI instructors and thereby permit the ENI students to develop an exit profile to be able to better teach in schools. All the data gathered in the final evaluation demonstrates that the students graduating from the ENIs now potentially have a better ability to implement in practice what they have learnt from their instructors & from what they have gained from the research they have done in the RCs. 26 These opinions should, however, been seen as anecdotal as the final evaluation did not conduct an inquiry into the gradual improvement of ENI students performance as interns in schools since the TMT project was implemented. “The profile of the students coming out of ENIs has changed as before they didn’t have a high level of competence, but now they have something more” ENI Director, Djougou 28 5.3 ENI Resource Centers Each RC is located on the premises of the ENI itself (with the exception of ENI Djougou27) and is managed by a committee, which includes the ENI director, an instructors’ representative, a students’ representative as well as the ENI accountant. All the ENIs are operational, but infrastructure issues (mostly related to the regular availability of electricity) also affect each RC from time to time. Internet connectivity in Benin, as a whole, is not reliable and is often slow; thus, connectivity issues also, periodically, affect each RC. Despite the infrastructure issues, the data gathered in the final evaluation clearly demonstrates that the RCs in each ENI are very beneficial to the students and are being used for documentation and Internet research. The impact of the RCs includes,  Students have greater access to knowledge and information via Internet and documentation research  Students can learn from different types of resources that they did not have access to before  Instructors have more flexibility in assigning activities to students, for which they can obtain information from the RC  Students now have access to books that they cannot get anywhere else  Students are able to benefit from this greater access to resources to build a more profound knowledge about their subjects  Students are becoming used to using information and communication technologies (ICTs) for their learning and this is beneficial for their future learning and teaching  What students pay for the Internet and photocopying in the RC is much less than what they would pay outside28  Each RC follows guidelines provided by the project about (i) the roles and responsibilities of the RC including schedules, who can use it, what the RC has to do and (ii) regulations of each RC such as hours of operation, policy of computer use etc Questions posed in the survey to the ENI instructors (see graph 6 below) demonstrate that the majority of instructors are satisfied with the RCs and the benefits they bring to the students. However, the data pertaining to the statement ‘I make use of the Resource Center’ cannot be corroborated. As, information obtained from interviews with RC managers clearly demonstrated that ENI instructors, in fact, do not frequently visit or use the resources in the RCs. 27 The ENI in Djougou will move to a new building in October 2013 when the RC will be located on the ENI premises. 28 Students do not pay for the computer and Internet use in the RC as that is covered by the CFA 5,000 fee that they pay for the RC; however there is a photocopying charge. “The resource center permits the students to do research on a deeper level and build more profound knowledge” RC Manager, Porto Novo 29 Graph 6: ENI Instructors Use & Impact of Resource Centers 5.4 Challenges The TMT project has had a positive impact on the ENI instructors and students and the RCs are providing valuable resources, yet certain challenges exist. These include,  Mastering the theory and practice of knowledge-building pedagogies within the span of three week-long training sessions and with the support of a 200-page manual is challenging, and some instructors were of the view that certain concepts in the instructors manual and training were dense.  Training requires follow-up and reinforcing and many instructors expressed the need to build further skills and knowledge, through trainings, in relation to the Instructors Manual and OSEP.  Some of the activities listed in the manual require materials that the instructors and the ENIs do not possess; hence these activities cannot be realized with the students. The large size of the ENI classes also precludes the instructors from applying some strategies and activities that they were trained on.  ENI instructors were promised certifications after the completion of their trainings, but they still have not received them, despite the mid-term evaluation also recommending the project to do so. As one ENI instructor told the evaluator, “In Benin, if you have done a training and do not have a certificate then it means that you haven’t done the training.” EDC officials have now stated that all trained instructors will receive their certificates by July 2013.  A blog was created for ENI instructors so that they could pose questions to each other, share resources and build an online community. However, the instructors are not participating in this blog. The main reason for this non-participation is that the instructors state that they do not know how to use computers. However, instructors were provided opportunities to learn how to use computers by the project and the RC managers, but they chose not to attend these sessions. This issue of developing a network for instructors was also accentuated in the mid-term evaluation, but due to the reasons mentioned above the technical capacity of the instructors’ remains low and efforts to create an online community has not been successful. 0 5 10 15 20 25 I make use of theResource Center The pre-service teachers make use of the Resource Center The Resource Center has adequate teaching and learning materials and resources I am satisfied with the Resource Center Very much Somewhat Not so much Not at all 30  The mid-term evaluation recommended that an OSEP based subject specific lesson plan catalogue be developed for ENI instructors so that they could get more help in applying OSEP in the teaching of their specific subjects, but this catalogue was not developed. However, as stated in the methodology section recommendations based on mid-term evaluation were not compulsory deliverables for the project to implement & adding activities to the project would also have required changing aspects of the contract award.  First-year student teachers only encounter the OSEP criteria when they are observed and evaluated in schools (during internships) and the instructors do not specifically mention or discuss OSEP with them in classes. ENI instructors are well positioned to introduce OSEP to students, however, without a curriculum standard or other specification outlining such an introduction they do not do so (except in student evaluations).  The RC of the ENI in Dogbo has not had Internet connectivity for the past six months. This situation has nothing to do with the TMT project, but solely with the internal workings of the ENI where a new director still has not taken his place & the ENI accountant refuses to provide any finances to the RC until the new director takes up his responsibilities.  There are 600 students in each ENI, with 300 first year students taking classes and doing short internships and 300 second year students interning full time in schools. Thus, there are 300 first year students in each ENI in any given year that have six laptops available to them in each RC. The number of these laptops is clearly not adequate for the demand of the ENI students and it is normal for up to 4-5 students to wait to use each laptop. As a result, each RC has limited the time of laptop use for each student to 30 minutes.  RC managers feel that they need more training (and re-fresher trainings) to be better able to serve the ENI students. Another critical challenge in the ENIs was the slowness of MEMP in providing official approval for some of project activities (for example assigning RC officials). Further, the aim of the ENI directors was not always in synergy with the project, which led to further delays. In addition to these limitations, it should be pointed out that instructors who have not mastered new knowledge or practices in relation to the trainings, manual and OSEP are thus logically not able to impart any benefits to the students and thus are not contributing to improving pre-service teaching performance at the ENIs. 31 6 IMPROVING TEACHERS PERFORMANCE This chapter provides an analysis of the project activities in MAP and non-MAP schools and TMT’s impact on improving teachers’ performance. 6.1 MAP and Non-MAP Project Schools & Frequency of Trainings Out of the 3,112 school directors that were trained by the project29, 588 directors and their schools were chosen to take part in MAP over its two editions. See table 3 below, which provides an overview of the MAP award participants and winners. Table 3: MAP Participants and Winning Teachers & Schools Eligible Schools Number of Schools Which Participated Number of Eligible Teachers for Awards Number of Teachers Selected for Final Evaluations Award Winning Teachers Award Winning Schools MAP 1st Edition 196 145 573 291 120 24 MAP 2nd Edition 392 210 795 240 120 24 The final evaluation data reveals that the following steps were generally followed in schools that were eligible and chose to participate in MAP,  After the trainings, school directors informed all their teachers about MAP and the 16 OSEP criteria  School directors discussed the criterion with the teachers  School directors evaluated teachers on the 16 criteria and counseled them in the criteria in which they were not performing well  Teachers practiced to improve in the criteria  School directors conducted a round of ‘formal’ observations of their teachers using OSEP and send the evaluation forms to the TMT project  If a teacher was considered eligible for a MAP award, a trained CP visited the school and observed the teacher (using OSEP). Based on the CPs evaluations the teachers who were considered eligible for a final award were observed by TMT master trainer (using OSEP)  Throughout the second and third phase of the evaluation of a teacher (by the CP and TMT master trainer) the school director continued to provide counsel to the teacher on OSEP School directors coached, counseled and advised their teachers to incorporate and perform better in the 16 criteria to win MAP awards and thus contributed directly to improving their teaching performance. Hence, for a teacher in a MAP school to improve his/her performance, effective teaching/training of the teacher by the school director on OSEP was required and critical. Teachers in MAP schools were also motivated to learn, improve and to win prizes; so they were receptive to the school director when coached/counseled on OSEP. Further, CPs also observed and evaluated the teachers using OSEP and thus gave them feedback and advice on how to improve on the 16 criteria. Therefore in MAP schools 29 See Table 2 in Chapter 4. 32 there was a continuous reinforcement of learning (based on OSPE) by the school director and the CP, which led to the teachers improving their performances. MAP led to a more profound change in teaching practice stemming from the training in OSEP provided to the school directors. Directors guided teachers in the use of OSEP criteria for lesson planning in their classrooms. The school directors training along with the teachers’ motivation to succeed and win prizes served to develop the teachers and the teaching of their curriculum. Integration of OSEP into teacher evaluations by CPs further supported both teachers’ development and the improvement of classroom instruction. MAP increased the teachers’ motivation to improve their performance through the provision of awards to teachers and to schools and in doing so, MAP accomplished its primary objective. Nonetheless, a key issue to accentuate here is that the teachers’ performance improved because the school directors were pro￾active and made efforts to counsel and advice their teachers. School directors who did not have the volition and who did not spend time and effort on the teachers (with OSEP) were not able to act as conduits in improving their teachers’ performance in MAP schools. *** Of the 3,112 school directors that were trained by the project, 2,524 school directors (and their schools) did not participate in the pilot program, the MAP. The directors of these schools were expected to observe their teachers using OSEP and to counsel and advice the teachers on improving on each of the 16 criteria. The key for improving teaching performance in the non-MAP schools thus was, again, the school director who had to be pro-active and spend time and effort to observe, counsel and advice the teachers. However, certain key ingredients were missing in non-MAP project schools including,  In MAP schools directors observed teachers, counseled them on how to improve in the OSEP criteria and then returned after some time to observe if the teachers had made improvements in the criteria. Such a systemic approach to OSEP was lacking in non-MAP schools.  There was no incentive (prizes) or a motivation scheme for the teachers to incorporate the 16 criteria in non-MAP schools. This is not to deny that many teachers in non-MAP schools wanted to improve their teaching methods to further aid the learning of students, yet these teachers were not evaluated regularly by their school directors and there was no outside mechanism promoting their use and inculcation of the OSEP criteria.  All the CPs in Benin had been trained in OSEP, yet in non-MAP schools they were not required to observe and evaluate teachers using OSEP as part of any program, thus there was no further systemic reinforcement of OSEP on the teachers. Another very critical issue pertains to the how many round of trainings each school director received. Eight hundred and fifty (855)30 school directors received two rounds of trainings from the project and the second round of training (which lasted three days) renewed the concepts and information from the first 30 See Table 2 in chapter 4. “It was after the second round of trainings that I was able to get a better understanding of OSEP and what I had to do with the teachers. I then felt I could even train other school directors on OSEP” School Director, Djougou Center A 33 round of trainings and provided the directors with a better platform to use OSEP.31 The final evaluation data clearly demonstrates that those school directors who received two rounds of trainings were in a much better position to be able to observe, evaluate and counsel their teachers on OSEP and thereby help them in improving their performance. On the other hand one round of training on OSEP was not enough for school directors to master and use OSEP in their schools (unless they were in MAP schools). Consequently, school directors who either (i) did not participate in MAP or (ii) did not receive a second round of training were the least capable of providing OSEP based counsel and support to their teachers. Thus, it logically followed that teachers who were in MAP schools or those who were in schools where the school director had undergone two rounds of trainings were more proficient in the use of OSEP and thereby improved their performance; provided that the school director was pro-active, had the volition and spent time observing and counseling the teachers on OSEP. 6.2 TMT Impact in Schools In both MAP and non-MAP schools, the data gathered by the final evaluation shows the TMT project had a positive impact at the level of the directors, teachers and students; however this impact was predicated on the key contextual issues raised in the preceding section. School Directors The project’s impact at the level of school directors included,  All school directors are also teachers and by incorporating the OSEP criteria they saw their own teaching and the students’ learning improve  More tools and information were provided to them on how to observe and evaluate the teachers in their schools  They learnt how to better counsel and advice the teachers in classroom practices and teaching  They got more knowledge about pedagogy  They improved their capacity on how to manage and supervise the teachers and the school Applying the MSC method, the following phrases provide an insight on what aspects of the gains from the project the school directors considered as being most significant,  How to teach  Improving the knowledge of teaching  Pedagogy of teaching  Stimulating teachers  Concretize teaching  Reinforcement of capacity  Improving the competence of teacher  Facilitating the learning of students 31 The final evaluation cannot state how many of these 855 school directors were part of MAP schools, as this information was not provided by the project. “I feel I have a better ability to supervise and advice my teachers with OSEP as compared to the other directors in the school complex who were not trained by TMT.” School Director, Quartier B, Bassilla 34 Teachers The project’s impact at the level of teachers included,  They received beneficial advice/counsel on their teaching practices and strategy  They got opportunities to practice improvements in their teaching  They were better equipped to manage the planning and sequence of their classes, to ask questions and to enable both boys and girls to participate in classroom discussions  They were better able to teach the content of their subjects to students  They improved their knowledge of pedagogy  OSEP helped then think of practices beyond the classroom (for example to clean up their school toilets & to stop using their cell phones during class)  As the project made an impact on the teachers’ it directly helped the students, as employing the OSEP criteria helped the students to be able to better express themselves and to improve their learning Applying the MSC method, the following phrases provide an insight on what aspects of the gains from the project the teachers considered as being most significant,  Management of class  More equitable treatment of girl students  Use of open questions  Effective structuring of student groups  Providing more knowledge and learning to students  Improving quality of teachers CPs As has been previously stated, CPs in MAP schools were required to use OSEP to observe and evaluate teachers who were selected for MAP awards. Yet, even after MAP was discontinued as well as in non-MAP schools, some trained CPs continued to use some OSEP criteria, along with their own national teacher level checklist, to observe and evaluate the teachers as part of their regular duties. While the final evaluation cannot say with any certainty that this use of OSEP for evaluating teachers by CPs is currently constant or systematic in the TMT project schools, nearly all the school directors who were interviewed said that the CPs who come to their schools continue to use some OSEP criteria when evaluating the teachers. This practice demonstrates that OSEP, to some degree, has permeated into the CPs evaluation system for teachers. It should also be mentioned that OSEP does not replace the CPs national teacher level checklist as this checklist was used before OSEP and CPs continue to use it in schools. However, OSEP improves the evaluation of teachers by  Establishing standards of comparison as by providing five concrete indicators for each of the 16 criteria, OSEP establishes a matrix for observation and evaluation that can be applied regularly and objectively (especially by the trained and experienced CPs).  Providing concrete examples of classroom practice as the five indicators under each of the criteria are linked to narratives of teacher and student activities. These short descriptions suggest specific and practical ways of teaching and of facilitating learning. “At first I felt that TMT would be very demanding, but once I started applying the OSEP criteria I saw the benefits to the management of my class and to my students.” Teacher, Doutou B, Houeyogbe 35  Enabling teachers to assess their progress as each completed OSEP grille becomes part of a teacher’s record of observation, evaluation and achievement. In subsequent evaluations, teachers and evaluators can refer to these records, address specific areas of weakness and chart progress (this however requires active work by the school director). *** TMT in Dekpo A, Aplahouse Mr. Elis Yao Abeni is the school director of Dekpo A primary school and was one of the first directors to be trained by the TMT project in 2009. His school was chosen to participate in MAP and over both its editions, it had an award-winning teacher and school itself is a MAP award winning school. Mr. Abeni was very exuberant about the project and said that even though the TMT project is going to end; he has, nonetheless, systemized the use of OSEP in his school. At the start of the academic year, he goes over the OSEP criteria with all his teachers and he spends extra time with each new teacher to train them on OSEP. He feels that OSEP allows school directors to easily see what teachers are doing in class and to tell them what they are not doing well. He said that OSEP equips teacher with tools to manage the sequence of their classes and permits them to easily pass on learning to students, as students participate in class if the OSEP criteria are followed. He is also of the view that when students get to participate in their own learning they work collaboratively and cooperatively. He was also quick to add that he believes that with OSEP and the hard work he and his teachers put in, the students in his school have gotten the best exam results for the past couple of years compared to all the other primary schools in the area. However, he also pointed out that it is not possible to apply all the 16 OSEP criteria in one class and for some subjects (such as French) it is harder to apply the criteria than it is for other subject (such as Math and Science). When asked if the TMT project has achieved its goal, Mr. Abeni was reflective for a moment and then said that the goal of TMT was to improve teachers work in their class and this goal was attained in his school. He continued by cautioning that he could not say if this goal had been achieved in all TMT schools as he did not know how much work and effort the other school directors had put in. The goal of the TMT project was to improve teachers’ performance using the OSEP and to motivate teachers (in MAP schools). The final evaluation data demonstrates that the project achieved its goals in schools where the school director was pro-active and took time to teach, counsel and advice the teachers. In these cases, the key outcome of the TMT project was that school directors and teachers actively integrated OSEP into their activities. School directors used OSEP as a tool for observation and evaluation of teachers’ classroom practices and teachers used it to improve their teaching and employed the 16 criteria as a guide for lesson planning in their classrooms. However, in other project schools where the school director was only trained once or where the school director was not pro-active in imparting OSEP, the impact of the TMT project was not as significant. 36 6.3 Challenges The TMT project had a positive impact on teacher’s performance yet certain challenges exist. These include,  School directors need at least two rounds of trainings to master OSEP and to be able to use it successfully to observe, evaluate and counsel teachers to improve their performance. Directors who did not receive two rounds of training had difficulties to successfully impart OSEP in their schools, even if they were pro-active and willing to do so.  School directors teach classes themselves so it was not easy for them to leave class and observe other teachers in their schools with OSEP. Further school directors cannot evaluate teachers on all 16 OSEP criteria in one observation.  Applying all of the OSEP criteria in class is difficult and certain criteria can be applied much more easily than others (depending on the subject being taught)  Lack of materials and resources, class size and lack of proper buildings in schools affect the application of all the OSEP criteria in the classroom  After MAP was discontinued and in non-MAP schools, school directors have not regularly continued to use OSEP to evaluate their teachers. Pro-active directors discuss the OSEP criteria with their teachers, but a formal evaluation process using OSEP is not being generally applied.  Non-MAP schools did not have anyone associated with the TMT project come to their schools to see how the trained school directors were using OSEP, how they were advising and counselling their teachers. Thus, non-MAP school directors did not get any feedback about their use and application of OSEP (this was also the case with MAP schools who did not have any teachers or the school being considered for MAP winning awards).  Younger students have different instructional needs and they pose great challenges to teachers attempting to use active-learning pedagogies. Younger students do not speak French at home and do not come to school with French-language skills. To address this gap, teachers with younger students tend to fall back on whole-class activities based on repetition.  The mid-term evaluation recommended that the project should develop a subject specific lesson planning catalogue (based on the OSEP criteria) to help teachers accelerate and improve the practice of active-learning pedagogies in their classrooms, but this catalogue was not developed. However, as has been previously stated these recommendations were not requirements for the project and they would also have required certain changes in the contract.  Pro-active MAP school directors did all the work to train their teachers in the OSEP criteria yet the first edition of MAP did not provide any prizes for school directors. This led to many school directors resenting the set up of MAP. In the second edition of MAP, nearly 80 school directors were given some prizes, but by that time the majority of school directors felt that MAP was not adequately compensating the hard work of the directors and many lost interest in the program. As one school director who lost interest in the program said, “When the head is not motivated the base is not motivated.”  The engagement of MEMP officials in MAP was not as expected even though a committee with these officials was formed to oversee the program. However these officials did not regularly attend any meetings. Certain MEMP officials also wanted more from MAP on a personal level and when they did not receive anything they lost interest in the program. Another point to note is that parents of students and the communities around the TMT project schools were not very aware of MAP or of the benefits that OSEP was bringing to teachers and students (the OSEP tool itself does not involve parents and the community in the school). However, it should be mentioned that due to the cut in funding the media campaign for MAP and the project itself was never 37 launched, which would have had an impact on disseminating information about the project to parents and the community. 38 7 CONCLUDING DISCUSSION The TMT project achieved significant results in improving per-service teachers’ training and teachers’ performance in schools, yet the project faced challenges. This concluding chapter provides an overview of the achievements of the project, analyses the issues facing the primary education sector in Benin and ends by discussing the sustainability of the impact of the TMT project in the country. 7.1 Achievements of the TMT Project In addressing several objectives, including improving pre-service teacher’s training and teachers’ performance the TMT met a range of acutely felt needs in the primary education sector in Benin. The key fundamental intervention of the project was OSEP (and its 16 criteria), which was adopted as an observation and evaluation tool for teachers, as a framework for teacher development and as a tool for lesson planning in classrooms. The main contribution of the TMT project can be thus viewed as the introduction of OSEP, and integration of OSEP across Benin’s education system, including teacher supervision, evaluation and classroom instruction. Performance Indicators All the results/targets pertaining to the performance indicators were achieved by the TMT project including,  Four baseline and feasibility studies were conducted  All 117 permanent and temporary instructors at the 5 ENIs were trained  300 copies of the Instructors Manual were provided to the 5 ENIS and MEMP.  Two MEMP officials were trained to become the manager and assistant manager for each RC  All the 5 ENIs have functioning cost-recovery systems for their RCs  Students regularly make use of the RC  Instructors in all the 5 ENIs performed better at the end of the project compared to the observation results made during the baseline  Nearly half of all the primary school directors in the country 3112 (in the 1st round) and 855 (in the 2nd round) were trained in OSEP along with all the CPs in the country (233) and 70+ CCs. Improvements in Pre-Service Teachers Training The goal of the TMT project was to improve the quality and competence of ENI instructors and hence the quality of pre-service teachers’ training. This goal was achieved. ENI instructors were highly satisfied with the Instructors Manual and their trainings and the unanimous consensus was that the TMT project helped the instructors to improve their skills, quality and teaching abilities substantially. OSEP also aided them to improve their classroom practices and to observe and evaluate their students. As a consequence of these impacts, ENI students are now able to develop an exit profile to be able to better teach in schools and are benefitting from learner-centered instruction. Further, they are also benefitting from the RC, which is enabling them to be exposed to new information and knowledge. Certain challenges, however, do exist in the ENIs including,  The trained ENI instructors have not received their certificates and certain temporary instructors have not received the final copy of the manual.  ENI instructors need follow-up training 39  Some of the activities and strategies in the manual cannot be realized in the ENI classrooms  ENI instructors are not making use of the instructors blog as they are challenged by technology and hence are not able to benefit from any follow up mechanisms to support their continued development  The number of laptops for students use in the RCs are insufficient Improvements in Teachers’ Performance in Schools Teachers in MAP schools were motivated to perform better and to win prizes, but for teachers to improve their performance, effective teaching/training of the teacher by the school director on OSEP was required. Further, in MAP schools there was a continuous reinforcement of learning (based on OSEP) by the school director and the CP, which led to the teachers improving their performances. However, in non-MAP project schools, teachers were not evaluated regularly by their school directors so there were no supporting mechanisms promoting their use and inculcation of the OSEP criteria in the classrooms. Another very critical issue pertained to the how many round of trainings each school director received. A minimum of two rounds of training was required for school directors to become comfortable in using OSEP and counseling their teachers on its 16 criteria. Thus, it logically followed that teachers who were in MAP schools or in non-MAP schools where the school director had undergone two rounds of trainings were more proficient in the use of OSEP and thereby improved their performance; provided that the school director was pro-active, had the volition and spent time observing and counseling the teachers on OSEP. In these cases, the project had a strong impact including,  Allowing the school directors to improve their own teaching  Providing the school directors with a greater ability to observe, evaluate and counsel their teachers on improving their performance  School directors and teachers got more knowledge about pedagogy  Teachers got beneficial advice/counsel on their teaching practices and strategy  Teachers were better equipped to manage the planning and sequence of their classes and to better teach their subjects to students  As the project made an impact on the teachers’ performance it directly helped the students learning process Certain challenges, however, do exist in the TMT project schools including,  Directors who did not receive two rounds of training had difficulties to successfully impart OSEP in their schools, even if they were pro-active and willing to do so  Applying all of the OSEP criteria in class is difficult & lack of materials, resources and proper building along with large class sizes affect the application of the OSEP criteria in the classroom  Non-MAP school directors did not receive any feedback in their use of OSEP  The engagement of MEMP officials in MAP and non-MAP schools was not as expected “The difference between teachers who teach in MAP schools versus those who don’t can be observed in their work and behavior in class. Their class is better managed and more targeted to the students.” School Director, Legbanou, Dogbo (Transferred from a non-TMT school to a MAP school in 2012) 40  The cut in funding of the project precluded TMT’s media campaign, which would have greatly aided in the dissemination of its impact and results *** The TMT project was exemplary in many ways and the project’s implementation, despite facing many challenges, met its results/targets. Yet EDC officials also felt that more could have been done. The TMT project was limited do some degree by its scope and budget, but its biggest impact was based on OSEP, which was one constant and consistent factor across all its activities. However, the introduction of OSEP by itself was not enough and all those trained (in OSEP) needed to be trained again, but the project’s budget precluded this option. Further, it was also not possible to expand MAP to a greater number of schools due to the budgetary and logistical reasons, as CPs and TMT master trainers would not have been able to evaluate many more additional teachers for awards. Some other issues to consider include,  The cut in funding of the project led to the discontinuation of MAP, the removal of the project’s media campaign and work in the primary education policy area with MEMP. While EDC officials admitted that they needed more time to be able to do any work in the policy area, it would have been beneficial if the project had been able to contribute to update the primary school teachers’ recruitment/merit/transfer policies (which date from the 1960’s).  The work done by foreign professors in writing the ENI Instructors Manual and in conducting the trainings (with local master trainers) was tremendous but insufficient. As the professors came to recognize that the instructors needed more help in content (subject based) work besides just process help (to teach).  The TMT project had three CoPs in four years and despite these changes, the project achieved its deliverables. However, there is no doubt that every time there was a transition and a new CoP came to take charge, extra time was needed for the CoP to acclimatize with the project and the local context. 7.2 Issues Affecting the Primary Education Sector in Benin The TMT project had significant achievements over its four years during which there were changes in the MEMP and in the primary school education sector in Benin, nonetheless, the project successfully continued with its activities and attained all its deliverables. However, certain other contextual factors currently also exist that will play a determining role in the sustainability of the project’s achievements in the future. These include,  Some trained permanent ENI instructors are retiring and other trained temporary instructors have left the ENIs. A key issue thus is who will train the new ENI instructors. This situation is quite significant as demonstrated by the example of ENI Kandi where of the current 13 temporary instructors only one was trained by the project and the remaining 12 were not (as they only started working in the ENI after the project trainings).  Current fees provided by the ENI students are adequate for the functioning of the RCs, but there are no mechanisms in place whereby additional funds will become available (in the future) to pay for the renewal of journal/magazine subscriptions & for buying additional computers/books.  The internal administration and management of ENIs can the hinder the functioning of RCs (as is the case currently with ENI Dogbo) and there are no mechanisms in place that will ensure that the RCs will continue to function and benefit the ENI students.  The proliferation of private ENIs has had an impact on the quality of teachers as, unlike the public ENIs, no entrance is required to enter a private ENI. The quality of instruction in the 41 private ENIs is also inferior to the public ENIs. Further, the project has not targeted any private ENIs and hence has no impact on pre-service teaching in their domain.  There is a culture against change or innovation that exists in schools and ENI students who have been impacted by the project will face adverse teaching environments if they are hired to teach in schools, which have not been targeted by the project.  Personnel changes in schools have a range of negative impacts on the durability of learning and changed practice inspired by the project.  ENI instructors and school directors need re-fresher trainings so that they do not loose what they have learnt.  Education for All (EFA) had led to an increase in the number of children who attend school, but the quality of teachers to teach the students is still far from adequate. Compounding this situation is the fact that the education system in Benin is very politicized and teachers strikes are common (leading to great disturbances in teachers performances and students learning)  Critically, MEMP has not hired any new teachers for primary schools in the past two years and it is not clear if any new teachers will be hired in the next academic year. This has a huge impact on the ENI students who have graduated from the ENIs after the project was implemented.  A cadre of MEMP officials who were committed to amplify the benefits of the project was transferred & other senior MEMP officials show a lack of interest/knowledge of the TMT project. 7.3 Sustainability of the Achievements of the TMT Project The TMT project spent time and effort in trying to build relations with MEMP to ensure the project’s sustainability. Yet the challenges the project faced, the disinterest of some MEMP officials along with the removal of the project’s media campaign had a huge impact on its sustainability. Nonetheless, data gathered for the final evaluation demonstrates that the achievements of the project are sustainable, and this sustainability can be demarcated into two overlapping and synergetic domains. Agents At the level of individuals, the following factors bode well for the sustainability of the TMT project.  All the CPs in the country along with a larger majority of CCs have been trained and these officials have acted as master trainers/change agents to further spread OSEP in schools. These officials do not need the TMT project anymore to train other people in OSEP. In other words, their capacity has been built and will not simply dissipate.  What the trained ENI instructors have gained from the trainings and the manual they will keep on applying and transmitting to the students, thereby consistently improving the ENI students’ exit profiles to teach in primary schools.  Pro-active school directors will continue to use the new skills they have gained to supervise their teachers, even if no outside mechanisms exist motivating them to do so.  Pro-active teachers will keep what they have learnt forever and continue to use this new knowledge (based on the OSEP criteria) to better teach students in the future, even if no outside mechanisms exist motivating them to do so. “OSEP is a good gift that will stay with the director and teachers forever. You can evaluate and teach with it every year.” School Director, Kandi B 42  Certain pro-active MEMP officials do exist who support the project and are convinced of the benefits of OSEP.  Any other project that comes in the future that builds on learner centered instruction, motivation, teacher observation and evaluation will benefit from having many MEMP officials, ENI instructors, schools directors and teachers in place who will know how to adapt to it. Systemic Level The TMT project at the mirco level affected the ENI instructors and students; the school directors, teachers and students and; CPs, CCs and other MEMP officials. At the macro level, innovations were brought into the primary education system primarily through OSEP; and if nothing else remains from the project, OSEP is sufficient as it enabled the contextualization and the spread and improvement of pedagogical activities in schools. Thus at the level of the primary education system, the following factors bode well for the sustainability of the TMT project.  OSEP has been ingrained in the primary education system via the trained CCs, CPs, school directors and the ENIs  In MEMP the ‘Harmonization of Supervisory Instruments and Pedagogic Control’ based on the OSEP criteria is underway  The Benin 2013-15 Education Sector Plan calls for improving quality of education and better evaluation of teachers. The departments carrying out these tasks at MEMP are incorporating OSEP in their work In MEMP there is currently a move for ‘Harmonization of Supervisory Instruments and Pedagogic Control’ (Harmonization d’Instruments d’Encadrement et de Control Pedagogic) under which efforts are underway to integrate the OSEP criteria in a new system that CPs will use when they supervise and evaluate teachers in schools. There are six stages in this effort,  Inventory of all current instruments CPs use in the field  Determination of fundamental instruments  Elaboration of standard instruments (based on OSEP)  Validation of instruments elaborated  Training of CPs in these instruments  Distribution and utilization of instruments (which include OSEP criteria etc) A 16-person team is overseeing this effort, which has reached the fourth stage and they are now waiting to train the CPs in the new instruments (with funding provided by the German government). However, the team is also currently waiting to get full approval from the Minister and other high officials at MEMP to continue with this task. It is expected, however, that once this approval is granted the harmonization will be finalized and all CPs will start to use the new instrument. The Benin 2013-15 Education Sector Plan, on the other hand, calls for the improvement of the quality of teaching and better evaluation of teachers. As part of this plan, studies for the actualization of a primary curriculum is planned. These studies will determine what primary schools in Benin should be like, what is missing in the schools, what new themes should be included, how to evaluate students, what students should learn etc. The department of Direction de l’Inspection Pédagogique at MEMP is partly “Some key stakeholders in MEMP are convinced that OSEP will become part of the education system as it is already part of it, and those using it know its benefits.” Member of Cabinet, MEMP 43 responsible for these studies, and officials in this department plan to include the learning from OSEP and the benefits it brings to teaching and learning (along with the problems teachers and students have in teaching which OSEP addresses) to inform the studies. All the results and insights from OSEP will thus have an influence on improving the quality of primary education through the studies that are called for in the Education Sector Plan. In this regard, the reach of the TMT project can be further extended if MEMP were to undertake comparative studies that demonstrate the difference between TMT and non-TMT schools. The results of such studies will not only highlight the benefits of the incorporation of OSEP, but will also provide the Government of Benin concrete data and systemic recommendations for high-level global engagement, leading to potentially more international funding for future teacher training projects. *** The future of teacher related programming in Benin is uncertain. While the government has plans to re￾examine the primary education sector, there are no teacher related programs waiting to be implemented. Hence, a sense of instability permeates the system as teachers and school directors do not know what will happen next. What is certain, though, is that there is a huge need and demand for more rigorous trainings to improve the quality of teachers. Reflecting on the TMT project in this scenario, the final evaluation has clearly demonstrated that with its limited budget and funding cuts, the project clearly addressed certain critical needs of the primary education system in Benin. While the project faced limitations, it was able to make a positive impact on improving pre-service teachers’ trainings and teachers’ performance in schools. The factors affecting the primary education sector in Benin are complex and the final evaluation acknowledges that the project faces challenges if its achievements are to be sustainable. However, supporting factors at both the individual and systemic level exist that bode well for the durability of the TMT project’s impact. Summing up, while it is clear that the TMT project made a contribution, it is also obvious that more work needs to be done to improve the training and performance of primary school teachers in Benin. 44 ANNEX A: OSEP Tool RATING The Teacher: 1 2 3 4 5 1. Manages instructional time effectively (Gère efficacement le temps d’enseignement) 2. Demonstrates effective classroom management skills (Fait preuve d’habiletés et d’efficacité dans la gestion de la classe) 3. Makes effective use of different instructional resources and strategies to explain and model subject matter concepts and skills (Utilise efficacement différentes ressources pédagogiques et stratégies d’enseignement pour construire des connaissances) 4. Engages students in carefully structured cooperative learning experiences (Implique les élèves dans des activités d’apprentissage coopératif soigneusement organisées) 5. Implements instruction that targets the development of students’ social and collaborative skills (Met en œuvre un enseignement qui se fixe comme objectif le développement par les élèves d’habiletés sociales et collaboratives) 6. Actively ensures the participation of all students in learning activities irrespective of their sex, achievement level, special needs, giftedness and other differences (Fait participer activement tous les élèves aux activités d’apprentissage quels que soient le sexe, le niveau de réussite, les acquis antérieurs, les aptitudes, les besoins spéciaux et autres différences) 7. Uses diverse instructional strategies to promote active student participation in learning (Utilise des stratégies d’enseignement variées pour promouvoir une participation active de l’élève à son apprentissage) 8. Effectively asks probing and open-ended questions that encourage thinking, and help students explicate their thinking (Pose des questions ouvertes qui incitent les élèves à la réflexion et facilitent l’explicitation de leur pensée) 9. Encourages students to have a voice in the learning environment (Encourage les élèves à s’exprimer librement quand ils se retrouvent dans l’environnement pédagogique) 10. Provides students with opportunities to build meaningful connections between different subject matter areas, and between these areas and everyday life experiences (Offre aux élèves des occasions d’établir des liens significatifs entre les résultats de leurs différents apprentissages et la vie courante) 11. Provides students with structured opportunities to apply their understandings and skills to everyday life situations and problems (Présente aux élèves des situations et des problèmes de la vie courante leur permettant de réinvestir les résultats de leurs apprentissages) 12. Provides students with structured opportunities to reflect on their own learning (Donne aux élèves des occasions de réfléchir sur leurs démarches et sur les résultats de leurs apprentissages) 13. Provides and helps students suggest ample, specific, and constructive feedback (Engage les élèves dans un processus constructif de rétroaction) 14. Uses student prior knowledge and experiences to plan and adjust instruction (Se réfère aux connaissances et aux expériences antérieures de ses élèves pour planifier et pour ajuster au besoin les activités d’enseignement et 45 d’apprentissage) 15. Provides students with opportunities to practice higher order and critical thinking skills (Offre aux élèves des occasions de faire des analyses, de faire des synthèses, de construire des preuves et de faire des déductions) 16. Provides students with opportunities to develop problem solving skills (Donne aux élèves des occasions de résoudre des problèmes) 46 ANNEX B: Documents Reviewed Sources for information about education in Benin include:  UNICEF Benin Statistics http://www.unicef.org/infobycountry/benin_statistics.html#90  UNESCO Education Statistics http://www.uis.unesco.org/Education/Pages/default.aspx  Nation Master Education Statistics Benin http://www.nationmaster.com/red/country/bn-benin/edu-education&all=1  Benin Education Documents (Government plans/policies in French) http://www.globalpartnership.org/document-repository/  World Bank Education Statistics http://web.worldbank.org/WBSITE/EXTERNAL/COUNTRIES/AFRICAEXT/EXTAFRREG TOPEDUCATION/0,,menuPK:444714~pagePK:34004175~piPK:34004435~theSitePK:4447 08,00.html  World Bank Benin http://web.worldbank.org/WBSITE/EXTERNAL/COUNTRIES/AFRICAEXT/BENINEXTN/ 0,,menuPK:322645~pagePK:141159~piPK:141110~theSitePK:322639,00.html  Yessoufou, A. 2012. Local actors in top-down implementation of curricular reform in Benin’s primary education system Lambert Academic Publishing.  Overseas Development Institute Engel, J., Cossou, E. and Rose, P. Benin’s Story: Benin’s Progress in Education: Expanding Access and Narrowing the Gender Gap. http://www.developmentprogress.org/sites/default/files/benin_education_progress.pdf), accessed on 5 May, 2013.  Benin Education Sector Plan, 2013-15, Phase III, Ministry of Education. EDC documents Background documents provided by EDC include, but are not limited to:  Benin Teacher Motivation and Training Project Annual and Quarterly reports (2010, 2011, 2012)  Benin Teacher Motivation and Training Project: Performance Management Plan (EDC) – 2009 (TMT PMP)  Guide to ENI Trainers Manual (pdf guide of 20 pages in French)  Les echoes TMT (USAID newsletters) 2012, 2013  OSEP/SCOPE Workshop Training (power point presentation in English)  USAID/EDC. 2010. Outil de Suivi et d’Évaluation Pédagogique (OSEP).  Présentation des résultats de l’Etude de base sur la performance des formateurs des ENI au Bénin (EDC presentation of baseline information for ENIs)  Présentation des résultats de l’Etude de base sur la motivation la performance et la supervision des enseignants du primaire dans 100 écoles pilotes au Bénin (EDC presentation of baseline information on teacher motivation)  Rapport de l’étude de base sur la motivation, la performance et la supervision chez les enseignants au Bénin (EDC report)  Rapport de l’étude de base sur la performance des formateurs d’ENI au BENIN (EDC report)  SCOPE Short Description (pdf file in English) 47  Standards-based Classroom Observation Protocol for Educators (SCOPE) (Observation form including 16 criteria in grid for scoring) 48 ANNEX C: List of ENIs and Schools Visited Schools Location School Type of School Meetings 1 Kandi Kandi A MAP winning school  Interview with Director (trained once) 2 Kandi Kandi B MAP winning school  Interview with Director (trained twice)  FGD with 4 MAP award winning teachers (2 from Kandi A and 2 from Kandi B) 3 Kandi Fafa Non-MAP school  Interview with Director (trained twice); she had been transferred to another school, but she came to Kandi B for an interview  New Director of school not trained 4 Kandi Kassakou B Non-MAP school  Interview with Director (trained twice) 5 N’dali Treboun C Non-MAP school  Interview with Director (trained once) 6 Djougou Djougou Center A MAP participating school  Interview with Director (trained twice)  Interview with 2 teachers 7 Basilla Quartier B MAP winning school  Interview with Director (trained twice)  FGD with 5 teachers (including 1 MAP award winning teacher) 8 Djougou Gosso MAP winning school  Interview with Director (trained twice)  Interview with 1 MAP award winning teacher 9 Bohicon Vehou A MAP participating school  Interview with Director (trained once) 10 Bohicon Vehou C MAP participating school  Interview with Director (trained once) 11 Abomey Sogbo Aliho B MAP participating school  Interview with Director (trained twice)  Interview with 1 teacher 12 Abomey Ahouaga B MAP participating school  Interview with Director (trained once)  Interview with 1 teacher 13 Abomey Djime A Non-MAP school  Interview with Director (trained once)  Interview with 1 teacher 14 Aplahoue Dekpo A MAP winning school  Interview with Director (trained once) 15 Houeyogbe Doutou B MAP winning school  Interview with Director (trained once)  Interview with 1 MAP award winning teacher 16 Dogbo Dogbo Tota Non-MAP school  Interview with Director (trained twice) 17 Lokossa Lokossa A MAP participating  Interview with Director (trained twice)  Interview with 1 teacher 49 school 18 Porto Novo Kandevie A MAP participating school  Interview with Director (trained once)  Interview with 1 MAP award winning teacher 19 Porto Novo Kandevie B Non-MAP school  Interview with Director (trained once) 20 Porto Novo Kandevie C Non-MAP school  Interview with Director (trained once) 21 Porto Novo Hinkoude B MAP winning school  Interview with Director (trained once)  Interview with 1 MAP award winning teacher ENIs ENI Director Instructors Student Teachers Resource Center Kandi Interview 1 FGD with 5 permanent instructors No 1st or 2nd year students were present at the ENI. Interview with resource center manager and his assistant Djougou Interview 1 FGD with 3 permanent and 3 temporary instructors No 1st or 2nd year students were present at the ENI. Interview with resource center manager (no assistant) Abomey Director was not present. Interview was conducted with Chief of Educational Services & Documentation (2nd ranking official in ENI) 1 FGD with 4 permanent and 2 temporary instructors FGD with group of 8 1 st year students Interview with resource center manager and his assistant Dogbo Director was not present 1 FGD with 4 permanent instructors FGD with group of 8 1 st year students Interview with resource center manager and her assistant Porto Novo Director was not present 1 FGD with 9 permanent and 5 temporary instructors FGD with group of 8 1 st year students Interview with resource center manager (no assistant) 50 ANNEX D: Final Evaluation Indicators TMT Performance Indicators The TMT project activities have two linked programs or Results. Result 1: ‘Improvement of pre-service teacher training,’ has three requirements. Result 2: ‘Improved Teacher Performance,’ has eight requirements as shown in Table A below Table A: TMT Requirements for Results 1 & 2 Number Requirements Result 1 Improved pre-service teacher training 1.1 Develop and publish the TTC Instructor manual, supporting the competency￾based National curriculum and on modern principles of teaching teachers. 1.2 Provide intensive technical training sessions to all TTC instructors, using experienced foreign professors specialized in teaching teachers, competency￾based curricula and child-centered teaching methods. 1.3 Provide technical and material assistance to set up library/resource centers in all operational TTCs. Result 2 Improved teacher performance 2.1 Feasibility and baseline study of teacher performance, motivation and supervision to determine factors such as rates of teacher absenteeism, mechanisms for supervision, levels of teaching skills, etc 2.2 Develop a system of improved teacher supervision and in-service technical support teacher performance at central and local levels 2.3 Develop a system for improved teacher performance evaluation at central and local levels 2.4 Pilot a merit-based awards program for the best schools and their teachers. 2.5 Execute annual nationwide media campaigns to reinforce the importance of education and teachers and ensure that all primary school teachers know about the merit 2.6 Recruit sponsors for the reward program from other education donors, the private sector and the GoB to donate additional prizes for outstanding schools and teachers; create broad-based support for the program. 2.7 Collaborate with the MEMP to draft and ratify national policies governing teacher motivation, supervision and evaluation, with guidelines and procedures 2.8 Develop and implement a plan to transfer the merit awards initiative to the MEMP by end of project. Based on these requirements for the two Results, performance indicators were enumerated in the PMP and consist of standard USAID indicators and custom indicators (applying specifically to TMT). These performance indicators represent milestones to be achieved and specific activities to be undertaken and are shown in Table B below. These indicators were assessed based on the data and documentation EDC Benin has compiled over the duration of the project & the data gathered from the field visits. 51 Table B: TMT Performance Indicators Number Indicator Result 1 Improved pre-service teacher training Required Standard Indicators 1.1 Number of teachers trained with USG support 1.2 Number of educators/instructors trained with USG support 1.3 Number of textbooks and other teaching and learning materials provided with USG assistance Custom Indicators 1.4a Number of students and teachers using the ENI library/resource centers 1.4b Number of ENI library/resource personnel trained in library and center management 1.4c Number of ENI library/resource centers with functioning cost-recovery systems 1.4d Percentage/number of ENI instructors with improved teaching skills Result 2 Improved teacher performance Required Standard Indicators 2.1 Number of baseline or feasibility studies 2.2 Number of administrators and officials trained 2.3 Support for education system policy reform 2.4 Number of laws, policies, regulations or guidelines developed or modified to improve equitable access to or the quality of education services 2.5 Number of people trained in monitoring and evaluation Custom Indicators 2.6a Monetary amount (or equivalent) provided by other award sponsors 2.6b Number of schools and teachers submitted to the selection committee 2.6c Number and type of award sponsors 2.6d Percentage and number of teachers showing improved attendance 2.6e Percentage/number of teachers showing improved classroom performance Please note: Within the last year (2012-2013) the funding for TMT was curtailed by USAID leading to certain changes in the project and to the removal of requirements 2.5 to 2.8 (see Table A). As a result of these developments the following performance indicators (see Table C below) also have been removed, Table C: TMT Performance Indicators Removed 2.3 Support for education system policy reform 2.4 Number of laws, policies, regulations or guidelines developed or modified to improve equitable access to or the quality of education services 2.5 Number of people trained in monitoring and evaluation 2.6a Monetary amount (or equivalent) provided by other award sponsors 2.6b Number of schools and teachers submitted to the selection committee 2.6c Number and type of award sponsors 2.6d Percentage and number of teachers showing improved attendance 2.6e Percentage/number of teachers showing improved classroom performance 52 The indicators removed pertain to:  Drafting and ratifying polices governing teacher motivation, supervision, and evaluation  Training of partners in M&E  Piloting the MAP, its media campaigns and recruitment of sponsors  Activities related to the transferring of MAP to the GoB and MEMP  Teachers showing improved attendance and improved classroom performance as a result of MAP Hence, the final evaluation did not assess performance indicators 2.3 to 2.6e that were removed. However, the final evaluation did qualitatively examine the MAP component of the TMT project over its two editions and provide an assessment of its impact. Mid-Term Evaluation Recommendation Indicators The mid-term evaluation of the TMT project carried out in April-May 2012 contained certain recommendations. These recommendations were essentially made to aid the implementation of the project activities. However, these recommendations were not compulsory requirements that the project had to implement, as they were not included in the contract with USAID. In the final evaluation of the program, the following indicators based on the recommendations of the mid-term evaluation were assessed to determine if (and how) they had been incorporated. Table D: TMT Mid-Term Evaluation Recommendation Indicators Number Indicator Objective Improving the Quality of Pre-Service Teacher Training 3.1 Communities of practice for ENI instructors are formed 3.2 Network for ENI instructors is developed and newsletter, email lists, blogs etc are initiated to support it 3.3 OSEP based subject specific lesson plan catalog is developed for ENI instructors 3.4 All ENI instructors (permanent & temporary) receive final editions of the instructors manual 3.5 All ENI instructors (permanent & temporary) receive certificates of training completion Objective Improving Teacher Performance 4.1 School directors received some form of post-training support 4.2 Catalog of subject specific lesson plans developed for teachers Objective Sustainability of TMT 5.1 Outreach to MEMP renewed & advocacy campaign for TMT project initiated 5.2 OSEP related data and analysis of teachers shared with MEMP Please note: the mid-term evaluation contained recommendations for MAP, however as this component of TMT has been discontinued the final evaluation did not asses any MAP related recommendations. Impact indicators Impact indicators were developed expressly for the final evaluation; these indicators are designed to address the final evaluation’s goals and objectives (as demarcated in the SoW). These indicators include, 53 Table E: TMT Project Impact Indicators Number Indicator Objective Improving the Quality of Pre-Service Teacher Training 6.1 ENI instructors express satisfaction with the training they received and the Instructors Manual 6.2 ENI instructors express change in their quality and teaching skills as a result of TMT project 6.3 ENI instructors express the TMT project helped them better teach and prepare pre-service teachers 6.4 ENI instructors express the TMT project made an impact on pre-service teachers 6.5 Pre-service teachers are taught topics covered in the ENI instructors manual 6.6 ENI directors express the TMT project made an impact on ENI instructors and pre-service teachers 6.7 ENI directors, instructors and pre-service teachers express usefulness of ENI resource centers Objective Improving Teacher Performance 7.1 CPs, school directors and teachers express change in their quality and skills as a result of the TMT project 7.2 CPs, school directors and teachers express impact of TMT project on teachers teaching skills 7.3 CPs, school directors and teachers express impact of TMT project on teachers supervision 7.4 CPs, school directors and teachers express impact of TMT project on students 7.5 CPs, school directors and teachers express impact of OSEP 7.6 CPs, school directors and teachers express impact of MAP Objective Goals and Objectives of TMT 8.1 The project beneficiaries and stakeholders express satisfaction with the TMT project achieving its goals and objectives Objective Satisfaction with TMT 9.1 The project beneficiaries and stakeholders express satisfaction with the TMT project Objective Sustainability of TMT 10.1 The project beneficiaries and stakeholders express satisfaction with the sustainability of the gains and benefits of the TMT project Objective Recommendations for Future Teacher Related Programming in Benin 11.1 The project beneficiaries and stakeholders express ideas/opinions on the future teacher related programming in Benin Please note: The indicators presented here do not include measurable or time-referenced outcomes (e.g., “ENI instructors have addressed OSEP criteria in their teaching 3 or more times in the past six months”). 54 ANNEX E: Evaluation Instruments ENI Instructors FGD Questions 1 Are you satisfied with the trainings? What benefits did they bring? What were the challenges with them? Please give examples. 2 Are you satisfied with the Instructors Manual? How well does the Instructors Manual address your needs and interests as an instructor? What aspects of the manual are the most valuable? What aspects of the manual are the most challenging? In what ways can the manual be improved? Please give examples. 3 What impact has the TMT project had on your quality and (teaching) skills? What specific changes did the project bring to your teaching? Please give examples. 4 Did the project help you better teach and prepare the pre-service teachers? 5 Were pre-service teachers taught topics/modules covered in the Instructors Manual? What topics/modules were the most useful and which were the most challenging? 6 What impact did the project have on student-centered based instruction (apprentisage basee sur l‘apprenant)? 7 What impact has the project had on preparing pre-service teachers to teach in schools? How are these pre-service teachers different from the other pre-service teachers who were not impacted by the TMT project? 8 When the pre-service teachers are posted to schools, what are the biggest challenges they will face? How has the project prepared them to face these challenges? 9 What is the condition of the library/resource centers at the ENI? What is its use and value? What are the challenges with it? How will this center be sustainable in the future? Who will pay for it? 10 Have you used OSEP? What are the benefits and challenges with it? 11 Were any communities of practice and/or networks of instructors (using emails, blogs etc) developed at the ENI? 12 Was any OSEP based subject specific lesson plan catalog developed for the ENI instructors? 13 Did all ENI instructors (permanent & vacataire) receive final editions of the Instructors Manual and certification of training completion? 14 Where the project’s goals and objectives clear to you? Did the project meet its goals and objectives? 15 What were the main strengths and weaknesses of the TMT project? 16 What is the most significant change that the project has brought to your ENI and the Beninoise education system? 17 How satisfied are you with the TMT project? List areas where you are satisfied and others where you are not satisfied with the program? 18 Are the benefits and gains from the TMT project sustainable? What are the signs that they are sustainable? What are the challenges to this sustainability? 19 What impact did the project have on education policies in Benin? Did any of the project activities support any education policy reform in Benin? 20 What is the future of teacher related programming in the Beninoise education system? Where has progress been made? What role has TMT played? What are the challenges? 55 TMT Project Survey for ENI Instructors, Benin 2013 Date: ENI: Dear participant, Please take a few minutes to complete this survey about your experience with the TMT project. Thank you. 1 Are you? (Check one box) 2 Are you? (Check one box) 3 How many years have you been teaching at the ENI? ________ 4 Answer the following questions related to your trainings for the TMT project. (Check one box for each item.) 5 Answer the following questions related to the Instructors Manual. (Check one box for each item.) 1 Permanent 2 Vacataire 1 Male 2 Female Not at all Very Less Some￾what Very Much a The trainings helped me to improve my skills and knowledge 1 2 3 4 b The trainings helped me to incorporate learner-centered instructional methods into my teaching 1 2 3 4 c The training helped me to better teach and prepare the students 1 2 3 4 d I am satisfied with the trainings 1 2 3 4 Not at all Very Less Some￾what Very Much a The Instructors Manual has helped me to improve my skills and knowledge 1 2 3 4 b The Instructors Manual has helped me to incorporate learner-centered instructional methods into my teaching 1 2 3 4 c The Instructors Manual has helped me to better teach and prepare the students 1 2 3 4 d I refer to the Instructors Manual 1 2 3 4 e I make use of the OSEP in my teaching 1 2 3 4 f I am satisfied with the Instructors Manual 1 2 3 4 56 6 Answer the following questions related to the Library/Resource Center at the ENI. (Check one box for each item.) 7 Answer the following questions related to the TMT project. (Check one box for each item.) 8 How satisfied are you with the TMT project? (Check one box) 1 Not Satisfied 2 Less Satisfied 3 Don’t know 4 Satisfied 5 Very Satisfied Not at all Very Less Some￾what Very Much a I make use of the Library/Resource Center 1 2 3 4 b The élève maîtres make use of the Library/Resource Center 1 2 3 4 c The Library/Resource Center has adequate teaching and learning materials and resources 1 2 3 4 d I am satisfied with the Library/Resource Center 1 2 3 4 Not at all Very Less Some￾what Very Much a The TMT project has made an impact on my knowledge and teaching skills 1 2 3 4 b The TMP project has helped me to incorporate learner￾centered instructional methods into my teaching 1 2 3 4 c The TMT project has made an impact on better preparing the students in the ENI 1 2 3 4 d The TMT project has made an impact on the teaching system in Benin 1 2 3 4 e The benefits from the TMT project are going to be sustainable in Benin 1 2 3 4 57 ENI Director Interview Questions 1 Are you satisfied with the trainings provided and the Instructors Manual for the ENI instructors? What benefits did they bring? What were the challenges with them? Please give examples. 2 What impact has the TMT project had on your quality and (teaching) skills of the ENI instructors? What specific changes did the project bring to their teaching? Please give examples. 3 Did the project help the ENI instructors better teach and prepare the pre-service teachers? 4 How did the project impact student-centered instruction (apprentisage basee sur l‘apprenant) in the ENI? 5 When the pre-service teachers are posted to schools, what are the biggest challenges they will face? What challenges will the pre-service teachers face in terms of using approaches based on student-centered learning? 6 What impact has the project had on preparing pre-service teachers to teach in schools? How are these pre-service teachers different from the other pre-service teachers who were not impacted by the TMT project? 7 What is the condition of the library/resource centers at the ENI? What is its use and value? What are the challenges with it? How will this center be sustainable in the future? Who will pay for it? 8 Where the project’s goals and objectives clear to you? Did the project meet its goals and objectives? 9 What were the main strengths and weaknesses of the TMT project? 10 What is the most significant change that the project has brought to your ENI and the Beninoise education system? 11 How satisfied are you with the TMT project? List areas where you are satisfied and others where you are not satisfied with the program? 12 Are the benefits and gains from the TMT project sustainable? What are the signs that they are sustainable? What are the challenges to this sustainability? 13 What impact did the project have on education policies in Benin? Did any of the project activities support any education policy reform in Benin? 14 What is the future of teacher related programming in the Beninoise education system? Where has progress been made? What role has TMT played? What are the challenges? 58 V School Director Interview Questions 1 Are you satisfied with the project trainings? What benefits did they bring? What were the challenges with them? Please give examples. 2 What impact has the project had on your quality and skills? What specific changes did the project bring to your? Please give examples. 3 What impact did the project have on the teachers teaching skills? 4 What impact has the project had on teachers’ supervision? What specific changes did the project bring for teachers’ supervision? Please give examples. 5 What impact did the project have on students in the school? 6 OSEP is the scheme of 16 criteria for observing and assessing the performance of teachers in the classroom. Have you discussed these criteria with the teachers? 7 What impact did OSEP have? What were the benefits and challenges of using OSEP? Will OSEP continue to be used in the school? 8 What are the main difficulties that you face in using the OSEP criteria to observe and evaluate teachers? 9 How did the project impact student-centered instruction (apprentisage basee sur l‘apprenant) in the ENI/school? 10 What impact did MAP have on your school (even if your school did not win)? What impact did MAP have on the teachers and students? What changes have you observed among teachers at your school as a result of your school’s participation in the MAP program? Did MAP have an impact on teacher absenteeism? What were the challenges with MAP? 11 How knowledgeable are the community and the student’s parents about the TMT project? 12 Was any catalog of subject specific lesson plans developed for teachers? 13 Did you receive any form of post-training support? Did you attend the supervisory institute 3 day trainings? What did you learn from that? What were the challenges with it? 14 What are the biggest challenges schools face in Benin? What skills/education do teachers need to face these challenges? Has the project helped you and your teachers to face these challenges? 15 Was any OSEP related data and analysis of teachers shared with MEMP? 16 Where the project’s goals and objectives clear to you? Did the project meet its goals and objectives? 17 What were the main strengths and weaknesses of the TMT project? 18 What is the most significant change that the project has brought to your school/the beninoise education system? 19 How satisfied are you with the TMT project? List areas where you are satisfied and others where you are not satisfied with the program? Provide some specific examples of the impact of the TMT program in your ENI/school? 20 Are the benefits and gains from the TMT project sustainable? What are the signs that they are sustainable? What are the challenges to this sustainability? 21 What is the future of teacher related programming in the Beninoise education system? Where has progress been made? What role has TMT played? What are the challenges? 59 VI School Teachers FGD Questions 1 What impact has the project had on your quality and (teaching) skills? What specific changes did the project bring to your teaching? 2 Did the project have an impact on your ideas about teaching and how you teach? What changes have you made in your teaching as a result of feedback from your school director and CP? What role did the 16 criteria for OSEP in influencing your teaching? 3 What impact has the project had on teachers’ supervision? What specific changes did the project bring for teachers’ supervision? 4 What impact did the project have on students in the school? 5 What impact did OSEP have? What were the benefits and challenges of using OSEP? Will OSEP continue to be used in the school? 6 How did the project impact student-centered instruction (apprentisage basee sur l‘apprenant) in the school? 7 What impact did MAP have on your school (even if your school did not win)? What impact did MAP have on the teachers and students? What changes have you observed among teachers at your school as a result of your school’s participation in the MAP program? Did MAP have an impact on teacher absenteeism? What were the challenges and limitations with MAP? 8 How knowledgeable are the community and the student’s parents about the TMT project? 9 Was any catalog of subject specific lesson plans developed for teachers? 10 Where the project’s goals and objectives clear to you? Did the project meet its goals and objectives? 11 What were the main strengths and weaknesses of the TMT project? 12 What is the most significant change that the project has brought to your school/the beninoise education system? 13 How satisfied are you with the TMT project? List areas where you are satisfied and others where you are not satisfied with the program? Provide some specific examples of the impact of the TMT program in your ENI/school? 14 Are the benefits and gains from the TMT project sustainable? What are the signs that they are sustainable? What are the challenges to this sustainability? 15 What is the future of teacher related programming in the Beninoise education system? Where has progress been made? What role has TMT played? What are the challenges? 60 MEMP Officials Interview 1 Where the project’s goals and objectives clear to you? Did the project meet its goals and objectives? 2 How has the project helped the ENIs and schools? 3 When the pre-service teachers are posted to schools, what are the biggest challenges they will face? What impact has the project had on preparing pre-service teachers to teach in schools? 4 Was any outreach to MEMP and an advocacy campaign for TMT project initiated? 5 Was any OSEP related data and analysis of teachers shared with MEMP? 6 What were the main strengths and weaknesses of the TMT project? 7 What is the most significant change that the project has brought to your the beninoise education system? 8 How satisfied are you with the TMT project? List areas where you are satisfied and others where you are not satisfied with the program? 9 Are the benefits and gains from the TMT project sustainable? What are the signs that they are sustainable? What are the challenges to this sustainability? 10 What impact did the project have on education policies in Benin? Did any of the project activities support any education policy reform in Benin? 11 What is the future of teacher related programming in the Beninoise education system? Where has progress been made? What role has TMT played? What are the challenges?