ACCESS TO ENGLISH An Evaluation of USAID/EGYPT English Language Training APRIL 2001 Submitted to: Office of Human Development and Democracy USAID/Egypt Cairo, Egypt Submitted by: Aguirre International 1156 15th Street, N.W. Suite 1000 Washington, D.C. 20005 Prepared by: Heide Spruck Wrigley, Chief of Party Bennett Lindauer Mark Sweikhart Ronald Schwarz This report was prepared for the Office of Human Development and Democracy, U.S. Agency for International Development Mission in Egypt, by Aguirre International under GEM Task Order 802, Contract No. FAO-I-00-99-00010-00. The findings, conclusions, and recommendations are those of its authors and do not necessarily reflect the views and/or policies of USAID. p. ii Aguirre International Aguirre International p. iii TABLE OF CONTENTS Table of Contents................................................................................................................ iii List of Abbreviations ...........................................................................................................v Introduction........................................................................................................................ vii PART I: IELP-II Executive Summary.............................................................................................................1 I. Overview........................................................................................................................3 II. Overall Findings.............................................................................................................4 A. Assessment of Effectiveness and Quality................................................................4 1. Introduction........................................................................................................4 2. Program Impacts................................................................................................5 a. In-Service .....................................................................................................6 b. Pre-Service ...................................................................................................6 c. Testing Reform ............................................................................................6 d. English for Specific Purposes and English for Occupational Purposes.......6 e. Professional Enhancement ...........................................................................7 f. Technology...................................................................................................8 3. Barriers to Effectiveness....................................................................................8 4. Promising Practices..........................................................................................10 5. Lessons Learned...............................................................................................12 a. In-Service ...................................................................................................12 b. Pre-Service .................................................................................................13 c. ESP/EOP....................................................................................................14 d. Professional Enhancement .........................................................................14 B. Findings and Recommendations by Component ...................................................15 1. Pre-Service: Findings.......................................................................................15 2. Pre-Service: Recommendations .......................................................................17 3. In-Service: Findings.........................................................................................18 4. In-Service: Recommendations .........................................................................22 5. ESP: Findings...................................................................................................26 6. ESP: Recommendations...................................................................................27 7. EOP: Findings..................................................................................................27 8. EOP: Recommendations ..................................................................................29 9. ESP and EOP: Recommendations ...................................................................29 10. Testing Reform: Findings ................................................................................29 11. Testing Reform: Recommendations ................................................................30 III. Final Recommendations...............................................................................................33 IV. Follow-on Activities Recommendations......................................................................34 V. Conclusion ...................................................................................................................36 p. iv Aguirre International PART II: ELTT I. Overview..............................................................................................................39 A. Study Methods.................................................................................................39 II. Overall Findings....................................................................................................40 A. English Language Training Program...................................................................40 1. Effective and Timely Preparation of Individuals for Training .........................40 2. Meeting the Needs of Different Categories of Customers............................40 3. Feedback to Project Officers.....................................................................40 4. Systems for COTR to Monitor In-Country English Language Programs...................................................................................................41 5. Program Flexibility.....................................................................................41 6. Program Hardware and Software...............................................................42 7. Supervisory Support for ELT and Participants’ Commitment to the English Training Program............................................................................42 B. English Language Testing Program....................................................................43 C. English Language Training and Testing Concerns of Stakeholders......................44 1. English for Professional Purposes................................................................44 2. ALI/GU EPT and CEPA............................................................................44 3. English Language Training Outside of Cairo ................................................44 D. Concerns of Consultants...................................................................................45 1. Focus of English Language Training ............................................................45 2. Need for Security of ALI/GU EPT and CEPA............................................45 3. Qualifications for EFL Teachers for Programs Outside of Cairo...................45 III. Recommendations................................................................................................46 A. Short-term Recommendations That Can Be Implemented within DT-2...............46 1. Promote Common Understanding between AUC and Its Clients Regarding the English Skills and Proficiency Levels Needed for Training Programs......................................................................................46 2. AUC Should Communicate the Length and Extent of ELT Needed if Participants Are to Develop Their Skills Sufficiently in the Time Programmed for ELT.................................................................................47 3. ALI/GU EPT and CEPA Call-Forward Scores..........................................48 4. Length of Time CEPA Scores Remain Valid ...............................................48 5. Repeated Scheduling of CEPA...................................................................48 6. Increase the Information Available about CEPA..........................................48 7. Update Hardware and Software for ELT....................................................48 B. Long-term and Follow-On Recommendations after the End of DT-2 .................49 1. English Language Testing............................................................................49 2. Separating Participant and Counterpart English Language Training...............49 3. Distance Learning Courses.........................................................................49 4. Qualifications of EFL Instructors.................................................................50 5. Decentralization of ELT under DT ..............................................................50 6. Criteria for Selection of Provider for Participant English Language Training...........................................................................50 Aguirre International p. v LIST OF ABBREVIATIONS The following abbreviations are use in the text of the evaluation report and are included here for ease of reference. ALI/GU American Language Institute/Georgetown University AUC American University in Cairo BELI Basic English Language Improvement CALL Computer-assisted Language Learning CELT Computer-mediated English Language Training CEPA Communicative English Proficiency Assessment COTE Course for Overseas Teachers of English CSA Communicative Skills and Methodology CSE Centers of Sustainable Excellence CSM Communicative Skills and Methodology CRM Communicative Reflective Methodologies DT-2 Development Training II Program EFL English as a Foreign Language ELTT English Language Testing and Training EOP English for Occupational Purposes EPT English Proficiency Test ESP English for Special Purposes FOA Faculty of Art FOE Faculty of Education GDIST General Directorate for In-Service Training HR Human Resources IELI Intermediate English Language Improvement IELP Integrated English Language Program INSET In-Service Training Center IT Instructional Technology IVC Interactive Video-Conferencing KSA Knowledge, Skills, Attitude MIS Management Information System MOE Ministry of Education NCEEE National Center for Examination and Educational Evaluation RFA Request for Application RFP Request for Proposal SBT School-Based Training SLEP Secondary Level English Proficiency SPEER Spotlight on Primary English Education Resources TA Technical Assistance TESOL Teaching of English to Speakers of Other Languages TOT Training of Trainers TTI Teacher Training Institutes p. vi Aguirre International Aguirre International p. vii INTRODUCTION Background and Objective This study has been designed to evaluate the effectiveness of two English language activities placed under the Development Training II (DT2) Program, Integrated English Language Program II (IELP-II) and the English Language Testing and Training Program (ELTT). The evaluation sought to capture to what extent project goals were achieved and to what extent processes and products put in place where likely to be sustained so that providers’ preparation can continue. We also identified promising practices as well as lessons learned and looked for models that could be applied to future USAID funded projects related to English language teaching, teacher training, strengthening of public and private training providers and overall human capacity development in Egypt. The report is divided into two major parts: an evaluation of (1) the Integrated English Language Program (IELP-II) and (2) the English Language Training and Testing Program. Study Methods The evaluation used a qualitative design with a “utilization-focused” approach designed to provide information useful in decision making and future planning. We also tried to include elements of “responsive evaluations,” modifying questions and selecting additional people to interview as new issues arose and concerns needed to be checked further. The evaluators worked as a team. The IELP-II evaluation team included Heide Spruck Wrigley, from Aguirre International, Ron Sweikhart, a private consultant, and Ron Schwarz from the University of Maryland. The ELTT team consisted of Bennett Lindauer and Heide Spruck Wrigley. Mr. Schwarz took the lead in examining issues related to English proficiency (for targeted teachers and learners) and Mr. Sweikhart focused on sustainability issues in pre-and in-service training. Dr. Wrigley, the Chief of Party, split her time between the two components of the evaluation. The evaluation used multiple data sources. We analyzed information and triangulated responses from three major sources: (1) document reviews, (2) interviews with stakeholders, and (3) observation of participant training. The evaluation took place from January 3rd to January 30th and included site visits to Alexandria, Beni Suef and Minya (the majority of the time was spent in Cairo). Interviews included discussions with staff, education partners from the Ministry of Education (MOE) and the Faculties of Education (FOE), contractors, beneficiaries, and consultants. Site Visits were conducted at GDIST centers, satellite centers, and English for Special Purposes and English for Occupational Purposes (ESP/EOP) centers and trainings set up by IELP and ELTT were observed. Draft recommendations were submitted and discussed with USAID staff before leaving Cairo. IELP-II issues and recommendations were discussed in March with the project officer, Mona Zikri during Teaching of English to Speakers of Other Languages￾TESOL 2001. (Dr. Zikri was in Washington during the final part of the evaluation in Cairo.) ACCESS TO ENGLISH Part I An Evaluation of the Integrated English Language Program II (IELP-II) PART I: IELP-II Aguirre International p. 1 EXECUTIVE SUMMARY The Integrated English Language Program (IELP-II) is an ambitious effort with the stated goal of “increasing the number of qualified English teachers” in Egypt. IELP-II works with a variety of partner agencies, including the Ministry of Education (MOE) and its teacher training branches, universities located in various parts of Egypt, local governates, and a sprinkling of private providers. There are over 150 activities carried out each year that deal with issues related to teaching, training, testing, planning, and management of services, as well as marketing and assessment. Areas to be addressed in the work of IELP-II include: pre-service training, in-service training, English for Specific Purposes, English for Occupational Purposes, testing reform, and professional enhancement for teachers, including technology. An effort to evaluate the impacts of IELP-II training on teacher behavior was added this year. Over time, the goal of IELP-II has included using partner collaboration to build systems and processes that can be sustained, and to assist Egyptian agencies take over key activities. Although this effort was not part of the original design, and there are no objectives associated with it, it remains an important aim that should continue to be supported. This report presents the evaluators’ overall comments on the quality of the project, its achievements and general effectiveness based on pre-established goals and objectives. We rated the accomplishments of IELP-II very high in all of these areas and were impressed with the management model that is used. It serves as a model to others. Other promising practices deserve mention as well, particularly the notion of having agencies identify the changes they want to make in their local areas through a Request for Application (RFA) process, and bringing partners together through collaboration and production of a final product (in this case a handbook) to be used by various agencies. We were impressed with the way the projects were able to bring together a cadre of “early adopters” of new technologies, train them, and provide them with opportunities to showcase what they have done and in turn train others. We noted a number of significant barriers that limit the effectiveness of IELP-II to build a system that can be sustained through local efforts. These barriers are largely due to the intransigence of established hierarchies, lack of experience in strategic planning, and a general reluctance on the part of established agencies to try new ideas. Given these obstacles, IELP-II will be able to effect significant change in some areas (transfer of training models to Ministry of Education (MOE) funded teacher training institutions, for example) but perhaps not in others (testing reform; building strong collaborations among universities). Helping to build widespread collaborations between pre-service and in￾service training institutions has proven difficult, and Egypt is far from having a seamless system of teacher training. We think the establishment of a lab school that brings together universities and local governates is an idea worth pursuing, particularly since such a site would provide training opportunities for a wide range of teachers, not just those teaching English as a Foreign Language (EFL). Including technology-based learning and teaching in such a site might make the model even more appealing and might attract additional funders. PART I: IELP-II p. 2 Aguirre International Building the capacity of providers to sell services to industry has proven to be a challenge as well. Given that employers do not yet seem to see the value of these services, IELP-II may have been ahead of its time in trying to build public-private partnerships in this area. Our recommendation here is to allow English for Special Purposes/English for Occupational Purposes (ESP/EOP) providers to build local capacity (again through the RFA process), while IELP-II serves as an Egypt-wide clearinghouse for information, resources, tools, provider lists, and the exchange of ideas. We suggest that this be a “virtual clearinghouse” developed on-line and accessible through Internet access to the entire world. In conducting this evaluation, we were very much surprised by the generally low levels of English proficiency of classroom teachers in Egypt. Since a teacher’s proficiency in the language to be taught is the cornerstone of the communicative approach, we call for a new model of teacher training that combines the acquisition of English communication skills (focused largely on speaking and listening) with experience in new methodologies that build these skills. This constitutes a “get two for the price of one” approach, since teachers learn how to implement various methods of teaching English while at the same time getting a chance to upgrade their own English skills. In examining overseas training, we were also surprised by the lack of enthusiasm we heard from some of the teachers regarding their participation in US-based training (technology training in Oregon was a notable exception). The experienced teachers we interviewed sometimes found the training not challenging enough, while the less experienced teachers did not always see the connection between the approaches that were emphasized in the U.S. programs and the realities of their own classrooms in Egypt. We suggest that if teachers at various levels of experience and English proficiency are to fully benefit from the opportunities that U.S. based training can provide, the content and focus of this kind of training will need to be reconsidered and in some cases redesigned. Taking greater advantage of the in-country EFL methodology training offered in-country by AUC should also be considered. All in all, IELP-II has been successful in almost all respects. The project is on target in meeting its objectives and has met or surpassed all milestones to date. In addition, IELP￾II has laid the groundwork for a system that can be sustained if there is the necessary call to action at the Ministry of Education and other agencies. We highly recommend continued funding for years 5 and 6. Our suggestions center around changes that should be made to focus the project during the next two years so the effects of IELP-II are deepened, rather than scattered. PART I: IELP-II Aguirre International p. 3 I. OVERVIEW The Integrated English Language Program (IELP-II) is a four to six year technical assistance and training program funded by USAID and administered by the Academy of Educational Development (AED) and its subcontractor AMIDEAST. The contract was awarded in October of 1997 and completion of the core contract is slated for September 30, 2001. IELP-II has a total staff of 66 (42 AED staff, and 24 AMIDEAST staff). The project staff is made up of 13 expatriates and 53 local hire. The project is organized into three divisions: Program Design and Implementation: Monitoring and Evaluation; and Finance and Operations. During the year of our site visit (Project Year 4), 200 program activities were planned, those included activities related to (1) pre-service, (2) in-service; (3) in-service supervisor training, (4) participant training; (4) testing; (5) English for Specific Purposes; (6) English for Occupational Purposes. Although sustainability was not an explicit goal of the original contract and there are no objectives related to that particular end, the project works closely with its Egyptian partners to plan, implement, and evaluate all project activities. To the extent possible it builds on existing systems and seeks to integrate pre-and in-service training (where little collaboration existed previously). IELP also makes efforts to use Egyptian education efforts and resources. By the end of the program, expected results include: ß Improved skills for the Ministry of Education inspectors supervising and providing in-service training to English teachers; ß Improved teaching practices of university faculty training future English teachers; ß Improved language and teaching skills of current English teachers; ß Improved language and teaching skills for future teachers; ß Test instruments developed for measuring teachers’ English language proficiency; ß Test reform at the 4th and 5th grade level; and ß Monitoring and evaluation capacity integrated into the Ministry of Education. The anticipated long-term impact of the IELP-II project includes: improved English language proficiency for the current and future work force of Egypt; sustainable quality in-service and Pre-Service teacher education; and sustainable capacity to produce English language assessment instruments conforming to internationally recognized standards. PART I: IELP-II p. 4 Aguirre International II. OVERALL FINDINGS A. OVERALL ASSESSMENT OF EFFECTIVENESS AND QUALITY 1. INTRODUCTION IELP-II is well designed, extremely well organized, and to date has met its stated goals and milestones in all areas. The project has reached its intended audience. There remains, however, a tremendous need to continue to upgrade the skills of EFL teachers throughout Egypt. IELP’s success in implementing over 180 activities a year is due to a staff who, at all levels, appear responsible, conscientious and committed to project goals. We were impressed with the professional competence and behavior of staff at all levels, including coordinators, drivers, and secretaries. The project runs smoothly, due in large part to the top administrative staff who are effective and efficient bringing staff together in a joint effort to make IELP a success. The project uses a continuous improvement model to meet its goal, including participatory decision-making, joint development of action plans, and ongoing assessment of what works and what needs to be changed. The management design and practices serve as outstanding models for other organizations. Project staff at all levels are also to be praised for their willingness to take a critical stance. They are proactive in responding to issues and problems as they arise and are willing to advocate vis-à-vis USAID for changes in activities that are not as successful as they could be. In our work it is fairly unusual to find projects like IELP-II who are not content with merely meeting milestones, but are genuinely concerned about making a difference through the work they do. This willingness to take a hard look at what is working and what is not, to respond to needs as they arise, and to advocate for changes that will improve the quality of services should be supported. USAID is to be commended for its flexibility in responding to IELP’s suggestions for change and its willingness to make changes that are within the spirit, if not the letter, of the original program goals. Following are some additional indicators of Program Quality that can serve as models for other projects and should be maintained. · The project has taken a system’s view in completing project tasks, although the original objectives focused on discrete milestones to be accomplished. Management is strategy-based and forward looking, focusing on questions such as, “What does it take to make a difference, how will we get there, and how will we know if we have succeeded?” · The project has taken the initiative in implementing efforts to build the capacity of Egyptian staff. Administrators and coordinators clearly care about issues that matter, and communication between the American expatriate staff and their Egyptian counterparts appears very open and respectful. We saw little distinction between the two sides, a phenomenon that is by no means the norm in other projects or in other countries. The project makes extensive use of Egyptian education experts and PART I: IELP-II Aguirre International p. 5 resources to strengthen and support local efforts. Trained Egyptian staff are then re￾utilized in programming to build local skills, confidence and broaden expertise in the field. · The project’s efforts to work with the Ministry of Education have been impressive. This is particularly true given the many challenges and barriers to effectiveness that exist in working with local bureaucracies. We saw a demonstrated willingness on the part of IELP to explore problems with partners and work together to solve problems. Nevertheless, many of the system’s problems are not amenable to the kind of solutions that IELP can help provide. Staff continues to work for change in spite of the barriers they face in working with a hierarchical bureaucracy where language education is not a priority. · The project design and implementation model leadership and participatory management. IELP staff employs approaches designed to build the capacity of Egyptian staff. Individual decision-making and accountability are promoted and effective cooperative planning is instituted. Team-based approaches predominate. We noticed that top administrative staff are extremely capable in explaining their project, discussing the shifts in strategies to best meet needs, and in laying out future goals. As a result, in-country staff have the opportunity to be part of a learning organization open to input and critique. This is particularly important in a country where joint decision making is not the norm. Through this experience, in-country staff are able to develop professional management and leadership skills. 2. PROGRAM IMPACTS As stated above, IELP-II has met all of its objectives, all milestones have been achieved and its accomplishments have been impressive (see Project Reports). Given the objectives of the project and reluctance of the Egyptian bureaucracies to embrace change, the project has had the anticipated outcomes, in terms of achievements. As for impacts, the short term impacts that have been under the control of the project are quite solid: teachers and faculty members have been trained in communicative language teaching and are starting to conduct training on their own. To what extent trained teachers will actually use the knowledge, skills, and strategies they have acquired through participation in IELP-II remains to be seen. The planned impact evaluation should shed light on these changes. The emphasis on the development of a joint product (a teacher training handbook) has brought together staff from both in-service and pre-service institutions and has involved teachers and supervisors from the field itself. This process has set the stage for a model of collaboration that can and should be continued. Similarly, the collaboration between IELP-II and staff from the Ministry of Education has laid the groundwork for joint planning, development of teacher training, and evaluation of these efforts. It is too early to tell to what extent MOE staff will “pick up the ball” and organize such training and actually use the strategic planning processes that were jointly developed with IELP-II. PART I: IELP-II p. 6 Aguirre International a. In-Service IELP-II has been successful in exposing in-service teachers to new ways of English teaching. In their training of non-specialist teachers who were called upon to teach English classes (although they had no experience and their English proficiency was minimal), they have been successful in improving the English skills of teachers to some extent, as well. However, the English proficiency of in-service teachers we talked with remains very low (master teachers, on the other hand, were highly proficient). Extensive upgrading of proficiency skills will be necessary if these teachers are to be considered qualified to implement a communicative approach effectively. Given that research indicates that it takes 6-7 years for someone to become proficient in another language, the short term training provided by IELP-II cannot be expected to have a significant impact on English proficiency. Similarly, the Basic English Language Improvement (BELI) training that MOE/GDIST is expected to provide will serve to improve proficiency to some extent, but most likely will not be sufficiently intense to meet the overwhelming need for English teachers who speak English. b. Pre-Service The RFA process developed by IELP-II has resulted in universities having to compete for projects, and so far the results seem promising. The technology center in Alexandria, for example, was not yet operational when we visited, due to delays in getting the necessary infrastructure. Other efforts had not yet gotten off the ground as well. However, a number of universities did respond to the RFAs and received a great deal of technical assistance in writing proposals and in aligning objectives with implementation, a new process for many of the faculty members. The experience gained here should help faculty members in the development of other educational projects as well. c. Testing Reform It remains to be seen what the long-term effects of the collaboration with teachers, supervisors, and with administrators from the National Center for Examination and Educational Evaluation (NCEEE) have been. To date, IELP has modeled both processes and products of test development. Processes have focused on understanding notions of validity and reliability (the underpinnings of any sound test) and putting these to use. IELP-II has also collaborated in the development of an item bank that can form the basis for test of the English proficiency of both teachers and trainers. IELP-II has also worked with a cadre of classroom teachers and supervisors to discuss language assessment of students, particularly the assessment of communicative abilities, the cornerstone of the USAID-funded teacher training. Yet, while discussions at NCEEE about test reform continue, there is no clear indication that significant efforts are being made to change the testing of students. PART I: IELP-II Aguirre International p. 7 d. English for Specific Purposes and English for Occupational Purposes IELP-II has been successful in helping ESP and EOP providers become aware of what is involved in providing quality services in these areas. Universities seem ready to develop ESP curricula that use a content-based model or a model of “sheltered ESP” where various strategies are used to make information in the content courses (science, medicine) accessible to students who are not proficient in English. English for Occupational Purposes has been a thornier issue. Industry still seems reluctant to invest in language training for their employees and smaller local providers experience difficulties in competing with larger foreign providers. All in all, IELP-II may be ahead of its time in trying to push for private sector involvement when the needs for English training of employees, while obvious, is not readily recognized. Again, the impact has been largely on providing a foundation for Egypt to build on. Instructional models have been demonstrated; issues of planning, implementation, and evaluation of English for Occupational purposes have been discussed; marketing strategies have been shared. In response to requests by providers, networking meetings will be set up that offer opportunities for industry and providers to share common concerns and match problems with solutions. e. Professional Enhancement There have been significant personal impacts of the program on participating teachers, supervisors, faculty members and partners at the Ministry and at universities. Master teachers reported benefiting greatly from the training provided by IELP-II. However, they were mixed in their assessment of overseas training: A significant number felt that US-based training providers underestimated both the background knowledge and the commitment to advanced learning that Egyptian teachers and faculty bring with them. A number of the master teachers apparently found the U.S. training too basic for their needs and resented the time spent on “tourist” activities outside of class. In classes, they would have preferred information presented in greater depth (the Technology Training in Oregon was a notable exception and received high marks). One highly proficient and experienced respondent explained that she and her peers found “off the shelf courses” superior to “customized classes” developed specifically for Egyptian teachers and coordinators. The difference is largely due to the fact that established courses have to meet certain university standards and are taught by regular faculty whereas customized courses are offered through continuing education where standards may be less stringent and instructors less experienced. Teachers participating in customized classes also felt that they did not get sufficient opportunities to use English since they spent most of their time speaking Arabic with their Egyptian peers. Somewhat surprisingly, a number of the “regular teachers” whom we interviewed found U.S. training somewhat lacking as well. While they very much appreciated the opportunity to visit the United States, they criticized some of the training itself. There seems to be a significant mismatch between the teaching techniques presented in the training (many focused on small group interactions) and the opportunities for using these PART I: IELP-II p. 8 Aguirre International strategies in Egyptian classes where it is not unusual for the teacher to have to manage 60 students in an overcrowded classroom. Among both master teachers and regular teachers there seemed to be a consensus that the in-country training provided by the American University in Cairo was in some ways superior to overseas training. Apparently, teacher trainers at AUC, familiar with Egyptian teaching contexts, were better able to help teachers see how they could integrate new language teaching strategies into their existing curriculum. Interviewees did not suggest that U.S. based training should be discontinued but rather they recommended that this component of IELP-II should be re-examined. Interviewees indicated that training for experienced teachers/staff developers should focus on specific issues important to Egyptian education such as testing or technology. Many suggested that programs for conventional classroom teachers should be more carefully selected and, where necessary, redesigned to better meet the needs of teachers in the field. Such a re￾examination can help both IELP-II and the Ministry make informed decisions on who could most benefit from what kind of training (U.S.-based or in-country). Such a process would also allow for determinations of the extent of the need for overseas training among different groups of teachers so that the number of teachers to be sent to the U.S. is reflective of the pool of applicants and their readiness to benefit from such training. If overseas and in-country training are redesigned so that opportunities for English acquisition are combined with exposure to new methods and if the threshold levels for English proficiency are kept flexible (depending on the demands and nature of the program) the number of teachers eligible to participate is likely to increase, rather than decrease. It was too soon to evaluate the impact that IELP-II has had on the curriculum of the One Room Schools. However, the materials developed reflect what we know about sound English teaching and effective learning at the primary level. The attractiveness of the materials (along with the “fun factor” built into the curriculum) should go a long way toward making teachers and learners want to use the materials. IELP’s strategy of involving a wide range of teachers in field testing also helps increase the likelihood of the new curriculum being used. f. Technology Providing opportunities for master teachers to upgrade their technology skills has had a tremendous impact on participant and has helped to breathe new life into training (The ongoing emphasis on communicative language teaching had started to feel a bit “tired” and repetitious to some of the master teachers). University-based teachers in particular embraced the advanced training and made significant changes in their teaching. 3. BARRIERS TO EFFECTIVENESS Projects, such as IELP-II, designed in part to build capacity that can be sustained and affect changes in the overall system, face significant external barriers. Within that context, IELP-II has achieved a great deal in terms of teachers being trained, PART I: IELP-II Aguirre International p. 9 collaborations established, and change processes being instituted. However, in trying to effect lasting change within the system, the project has faced daunting obstacles. Among the most significant are the following. · Within the MOE run system, there is little solid foundation to build on. INSET Centers, for example, are not yet strong, and EOP is not yet an established field. Efforts run the danger of being diluted because the needs are so great, running both deep and wide. Besides the need to upgrade the skills of thousands of teachers, significant needs include management skills (in terms of designing, implementing, and evaluating training programs), supervisory skills (to support teachers who have been trained and select and supervise trainers), and Management Information System (MIS) tracking and monitoring (to document who has been trained and who is available to do the training). · The educational system is test-driven. The ever-present influence of the examinations pose barriers at many levels—defining the expectations of students of the learning process, shaping the thinking of teachers about what and how they should teach, and limiting reforms in the Pre-Service education of teachers at the Faculties of Education (FOEs). At the primary and secondary levels, the unchallenged status of the examination process means that what counts in the EFL classroom is not being able to communicate in English, but rather knowing about English, since these are the skills being tested. Parents also, appear to be less concerned that their children learn to communicate in English than their children’s ability to pass the test, creating further barriers to the integration of new methodologies. Finally, many teachers teach students in private lessons outside of their regular classes and do so in a very “classical, grammar-based approach” in order to help students pass the existing tests. These teachers are not likely to use a communicative approach in their regular classroom and in their private tutoring. Unless Egypt moves to a testing system that assesses communicative abilities, all efforts to change teaching methodologies can be expected to have limited success in reforming the instructional system. · The Ministry of Education has little experience in strategic planning. MOE/GDIST lacks a comprehensive, strategic planning process to define and respond to training needs and a process to identify and make best use of effective, well-trained staff for in-service training. To date, there is little experience in collaborative decision making, management by objectives, nor ongoing assessment of what works. IELP-II has laid the groundwork by modeling and discussing these processes in joint planning meetings, but the established practice of waiting for specific directives from on high (where strategic planning is a new concept as well), presents significant barriers to change. It remains to be seen to what extent MOE/GDIST is able to move ahead with the now agreed upon plans to take over the training of novice EFL teachers. We can expect that a great deal of support will still be needed to assure quality of services through planning, monitoring and evaluation. · There is limited coordination among agencies within the Ministry of Education. Discussion with the ministry and with other stakeholders has shown that there is little PART I: IELP-II p. 10 Aguirre International articulation of functional roles and multiple entities exist to perform similar or overlapping functions. There are few demands or incentives for collaboration. As a result, effective coordination and sharing of facilities and resources is an ongoing issue. A frequently cited example was access to existing computer facilities. The computers available through the Technology Development Centers are not linked to GDIST and EFL students and teachers cannot take advantage of their existence. Similarly, the INSET Centers cannot use these centers for much needed teacher training. · There is limited collaboration between the Ministry of Education and Faculties of Education and Faculties of Art in trying to build a cadre of “qualified teachers.” In fact, in some cases, there is a total absence of communication between MOE/supervisory staff and educators in the faculty of education. Quite often teachers leave FOE with little real practical experience in the classroom and little support from local districts upon arrival at their assigned school. New teachers frequently lack the practical training needed to be effective in the classroom and are particularly under￾prepared when it comes to teaching overcrowded classes. 4. PROMISING PRACTICES We identified a number of Promising Practices in the work of IELP-II. We consider the topics noted below worthy of replication in other USAID-funded projects related to language teaching: · Collaboration across institutions. A number of collaborative projects have been established that bring together key players from different agencies include faculty of Faculty of Education (FOE) and Faculty of Art (FOA), and individuals from the Ministry of Education, university faculty involved in ESP/EOP, and private providers. Bringing teams together across horizontal lines to do joint work focused on outcomes appears to be especially promising (e.g., the development of instructional materials, and standards to guide implementation of training plans). Asking agencies to collaborate in the development of products (development of a teacher handbook, for example) works since it focuses energies on production, not just processes. Such a process is vastly superior to a series of meeting where only needs and issues are discussed, vague plans are made, and there is no real commitment to action. Similarly, involving a wide range of beneficiaries in the field testing of products (e.g., the handbook and curriculum materials for the One Room Schools) has worked well. Field testing has put these products into the hands of teachers and supervisors who engaged the materials and tried them in their classrooms. This has been an excellent way of exposing faculty members, teachers and supervisors to new ideas while getting “buy-in” into systems changes. · Evaluation focused on outcomes and impacts. The project has taken the initiative in implementing a system for evaluating the impact of its staff development efforts. The evaluation design is sound and efforts to involve local inspectors as data PART I: IELP-II Aguirre International p. 11 collectors is politically wise. The pilot project to be conducted this Spring should provide valuable information for focusing future activities and for moving the impact evaluation up to scale. Efforts to evaluate outcomes and impacts should be an integral part of all future projects of this kind. · Computer-based systems for tracking and monitoring (MIS system). IELP-II has instituted a data collection system for collecting background information on teachers who are participating in training and in tracking the outcomes of the training. They are ready to train GDIST in the implementation of such a system. This is a significant start in developing an effective training system, since it allows training institutions to match trainers with potential clients (e.g., in-service teachers), monitor success, and identify unmet needs in a systematic way. Such a system will allow training institutions to see at a glance which areas are being served (and to what extent), and which are not. It helps them to see whether they are achieving the necessary gender balance among participants and whether rural areas are receiving their share of services. By collecting background information on students (English proficiency skills, for example) and matching them to outcome data, they will be able to see which groups are succeeding and which need additional support. · Comprehensive training models that reflect what we know about effective staff development. Research in staff development is quite clear on the limits of “one-shot workshops” in effecting change in teacher behavior in the classroom. IELP-II has realized these limitations and has built a model that surveys teachers before trainings are held so that needs can be identified and discussed. Post-training workshops are held as well. Teachers develop action plans on how they plan to use the knowledge and strategies gained in their classes, and they receive feedback on these plans. Finally, the use of teaching demonstrations and the use of “micro-teaching” where trainers and teachers teach side-by-side is an effective model for getting training “to stick.” Teacher observation and feedback on actual teaching in the local context are built into the model as well. However, since neither IELP-II staff nor trainers are allowed to observe classes, this component of the training is up to local supervisors who may or may not have participated in training on how to observe and provide positive feedback. · Technology that follows the maxim “invest in the zealots.” Experience in technology training for teachers has shown that it pays to invest in a core group of people who are ready and eager to embrace innovations, rather than trying to institute large-scale reform in a system where many teachers are still resistant to innovation. IELP-II has provided opportunities for individuals to become knowledgeable in the use of technologies that promote language learning. Most participants have been from university faculties where there is both greater access to technology and greater freedom in using new methods, although some in-service teachers were involved as well. The training provided was hands-on and project-based, focusing on the application of the technology for English language use and English learning. The training also provided faculty members and teachers with technology-based resources for language teaching to be used either by teachers or students. Finally, participants in PART I: IELP-II p. 12 Aguirre International the training learned how to use technology-based presentation skills (such as PowerPoint), for presentations in professional conferences in-country and in conferences around the world. By focusing on those individuals who were eager to tackle technology, IELP-II created a cadre of professionals who are now ready to act as change agents as they model effective use of the technology with their students, peers within their work places, and professionals from other countries. By focusing on the new media, including effective use of the internet, IELP-II has in essence leap-frogged over conventional Computer-assisted Language Learning (CALL) technologies, such as computer-based integrated learning systems and “skill and drill” software. Since conventional software of this kind is not effective in improving language use (a key component of communicative competence), IELP-II made the right decision in being forward looking. Introducing the “zealots” to new media and providing them the opportunities to share their knowledge through conferences and workshops has produced an enthusiastic core of practitioners whose knowledge can cascade down to others. A natural next step would be to set up pilot projects that link technology to different kinds of learning (English, science, math), and train an entire school or lab site in effectively integrating technology into teaching. · Collaboration focused on products, not just processes. Asking agencies to collaborate in the development of products (development of a teacher handbook, for example) works since it focuses energies on production, not just processes. Such a process is vastly superior to a series of meeting where only needs and issues are discussed, vague plans are made, and there is no real commitment to action. Similarly, involving a wide range of beneficiaries in the field testing of products (e.g., the handbook and curriculum materials for the One Room Schools) has worked well. Field testing has put these products into the hands of teachers and supervisors who engaged the materials and tried them in their classrooms. This has been an excellent way of exposing faculty members, teachers and supervisors to new ideas while getting “buy-in” into systems changes. · Support of fledgling institutions. IELP staff have acted as change agents in helping to develop local efforts, such as TESOL Egypt and its Instructional Technology (IT) Interest Section. This support has allowed participants of Computer-mediated English Language Training (CELT) to put together a symposium and conduct workshops in local areas. Collaboration of this sort where partners are supported, but also encouraged to take on increasing responsibility, constitutes a sound model. 5. LESSONS LEARNED a. In-Service It is much easier to build skills of individuals, hoping for a trickle down effect, than it is to effect changes within a system. System change requires ongoing collaboration around meaningful issues. Processes such strategic planning and management by objectives need PART I: IELP-II Aguirre International p. 13 to be modeled, and technical assistance should be available on an as-needed basis. In the end, there needs to be a certain push to encourage in-country agencies to take the steps necessary to move change forward. Ongoing support by USAID should be linked to such steps being taken. In other words, now is the time for the Ministry of Education and GDIST to “walk the talk” and put training processes in place. Issues are similar when it comes to testing reform. Projects, such as IELP-II can develop testing banks and train in the areas of standards and procedures for sound testing, but change in the system must come from within. Ongoing technical assistance in test development will be most effective if implemented on an “as needed” basis. US-based teacher training is not necessarily superior to in-country training. Exposure to U.S. culture and democratic principles can have far-reaching effects and should be maintained although more attention should be paid to the kinds of experiences that are provided outside of the classroom with less of a focus on tourism and more of an emphasis on “civic life,” including conversations with U.S. peers and/or structured activities that call for cross-cultural comparisons. US-based training needs to be re￾examined so that it is better able to respond to the needs of Egyptian teachers. Upgrading the English proficiency skills of Egyptian teachers is a complex task which cannot be achieved through a series of workshops dedicated to methodology alone. New ways must be found so that teachers can gain the English proficiency needed to teach the communicative approach effectively. If teachers with limited proficiency in English are to be comfortable using this approach, they will need additional opportunities beyond methodology workshops to communicate and use English. b. Pre-Service Maintaining flexibility in a contract and keeping objectives fairly open works. Rigid adherence to pre-specified objectives can be counter-productive. Flexibility in the initial contract allowed IELP-II to abandon the original idea of creating university-based Centers of Excellence, designed to offer leadership to other universities. These were replaced with more successful models, geared toward local improvements at Faculties of Education and Faculties of Art (where English literature is taught). The idea of providing up-front time in a contract to get to know partners and gain a sound understanding of political concerns and turf issues is of merit and deserves further consideration. Collaboration among agencies around the development of products works. Expecting universities to collaborate on their own accord does not work, even if outside support is available. Asking universities to compete for projects works, although the efforts needed to help faculty design projects that show promise has taken much more work than previously imagined. It has taken these universities much longer than expected to get projects off the ground, and on-going encouragement and monitoring is needed to get these projects under way. PART I: IELP-II p. 14 Aguirre International c. ESP/EOP Expecting collaboration between university departments which face turf issues does not work (an expectation not unique to Egypt). Strengthening the capacity of individual centers and the expertise of individual faculty members to provide quality services works. In the end, the various departments may need to fight their own battles with technical assistance being available as agreements are being reached. It is possible to be ahead of one’s time, and EOP is a case in point: Private providers are eager to market their services to industry and establish on-site classes. Industry does not appear to be ready to buy these services. Forcing a system that is not ready for change might not be a good idea. The time seems right to allow private providers and universities to compete for a Technical Assistance (TA) project that continues to build the capacity of the EOP effort. IELP-II can serve as a virtual clearinghouse for information and resources and can continue to create materials, such as “EOP Digests” for the field. After two years, if continued funding exists to support such an effort, both the clearinghouse and the TA function can be taken over by a local university or a private agency. d. Professional Enhancement Creating a cadre of change agents for technology works, as does helping them to set up local conferences and providing workshops for local area universities and schools. Using distance learning technologies to bring new ideas to local district teachers was too ambitious an undertaking for this project and has not worked particularly well. Facilitators at the local Satellite Centers where downloads occurred were not as effective as they might have been. We heard several reports of local teachers not being interested in the video presentations given by other teachers and paying little attention to the screen. Using the video technology to present ideas that might interest the teachers, using multi￾media, and presentations by dynamic speakers might be better ways to introduce this technology to the field at large. PART I: IELP-II Aguirre International p. 15 B. FINDINGS AND RECOMMENDATIONS BY COMPONENT 1. Pre-Service: Findings How well are the university Faculty of Education Centers of Excellence being strengthened? How valid is the current approach to the technical assistance to the Centers in terms of capacity building, serving as resources to other institutions, and impact and sustainability? IELP-II originally envisioned the development of a program that would strengthen the capacities of a number of FOEs and would be integrated into a select number of universities creating “centers of excellence” designed to offer technical assistance, act as a resource to other institutions of higher education and possibly provide leadership in issues related to EFL teaching. Lessons learned early on suggested that while the concept of building the capacity at key universities deserved merit, the turf issues common among universities along with the tradition of being autonomous (as opposed to collaborative) required a rethinking of the implementation strategy. In response to signs indicating a reluctance of universities to collaborate, IELP-II, with full support from USAID chose two alternative strategies designed to meet the original goal: building individual and institutional capacity within the Education and Art faculties. Both strategies were sound and demonstrated success. In the first, the IELP-II training activities focused on promoting the integration of communicative methodology into a content course that is project-based. The effort concentrates on the development of a collaborative product, a handbook for teachers, called Spotlight on Primary English Education Resources (SPEER). This strategy has the advantage of drawing out capable educators and change-makers regardless of their institutional affiliation and guiding them in process of collaboration and co-operation focused on a common end product useful to the field. The second strategy allowed universities to identify problems and challenges common to the field and obtain technical assistance from IELP-II. Encouraging institutions to address these challenges locally carried with it the expectation that both products and processes being developed could act as models and change might spread to other institutions. This second strategy became the cornerstone of the Pre-Service English Teacher Education Improvement Program. As part of this strategy, a competitive RFA process was established through which universities were asked to identify problems, suggest solutions, and identify the strategies they planned to implement in response to these problems. Since Egyptian universities were new to the notion of having to bid for technical assistance, a great deal of support was provided by IELP-II to guide them through the process of establishing objectives and writing a project plan. The results became support for locally identified efforts to improve the pre-service training and preparation of English teachers. It should be noted that this targeted approach does not directly build FOE capacity to serve as resources for other institutions, but rather it supports selected FOEs in embarking on improvement efforts. Both of these new PART I: IELP-II p. 16 Aguirre International strategies are promising models in terms of their impact on pre-service education and should be continued and supported. In addition, completing the RFA process provided valuable experience related to strategic planning, an effort worthwhile in itself. What efforts are being made to improve institutional capacity at these centers to design, deliver and manage training in the future? What efforts have been made to build Knowledge, Skills, Attitude (KSA) and expertise to provide FOEs with qualified staff and to provide communicative English Language and effective interactive teaching skills to future teachers? To what extent are the other Faculty of Education target activities of IELP-II demonstrating impact? How well are they being institutionalized? Efforts to improve the individual faculty capacity have included a series of trainings focusing on child centered language acquisition, materials development, assessment, and the application of new methods in the classroom through action planning. Participants in Institute training programs during years 1 and 2 have been subsequently involved in a process of materials development/readaptation, field testing, and review. They will be eventually involved in the dissemination of a teacher training handbook for both methodologists and supervisors with materials that are relevant to the Egyptian context and are user-friendly. Importantly, both FOE and MOE staff have participated in this process. Efforts to date to improve FOEs at the institutional level through competitive processes have resulted in five centers being supported. Faculty in Alexandria are being supported in the development of a CALL lab; those in Beni Suef and Helwan in the improvement of teaching practice; in Assuit in course design; and in Suez in testing. A newly issued RFA to support a new round of additional institutions focuses on performance improvement at FOEs in areas such as enhancing management skills (strategic planning, consensus building, interdepartmental coordination); development of performance standards for staff and student teachers; improvement of coordination with MOE and teaching practice; development of materials; and design of new curricula. The project wishes to build on the successes to date in improving collaboration across faculty (FOE/FOA) and between FOEs and the MOE. Our understanding of effects on this project component to date comes mainly through anecdotal evidence. In terms of effects on knowledge, skills, attitudes (KSA), numerous interviewees reported changes among colleagues in their methods and ideas about teaching language. Others reported that exposure to communicative methods had increased their repertoire of teaching techniques–promoting their use of group work, shared readings, and worksheets; and increasing their abilities to deal with large groups and promote their students’ involvement. Another person reported improved presentation skills and enhanced abilities to formulate and implement an action plan. Procedural changes included the introduction of micro-teaching approaches and the use of observation checklists at one institution; and the introduction of mentoring, teaching practice conferences, and smaller teaching practice groups in another. PART I: IELP-II Aguirre International p. 17 Though the three projects conducted under the competitive process are in their early stages, at one site, course design efforts have resulting in dramatically improved communication between FOE and FOA departments. Now a joint planning process is underway to redesign five courses which affect the language proficiency of future teachers. IELP staff note that exposure to the RFA process has increased faculties’ capacity to identify their key challenges; better distinguish between those issues which can be addressed through training and which cannot; better prioritize their objectives; focus on process as well as product; and accept to a much greater degree the value of cooperation with MOE to achieve their objectives. A clear picture of the effects of the two key strategic approaches of the IELP-II pre￾service component hopefully should come through a more comprehensive effects evaluation, yet to be developed, but planned for year four. How might the model of the Centers of Excellence be transferable to other academic specialties? The two-alternative strategy developed by IELP-II to strengthen the capacity of universities is a promising model that can easily be used for improvement of the education of university students in other faculties. The basic principles and processes are not specific to English learning and teaching and will easily transfer to other academic specialties. Both strategies require an up-front assessment of the critical issues and the training needs across faculty at different universities. They also demand the active involvement and participation of faculty in the process. The project-based model, focusing on collaboration of faculty across-universities (as opposed to collaboration among the institutions themselves), can easily be transferred to other areas. Guiding faculty members in developing products that move teaching in new directions has applications not only to the humanities but to the sciences as well, resulting not only in materials to guide the field, but in increased communication among faculty members from different universities. The competitive process model also is not FOE/FOA-specific and is applicable to various departments and academic specialties. Faculties can identify their key areas targeted for improvement, their plans for improvement, and the kinds of support they require. Faculties have ownership and (perhaps unlike units within the MOE) a sufficient level of independence and autonomy to carry out a process of change. 2. Pre-Service: Recommendations · Continue both the collaborative strategy and the RFA process but require project applicants to identify more specifically how their proposed project will effect change at the level of the primary, preparatory, or secondary classroom. Encourage applicants to identify expected impacts and outline strategies for assessing both short-term and long-term outcomes. While the results may not be measurable at this level, during the span of each individual project, the priorities outlined by the funder in the competitive RFA process can have a significant role in heightening awareness of IELP-II’s ultimate goal of having impact upon the quality of English language instruction in the Egyptian classroom. PART I: IELP-II p. 18 Aguirre International · Develop more effective evaluation tools to assess the success for individual and institutional capacity-building activities in the Pre-Service program. The Pre￾Service project to date lacks adequate and effective evaluation, and tends to be weak even at the level 1 and 2 levels. Given the shift in the component to more project￾based approaches, both with the Spotlight on Primary English Education Resources (SPEER) handbook and with the newer competitive RFA approaches, more careful consideration should be given to the evaluation of outcomes. Evaluation of pre￾service improvement projects should, of course, be linked to the internal assessments of success developed by the universities submitting proposals. Special attention should be given to the concept of formative evaluation, allowing IELP to assess success to date and work with participating universities to help them make necessary changes and improve their work, based on a systematic assessment of successes, challenges, and outcomes. · Consider new ways of linking MOE and FOEs. Consider the establishment of an EFL lab site that is based on collaboration between MOE-funded schools and FOE￾trained teachers. Such sites can provide a practicum for novice teachers as well as a learning ground for more experienced EFL teachers, demonstrating how communicative approaches can be established within Egypt. This model can also provide research opportunities for FOE faculty and serve as a means for IELP-II to measure the impact of its training. Such a lab site should be focused on 1 or 2 particular aspects of EFL, so as not dilute the training efforts. These aspects might include technology-based learning; EFL for science and technology; English for international communication, with a focus on verbal fluency; and face-to-face communication; and ESL through video, etc.1 A study tour for Master teachers, MOE administrators, FOE faculty members, and USAID staff can provide opportunities for examining additional options for strengthening pre-service teaching through a practicum that links local school districts and teacher preparation at the university level. 3. In-Service: Findings How well is IELP-II creating a basis for sustainability of the model and improving institutional capacity within GDIST to design, deliver, and manage future training? Will GDIST (the in-service training department of the Ministry of Education) be able to take over the activities and expand them? What will it take to bring this about? Core of In-Service Programming and Resources Available. Working with Egyptian partners in years 1-3, IELP-II has successfully crafted and is continuing to develop a core of valuable training courses and materials (1) that target the development of English proficiency and the use of appropriate methodologies by the classroom teacher, (2) that 1 The ESL lab school jointly run by Portland State University and the Portland Community College and funded through the National Center on the Study of Adult Literacy and Learning might provide some insights into what it takes to set up such a collaboration. PART I: IELP-II Aguirre International p. 19 develop the critical skills needed by supervisors and senior teachers to conduct teacher training sessions and provide follow up support, and (3) that strengthen the abilities of Inspectors General and supervisors to plan and design appropriate training programs. We found, on the whole, that the courses are sound pedagogically, are responsive to local needs, and have benefited from the active involvement of Egyptian educators. Increased GDIST Visibility and Role. GDIST increasing involvement in the administration of the effort has played a significant role in raising its visibility among MOE supervisors, a number of whom have commented that they had never heard of GDIST prior to IELP involvement and that they had no clear concept of the role of in￾service training prior to IELP. A Cadre of Trainers to Draw Upon. Given the IELP-II effort, GDIST and its INSET centers (as well as local training programs conducted in the governates) can now draw upon a significant cache of resources and a cadre of almost 700 trainers. Importantly, the project has developed effective processes for interaction between key MOE players, and systems for needs assessment, planning, program design, and evaluation. A Transition Scheme Developed. A transition scheme to transfer programming to GDIST has begun. This year funding and management of Basic English Language Improvement for primary teachers (BELI), Communicative Skills and Methodology for prep and secondary teachers Communicative Skills and Methodology (CSM), and School-Based Training (SBT) have been included in the central plan of GDIST and thus are on the “training map.” CSM has already been “transferred” to GDIST with pilots of the course now being administered at four centers where INSET capacity and the quality of the local trainer pool tends to be higher. Current numbers under the four pilots represent perhaps only a third of comparable yearly numbers under IELP-II, though plans to extend to the nine INSET sites are planned in Year 5. GDIST will also pilot BELI courses at four INSET centers this summer though it should be noted that at this time it is not clear if GDIST will be administering an language assessment tool (presently the Secondary Level English Proficiency—SLEP) to appropriately screen applicants. To date, it cannot be said that GDIST has the capacity to sustain a comprehensive program of in-service training. The sustainability of the model will depend not only upon GDIST’s ability to manage and administer the current and future courses of IELP-II, but also upon its ability to offer an ongoing program of in-service training as teacher needs change. This entails GDIST’s ability to: · Assess training needs based on standards of teaching performance at the various levels within the system; · Develop annual training plans that are responsive to regional needs and input of INSET centers; · Design appropriate training programs; · Develop or use tools that will discriminate which candidates can best benefit from the programs; · Evaluate training programs and refine or modify them as necessary; and PART I: IELP-II p. 20 Aguirre International · (optimally) to evaluate results system-wide. In addition, given the critical role of in-service training to improving educational quality, MOE must ensure that GDIST has: · Finances to sustain a broad-based program of in-service training; · Increased numbers of INSET centers that can provide more equitable coverage of training opportunity; · Adequate local facilities to deal with both small and large audiences; and · Technological capacity to maintain a shared system-wide database and the ability to utilize this information for informed decision-making A Targeted Program of Institutional Capacity Building Required: While current Year 4 IELP plans include efforts to build evaluation skills among MOE supervisory staff through training and through the inclusion of MOE staff in the IELP-II Effects Study, no evaluation training targeted specifically to GDIST and the INSET centers as institutional units is planned in the current year. This year IELP does plan to pilot the transfer of information system technology to GDIST and three INSETs (Cairo, Alexandria and Tanta). Such a transfer would include the customization of a database for GDIST and training to develop appropriate collection tools, queries, and reports, and their use for decision making. Transfer and use of this technology in GDIST and the INSETs currently rest on assumptions of support from other funders for the hardware and software. Complete GDIST readiness to serve as a sustainable in-service unit providing for system￾wide training support would require (in addition to the MOE factors noted above) a more concerted institutionally-targeted program of capacity-building support by IELP-II in years 5 and 6 than is currently in place. What efforts are being made to build individual KSA and expertise to provide MOE with qualified staff to continue teacher and supervisor training? How well are the GDIST satellite In-Service Training Centers being strengthened? How well are training activities being transitioned and supported by IELP-II to GDI ST and its satellite centers? What are the challenges facing the satellites in taking over training of supervisors and other IELP-II activities in the future? What is the level of motivation of the satellites to take over more training responsibilities? How will GDIST be able to continue the capacity building of the satellites? Qualified Staff for Continued Teacher and Supervisor Training. A key element of IELP strategy for KSA capacity building has been at the individual level. Almost 10,000 MOE teachers, supervisors and manager/specialists have been trained. Training programs have been based on a training of trainers model that has built significant capacity at the supervisor level (73% of participants who have received Training of Trainer and/or Master Trainer Training have been supervisors). As most centers may have only one English Specialist who will, in most cases, be unable to provide the wide range of training required, it is upon members of this IELP-II-trained pool that GDIST and INSET centers can call upon when trainings are conducted. Current concerns among many supervisors are that this valuable human resource will not be adequately recognized and PART I: IELP-II Aguirre International p. 21 utilized in developing and shaping INSET programming. As a start, the IELP-developed information system to be customized and transferred to GDIST and its INSET centers beginning this year will enable them to identify trainers from the database by demonstrated skills, location, and a number of other factors. Difficulties with Institutional Capacity Building Targeted to the INSETs. Given that transfers of the initial courses of BELI and CSM to the pilot INSETs are only beginning this year, it is not possible to comment fully on the how the transition to the satellites is proceeding. While the project is gradually shifting its focus to more institutional-level capacity-building, a more natural outgrowth of these efforts have centered at the governate level in building Inspector General/Senior Supervisor teams with potential for new members that can better plan and design effective local training. This area is in fact a separate stream of in-service training provision. In part, a complicating factor to support for INSET centers is a lack of delineation as to the role of local training vs. that at the INSET centers. The earlier individual capacity-building focus of the project lends itself to proceeding up the local training “stream.” Among INSET centers, capacity for conducting training, equipment and resources, and motivation vary. Clearly the latter factor may be significantly affected by the other factors. Given that INSET center staff size is small, the development of institutional capacity requires that not only the skills of the few center staff must be enhanced, but also that each center must be assured of access to a sufficient number of locally available trainers with a diverse range of skills to meet training needs. Additionally, the relationships between these local players (INSET and non-INSET staff) must be strengthened through supportive processes that would develop this more encompassing team’s abilities to do the activities required (joint assessment, planning, design, evaluation, etc.). This assumes that INSET service provision, though overseen by GDIST Central, would be decided through a more cooperative planning process than now exists. Presently, like GDIST, the INSET centers on the whole do not have the capacity to conduct a sustainable in-service training program that meets the criteria set forth earlier. It should also be noted that while GDIST holds the purse regards the training that occurs at the INSETs, capacity, at least at one center visited, suggests that the system should be open enough to encourage and motivate innovation at the local sites. A centrally planned program of training without local input would inhibit innovation. Also required are more clearly defined demarcations between the roles of the INSET centers and local training efforts within each governate. Will INSET centers, for example, only be conducting system-wide “generic courses,” that will be complemented with local training that highlights/reinforces site-specific issues? This issue may have relevance to the roles of individual INSET centers in defining and shaping the GDIST system-wide annual plans. What will it take to enable GDIST to undertake the upgrading of the 500 mid-level administrators by means of the IELP-II model, instead of massive U.S. training? The nature of the IELP program has necessitated two core assumptions to making improvements in the teaching of English in the classroom. First, to reach the magnitude of teachers required, a trainer of trainers model is needed. Second, given the difficulties PART I: IELP-II p. 22 Aguirre International inherent in system-level change, investing in building the capacity of key change agents in the system facilitates the change process. While they have targeted efforts at multiple layers in the system, their limited project scope (English language improvement vs. instructional improvement in multiple subjects) has necessitated a more bottom-up approach. Additionally, the significant redundancy in the Egyptian education structures, has required an emphasis on bridge building between disparate elements to promote support for their efforts and improve overall coordination. The project has learned that effective training can occur in-country (supported by both internal and external experts), though selective overseas training of key personnel to obtain high levels of technical proficiency has its place. Thus, lessons from the project suggest that a predominantly locally-based effort would be preferable in terms of long-term results in the system and in its cost-effectiveness. Despite significant project achievements in the first three years, the emphasis is only beginning to impact upon GDIST/INSET capacity to sustain an ongoing in-service training effort. Therefore, without significant financial and technical support from both overseas and local experts, GDIST does not have (nor would it have at the close of IELP￾II) the capacity to conduct the upgrade of such a large number of administrators from a group that would presumably extend beyond English Language–related professionals. Constructing such a model of training would necessitate knowing who these administrators are and what institutional structures tie them together. 4. In-Service: Recommendations · Ensure a sound transfer of training from the project to the MOE by developing a specific plan for institutional strengthening for the final years of the contract. Work collaboratively with all partners to develop strategic plans that outline who is responsible for which part of the language training system. Outline the kind of support that will be provided by IELP-II, but link this support to definite steps being taken by partners. Use a phased-in approach of technical assistance, where support “kicks in” when certain conditions have been met or pre-established milestones have been achieved. Keep the Technical Assistance (TA) model flexible enough to respond to needs as they arise. When they do, continue a model of collaborative planning that stresses shared responsibility and fixes the roles to be played by each institution and the individuals within. · Find new ways of upgrading the proficiency skills of EFL teachers who still struggle with English. The current training models are not sufficient in scope or intensity to upgrade the skills of the many teachers who are not comfortable communicating in English. In order to effectively teach a communicative approach, these teachers will need the opportunity to significantly improve their speaking and listening skills so they can feel comfortable using the methods that IELP-II supports. If these teachers are to be effective in teaching English to their students, they will need the opportunity to immerse themselves in English while absorbing new methodologies through a process of natural language acquisition. In other words, they need a chance to improve their English through methods that reflect the approaches PART I: IELP-II Aguirre International p. 23 we want them to use with their students. Two models for achieving this aim come to mind: o An in-country intensive English language immersion model designed specifically for English teacher. In these courses (offered 20 hours a week for 6 weeks, for example), teachers will get the opportunity to build their communication skills by listening to English and by using English in an interactive setting, focused on conversation and discussions on topics of interest. Interactions with native speakers and opportunities to use English outside of the classroom can be built into such a course. The model should be designed so that it improves the teachers’ instructional skills along with their competency in English: It will give them a chance to experience new methods of language learning and teaching first-hand and will provide opportunities for discussions on what works and what does not in English teaching and learning. Teachers should be encouraged to reflect on what helps them learn and what makes learning difficult, followed by discussions of the methods they themselves use in their classrooms to assist their students in the acquisition of English. o A U.S. based teacher to teacher model focused on interaction with other teachers, observations of classrooms, and rich in opportunities for conversations in English. Such a model would provide hands-on, practical experience in language learning and teaching, rather than academic study of methodology. Egyptian classrooms could be matched with ESL programs in the U.S., and a US-based program can act as a host to a small group of teachers. These teachers would build one-on-one relationships and would exchange ideas about their students, cultural differences and similarities and the challenges of teaching EFL/ESL. Such a model would provide Egyptian teachers with myriad opportunities to listen to and use English as they observe classes, talk with the students, and discuss ideas with other teachers. Again, it would improve their English communication skills while allowing to absorb and discuss new methodologies. A team of co-facilitators (one Egyptian master teacher, one US￾based) can lead discussions on how methods could be adapted to an Egyptian context and help Egyptian teachers make the connection to their own classrooms. Such a model represents a form of “sheltered content learning” in as much as information about teaching will be made accessible through observation and conversations with U.S. peers, rather than through academic learning. Such a teacher-to-teacher model of acquiring new methodologies does not require the same high proficiency than courses at a university-based methods course. Even teachers with high beginning and low intermediate English skills can significantly benefit from classroom observations and conversations with English speaking teachers. The CEPA threshold levels currently set for participation in U.S. training should therefore not apply. However, participating teachers should still be Secondary Level English Proficiency (SLEP)-tested to assure that they have the minimal proficiency necessary to hold a simple conversation in English. PART I: IELP-II p. 24 Aguirre International · Use the current data available and the results of the June 2001 Effects Study to examine the coverage and impact of efforts. Determine if a focus on particular geographical regions, system levels, or institutional units might be worth a concentrated effort for the last years of the current project. Emphasis might be placed on such activities that potentially afford insights to “mini-models” of sub-system change. EFL lab schools and pilot projects focused on technology are examples of a focused approach of this sort. Deepening involvement in select areas would allow for experiments in sustainability within the system (where opportunities for system/structural impact seem promising), especially since that large-scale, system￾wide change is probably beyond the scope of an English language improvement program. Sustainable system-wide change is not a realistic expectation for a single project, especially given the multitude of barriers that inhibit such change. · Use the piloting of in-house Information Systems at GDIST and INSETs as a key avenue to institutional capacity building effort. Link this activity with other IELP efforts (those already planned as well as those suggested above) to support GDIST/INSETs in the management and operation of the Year 4 courses. It would be unfortunate if the significant processes developed by IELP and the learning gained from the experience were not sustained by GDIST due to inadequate attention to the transfer process at the final stages. For example, it appears that current plans by GDIST to administer the BELI course will not include the use of a SLEP test (the current tool used by IELP-II) to properly evaluate teacher language proficiency. This fact reinforces concerns expressed by IELP-II-trained trainers that fair screening and selection practices will not be used by GDIST to determine potential trainees. Concern is required by IELP in this matter, given that such courses’ effectiveness is based on their appropriateness to language level. The basic proficiency testing system and the Information System technology designed by IELP have all been integral to effective in-service training processes, in terms of selecting candidates, assessing need, tracking training and trainer expertise, etc. These critical processes should be an integral part of the “course transfer.” It serves as a corner stone in building the knowledge and skills among these institutional units in collecting information, analyzing it, and in making informed decisions on policy. IELP-II should highly recommend the use of the tools that have been developed: a system for English language proficiency testing; a data collection system to track, monitor and assess success; and a matrix of trainer skills and experience designed to match training expertise with needs. These recommendations should be put into writing so that future support can be linked to the use of these systems. · Re-examine IELP-II outreach efforts to see if approaches inadequately address pre-selection effects (cultural issues, geographic isolation, etc.) that might limit the training participation of specific groups. For example, current IELP data suggests that only 38% of primary level SLEP-tested teachers are female. While the percentage of female English teachers at the primary level is not available, it is likely to be over 38%, given the large numbers of female teachers at that level. Investigating possible gender bias in training, along with its causes and exploring remedies to increase awareness and participation in the project by women seems warranted. A PART I: IELP-II Aguirre International p. 25 more effective outreach approach would be a critical piece of the training model transfer to MOE and it would support the goals of other education projects in Egypt to increase the participation of women and girls in education. · Review IELP-II system data to ensure that adequate training capacity exists among supervisors at the local levels. Ongoing efforts are needed to ensure that staff members at each INSET (and soon-anticipated INSETs) have the depth of experience needed to support quality training once full responsibilities for course management and administration are transferred. Training in how to use the trainer skill matrices will allow INSET staff to select the right kinds of trainers for each training. The system allows administrators not to only to track trainers by number, but also by the type of training that can be provided and geographic region in which the trainer can serve. It also captures the skills sets that trainers have demonstrated, thus allowing the best trainers to surface and matching expertise with need. Plans in Years 4, 5, and 6 should take training in the use of these matrices into consideration in its supervisory/management workshops. · Re-examine the TOT/Master Training process to ensure that Egyptian trainers leave at the close of the training with the necessary tools and knowledge. Our interviews and observations have shown that trainers are often not prepared to effectively translate the methods discussed in training to the Egyptian teaching contexts (where classes are large, desks are fixed to the floor inhibiting pair and group work; the “Hello English” represents the curriculum, and grammar and vocabulary tests reign supreme). Many Egyptian master teachers are at a loss on how to teach these methods so that they match the realities of the classrooms in which their trainees find themselves. Comments from numerous in-service trainers and evidence from training evaluations suggest that more direct focus is needed during trainings to assist trainees in translating their new learning to the specific contexts they face. We offer a few suggestions: Perhaps an Egyptian working group among trainers to share their particular approaches would be a starting point in better identifying what the issues are, what specific advice these practitioners have in addressing the barriers, and what concrete activities might be appropriate to include as part of future trainings. Perhaps in the final days of a TOT workshop the trainees could divide up into groups with particular trainers to address the key barrier each individually feels is his/her special challenge. Alternatively, a follow-on for trainers to tackle this specific issue of adaptation could serve this purpose. Note that a distinction is being drawn between what appears to be the current practice of opening this issue up to general discussion versus a more targeted, conscientious alternative that focuses specifically on lesson strategies drawn on lessons from trainers in the field and that affords capable trainers/participants the opportunity to share/demonstrate them either during the training or in a specially designed follow on. Such an approach would assist those who honestly would like to try the new methods and perhaps reduce resistance by those who see the methods as lacking transferability due to their particular school settings or to the limitations of their own language proficiency. These comments PART I: IELP-II p. 26 Aguirre International seem especially relevant for the BELI course now to be managed and administered entirely by GDIST. · Educate new outside consultants from the start (especially with new courses like Intermediate English Language Improvement—IELI, and Communicative Reflective Methodologies—CRM) on the special issues faced by the Egyptian teachers. Bring IELP-trained trainers into the process early on so that transfer elements can be integrated into the initial “TOT pilot” early on. This would greatly assist trainers in their ability to directly transfer the learning to their trainees, who in turn would have ways of showing in-service teachers how to adapt new ideas to the classroom. While some of the trainers being trained are more advanced and are capable of thinking of transfer strategies on their own, evaluator observations of training sessions make this clear that this is not the case among all. 5. ESP: Findings What efforts have been made to improve institutional capacity at selected ESP centers to design, deliver and manage future training? What efforts have been made to build individual KSA and expertise to provide ESP centers with qualified staff to quality targeted ESP courses to both internal (university EAP) clients and to external (private EOP) clients in the future? In the initial IELP-II proposal, ESP outputs emphasized the training of content instructors (who taught in English to university students) in intensive English courses. Yet it was learned early in the project that greater impact was to be obtained through “working with ESP staff to train future ESP practitioners.” Five sites were to be identified in the first year of ESP (IELP Year 2) and 30 teachers/year trained from these centers. However, ESP services tended to occur throughout the university in many forms, at FOEs and FOAs, ESP centers, etc. To assist all of these providers, the initial focus was on individual capacity-building. Needs analysis and materials development became priority topics. After the first year working with the five autonomous centers, IELP-II learned of these centers’ interests in approaching the EOP market and subsequent efforts have included a greater focus on joint activities with the EOP component on issues such as marketing and the role of managers in taking advantage of teachers’ skills. In some centers (Mansoura, for example), ESP centers had to compete internally with the FOEs for university clients like the Faculties of Medicine, so these skills were equally relevant [Ghada]. In the second year (IELP Year 3) five new centers were added. Among the trainings offered thus far in the ESP component and that target KSA are: an ESP Institute in 1999; and an ESP Institute for Novice Teachers and ESP Summer Institute for Experienced Teachers in Testing and Evaluation in 2000. Additionally a US￾based program in 2000 that involved both ESP and EOP practitioners (and included pre and post activities) was offered for ESP trainers in Course Design and Materials Development. To encourage institutional strengthening, participants had to commit to PART I: IELP-II Aguirre International p. 27 being trainers for their institutions on their return, and managers had to acknowledge this future role. Four top achievers on their return from the US (three from the ESP centers at Mansoura, Helwan and Beni Sweif) worked on the development of an ESP course that trained 18 novices. Other participants followed up with five local teacher education seminars in IELP Year 4. Current activities offered in coordination with EOP (one observed: the ESP/EOP Partner Days) have continued to reinforce topics related to strategic planning, greater management involvement in monitoring and evaluation activities, customer services and improved marketing. Some high quality materials and resources have been distributed as part of these sessions. Activity evaluations for the ESP component of the project are limited, but those interviewed as part of the evaluation noted personal improvements in their abilities to use various teaching methodologies, tailor an ESP course, adapt materials, and develop tests. These skills are contributing to the centers for which they work. Some participants (even those that said the content was “nothing new” for several events they attended) highlighted the practical aspects of several of the IELP courses with their focus on project-based approaches, development of an actual course during the trainings, and mini-lessons with co-teaching opportunities. One interviewee highlighted the valuable practical experience she had gained in presenting a paper to her peers. 6. ESP: Recommendations · Use the RFA process to continue building ESP capacity at the university level. Set an invitational priority for universities wishing to provide TA services in ESP. Invite faculty to develop services in a number of ESP areas, including tools for needs analysis, course development in high demand areas, hosting of local conferences and network meetings, quality standards; participant assessment and course evaluation; and marketing to internal and external clients. IELP can provide technical assistance in any one of these areas on an as-needed basis. 7. EOP: Findings What is the prognosis for this activity, given the slow pace thus far? Have original strategies been substantially altered? Have new programs and strategies been initiated? Have proposed programs or strategies been altered or eliminated? Have the new programs added value? If there are significant changes, has there been a rationale for them? Have the changes contributed to program objectives? How useful might the model of offering technical assistance and capacity building of private sector trainers for public and private sector targets be in other activities, such as in school-to-work and workforce development? To what extent could a future IELP￾III expand this private sector model? The original plan for strengthening the EOP sector had to be significantly revised since the original project proposal was based on a false picture of the sector. It assumed a larger number of possible EOP centers and assumed their need for targeted institutional PART I: IELP-II p. 28 Aguirre International assistance. In reality, there were only a few centers of this nature, so early on even the definition of “EOP center” needed to be reconsidered since it was not clear if a center meant a self-contained institutions or if it could include in-house training departments in industry or a loose group of free-lance teachers working with various clients (or trying to). The original project design also assumed a more homogenous group of training providers when, in fact, they tended to fall in two main groups: high-powered providers either international or locally-run with strong experience and good client connections, or small providers whose staff included a manager, with perhaps one full-time teacher and another 5-6 teachers on call. Changes were thus made to work with a more diverse group and replace the notion of a center with that of an EOP service provider. EOP in Egypt faces another daunting challenge. As an IELP-II survey of 200 exporters suggests, English for Occupational Purposes is not seen as a priority for industry and there is a limited willingness by Egyptian businesses to purchase EOP services.2 Recent moves to hold sector-specific business lunches with providers and clients show promise in lowering some of these barriers. The term EOP providers has expanded to include a wide range of audiences, not only private sector providers, but universities, ESP institutes, and in-company training units. Geographically, the focus has moved from an initial emphasis on Cairo to include Alexandria. The geographic focus now includes providers in Ismailia, Port Said, and Suez. At the time of our visit, newer investigations were being conducted in Upper Egypt with businesses such as tourism and banking. Overall, the project has incorporated a greater focus on strategic planning, small business training, evaluation and marketing. The result of a recent EOP consultant report has been heeded.3 Given the infancy of EOP in Egypt, the approach to explore strategies for partnerships between the private sector and ESP providers has been appropriate. Full scale success (as defined by a wide variety of ESP services being purchased by industry), however, seems elusive given the reluctance of industry to hire consultants. It may be necessary to take a half-step back and allow individual centers and providers to build their capacity while waiting for industry to catch up. In the end, helping private companies and quasi￾governmental agencies to see the need for EOP services and convincing them that English language training for their employees is a sound investment may require a marketing campaign that is outside of the scope of a single project such as IELP-II. 2 This is not unusual or limited to Egypt. A similar picture exists in the United States where business are reluctant to spend money on building the English skills of their immigrant workers. There is, however, some investment in building the foreign language skills of employees slated to go overseas, although in most cases, large scale experienced providers, such as Berlitz are selected to provide this service. 3 The report offered guidance to the program suggesting their focus on a) the management skills of a select group of EOP providers, b) the development of a network of qualified and experienced EOP teachers, c) the provision of business materials adapted to Egyptian business needs, and d) efforts to foster client interest in EOP services and better define their own needs. PART I: IELP-II Aguirre International p. 29 8. EOP: Recommendations · Use the RFA process to further build the capacity of providers to offer EOP services. Allow university centers, private industry and private providers to compete by mapping out a plan for the development of tools and services that build their own capacity while offering important strategies to other providers. Set an “invitational priority” for building a partnership. Continue providing technical assistance in the EOP area by strengthening Centers such as the Alex Pilot Reform Project and helping to disseminate findings. · Discontinue the current strategy of working with individual providers and consider the adoption of a strategy for EOP that is closer to the ESP/EAP model. · Work with USAID to help sponsor an international conference on the role of EFL in in-country employment and international trade. Offer various strands focused on workplace English, EOP/ESP, and school-to-work. Allow promising practices and new ideas to emerge from such an exchange and use these ideas to shape follow-on projects. 9. ESP and EOP: Recommendations · During the next two years, IELP-II should set up an internet-based “Virtual Clearinghouse for ESP and EOP in Egypt.” A website should be designed that contains all the ESP/EOP tools developed so far, links to other sites that provide pertinent information, and a listing of providers along with their specialty. Such a website should also include a page designed specifically for potential clients where they can access information on why an investment in EOP makes sense, what to look for in a potential client, how to judge the quality of a proposal, and what outcomes can reasonable be expected from an EOP course. IELP-II should also prepare a series of Digests or Fact Sheets that summarize key points to consider in key areas (e.g., needs analysis, outreach and marketing; quality indicators, elements of effective programs, etc). Promising Practices can be highlighted in specific areas (such as partnership building or innovative use of technology), and new collaborations can be featured in a case study section. Training pieces could be included through streaming video and an interactive Question and Answer period could provide important information to both potential providers and potential clients. 10. Testing Reform: Findings How well is the capacity of the National Center for Examination and Educational Evaluation being strengthened? What effect will the low number of individuals involved in the test reform activities have on the likelihood that the Ministry and NCEEE will buy into the reforms? What kind of challenges are they facing and how can they be surmounted? PART I: IELP-II p. 30 Aguirre International IELP-II has made significant inroads into “Testing Reform” by laying the foundation upon which both the NCEEE and local governates can build. The project’s data bank for testing the proficiency of teachers (through the Saqara test) can serve as a basis for the professionalization of the teaching force, since it allows for the setting and assessment of minimal competencies for teachers at various levels (primary, prep, secondary). If linked with the MIS system being developed by IELP-II, such teacher proficiency testing can provide a rich picture of current abilities while identifying both the need for language training and documenting the relative success of such training. In working with teachers and supervisors around issues of learner assessment, the project has built awareness in participants of validity and reliability and other elements of sound testing. There are some indications that at least some supervisors are heeding the call and are developing school-based tests that are more closely in line with the principles of language testing. As we examined the array of testing activities being undertaken, two issues worried us: · We saw little movement at NCEEE or at the MOE that would give us confidence that Egypt is making a strong commitment to move testing reform from discussion to action. This includes both the proficiency testing of new teachers through the proposed Saqara test and changes in the testing of students. As we know from other countries, testing reform is best accomplished in conjunction with learning standards and curriculum reform. We saw no indications that large scale educational reforms were being considered. · As mentioned previously, we were concerned about the low English proficiency of the current teaching force. We had also heard reports from pre-service faculty that many of the graduating teachers had only marginal English skills (as a rule, universities do not test the proficiency of either incoming or graduating teachers). Given their current levels of English proficiency, it is likely that a large percentage of new teachers will not be able to pass the Saqara, in terms of reaching the minimal threshold levels that are being established. Discussions with MOE and NCEEE are needed to assure that there will be an ongoing flow of teachers into the system while the current teacher training system is being upgraded to ensure that all graduating teachers will be fully proficient in English. 11. Testing Reform: Recommendations · During the rest of Year 4, continue to work with both MOE and NCEEE. Provide agencies with background information on ways to link student performance standards (what students should know and be able to do at various levels) with student assessment. Continue to participate in the development of a strategic plan that links curriculum standards (what should be taught and learned) with testing standards. Continue to foster collaboration between NCEEE and MOE teams around testing reform. PART I: IELP-II Aguirre International p. 31 · Set an end goal for participation in testing reform by the end of Year 4. Link continued involvement in NCEEE testing reform to indicators that plans are moving from theory to practice. Outline the support that will be triggered if certain milestones in reform are being achieved (such as a mandate from the MOE to develop new student achievement tests). At the university or MOE levels, wait for a commitment from these institutions to use the Saqara with incoming students or GDIST trainees. · As teacher proficiency testing is discussed seriously, work with the Ministry of Education, FOEs and FOAs, USAID, and with other donors. Outline a menu of options that can be used to upgrade the skills of teachers currently in the system. Outline a system of incentives and rewards (such as stipends and eventual salary increases) to be provided teachers willing to upgrade their skills and be tested and certified. Consider intensive English training (about 3 months) either in-country or overseas as one of the options for skills upgrading. Make it clear that the training of non-specialists who were reassigned as English teachers was considered a stop-gap measure and that USAID can no longer afford to support this practice. · Consider using the Saqara Test internally as a way to assess the proficiency of trainers. Carry out all necessary work (such as testing of item difficulty) so that a pilot test can be field tested by the end of the year with all future IELP participants currently being tested by SLEP. Seek a sample of teachers willing to be tested with both the SLEP and the Saqara test so that results can be compared and feasibility of using the test can be established. Convene a panel of experts and seek advice on using the test system-wide. · Seek the collaboration of universities and of the MOE in continued field testing of the Saqara. Using the competitive process, seek involvement of FOEs and FOAs willing to field test the proficiency test with their teachers in training. Work with GDIST to field test the Saqara with the trainers who will conduct IELP-developed courses (BELI, etc.). · Continue to build the capacity of teachers and supervisors to recognize and develop sound student achievement tests in English. Consider the development of a handbook, similar to the SPEER handbook for teaching EFL as a marketing tool to help teachers, students and parents become aware of what it means to “speak English” and how such proficiency can best be assessed. · Integrate strategies for classroom-based learner assessment into current TOT training. Allow teachers to see how what is taught can be aligned with what is being tested. Focus particularly on ways of low-burden ways of assessing speaking and listening skills. Model the use of Can-Do lists for self-assessment and other checklists built on observations of proficiency to build awareness of low-burden alternative assessments that provide insights into who is learning what and who is having difficulties. PART I: IELP-II p. 32 Aguirre International · Provide opportunities for a core of potential testing experts to be trained in language testing. Include possibilities for both short-term programs (one or two courses), as well as Master’s levels programs. · Work with USAID to sponsor an international conference that addresses issues of educational reform and testing reform. Offer a strand that looks at reforms in English language teaching and testing. Such a conference can make links between different disciplines apparent (current testing reforms in math may have features similar to those in foreign language teaching since both tend to be focused on the application of skills). PART I: IELP-II Aguirre International p. 33 III. FINAL RECOMMENDATIONS In addition to the recommendations specific to each component, the following suggestions should be considered. · Operationalize the concepts that form the basis for current work. Explain what is meant by “qualified teachers” and “communicative language teaching” so that success can be more easily evaluated. · Work with appropriate groups to identify standards for training, teaching and testing in the teacher education area, with a special focus on EFL. Start by developing outcome standards and identify what trainers and teachers “should know and be able to do” as a result of IELP-II-sponsored activities. Link evaluation of impacts to these performance standards. · Use language learning and teaching standards as a basis for selecting and designing appropriate training. Expand the use of in-country training, offering different models for novice and for experienced teachers (since their needs differ significantly). · Focus on sustainability through the development of strategic plans and the design of a comprehensive training models for GDIST and the INSET Centers. Use the result of the Evaluation Study currently underway to focus these efforts. As part of this process, stress the establishment of performance indicators for these agencies and link continued support and technical assistance to these indicators. · Expand the evaluation of impacts at various levels and use information to pinpoint areas where additional training is needed. Share information with partners and insist that proposals submitted through the RFA process discuss performance objectives and expected outcomes. Continue the involvement of supervisors and inspectors in the evaluation effort. Develop classroom observation tools and teacher feedback forms. Field test these tools widely to assure maximum use within the system. · Continue development of an MIS system and strongly encourage its use by all agencies supported by IELP-II. The system can and should be used for program planning, participant tracking, monitoring of training, and evaluation of outcomes and impacts. Provide training in the use of the system for MOE and for FOEs and FOAs, as appropriate. PART I: IELP-II p. 34 Aguirre International IV. FOLLOW-ON ACTIVITY RECOMMENDATIONS Design Issues · Any follow-on project should be dependent on commitment from agencies in Egypt to share responsibilities in making significant changes in the teacher education system. Follow-up projects should focus on ways of designing and sustaining a system that works on all levels and institutional performance improvements should be the ultimate goal. A technical assistance project of this sort should remain separate and not be subsumed under DT2. A name change might be appropriate.4 · A focus on impacts (rather than outputs) should be continued, including both short- term outcomes and expected longer-term results. Sufficient resources must to be available for data collection and analysis. Partners should agree to share necessary data or facilitate independent data collection (access might be an issue). · There should be a extensive period of time dedicated to deciding what shape a follow-on project might take, so that final efforts make a significant difference. Other USAID contractors, donors and organizations working with the same partners should be involved in building the vision, setting goals and identifying expected impacts.5 USAID should take the lead in bringing together this group. · USAID should consider sponsoring a series of conferences around the issues raised by the group. These might include technology integration; testing reform; curriculum reform and student performance standards; building a workforce that is competitive in a global market; and developing a system that builds world class foreign-language skills. It is important to situate English language teaching and learning within a broader context of reforms in education and training. · Decisions must be made in terms of the directions that a new project should take. A feasibility study might be commissioned to determine to what extent large scale changes are likely to happen. If there appears to be little commitment to system-wide change on the partner side, efforts should move in the direction of smaller, more focused projects, such as pilot programs for technology-mediated language learning, public/private collaborations around workplace literacy (with a focus on EFL), or a series of lab schools. 4 One individual suggested “STEP” (Sustainable Teacher Education Program). 5 These might include the World Bank/European Union, JICA, the British Council, AUC, CDC, MES, or Longman’s Publishing, all connected to USAID efforts. PART I: IELP-II Aguirre International p. 35 Contracting Issues · Future contracts should involve all partners at the initial project design stage. Partner roles and responsibilities should be clearly defined from the outset. Partners should be actively involved and responsible for developing strategies and work plans, along with implementation of such plans. It should be clear to all that technical assistance will be focused, rather than broad-based or overly general, on helping partners achieve these plans. TA will be triggered (or can be accessed) as certain milestones are met. · There needs to be sufficient flexibility in the original design to examine the practicality of the proposed model. A follow-on project should use a phased in approach that starts with a planning stage to test assumptions by a small team. This team should work with partners to develop a multi-year strategy and a two-year implementation plan. This plan will need to be discussed with USAID and approved before final staff hiring is carried out. · Even in the final design, objectives need to be sufficiently open to allow contractors to respond to needs as they arise and make changes as appropriate. · USAID should consider issuing “performance-based contracts” that link payments to submission of deliverables and achievement of milestones by all partners. Monitoring and Evaluation Issues · A “participatory evaluation model” should be considered. Such a model allows for input from all stakeholders during the design stage, along with feedback on all stages of the implementation. In this model, contractors and partners would work with USAID to set priorities, develop the statement of work, and select evaluators. Findings and recommendations are discussed with all partners and changes to be made are jointly agreed upon before the evaluation report is finalized. PART I: IELP-II p. 36 Aguirre International V. CONCLUSION An effective teacher training system can serve as a model for efforts to improve practices and reform the system. The collaborative models that bring together MOE and Faculties of Education can be transferred not only to the teaching of other languages but across disciplines as well. Sound tests, designed to assess the proficiency of English language teachers, can be adapted to other languages. Establishing pilot schools for technology and designing school sites to act as laboratories for teaching practice and research offer broad-based implications as well. Efforts to interest employers in upgrading the skills of their workers can be extended to other areas of workforce development (e.g., computer training; international business relations). Similarly, the processes associated with ESP and EOP, such as needs analysis, materials design, and evaluation of participant outcomes, will serve as examples for other employment-related projects, such as “school to work.” The successes that IELP-II has experienced, the challenges that the project has faced, and the strategies that have been used to overcome barriers deserve discussion among a wide group of stakeholders. Given the resources that have been spent on this effort, it is important to share lessons learned so that others involved in similar efforts inside and outside of Egypt can benefit as well. ACCESS TO ENGLISH Part II An Evaluation of the English Language Testing and Training Program (ELTT) PART II: ELTT p. 38 Aguirre International PART II: ELTT Aguirre International p. 39 I. OVERVIEW The American University in Cairo (AUC) Center for Adult and Continuing Education (CACE) has provided English Language Testing and Training (ELTT) services to USAID since 1989 under two successive contracts. Two groups of clients are served by the training program: prospective participants for technical or academic training in the U.S. or third countries; and “counterparts” (personnel in Egyptian Ministries who need to improve their English language skills in order to accomplish their work more effectively). Under the current contract, which was awarded on March 14,1997, AUC also maintains USAID’s English language proficiency testing program using the American Language Institute/ Georgetown University (ALI/GU) English Proficiency Test (EPT) and Communicative English Proficiency Assessment (CEPA). AUC/ELTT will complete the first of three option years on March 13, 2001. Study Methods The assessments and recommendations made in this report are based on a range of activities. An examination of all AUC/ELTT program facilities was conducted including the Open-Access Center, Computer Lab, classrooms, test storage facilities, testing room, and program resources including print, audio-video, and on-line instructional materials. In addition, the consultants held frequent, in-depth talks at AUC with the ELTT Program Manager, Director of Courses, and Director of Testing. Discussions were also held with the ELTT faculty and a group of participants currently enrolled in classes. The consultants observed three General English language classes (01A, 01B, 02B), one Auxiliary Course (English for Workplace Communication), and a Computer Assisted Language Learning (CALL) class in the Computer Lab. Two CEPA test administrations were observed, one at AUC and one in Minya. The consultants also visited an AUC/ELTT sub-contracted English program in Minya for participants in USAID’s Master Teacher Exchange Program (MTEP). Discussions in Minya were held with the local English language teachers and with participants currently enrolled in classes. Training facilities were also visited. In addition to briefings with COTR and James van den Bos, a meeting was arranged for USAID project officers and representatives from contractors to discuss their concerns related to the English language services provided by AUC. AUC’s Program Manager and Director of Courses also attended this meeting. Additional meetings were held with customers of training and testing services including representatives of IELP-II and the Egypt Environmental Affairs Agency. PART II: ELTT p. 40 Aguirre International II. OVERALL FINDINGS A. ENGLISH LANGUAGE TRAINING PROGRAM An examination of program documents, program facilities, and class observation reveals that the ELTT contractor is in full compliance with the terms of its contract with USAID. Specific areas of examination as requested in the Scope of Work follow. 1. EFFECTIVE AND TIMELY PREPARATION OF INDIVIDUALS FOR TRAINING For prospective participants, the six-level general English program is founded on a communicative curriculum with performance objectives attainable in the 12 weeks allotted for each level. This is born out by the fact that the vast majority of participants who complete the fourth level (2-B) achieve the required CEPA score required for short￾term training programs in English. Trainees who begin at Level 01-A will need 48 weeks of language training. All stakeholders need to consider whether it is cost effective to admit prospective participants at this level recognizing the length of time needed to reach call-forward. Consideration should be given to using other facilities for basic English language training, and selecting participants at higher proficiency levels for training at AUC. 2. MEETING THE NEEDS OF DIFFERENT CATEGORIES OF CUSTOMERS For counterparts, AUC has provided a variety of courses to meet general and specific English language needs of GOE personnel. English for Workplace Communication is appropriate for personnel from different Ministries who require English language skills to function more effectively in their jobs, both in their oral and written communication in English. In addition, if funding and scheduling allow, AUC has shown its ability to respond to specific needs of a particular workplace. English for Museums and Report Writing are two examples of ESP courses that AUC has offered. The current AUC core curriculum for participants is highly effective in preparing participants for training abroad. The teaching staff, management, instructional facilities including a Computer Assisted Language Learning Lab very likely make it one of the best facilities, if not the best, facility for ELT in Egypt. When AUC is asked to provide English for Specific Purposes (ESP) courses, it is able to do so. Yet this places a burden on staff and management. Consideration should be given to utilizing other training facilities in Cairo for such counterpart ELT. 3. FEEDBACK TO PROJECT OFFICERS Upon enrollment, AUC currently informs project officers of the placement of trainees in classes. Mid-way through the term, a Mid-Course Progress Report is sent to the project PART II: ELTT Aguirre International p. 41 officers with projections on whether the trainee will likely pass the course. An End of Course Evaluation Report is sent. This report indicates whether the trainee passed the course. Comments are also provided on motivation and general strengths and weaknesses. Information on the extent to which the trainee achieved the performance objectives for the course is lacking, however. AUC needs to provide more specific feedback to project officers on the language capabilities of the trainees in terms of performance criteria. Feedback of this type, however, will be meaningful only if the trainees’ future technical training plans are known. 4. SYSTEMS FOR COTR TO MONITOR IN-COUNTRY ENGLISH LANGUAGE PROGRAMS The current contract requires AUC to submit quarterly reports and annual reports, which summarize the year’s training and testing activities. In addition, the COTR is in frequent communication with the Director of the AUC/ELTT by phone, fax, and E-mail. These resources appear sufficient to enable the COTR to monitor the in-country English language program effectively. 5. PROGRAM FLEXIBILITY During the term of this contract, AUC has demonstrated its ability to offer a variety of courses not only in content but also in the venue of the training. At AUC/Zamalek, in addition to the general English courses, the program has offered a variety of ESP courses and Auxiliary Courses for counterparts. In addition, they are overseeing, through a sub￾contract with AMIDEAST, English training programs in Minya, Beni Swef, and Fayoum for participants in the Master Teacher Exchange Program (MTEP). Requests are often made for counterpart ELT to be offered at the Ministries. Offering a program off-campus has advantages and disadvantages for the trainees. The obvious advantage is convenience. A disadvantage is that they do not have use of AUC facilities such as the Computer Assisted Language Learning Lab and the Self-Access Center. Nevertheless, AUC has offered courses at various Ministries in Cairo including a current course at the Ministry of Planning. Regarding the balance between counterpart and participant training, the current contract states only that “a minimum of 10 courses, ELT and ESP, shall be offered each quarterly session.” It does appear, however, that the focus of the program is on participant ELT. The core curriculum includes not only English language skills training but also considerable cultural orientation, and this implies preparation for training in the U.S. Concerning the possibility of AUC’s offering the same courses to both prospective participants and counterparts, while some of the performance objectives will be the same, the overall goals of the two groups are different. There is clearly a growing need for professionals in Egypt to reach a level of English proficiency that will give them access PART II: ELTT p. 42 Aguirre International to both technical information available only in English and the general communication skills they will need to function more effectively in their work. 6. PROGRAM HARDWARE AND SOFTWARE AUC is now at the end of its first option year of the contract. The computers all run on Windows 95 and have 32 megabytes of RAM. One of the computers has a damaged hard disk. The audiocassettes have been heavily used in the four years of the program. Headphones for the CALL Lab are worn out. As a result, up-to-date software cannot be used in the program. Funding needs to be earmarked to upgrade the hardware and software not only for the CALL labs but also for the computers used by management, staff, and faculty. 7. SUPERVISORY SUPPORT FOR ELT AND PARTICIPANTS’ COMMITMENT TO THE ENGLISH TRAINING PROGRAM Levels of supervisory support for participants’ language training vary from Ministry to Ministry and supervisor to supervisor. Some supervisors may place little value on, or not be aware of, the demands of an intensive language program. In other cases, such the MTEP, a Ministry may give participants leave to focus fully on their language studies. In the majority of cases, supervisors support language instruction but understandably place work demands above studies. The level of a supervisor’s support clearly impacts on the commitment of participants to their studies. Practical issues, such as the need to commute to Zamalek from their workplace and the cost involved, may also affect their commitment to ELT. In some organizations, trainees undertaking classes arranged by their employer are required to pay a nominal tuition. If the trainees do not complete, or fail the course, they may be required to pay the tuition in full. In order for ELT to be successful, there needs to be a commitment to the training on the part of the supervisors as well as the trainees. The commitment must start with the supervisors who must communicate to the trainee the purpose of the training. With strong support from supervisors, trainees will more likely commit themselves to the language training. PART II: ELTT Aguirre International p. 43 B. ENGLISH LANGUAGE TESTING PROGRAM AUC is meeting the highest standards for maintaining a secure and valid testing program. All testing materials are kept in double-locked cabinets in a locked room. The only personnel with keys to the testing storage room and cabinets are the Program Director and the Testing Specialist. Test records are kept on a database that allows the Testing Specialist to produce a variety of reports on the CEPA and ALI/GU EPT. The consultants observed two CEPA administrations. The tests were administered under secure conditions. The testing program is most secure with AUC and should not be moved to another facility as long as USAID requires English language proficiency testing. PART II: ELTT p. 44 Aguirre International C. ENGLISH LANGUAGE TRAINING AND TESTING CONCERNS OF STAKEHOLDERS 1. ENGLISH FOR PROFESSIONAL PURPOSES As Egypt plays an ever-increasing role in the global economy, demands for English for professional purposes are likely to continue. There will be increasing demands for customized training. Increasingly, local Egyptian providers will be able to meet these needs (the ESP Center in Alexandria, for example seems well equipped to do so). AUC will most likely be the “Cadillac” of providers for courses that focus on workplace communication and should be able to make a good case for its services. Given the anticipated limited resources in the future and the need to streamline services, auxiliary courses might best be moved and become part of a general training system that addresses the overall needs of the Egyptian workforce, not just those related to English. 2. ALI/GU EPT AND CEPA There are major concerns that the current cut-off scores for CEPA and the ALI/GU EPT are too rigid and inflexible. Clients are not always clear as to what the testing options are and what is permissible under the current waiver policies. Furthermore, there is concern that the CEPA poses difficulty because of the question types that are often unfamiliar in the Egyptian context. Criteria need to be developed to allow a wider range of participants to move into the “call forward” category while still assuring that participants have the English skills necessary to fully benefit from the USAID-funded overseas training/professional development experience. Participants who sit for the CEPA must have a thorough understanding of the nature of the test and the strategies that will help them achieve the highest score for their proficiency level. CEPA preparation materials already exist. Training officers must make a greater effort to disseminate these materials to their staff. In addition, AUC can assist by preparing fact sheets for distribution. 3. ENGLISH LANGUAGE TRAINING OUTSIDE OF CAIRO There is a need to provide training to outlying areas beyond Cairo. The current contract and budget constraints limit AUC’s ability to respond to requests for services if the number of potential clients is limited. Alternatives should be considered if access to training is considered a priority need and more flexibility in terms of numbers may need to be adjusted. AUC and other institutions can be encouraged to develop distance learning courses now that computers with Internet access are becoming increasingly available throughout the country. PART II: ELTT Aguirre International p. 45 D. CONCERNS OF CONSULTANTS 1. FOCUS OF ENGLISH LANGUAGE TRAINING The focus of English language training and testing should be on the knowledge, skills, and abilities needed if participants are to fully benefit from the professional or academic experience provided in the United States. Clear training plans for prospective participants need to be developed and communicated to AUC. This will allow AUC to tailor instruction to individuals through their self-access learning laboratories and through Internet and on-line courses. The language program can then function more effectively in preparing trainees for training abroad. 2. NEED FOR SECURITY OF ALI/GU EPT AND CEPA With no new forms of the CEPA or ALI/GU EPT being produced, the maintenance and security of the testing program is of critical concern. AUC has a proven track record in maintaining the highest standards of test security and maintenance. We feel strongly that USAID should not take the risk of moving the testing program to another contractor. If only one test booklet is compromised, the entire testing program will be seriously jeopardized. 3. QUALIFICATIONS FOR EFL TEACHERS FOR PROGRAMS OUTSIDE OF CAIRO Under the current contract, the core faculty in the AUC program must be native speakers of English, have Master’s degrees, and have at least three years of teaching experience. If English language training is to be offered outside of Cairo, it will be difficult to find teachers who meet these criteria. Teachers may have to be recruited who are non-native speakers of English and who have experience and training but perhaps not a Master’s degree. PART II: ELTT p. 46 Aguirre International III. RECOMMENDATIONS A. SHORT-TERM RECOMMENDATIONS THAT CAN BE IMPLEMENTED WITHIN DT-2 1. PROMOTE COMMON UNDERSTANDING BETWEEN AUC AND ITS CLIENTS REGARDING THE ENGLISH SKILLS AND PROFICIENCY LEVELS NEEDED FOR TRAINING PROGRAMS There is a need for two or three different models that take into account (a) the benefits a group is expected to derive from the overseas training, (b) differences in participant groups (learner profiles), (c) the extent to which the host program requires high levels of English language competence if participants are to gain the knowledge and skills they are expected to attain, and (d) the functional English and cultural awareness necessary if participants are to thrive in a U.S. environment. For example, AUC and its clients should list the kinds out outcomes that certain types of programs can provide, e.g., greater understanding of the relationship between economic development and adult literacy; ability to design, implement and manage large scale data￾bases; knowledge of models commonly used in English as a Foreign Language Teaching and ability to implement them in the classroom. Similarly, AUC and its clients would develop profiles of the types of clients who seek overseas training or have been recommended for such training. These clients might include those who need a general overview of a subject, those who need hands-on skill development with a technical emphasis and those who are expected to effect change in administrative or policy levels. Such profiles would also include information on the kinds of training the applicant has completed in English and what functional English proficiency the person possesses. Thirdly, each course or program offered should indicate what English abilities are required to benefit from training. This might include the ability to glean information from academic lectures vs. the ability to follow hands-on instruction on a computer along with a sound knowledge of the underlying technical concepts and the terms used in English to express these concepts. And finally, host programs should indicate to what extent bilingual support will be available and to what extent participants are expected to negotiate the training program and the English speaking community on their own, to provide a better sense of the extent of the pre-departure and cultural awareness training is necessary. U.S.-based training providers should provide specific information on program outcomes (i.e. what participants are expected to take away from the training) and list the type of English competence is required for success. Similarly, they should indicate to what extent participants are expected to negotiate the training and community environment on their own and in English and to what extent bilingual support is provided. PART II: ELTT Aguirre International p. 47 Sending agencies on the other hand, should provide a more detailed profile of the applicant, including desired outcomes and present ability to negotiate training information in English. AUC and its clients can then design a model that matches participant information to program offerings. Such information is best provided in a flow chart or graphic format that can easily be grasped. We suggest the use of an “if-then” model that explains that “if you have this type of employee who has this type of background and is going for this kind of program, then the following training and testing options are available.” This should make it clear who needs to meet requirements for academic English and who would benefit from a program with more hands-on training and support and therefore would meet less stringent testing requirements. AUC should work with clients and other USAID funded contractors to lay out such a model. · The model should be discussed and designed in such a way that it provides optimal information for decision making by the various stakeholders. · The model should also identify alternatives to U.S. training, such as a combination of in-country training and study tours. · The model should be placed on the AUC/ELTT web page and the URL should be included in all communication with stakeholders. Develop fact sheets that summarize information on training and testing options in a bulleted format and distribute them each time a contract is issued or considered. Advertise the AUC web page as a central clearinghouse for all information on ELTT training and link the page to other pertinent web sites. 2. AUC SHOULD COMMUNICATE THE LENGTH AND EXTENT OF ELT NEEDED IF PARTICIPANTS ARE TO DEVELOP THEIR SKILLS SUFFICIENTLY IN THE TIME PROGRAMMED FOR ELT AUC should provide information to clients on student achievement in aggregate form as well as through individual reports. Link student outcomes to performance objectives. Report what individual students “know and are able to do” (in functional terms) after participating in a course, linking student outcomes to performance objectives. In this way, clients can be assured that participants have attained the knowledge, skills, and strategies identified as critical in the performance objectives. AUC should also consider developing or adapting a scale (with rubrics) that describes different levels of proficiency for each skill identified in the syllabus. To capture performance levels, consider engaging students in projects that allow them to demonstrate proficiency in various areas. PART II: ELTT p. 48 Aguirre International 3. ALI/GU EPT AND CEPA CALL-FORWARD SCORES Considering the increasing standards of English language proficiency required by U.S. universities, USAID should not lower the current ALI/GU EPT call-forward scores for academic training. For CEPA, consider implementing a system of conditional waivers for participants who score between 57 and 63. The waiver would depend upon successful completion of a short-term English language course to prepare participants for the language demands of training in English. 4. LENGTH OF TIME CEPA SCORES REMAIN VALID The length of time CEPA scores remain valid should depend on the score itself. We recommend that scores between 65 and 74 should remain valid for 6 months; scores between 75 and 84 should remain valid for 1 year; and scores above 85 should remain valid for 2 years. 5. REPEATED SCHEDULING OF CEPA Since CEPA is a standardized proficiency test, repeated administrations of the test will NOT result in significant score gains unless appropriate English language training is provided between test administrations. Project officers need to be aware of this fact and should be strongly encouraged not to send prospective participants for repeated CEPA administrations without English language training. 6. INCREASE THE INFORMATION AVAILABLE ON THE CEPA In addition to encouraging project officers to disseminate the CEPA Preparation Guide to participants before testing, USAID should request that AUC develop a fact sheet about the CEPA in Arabic that can be faxed to project officers or candidates. USAID should also request that AUC consider translating the Preparation Guide into Arabic and putting it on their web site. 7. UPDATE HARDWARE AND SOFTWARE FOR ELTT USAID should request that AUC upgrade their hardware and software. Specifically, AUC needs to: · purchase 10 industrial quality audio-cassette tape players with counters; · upgrade computers so that they can run Windows 2000 and support the most up-to-date EFL software; · upgrade Windows 95 to Windows 2000 for all computers; · purchase 15 headphones for use in the CALL Lab; · purchase one HP LaserJet printer for the office of the Program Manager; and · purchase a “data show” for presentations in the CALL Lab. PART II: ELTT Aguirre International p. 49 B. LONG-TERM AND FOLLOW-ON RECOMMENDATIONS AFTER THE END OF DT-2 1. ENGLISH LANGUAGE TESTING First, we strongly recommend that English language proficiency testing using the ALI/GU tests should continue under a future contract. These tests remain secure, valid, and reliable. Since they are USAID’s own tests, there is no cost incurred to USAID in administering the tests. Furthermore, if the program continues to be administered outside of the USAID Office, we strongly recommend that the contract be given to AUC. AUC has demonstrated its capacity to maintain the program according to the strictest standards of security thus ensuring that the tests remain valid and reliable. Moving the program to another site after the end of this contract would jeopardize the integrity of the program. Since the USAID testing program is now a “closed system,” i.e., no new tests are being developed, it is essential that every effort be exerted to maintain the security of the tests. 2. SEPARATING PARTICIPANT AND COUNTERPART ENGLISH LANGUAGE TRAINING · Separate English Language Training for Counterparts (auxiliary courses) from English Language Training for Participants who are slated for training in the United States. · Integrate all English language training for professionals in Egypt with other training activities offered under DT-3. · Allow providers who meet quality standards to compete. Consider a “voucher system” that provides agencies with training funds to be used for USAID-supported English Language Training courses. Agencies should be allowed to supplement these funds with their own moneys if they desire customized courses or seek courses that go beyond the levels funded by USAID. 3. DISTANCE-LEARNING COURSES Under a future DT-3 contract, the RFP should request a plan for developing and implementing distance learning courses for use both in and outside of Cairo. Such courses are best provided by universities that can set up courses within the university’s web site. Participants could “register” as students, be given ID numbers, and then have access to the variety of material and activities available only through software programs such as Blackboard or Course Info. We would not exclude non-university institutions; however, such institutions would have to demonstrate in detail how they would design and implement distance learning. PART II: ELTT p. 50 Aguirre International 4. QUALIFICATIONS OF EFL INSTRUCTORS Regardless of whether future ELT for participants is separated from training for counterparts, all teachers employed must be trained. We recommend, however, that other training options besides an MA in TEFL be considered, such as a Cambridge/RSA Certificate or Diploma. In addition, it is our belief that trained non-native speaker EFL teachers are preferred to non-trained native speakers. The regulation requiring native English speaker teachers should be eliminated from a future contract. 5. DECENTRALIZATION OF ELT UNDER DT For a Follow-on project, the Mission may want to consider incorporating English language training (ELT) under a general DT-3 Project. The following considerations should be taken into account before such a decision is made: · Will such an arrangement reduce the cost of ELT? · Will such an arrangement be more efficient than the current system? · Can USAID be assured of high-quality ELT if a direct contract is not awarded? · Can a sub-contracted ELT program provide the flexibility that the AUC/ELTT currently provides? 6. CRITERIA FOR SELECTION OF PROVIDER FOR PARTICIPANT ENGLISH LANGUAGE TRAINING The RFP issued for the current ELT contract lays out clearly the criteria for a high￾quality ELT program. The previous RFP, with modifications that take into account the proposed recommendations in this report, should be the basis for a future ELT program. Among the criteria to be considered are: · Communicative curriculum based on performance objectives with a strong U.S. cultural orientation component; base level should equate to AUC’s Level 02-A; · Available valid testing instrument for placing participants into appropriate levels; · Classroom space in the Cairo metropolitan area; · Computer Assisted Language Learning (CALL) Lab with at least 17 positions; · Self Access Resource room for trainees; · Resource room for teachers; and · Key staff include a Program Director with a minimum of a Master’s degree in TEFL or a related field, a minimum of 5 years of administrative experience in an EFL academic setting, and a minimum of 7 years of TEFL experience; a Director of Courses with a Master’s degree in TEFL or a related field and a minimum of 5 years of teaching experience; administrative staff; and faculty with a recognized qualification for teaching EFL (such as the RSA/CELTA certificate) and a minimum of 2 years of teaching experience.