Desk Review and Meta-Evaluation Study DRAFT REPORT March 2010 Report prepared for the United States Agency for International Development (USAID), Egypt Mission, Education Office, by the Aguirre Division of JBS International, Inc., under the Global Evaluation and Monitoring II Contract EDH-E-23-08-00003-00. Its author is Lynn Losert. Desk Review and Meta-Evaluation Study March 2010 The authors’ views expressed in this publication do not necessarily reflect the views of the United States Agency for International Development or the United States Government Contents List of Acronyms ................................................................................................................................................iv Executive Summary ............................................................................................................................................. v 1 Introduction and Purpose .......................................................................................................................ix 1.1 Purpose of the Study ..............................................................................................................................ix 1.2 Methodology .............................................................................................................................................x 1.3 Structure of the Paper ...........................................................................................................................xi 2 Background: Education in Egypt .............................................................................................................1 2.1 Context ......................................................................................................................................................1 2.2 Situation Analysis .....................................................................................................................................1 A. Access ..................................................................................................................................................1 B. Equity ...................................................................................................................................................2 C. Quality .................................................................................................................................................3 2.3 Decentralization .......................................................................................................................................5 2.4 Government Education Strategy ..........................................................................................................6 3 Review of Projects .....................................................................................................................................7 3.1 Education Reform Program 1 (ERP/ EQP 1) .....................................................................................7 Project Objectives .....................................................................................................................................8 Project Activities and Achievements .....................................................................................................8 Factor Analysis: ........................................................................................................................................ 11 3.2 Education Reform Program II (ERP/ EQP 2) .................................................................................. 12 Project Objectives .................................................................................................................................. 12 Project Activities and Achievements .................................................................................................. 13 Factor Analysis: ........................................................................................................................................ 15 3.3 Girls’ Improved Learning Outcomes (GILO)................................................................................. 16 Project Objectives .................................................................................................................................. 16 Project Activities and Achievements .................................................................................................. 16 Factor Analysis: ........................................................................................................................................ 18 3.4 New School Program (NSP) .............................................................................................................. 19 Project Objectives .................................................................................................................................. 19 Project Activities and Achievements .................................................................................................. 19 Factor Analysis: ........................................................................................................................................ 21 Egypt Monitoring and Evaluation Program Aguirre Division of JBS International, Inc. ii 3.5 School Team Excellence Awards Program (STEAP) .................................................................... 22 Project Objectives .................................................................................................................................. 22 Project Activities and Achievements .................................................................................................. 23 Factor Analysis: ........................................................................................................................................ 23 3.6 Technology for Improved Learning Outcomes (TILO) ............................................................... 25 Project Objectives .................................................................................................................................. 25 Activity Activities and Achievements ................................................................................................. 25 Factor Analysis: ........................................................................................................................................ 27 4 Review of Instruments ........................................................................................................................... 28 4.1 Critical Thinking, Achievement and Problem-solving (CAPS) ............................................. 28 4.2 Management Assessment Protocol (MAP) .............................................................................. 29 4.3 Standards-based Classroom Observation Protocol for Egypt (SCOPE) .......................... 30 5 Synthesis .................................................................................................................................................... 31 5.1 Decentralization .................................................................................................................................... 31 5.2 Gender Equity ........................................................................................................................................ 33 5.3 Teacher Training ................................................................................................................................... 34 5.4 Community Development .................................................................................................................. 36 5.5 Curriculum Development ................................................................................................................... 37 5.6 Policy Reform ........................................................................................................................................ 38 6 Conclusions and Recommendation .................................................................................................... 39 6.1 Conclusions ............................................................................................................................................ 39 6.2 Recommendations for Field Evaluation ........................................................................................... 40 Appendices ......................................................................................................................................................... 42 Appendix 1: Scope of Work ..................................................................................................................... 42 Appendix 2: Current Situation of Education in Egypt ......................................................................... 43 A. Access ............................................................................................................................................... 43 B. Equity ................................................................................................................................................ 44 C. Quality .............................................................................................................................................. 48 Appendix 3: Data for Appendix 1 Figures ............................................................................................. 53 Appendix 4: Additional Project Information in Alphabetical Order ............................................... 55 1. ERP 1 ................................................................................................................................................. 55 2. NSP .................................................................................................................................................... 59 3. STEAP ............................................................................................................................................... 60 Egypt Monitoring and Evaluation Program Aguirre Division of JBS International, Inc. iii Appendix 5: Compilation of Efficiency, Effectiveness, and Sustainability Factors from the Six Programs ........................................................................................................................................................ 61 Appendix 6: Six Component Matrix ....................................................................................................... 68 References ......................................................................................................................................................... 70 By Project ...................................................................................................................................................... 70 By Instrument ............................................................................................................................................... 72 Egypt Monitoring and Evaluation Program Aguirre Division of JBS International, Inc. iv List of Acronyms BOT Board of Trustees CAPS Critical Thinking & Problem Solving tests CAPMAS Central Authority for Public Mobilization and Statistics. CEAP Community Education Action Plan CEC Community Education Committee CET Community Education Team EGRA Early Grade Reading Assessment ERP Education Reform Program (USAID program) GAEB General Authority for Educational Buildings GILO Girls’ Improved Learning Outcomes Project (USAID project) GOE Government of Egypt ICT Information and Communications Technology Idarra District-level administrative unit of the Ministry of Education MAP Management Assessment Protocol M&E Monitoring and Evaluation MOE Ministry of Education NSP New Schools Program (USAID project) PPP Public-Private Partnership (Unit of the Ministry of Finance) SBTEU School-based Training and Evaluation Units SCOPE Standards-based Classroom Observation Protocol for Egypt SIP School Improvement Plan STEAP School Team Excellence Awards Program (USAID program) TILO Technology for Improved Learning Outcomes (USAID project) TIMSS Trends in International Mathematics and Science Study UNICEF United Nations Children’s Education Fund Egypt Monitoring and Evaluation Program Aguirre Division of JBS International, Inc. v Executive Summary The purpose of this study is to assist USAD to monitor and evaluate the current education portfolio in Egypt. This study represents the first of three tasks which in entirety form the Egyptian Monitoring and Evaluation Program (EMEP). This task takes the form of a desk review of literature to look at the current status of education in Egypt and a meta-evaluation of recent/on-going programs in the education portfolio to determine efficiency, effectiveness and sustainability of activities. This meta-evaluation, strictly a document study, will assist to lay the ground-work for Task Two of the EMEP, field-level evaluation in Egypt of the education portfolio, and make recommendations as too what the country-level assessment should take particular notice of. Observations made in this study will be ‘ground-truthed’ in the field. There is a distinct possibility that findings in the field will diverge from conclusions made in this report, understanding the different perceptions gained from a document review and the reality of implementation gained from ground-level observation. As requested by USAID, progress in the six intervention components of decentralization, gender equity, teacher training, community participation, curriculum development, and policy reform will be determined based on the degree to which current education programming has been implemented with efficiency, effectiveness and sustainability. The current education strategy ends in September 2010. Results from this evaluation will inform the design of the next USAID education strategy in Egypt. Summary: Efficiency, Effectiveness and Sustainability Efficiency Practices considered efficient focused on how programming and implementation affected or multiplied the use of program resources. Some examples of these and the programs they came from are: a) the use of school clusters (ERP 1, NSP), b) charging user fees for technology (NSP), c) using methods and materials from previous USAID programs (GILO, TILO), and d) partnerships with private sector resulting in large monetary contributions to reform efforts (STEAP, TILO). Restructuring or modifying activity implementation mid-program was considered to decrease efficiency (ERP 1, STEAP) as was limited intra-program cooperation at logical administrative junctures (ERP 1 and 2). Effectiveness The evaluation of effectiveness identified what mechanisms and methods were developed through implementation to enable intended results to be reached. Some examples of identified effective practices include: a) structures of support for teacher training at the school level (ERP 1, GILO, NSP, TILO), b) activities which increased girls’ education access and attainment (ERP 1, GILO, NSP), c) system and policy changes to guide reform (ERP 2) and d) materials and methods which standardize information use (ERP 1, ERP 2, STEAP, GILO, TILO). Some practices were considered not effective. These included: a) developing processes without input from all levels of stakeholders (ERP 2, STEAP), b) preparing models and materials for national use without prior quality checks and impact evaluations (ERP 1, STEAP), and c) unequally matching program inputs with expected outcomes (ERP 1, STEAP, GILO). Sustainability Sustainability was influenced by appropriate program planning to develop mechanisms which would continue long-term effects after funding support ends. Examples which support sustainability include: a) engagement of school staff, administrators and communities together in Egypt Monitoring and Evaluation Program Aguirre Division of JBS International, Inc. vi participation in school reform (ERP 1, GILO, NSP, STEAP, TILO), b) building capacity of government officials at lower administrative levels (ERP 1, ERP 2, GILO, NSP, STEAP, TILO), c) building of models and methods to support teacher training (NSP, GILO, TILO), and d) supporting school improvement planning to guide reform at the ground level (ERP 1,GILO, STEAP). Practices that were judged to have questionable sustainability were: a) the huge amounts of training required for decentralization efforts without adequate on-the-ground support (ERP 1) and b) building capacity of central ministry officials to work with local branch offices (GILO). Synthesis: The Six Components Decentralization A major strategy of USAID education programming, decentralization has benefitted significantly from the ERP 2 program in system/policy change and other program efforts to support engagement of communities and school staff/officials in the school planning process. Building capacity of government officials in several key ministry departments has also been identified as contributing to decentralization efforts, although continued efforts of these individuals after program support has ended may be an issue. Development of materials and components which aid provision of standardized and accessible information as the basis for decision-making is also seen to contribute to decentralization. The huge amount of training provided at the governorate/idarra juncture seems to have been confusing and may have weakened decentralization efforts at this point. The pace of decentralization seems very fast perhaps affecting stakeholders’ understanding of their roles in the process. Decentralization contributes to education quality improvement when parents are involved in school performance and individual schools have budget authority; therefore it would seem that continued efforts would be beneficial to the sector. Gender Equity This topic has been successfully addressed by three USAID programs. The strategies of school construction in rural areas, giving scholarship, and involving communities have made big contributions to increased school access for girls. The current GILO program could be the model for effective girl-friendly practices for insertion into other programs as a standard practice. The major issue at this point seems to be the need for formative evaluations to assess the issues girls face in education given the large numbers already in the system but the low attainment rates they seem to achieve. Teacher Training This issue has also received major attention from all USAID programming except for the STEAP program. Building structures of support for local, standardized in-service training delivery has been identified as making a valuable contribution to the continued development of improved quality of teachers. Despite major efforts however, improved teacher classroom behavior and subsequent increased student academic achievement have been difficult to discern. As the new teaching methodologies are in counterpoint to the rest of the education system in which a dense curriculum, centralized teacher deployment mechanisms, low salaries, and the competitiveness of the national exams continue to drive rote memorization methodologies, teacher training needs to be presented as one of a holistic array of identified strategies to improve quality. Community Development As part of the decentralization effort, communities have received much support from USAID programming. Formation of Community Education Teams and Boards of Trustees seemed to have offered an opportunity for consolidated leadership and representation in the school reform Egypt Monitoring and Evaluation Program Aguirre Division of JBS International, Inc. vii process, but then the extent of participation of the rest of the community at large was unknown. A couple of programs seemed to have fostered great participation and excitement among community members which is not always apparent in programs which work through Boards of Trustees. There is no clear understanding of just how much community members contributed to the school improvement planning process and if they continue to participate in the monitoring and evaluation of it. Continued support for communities would strengthen their understanding of their roles and willingness to take critical action against the issues which need attention. Private partnerships with schools offer valuable resources, support human resource development, local employment and community building, and promote responsibility for provision of quality education. Curriculum Development USAID programming was seen to offer the least attention to curriculum development. Efforts in this area include content enhancement of life skills, technology and individual curricular subjects, Early Grade Reading development, and technology-assisted education development. Technology in education is a high priority of the Ministry of Education although the positive impact of ICT on education quality has not been proven; therefore indicators of progress and end results should be well-understood and agreed on among stakeholders. Further efforts with the curriculum should be well-coordinated with the Ministry of Education efforts according to a well-defined implementation plan; otherwise efforts will be seen as fragmented and out-of-synch with other initiatives. Policy Reform USAID assistance made major contributions to the policy and system level of reform including development of a policy framework to guide attainment of standardized measures of improved performance of the sector. The National Strategic Plan is in place for system guidance but is very detailed and does not provide practical tools for application at the school level; it would benefit from refinement of a scaled set of practical indicators. The main factors limiting change and implementation of the policy framework may indeed be the MoE officials themselves because of lack of commitment and resistance to change, new laws and decrees that do not have accountability and transparency requirements for implementation, lack of vision and capacity at local levels, and lack of resources and knowledge of procedures. Continued efforts should be defined within the framework on a prioritized schedule in cooperation with the government and other donors. Recommendations for Field Evaluation Based on the desk study and meta-evaluation, the following recommendations are made to guide on-site evaluation of USAID education programs in Egypt: 1. The teacher training initiative and structures should be thoroughly reviewed. Teachers should be questioned/observed as to their ability to use new methodologies. Training structures should be in place and continue to be useful to support teachers in classrooms. USAID has put many resources and efforts into teacher training; to protect their investments, these efforts need to be assessed to guide future programming. Moreover, improved education quality is very dependent on improved quality of teachers. 2. As decentralization has also been a main focus of USAID programming, community-level leadership and participation should receive considerable attention from ground-level evaluations. Assessment should ascertain if Boards of Trustees are in place and Egypt Monitoring and Evaluation Program Aguirre Division of JBS International, Inc. viii effectively coordinating with schools to improve performance. Communities should continue to actively participate in the school improvement process also. 3. School leadership, i.e. administrators and supervisors, should be seen to be actively involved in improving quality at their schools. They should be knowledgeable and participative in the process. The School Improvement Plan should be an actively working document, developed with input from all stakeholders and clearly guiding improvement of the education process in schools and communities. 4. The issue of girls’ education attainment should be given close attention. According to the most current education data available for this report, there seems to be an emerging trend of improved enrollment and attainment for girls which should be checked out on the ground. 5. Education leaders on a system level should be assessed as to their willingness to support changes in the lower levels of the system and moreover, should be knowledgeable of current reform efforts and have an opinion as to what still needs to be done to improve education in the country. Egypt Monitoring and Evaluation Program Aguirre Division of JBS International, Inc. ix 1 Introduction and Purpose USAID has been a committed partner to Egypt in education reform for many years. Current assistance to the Government of Egypt supports sustained improvements in student learning outcomes by a) improving the quality of teaching and learning, b) expanding equitable access to education, and c) strengthening school management and governance. Guided by the strategic framework for foreign assistance under the objective of investing in people, USAID assistance builds capacity of the system to produce students with skills and knowledge necessary to take their place in the 21st century global economy. Egypt has made outstanding progress in increasing access to basic education in the last fifteen year and as a consequence, nearly universal enrollment has been attained in grades one through nine with an almost zero gender gap. The fast pace of enrollments however has negatively impacted the quality of education. Participation of Egyptian eighth graders in the international evaluation, the Trends in International Mathematics and Science Study (TIMSS), revealed a poor performance in 2004 and a decrease in math and science scores in 2008, indicating that despite focused efforts in the sector, quality of education is not improving. The sector continues to be troubled with low pass rates and poor acquisition of basic skills of literacy and math which have translated into low-paying job opportunities and high unemployment rates for graduates. Other identified challenges are: the proliferation of a private tutoring system, an extremely uneven teaching-administrator staff ratio, poorly trained teachers, a complex curriculum, a rigid tracking system at the upper secondary level, and difficulties of rural children to attain education. Through USAID programming, thousands of teachers and school administrators have received training and the Teacher Cadre law which provides a system of professional teacher development has been passed, school construction and scholarships have enabled thousands of children, particularly girls, to enroll in school, and books and technology have been provided. USAID assistance also supports emerging decentralization efforts by building capacity of education officials at all levels and encouraging community engagement in school planning processes. A continued strong partnership with Egypt will enable USAID to focus attention on remaining issues and facilitate the education system to address current political, social, and economic goals of the country. 1.1 Purpose of the Study USAID has requested assistance to monitor and evaluate the current education portfolio of six programs.1 This document represents the first of three tasks requested by USAID/Egypt which in entirety form the Egyptian Monitoring and Evaluation Program (EMEP). Outcomes of the EMEP are: 1 See the Scope of Work in Appendix 1. Egypt Monitoring and Evaluation Program Aguirre Division of JBS International, Inc. x Inform USAID of the efficiency, effectiveness, and sustainability of its education portfolio; Inform USAID of suggested steps for the design of its next education strategy; Improvement of the monitoring and evaluation of the education activities of USAID in Egypt. This first task will take the form of a desk study of recent education assessments to review the current status of education in the country and a meta-evaluation of the six programs to extract and roll-up major results of the portfolio to this point. In this meta￾evaluation, the six recent/on-going education programs will be critically reviewed to determine efficiency, effectiveness, and sustainability of activities. By noting the extent of these factors, insight into the progress of the six intervention components of decentralization, gender equity, teacher training, community participation, curriculum development, and policy reform can be gained. This report sets the foundation for the next step of the EMEP, field work in Egypt to assess ground-level performance of the six education programs in the USAID education portfolio in Egypt. In essence, this report identifies issues that should be the focus of the assessment on the ground. Findings made in this report will be verified by first-hand observation in the field thus allowing a more authentic evaluation of the current performance of the USAID portfolio. With this knowledge, USAID will have a better understanding of how successful recent education programming has been, what strategies have not been so successful, and what still needs to be done in these areas of intervention. This information will assist USAID in the design of its next education strategy in Egypt. A review of the three tests developed with USAID assistance (CAPS, MAP, SCOPE) which are being used as existing program performance indicators is also included in this study. 1.2 Methodology The review of literature for this analysis was gained from a desk study of secondary data which consisted of (a) government education statistics and indicators, (b) seminal research reports and assessments from major donors, and (c) relevant individual and NGO reports. Literature for the meta-evaluation of programs consisted largely of implementer quarterly and final program reports, a few program evaluations, some technical documents from trainings and presentations, web-based information from USAID and implementers, and Egyptian government sector documents. The review of the six education programs includes a brief description and program objectives and strategies. Thereafter, the program activities were listed in a table categorized according to which USAID education strategy they addressed. Achievements in the form of most current programs results/outcomes, if available, were listed in the same table if they were presented in program reports that way. Otherwise, they are presented in separate tables. Thereafter, based on program implementation and achievement of objectives, a determination of efficiency, effectiveness, and sustainability was made. The six programs reviewed for this evaluation are: 1. Education Reform Program 1 (ERP 1), Egypt Monitoring and Evaluation Program Aguirre Division of JBS International, Inc. xi 2. Education Reform Program 2 (ERP 2), 3. Girls’ Improved Leaning Outcomes (GILO), 4. New Schools Program (NSP), 5. School Team Excellence Awards Program (STEAP), 6. Technology for Improved Learning Outcomes (TILO). Analysis of efficiency, effectiveness, and sustainability are based largely on the author’s opinion from her experience and knowledge of education reform issues and tempered by findings in implementers’ reports. However as these reports are self-reported, they could contain some level of bias. The input from other evaluation team members, particularly those of the Egyptian team members, was also very valuable. Actual program evaluations were limited; therefore this analysis was not able to benefit from other objective assessments. For consistency and reliability, the definitions of efficiency, effectiveness and sustainability were taken from the World Bank and used for all programs.2The definitions used for this study are: Efficiency: Is a measure of how economically resources are converted to results. Resources can refer to funds, expertise, time, or other inputs. Effectiveness: Is the extent to which the objectives were achieved according to the development interventions, taking into account the relative importance of the objectives. Effectiveness can also be used as an aggregate measure of (or judgment about) the merit or worth of an activity.3 Sustainability: Is a continuation of benefits from a development intervention after major development assistance has ended. Analysis of sustainability can also project the probability of continued long-term benefits or resilience to risk of the net benefit flows over time. A description of the three assessment instruments, the Critical-thinking, Achievement, and Problem-solving Skills (CAPS) test, the Standards-based Classroom Observation Protocol for Egypt (SCOPE) and the Management Assessment Protocol (MAP), developed by the Education Reform Program to assist the Ministry of Education in measuring the progress of reform is also given. This is offered more as ‘food for thought’ in the possibility of using these instruments as standardized tests in Egypt. A review of reliability of the instruments and data collected from them is the objective of another task in the Egypt Monitoring and Evaluation Program. 1.3 Structure of the Paper The paper comprises six chapters: Chapter 1: Introduction and Purpose: The first section provides a background and description of the study as well as the list of study objectives. 2 World Bank (2004) Ten Steps to a Results-Based Monitoring and evaluation System by J.Z Kusek and R.C. Rist. 3 An example of this would be the extent to which an intervention has/is expected to attain its major relevant objectives efficiently in a sustainable fashion and with a positive institutional development impact. (World Bank 2004). Egypt Monitoring and Evaluation Program Aguirre Division of JBS International, Inc. xii Chapter 2: Background: Education in Egypt: The chapter provides the sector context in Egypt as well as identifies current issues in education, including a discussion on decentralization of education and presentation of the government education strategy from the National Strategic Plan. Chapter 3: Review of Programs: Chapter 3 provides a list of the six programs, reviewed by introduction, program activities and achievements, and factor analysis of efficiency, effectiveness, and sustainability of each. Chapter 4: Review of Instruments: This chapter reviews the three test instruments: Critical-thinking, Achievement, and Problem-solving (CAPS), Management Assessment Protocol (MAP), and Standards-based Classroom Observation Protocol for Egypt (SCOPE). Chapter 5: Synthesis: Chapter 5 synthesizes the attainment of the six components of decentralization, gender equity, teacher training, community participation, curriculum development, and policy reform by way of determination of efficiency, effectiveness, and sustainability of implementation of the programs from chapter 3. Chapter 6: Conclusions and Recommendations: This chapter provides concluding observations of the author along with recommendations for the next program design. Appendices: These contain more extensive explanations of education context, the programs, a list of efficiency, effectiveness and sustainability factors complied into one document, a matrix of the six components crossed-referenced by which program implements which activities in each, and a list of references used for this document. Egypt Monitoring and Evaluation Program Aguirre Division of JBS International, Inc. 1 2 Background: Education in Egypt 2.1 Context Education has been a top priority in Egypt since the 1990s, the backbone of the government’s efforts to enhance social stability, ensure national security, and increase economic competitiveness in the new millennium. Egypt espouses the current Education for All movement (in Egypt’s case: ‘Education for Excellence and Excellence for All’) and has committed to educational reform by abiding by principles of the international community. The government has strong partnerships with several multi- and bi-lateral donors, including USAID as well as the NGO community which have contributed hugely to its progress. The current National Strategic Plan 2007-2012 guides its continuing efforts to address the remaining challenges in the education sector. 2.2 Situation Analysis The strong commitment of the government to sector development has resulted in significant progress. Egypt has met many of the access challenges to education and now is able to provide education to most of its children in basic education, grades 1-9. Almost universal enrollment at this level has been attained. Gender parity has nearly been reached and girls make up a total 48% of enrollments in these grades. The quality of education in Egypt remains the major issue in the system today. Successful expansion in basic education enrollments has exacerbated the quality and lack of relevancy of education; quality remains low and unequally distributed further compounding the discrepancies between educational outcomes across income, geographic, and gender divisions. The generally low academic performance of students, as qualified by for example, the Trends in International Math and Science Survey (TIMSS)4, reveals that the education system is not preparing Egyptian students for the 21st century marketplace. The salient reform issues and their current status are reviewed here in order to set the backdrop against which the project profiles will be presented. For a more complete discussion of the education situation see Appendices 2. Appendix 3 contains data sets for figures in Appendix 2.5 A. Access About 90% of Egyptian students attend public education. In pre-university education, at the primary level, gross enrollment (GER)6 stands at 96% and at preparatory level (grades 7-9), at 93%. Meanwhile pre-primary education (for four-to-five year olds) did not exceed 18% in 2006- 4 Trends in International Math and Science Survey (TIMSS), a standardized test of math and science for 4th and 8th graders, is used to compare education achievement on an international basis. The exam tests student knowledge of basic math and science concepts which should be standard curricula content for grade 4 and 8 learners. http://timss.bc.edu/timss2007/index.html 5 Current data are difficult to obtain with the most complete set being available from 2005/2006. The most recent Ministry of Education data accessible for this report are from 2007/2008 6 Net enrollment ratio: Ratio of children of official school age for a particular grade or education level, enrolled in a particular grade or level, expressed as a percentage of the population in that same age group. (World Bank). Egypt Monitoring and Evaluation Program Aguirre Division of JBS International, Inc. 2 07.7 At the secondary level (combined general and vocational grades 10-12), enrollments are still significantly lower than that of the grades 1-9 at a 76% rate.8 B. Equity In the last 20 years, the greatest increases in enrollment and education attainment have been made by children from the most disadvantaged groups, the poor, and those who live in rural area and Upper Egypt. Inequity issues continue however among diverse population groups. Income Disparities A strong correlation between poverty and education exists in Egypt. Inequality across income quintiles is lowest for primary enrollment, where only 12 percentage points exists between the richest and poorest quintile (97% and 85%). However, secondary enrollment drops twice as fast as primary enrollments across quintiles and a gap of 30 percentage points exist between the richest and poorest population levels (74% and 44%).9 The system of private tutoring, pervasive in Egypt, also puts poorer students at a disadvantage when richer students are able to spend approximately ten times more for the service thus giving them a greater advantage in accessing more desirable education options. Geography and Gender Disparities Historically, in Upper Egypt where more than 50% of Egypt’s poor live, governorates showed both lower rates of enrollment than either Lower Egypt or metropolitan areas (except for Aswan). Extensive school construction efforts and gender-focused programming seem to have been successful at encouraging increased girls’ enrollment and attainment rates however to allow for current high primary and preparatory GERs. As of 2005, 73% of students from rural Upper Egypt managed to complete primary education as opposed to 87% from urban governorates and areas in Lower Egypt and 82% for those from rural areas in Lower Egypt and urban area in Upper Egypt.10 This situation might be changing however due to recent reform efforts which have focused on rural areas. The issue of difficulties in school enrollment and attainment appears to have been pushed up to the higher levels of education. Total enrolment in secondary education in 2007/2008 was 76% of which 52% were girls. According to the MoE, as of 2005, students living in rural areas accounted for only 24.2% of general secondary education while students from urban areas represented 75.8% of the total number. Once enrolled however, girls appear to be quite successful in transitioning through the grades, thereby justifying strategies to address gender and geographic differences. Another disadvantaged group, poor urban boys, made the least progress in educational attainment in the last eighteen years, attaining only 3 percentage points over that time and lagging behind in secondary completion rates of all groups except poor rural girls from Upper Egypt.11 School drop-out accounts for two-thirds of the failure of poor urban males to complete 7 UNESCO (2008) Education Support Strategy: Egypt. 8 Ibid UNESCO (2008). 9 World Bank (2007) Improving Quality, Equality and efficiency in the education Sector: Fostering a Competent Generation of Youth. 10 Egypt Ministry of Education (2007) National Strategic Education Plan for Pre-university Education 2007-2012. 11 Langsten and Hassan (2009) Education Transitions in Egypt: The Effects of Gender and Wealth. Egypt Monitoring and Evaluation Program Aguirre Division of JBS International, Inc. 3 11 years of schooling.12 Findings from the Egypt Household Education Survey indicated that the main reasons children may not attend school were a) monetary costs of schooling, 2) parent’s/guardian’s attitudes about schooling (in which formal schooling was not considered important for children or for girls especially) and 3) the opportunity costs of sending children to school thus sacrificing the financial contribution they would make to family income.13 Academic Disparities The system of tracking students at the secondary level into general and technical vocational education based on the results of the grade 9 exam contributes to system inequity by unequally dictating subsequent education and future employment opportunities. The approximate 33% of students with high scores in the preparatory end exam are offered general education opportunities which in most cases, translates into guaranteed access to university education and better future employment opportunities. Meanwhile, the remaining 56% of students14 are tracked into lower quality technical vocational education generally with no further opportunities for academic tertiary education.15 C. Quality Education quality includes the myriad variables which singularly and together impact the effectiveness of the teaching and learning process in a classroom, school, or system. Only the most salient quality issues are included here. Professional Staff As of the 2007/08 school year, the Ministry of Education reported a teaching force of 658,431 for all grades, pre-primary to secondary levels and a non-teaching work force (administrators, librarians, janitors, etc) of 435,187, therefore a ratio of 1: 0.7 exists. Teacher quality has been an issue as they most often adhere to the rote teaching methodology which does not encourage critical and creative thinking of their students. Pre-service training is of variable quality although currently there are many Faculty of Education initiatives and reforms to improve the service. Many training candidates complete pre-service training without having had practical in-classroom experience.16 Furthermore, novice teacher’s areas of specialization and teaching credentials are often not considered when they are hired which means literally for example, that a graduate of the art history faculty could be assigned to teach English. In-service training is guided by the Central Directorate of In-service Training with a branch in each governorate, but is reportedly of doubtful quality. In-service training is provided extensively by donor programs. To improve the quality of the education workforce, a Teachers’ Cadre, established in 2007 aims to standardize the process of professional teacher development, certification, and promotion. The process addresses both pre- and in-service teacher training. As of August 2008, 829,849 teachers, administrators and specialists had sat for Cadre placement tests and achieved a pass 12 Ibid. 13 USAID (2005/06) in UNESCO 2008. 14 The remaining 10% of students are in private secondary schools or Al Azhar education. 15 World Bank (2007) Improving Quality, Equality and Efficiency in the Education Sector: Fostering a Competent Generation of Youth. 16 Sheta, T.A. (2007) Novice Teachers: Current Situation and Future Directions. Egypt Monitoring and Evaluation Program Aguirre Division of JBS International, Inc. 4 rate of 92.45%.17 The Ministry of Education reported that the number of administrators in basic education schools has been reduced by 25% to enable a more realistic teaching-non-teaching staff ratio.18 The use of reform-based methods of teaching is now espoused as the most effective method to improve student performance.19 Professional development of school administrators is receiving much attention from the Ministry of Education now as the professional promotion track of the Teacher Cadre is nearing completion. Job descriptions for school principals and deputies are now in place while the process of developing the professional certification for school leadership and revision of the structure of school administration (to re-adjust the number of administrators at schools) are now in reform. Curriculum The general curriculum is reported to be rigid and lengthy.20 According to the National Strategic Education Plan, the main issues with the curriculum relate to the: a) predominance of traditional subject-centered teaching and learning methods; b) predominance of traditional assessment techniques that emphasize memorization; c) poor use of technology in the educational process; and d) the failure of the national curricula to meet the needs of different geographical environments.21 In the last few years, curriculum reform has been receiving attention from the government as part of a unified focus (including teacher development, textbooks, technology, and student assessment) on improving student academic performance. As of 2009, the MoE had completed curricula revisions to reflect ‘new modern’ methods that enhance comprehensive assessment and active-learning through ‘new standards’ for grades 1-4 and 7. However, the effectiveness of the new curriculum is still questionable, plus there has not been adequate teacher training for the new curriculum.22 Examinations The competitiveness to pass the national school-leaving exam at grade 12 is the main driver of the current in-class rote memorization method of ‘learning’ as students who pass are then able to attend university. Those who fail have few options beyond vocational school. Student performance on the TIMSS is low and did not improve between the two participation dates of 2003 and 2007. Typically, students do not learn and are not evaluated in cognitive, problem￾solving and critical thinking skills which are necessities to increase the competitiveness of Egypt’s workforce in the world market. Student assessment has also been the focus of much attention by the government recently, working hand-in-hand with improved teaching-learning processes in the classroom and curriculum development. The government plans to continue participation in internationally-recognized exams such at the TIMSS in order to track its own progress in education reform. Also the Critical-thinking, Achievement, and Problem-solving Skills (CAPS) instrument, developed by the Education 17 USAID (2009) ERP-EQUIP 2: Policy and system Change: Final Report. 18 Ministry of Education (2007). 19 In a move away from rote lecture and memorization methods, reform-based methods espouse interactive instruction and learning such as learner- or child-centered teaching, cooperative learning and active learning. 20 USAID (2008) New School Program: Final Evaluation. 21 Ministry of Education (2007 p97-99. 22 As reported from an Egyptian consultant. Egypt Monitoring and Evaluation Program Aguirre Division of JBS International, Inc. 5 Reform Program, could have utility as a standardized test but should be well-administered over a wide base to ensure reliability, standardization, and acceptance. Classroom Density While the calculated teacher-student ratio (the ratio of number of students to number of teachers on a country-wide basis) at primary level is now 25.6:1 and secondary level 14:1, many classrooms exist particularly in disadvantaged urban areas and rural governorates, where student numbers per classroom are high (above 40 or more). 2.3 Decentralization The argument for education decentralization is based on the political, geographic, and often cultural distance that separates centralized governments and local communities. Decentralization has become a popular recommendation from donors when education systems in developing countries produce poor outcomes, i.e. students of poor quality, positing that local actors are better able than distant central governments to respond to local education issues such as curriculum development, teacher training and deployment, and school management. More support for the process comes from the desire to increase the efficiency and effectiveness of educational management at school level.23 Additionally, accountability to near￾at-hand community members may encourage teachers and school administrators to be more responsive to education issues in the local schools. That said, however, the relationship between education decentralization and improved student outcomes is not always clear. The literature consistently states that decentralization does not always have a positive influence on education quality, rather has the potential to do so. Consistent evidence of the positive impacts of decentralization appears to exist when schools have budget authority and parents are involved in school governance.24 Decentralization should not be considered an end in itself but rather a process to enhance human development.25 Effective decentralization should be based on an assessment by the mid￾levels of government (in the case of Egypt, the governorate and idarra levels) to identify local priorities; thereafter, the governorate role would be to develop local/regional policies that reflect local situations while at the same time being in line with nation policies developed at the central level. 26 The decentralization issue has been on the radar screen of the Egyptian government since the 1980’s. The policy and financial frameworks are now in place and proponents have been pushing strongly for it, citing experiences from other countries of involving stakeholders at all levels as a means to achieve sustainable development and improve the quality and delivery of services. 23 UNESCO (2005) UNESCO Seminar on EFA Implementation: Teacher and Resource Management in the Context of Decentralization. 24 USAID (undated) EQUIP 2 Working Paper: Identifying the Impact of Education Decentralization on the Quality of Education. 25 UNDP (2004) Egypt Human Development Report. 26 Ibid UNDP (2004). Egypt Monitoring and Evaluation Program Aguirre Division of JBS International, Inc. 6 Initial decentralization efforts in education began in the late ‘90s with the initiation of several projects which had large community participation components.27 In 2000, two USAID programs, the New Schools Project (2000-2008) and Alexandria Reform Pilot Project (2002-2004) further supported initial education decentralization efforts by developing mechanisms (such as Community Education Teams and Parent Teacher Councils) which enabled community members to be involved in the management of schools and oversee responsibility of the school administrators. These two projects seem to be held as worthy examples on which continuing decentralization efforts can be made in the education sector. With USAID assistance (through efforts of the Girls’ Improved Learning Outcomes (GILO) project), fiscal decentralization was piloted in three provinces (Fayoum, Ismailiya, and Luxor) last year, with intentions by the Ministry of Local Development to initiate decentralization of selected activities on a national scale. According to the National Strategic Plan, the main decentralization strategy of the Ministry of Education is School-based Reform, which is a program of school self-assessment and performance planning. This program includes development of school improvement plans and by necessity, strongly supports school-based management as the means to achieve individual school progress. School-based reforms support good governance by increasing involvement of communities in managing education inside schools and calls for greater participation by staff and parents in collective policy- and decision-making processes of the school.28 2.4 Government Education Strategy The following chart outlines the main strategies of the Egyptian government for addressing education reform as contained in the National Strategic Plan for Pre-university Education Reform in Egypt 2007-2012. These strategies set the context for current reform issues in the country. Main Strategies of the Egyptian Government for Education Reform Curriculum Reform and Student Assessment School construction Professional Development of education staff Out-of-school children: girls, disabled, poor Decentralization and community mobilization Education for Special Groups Basic education: increased quality, equity and access Improved Monitoring and Evaluation for quality assurance ICT for education: management information use Secondary Education: increased quality and access School-based Reform for Accreditation Early Childhood Education 27 For example, UNICEF and CIDA Community School project, the MoE’s One Classroom School project, and the World Bank Education Enhancement Project (Ginsberg, M et. al. (draft 2009) Reforming Education Governance and Management in Egypt. 28 Ibid Ministry of Education (2007) Egypt Monitoring and Evaluation Program Aguirre Division of JBS International, Inc. 7 3 Review of Projects Each USAID project is assessed individually to determine the extent to which the factors of efficiency, effectiveness, and sustainability have been achieved through project design and implementation. Each evaluation includes an initial introduction of the project along with a description of what documents were reviewed for the evaluation. Project objectives and strategies follow. This section is followed by a synopsis of project activities, achievements, and results based on the information available in the documents that were reviewed initially. Where project activities are listed in table form, they are categorized according to the strategies of the USAID education strategy which they address. Thereafter, an analysis is given determining the achievement of efficiency, effectiveness, and sustainability in the project. In this study, efficiency is a measure of how economically resources are converted to results. Resources can refer to funds, expertise, time, or other inputs. Effectiveness is the extent to which program objectives were achieved according to the development interventions, taking into account the relative importance of the objectives. Effectiveness can also be used as an aggregate measure of (or judgment about) the merit or worth of an activity.29 Sustainability is a continuation of benefits from development intervention/s after major development assistance has ended. Analysis of sustainability can also project the probability of continued long-term benefits or resilience to risk of the net benefit flows over time.30 3.1 Education Reform Program 1 (ERP/ EQP 1) The Education Reform Program is a large reform initiative focusing on school-based change to support sustained improvements in student learning outcomes. The objective of the entire ERP was to improve access to quality education in selected governorates. The ERP 1 (an EQUIP I program) focused program implementation at the school level while ERP 2, the sister program (an EQUIP 2 mechanism), focused on policy and systems reform.31 Both programs shared the same central themes of decentralization of school management, increased community participation and enhanced training of teachers and school administrators but focused on different system levels to achieve these. The ERP 2 ended in 2009 while the ERP 1 will continue until late-2010, having taken over several program foci from its sister program, ERP 2.32 Each program will be reviewed individually in this document. Each EQUIP program was supposedly to provide services to the whole ERP. EQUIP I (ERP 1) was responsible for developing a program-wide communication strategy for the whole of ERP. EQUIP 2 (ERP 2) was responsible for ERP monitoring and evaluation which included submitting 29 An example of this would be the extent to which an intervention has/is expected to attain its major relevant objectives efficiently in a sustainable fashion and with a positive institutional development impact. (World Bank 2004). 30 Definitions from World Bank (2004) 31 EQUIP 1, 2, and 3 are USAID funding mechanisms. EQUIP 1 focuses on classroom- and school-level educational interventions that improve student learning and closely involve the community. EQUIP 2 targets policy and systems development, management, and education finance at the cross-community, district, and national levels. Ginsberg, M et. al. (2007) Active Learning Pedagogies as a Reform Initiative: The Case of Egypt. 32 These components focused on: the Professional Academy for Teachers, the M&E databases and work on the national education indicators, and work on the measurement instruments, CAPS, MAP and SCOPE. Egypt Monitoring and Evaluation Program Aguirre Division of JBS International, Inc. 8 reports for both ERP 1 and 2. Both EQUIP 1 and 2 programs collaborated in the development of the CAPS test, with EQUIP 2 managing the activity and EQUIP 1 providing technical expertise. Education Reform Program 1 The ERP 1 started in June 2004. Originally intended to end in June 2009, it was extended to September 2010. It originally worked in selected primary, preparatory, secondary and vocational schools in seven governorates: Alexandria, Cairo, Beni Suef, Fayoum, Minya, Qena, and Aswan. The total funding amount (combined with ERP 2) was $64.5 million. This analysis of the ERP 1 is based on early progress reports of the original program description of June 2004-July 2007, a research paper on active-learning pedagogies as a reform initiative, a revised program description from 2005, and several progress reports from 2009 after the ERP 1 had picked up the ERP 2 components. Project Objectives The ERP 1 implementation plan underwent several changes early in the program due to changes in USAID or Ministry of Education priorities. An initial program structure was modified to enable better coordination and integration between the two ERP programs (See Appendix 3.1 for the original program strategic components.); thereafter the ERP program as a whole focused on the five common strategies listed here with the ERP 1-specific components noted. 1. Capacity for Decentralization: improved capacity of government in a decentralized system to deliver quality education services with accountability; 2. Community Participation: Community-school partnerships empowered to promote standards￾based education through the following activities: (ERP/EQUIP 1 Components) Community Participation, Girls’ Scholarships, Multi-grade Classrooms/Schools, School Construction, School Governance – Board Of Trustees (BOT), Integrated Literacy, Early Childhood Development Education. 3. Standards: Decision-makers, education professionals, and communities use National Standards of Excellence to improve educational outcomes; 4. Professional Development: A in- and pre-service teacher training system which enables teachers to meet national standards for teacher performance through the following activities: (ERP/EQUIP 1 Components) Teacher Professional Development; Administration and Leadership Development; Standards Implementation. 5. Monitoring and Evaluation: Quality and accessible EMIS data inform decision-making at all levels and support recommendations for adaptable models and systems. Project Activities and Achievements The ERP 1 program continued to morph over the years of implementation, but the focus on the same five components above remained the same. 2005-2007 Egypt Monitoring and Evaluation Program Aguirre Division of JBS International, Inc. 9 The ERP 1 project activities and achievements for 2005-2007 (Appendix 3.1) attained relatively good results especially in terms of training outputs and number of scholarships dispersed. The ERP 1 was the only USAID program to employ strategies at all levels of the formal and non￾formal education system (i.e. pre-school, primary, preparatory, general secondary, vocational secondary, university-based teacher preparation, and adult education) in their project implementation group of ‘Family of Schools’. However, in the second phase of implementation, the Life Skills, Early Childhood Development, and School-to￾Work components were dropped due to a change in USAID programming priorities. 2007-2009 In 2007, ERP 1 again re-structured its programming, committing to a strong central theme of School Based Reform (SBR), targeting teachers, school leadership and governance structures, communities and parents, and the government systems that support schools. The program would accomplish this through focused attention on six interrelated and complimentary strategies: 1. Competent Teachers: Strengthening teacher competencies in active learning, critical thinking, subject-matter-integration, enhancing focus on early grade reading, introducing concept of reflective practice by teachers in ‘Quality Circles’ and using formative assessment approaches. 2. Empower Leadership: Improving principals and BOT leaders’ ability to provide inclusive leadership and decision-making, acquire effective instructional leadership skills, and use of leadership ‘Quality Circles’. 3. Community Participation and Good Governance: Strengthening ability of schools and BOTs to develop and implement quality School Improvement Plans (SIPs) designed to improve school quality necessary for accreditation. Increase schools’ ability to reach out to community and community to be more involved. More focused attention on disadvantaged students in the community. 4. Supportive Systems: Working closely with idarra and to a lesser extent with governorate levels to strengthen ability to provide effective support systems for schools. More support to teachers, principals, and BOTs. 5. Adult Literacy: Assisting Adult Education Authority (AEA) to increase capacity to independently provide quality literacy training, and expanding its coverage by working with civil society organizations. 6. Systematic Learning: Working toward a proactive documentation process to assure collection and use of meaningful data and new knowledge and assist MoE to take over the process. Many of the Phase 1 activities were carried over into Phase 2 (see Appendix 3.1 for a comparison of the two phases). However, whereas Phase 1 activities laid the foundation for school reform through selection and training of teachers, administrators, and communities, Phase 2 built on these efforts by strengthening the links between these groups of stakeholders. Under the theme of School Based Reform (SBR), the program capitalized on previous achievements but added increased responsibilities of local stakeholders to deepen and widen the scope of implementation of school reform. An added strategy was the increased focus on involvement of the idarra-level education authorities particularly in teacher professional development to assist in setting up clear pathways of responsibilities and progress towards Egypt Monitoring and Evaluation Program Aguirre Division of JBS International, Inc. 10 institutionalization of the process. Another new strategy was the focus on generating and using reliable school data in school reform by the school community in order to prove need and achievement in reform issues. The Adult literacy activity became a major program component in Phase 2 but went beyond the actual provision of literacy services to focusing on strengthening the capacity of the Adult Education Authority themselves; thereby enabling this government entity to become a more independent service provider by outsourcing the provision of the literacy services to civil society groups. 2009-2010 In subsuming some of ERP 2 activities in 2009 (those of Professional Academy for Teachers, M&E databases and the work on the national indicators, the focus on the CAPS, MAP and SCOPE instruments) the ERP 1 evolved away from directly supporting schools and communities towards efforts to institutionalize the core aspects of School-based Reform at the idarra level. The program would also focus more on efforts with the Ministry of Education to build systems that support school-level reform especially in teacher professional development, planning capacity of the governorate level for adult literacy, and increased emphasis on reporting and data management. Program objectives and several key activities in this phase include: 1. Family of Schools turned over to idarras as models of effective schools. Technical Support Units (idarra-level) supported to implement plans for increased professional development models at the school level. Social Work Department supported to support BOTs. Professional development structures (Teacher Learning Circles, Learning Resource Centers, School Based Training Units and Clusters) strengthened at idarra and school level. Supplemented reading and writing initiatives at idarra level. 2. Decentralized Adult Literacy planning and implementation instituted in nine governorates through the Ministry of Local Development and Governorates. Nine governorates supported to decentralize planning and implementation of Adult Literacy programs through Adult Education Authority and Ministry of Local Development officials. 3. Education idarras understand ERP/SBR and provide needed support to implement program. Central level MoE supported to support decision-making at governorate and idarra levels to develop School Improvement Plans. Idarra level strengthened to support School Based Reform through ToT, guides and training for administration of MAP and SCOPE. Support central MoE on development of comprehensive national assessment system (national indicators), teacher professional development (Professional Academy for Teachers) and EMIS (databases). 4. Successful strategies and practices in area of SBR and institutionalization are captured and shared. Quality reporting, documentation, communication, and data collection/verification activities maintained. Egypt Monitoring and Evaluation Program Aguirre Division of JBS International, Inc. 11 Based on a review of the most recent quarterly reports, the ERP continues to successfully reach program targets particularly those for building capacity of government staff at the idarra level but also at the governorate and central levels. The program has continued to develop new test items for the CAPS, MAP, and SCOPE assessments during 2007-2009. The program seems to have made a major impression in the sector because of the type and size of its contributions in decentralization, teacher training, and increasing girls’ school enrollment. Factor Analysis: Efficiency The ERP started at a wide base of implementation across all levels of the school system focusing on a diverse array of strategies apparently without a common aim. This constrained the depth of change that was able to be attained. Even the strong program focus on teacher development would not have been a common denominator as teachers’ needs (although not necessarily their skills) would be different at each level of the system. The project did not achieve projected targets in several activities in Phase 1. Efficient? The re-structuring every couple of years cost time and money to re-orient everyone in the program and could have confused beneficiaries on the ground level. Efficient? The use of school clusters as the delivery unit for professional development programs, instructional supervision, and teacher support is an efficient organizational mechanism. Efficient Effectiveness The professional development delivery mechanism was modified several times (from a cascade model, to a refined cascade, to direct training model, to collaboration with multiple levels) over the course of the program. Did it cause confusion among stakeholders or add value by being flexible and including many individuals across levels? Effective? The activation of School-based Training and Evaluation Units (SBTEU) to support teacher professional development in school clusters could allow for timely, consistent, and on-site training delivery. Effective The development of various Professional Development models (such as Learning Resource Centers and Teacher Learning Circles) and specific coursework (Professional Development: the Road to Quality) targeting educators at all levels supports implementation, sharing among professionals, and strengthens training structures although they are being prepared for national disbursement without evaluation or empirical evidence of improved teachers’ behavior. Effective? Training school administrators and supervisory staff to become instructional leaders in schools supports teachers’ use of new methods in classrooms. Effective Building capacity at central level for improvements in the national assessment system (national indicators), teacher professional development (Professional Academy for Teachers) and EMIS (databases) is necessary to strengthen the system but should reflect needs at all levels. Effective The scholarship program very successfully enabled many children (150,000) to attend school which is very effective, but the reality is that such an initiative is generally not sustainable by the host government in the longer run. Effective Sustainability Egypt Monitoring and Evaluation Program Aguirre Division of JBS International, Inc. 12 The increased focus on participation of idarra-level MoE staff in teacher professional development and School Improvement Plans is necessary to build the structure and ensure institutionalizing the process. Sustainable A process to engage school staff, administrators and communities equally together for participation and responsibility for effective school reform creates a sound foundation for continuing attention to quality education. Sustainable The program supported huge amounts of training at all levels, especially of teachers. Was there too much training and too little classroom support for implementation of new methodologies? Sustainable? Decentralization efforts (especially at the idarra and governorate levels) seem to be moving very quickly and are confusing (training for Social Work Support Teams, Social Work Department members, BOT Support Teams, BOTs!). Are these authorities also confused in their understanding of their roles and responsibilities? This process seems to employ/involve many individuals. Do they have redundant responsibilities? Sustainable? 3.2 Education Reform Program II (ERP/ EQP 2) The Education Reform Program 2 (also known as E2) shares the same introduction as the previous section, ERP 1, as indeed they were two halves of the same program. The ERP 2 used a systems approach to build a foundation for policy change and institutional capacity for replicable reform. The central themes of decentralization of school management, increased community participation, and enhanced training of school teachers and administrators continued in this program and the same focus on improved student learning outcomes was kept. The ERP 2 program also worked in the same seven governorates of Alexandria, Cairo, Beni Suef, Fayoum, Minya, Qena, and Aswan. The ERP 2 played a significant role in partnership with the MoE through a dynamic time of policy formation, system reform, and shifting strategic priorities. The program ran from April, 2004 to March 2009, during which some major and fundamental sets of reforms and rethinking of education in Egypt came about. As noted in the final program report reviewed here, without the structural changes that the ERP 2 facilitated in the system, school level improvement would not be possible. Project Objectives As with the ERP 1, the ERP 2 underwent several modifications in its implementation plan due to shifting USAID or Ministry of Education priorities and the changing political and social landscape of the time. Throughout its implementation time, the program managed to remain flexible and responsive to the changing institutional environment and continued to focus on the five common strategies of the ERP. 1. Capacity for Decentralization: improved capacity of government in a decentralized system to deliver quality education services with accountability; 2. Community Participation: Community-school partnerships empowered to promote standards￾based education through the following activities: 3. Standards: Decision-makers, education professionals, and communities use National Standards of Excellence to improve educational outcomes; Egypt Monitoring and Evaluation Program Aguirre Division of JBS International, Inc. 13 4. Professional Development: An in- and pre-service teacher training system which enables teachers to meet national standards for teacher performance. 5. Monitoring and Evaluation: Quality and accessible EMIS data inform decision-making at all levels and support recommendations for adaptable models and systems. ERP efforts affected all levels of the education system by the nature of what policy change was meant to do at each level. At the central level, the focus for E2’s work was on supporting ministry and government policy to recognize the system in all its parts and interconnectedness and ensured effective functioning of the whole.33 At the school level, ERP 2 contributed to policy and system change as it related to leadership, planning, quality assurance in schools, data gathering and reporting, and increasing the responsibility of school administration for effective management of the institution. Better teaching is increasingly seen as a key element for success in improving student learning outcomes and includes not only the human resources aspect of professional development, but also the resources required for teachers to be successful such as curriculum, teaching materials, and equipment. At this level, E2 contribution aided development of the Teachers’ Cadre and the Professional Academy for Teachers, the work with Faculties of Education, and Specific Education/Kindergarten, and active learning-training for Hand in Hand curriculum implementation. The ultimate goal of an effective education system is better learning i.e. improved student learning outcome which can be understood as preparation for the workplace, societal responsibilities, or further learning or can be taken to mean achievement on curriculum and program outcomes. E2 support at this level added technical assistance to national testing of student achievement, development of standards, and program components such Community Youth Mapping, Creative Science Education Initiative, and Hand in Hand. Project Activities and Achievements ERP 2 played a significant role in the efforts to improve government system components. These efforts included dialoguing, visioning processes, using highly experienced and reputable international experts, collecting and presenting data, researching, and strategizing across all levels to ensure that the end products were research-based, data-driven and truly representative of the vision the Ministry of Education has for Egyptian children. Throughout the process, the National Strategic Plan was the guide against which progress was measured. Results shown here were taken from the final program report, 2004-2009. Project Activities by USAID Objective: ERP 2 1. Improved quality of teaching and learning. Professional Academy of Teachers (PAT) and Teachers’ Cadre: A Teachers’ Cadre inclusive of a ranking matrix and salary system is made into law (2007). Job descriptions for teachers in five levels included in the Cadre. 829,849 teachers, administrators and specialist candidates sit for Cadre placement with a pass rate of 92.5%. The Induction and Mentoring Program for new teachers piloted, approved, and scaled up. 33 This section taken directly from USAID (2009) ERP-EQUIP 2: Policy and System Change: Final Report, 2004-2009 p10. Egypt Monitoring and Evaluation Program Aguirre Division of JBS International, Inc. 14 The Professional Academy of Teachers is established by Presidential decree (2008). Quality Assurance and Accreditation: The National Authority for Quality Assurance and Accreditation established by law (2006). National Effective School Standards set as basis for accreditation of schools. Quality Assurance Units in all 29 governorates and 260 idarra in the country established and provided with orientation and accreditation processes and materials. Faculties of Specific Education/Kindergarten: Pre-service performance standards for Faculties of Education (Arabic, Math, Sciences, and Social Studies) created. National Academic Reference Standards created for eight academic programs. Standardized Testing: Created MAP as a protocol to assess school management system. Created SCOPE as a protocol to assess teacher practices with reform-based teaching methods. Created CAPS as a protocol to assess student leaning outcomes. All tests based on prescribed outcomes according to the National Strategic Plan. 3. Strengthened school governance and management. Strategic Planning: The Egypt Household Education Survey (EHES) 2005-06 published and released. The National Strategic Plan (2007), the foundation for all subsequent reform efforts of the MoE, released. Twenty-seven Governorate Strategic Plans, representing all governorates in Egypt, completed and approved. Idarra Implementation Plans for all 75 Idarra in the seven ERP target governorates, completed. National Strategic Plan indictors for M&E of the implementation process in all governorates completed. Decentralization: Decentralization policy for pilot use in three governorates (Fayoum, Ismailia, Luxor) endorsed. Education system is first sector to start decentralization process. School level funding formula based on enrolment and pro-poor weighting endorsed by Minister of Finance (MoF). Fiscal decentralization policies give schools more control of own finances either endorsed or in the works. School/BOT Guide to Decentralized Education Finances prepared by MoF. For the first time, an Inter-ministerial Committee (MoF, MoE, Local Development, and Administrative development) meet to collaborate on decentralization efforts. Organizational Transformation: Functions, roles, and responsibilities re-worked and devolved to lower levels of the MoE as major part of decentralization system to support school based reform. The Organizational Transformation Plan, a major system change articulated in the National Strategic Plan (NSP), is approved. Leadership Program: A residential Leadership Training Program for MoE middle managers, initiated. Their training consists of ICT, English, leadership and management skills as well as education reform modules of the National Strategic Plan, decentralization, funding formulae, teachers’ cadre, accreditation, EMIS and data-based decision-making, and social marketing. 147 middle managers representing every governorate, successfully complete the year long program. They comprise the cadre of knowledgeable and skilled professionals that the MoE can move into senior positions as needed, and will provide leadership to the reform agenda. Education Management Information System (EMIS): National Education Indicators are for the first time, custom-designed, realistic and operational. Strategic Plan indicators make the Plan operational and measureable. Capabilities for indicator design, data analysis, and reporting were built at multiple levels. Disparate sets of MoE information systems unified under the EMIS Transformation Plan. System of School report Cards produced in 162 schools in two governorates and is being adopted by other governorates. Egypt Monitoring and Evaluation Program Aguirre Division of JBS International, Inc. 15 Reporting and Information Systems: State-of-the Art Information and Knowledge Management System for ERP as well as other USAID programs (TILO, GILO, NSP). Includes for example, a School Data Bank with information on all schools in the USAID programs. Systems transferred to MoE and other USAID projects. Verification of Policy Reform for Cash Transfers: Trained MoE in process of negotiating and demonstrating policy reform with conditional cash transfers of $150 million with USAID. Source: USAID (2009) ERP-EQUIP 2, Policy and System Change Final Report, 2004-2009. Factor Analysis: Efficiency As little information is available on the division of funding between the ERP 1 and 2, little can be ascertained about the efficient use of the money per program. However, in terms of time and degree of structural and policy change made possible by the facilitation of the ERP 2, the time would have seemed to be well-spent for the significance of the results produced. Efficient Effectiveness The use of the National Strategic Plan as the guide for improvement ensures consistency and standardization across the whole system. Effective Recognizing that system change will not happen quickly, the ERP focused on building the need, understanding and tools to support long-term educational reform. Effective System decentralization is difficult, encountering much cultural and political resistance. The current progress, including decrees for fiscal decentralization and devolving more authority to local Boards of Trustees, represents huge steps in a relatively short time and a volatile environment. However, policy and laws should be made with input from all levels of government to reflect needs and buy-in of lower level stakeholders. Effective? The development of the Professional Academy for Teachers and the Teacher’s Cadre represents a major step in the provision of standardized and structured professional development delivery. Effective Although the two ERPs focused on the same five common strategies, there did not appear to be much coordination of efforts between the two programs especially at the idarra level which seemed to be a logical meeting point of implementation (ERP 1 working up to and ERP 2 working down to the idarra level). Effective? Development of a comprehensive database/information tracking system assists in program monitoring and evaluation and provides a good model for host government use. (However, sustainability is at risk if the system is not continued and the MoE is not capable of using it after the program ends.) Effective Development of the CAPS, MAP and SCOPE assessments provides the host government with model instruments to track progress in school-based reform. Effective Sustainability The framework for reform is in place and even though the actual implementation and achievement of reform will take considerable time, the laws will endure. Sustainable. Egypt Monitoring and Evaluation Program Aguirre Division of JBS International, Inc. 16 The initiation of a Leadership Training program and availability of a continued supply of well￾trained staff assists to strengthen the system and perpetuates the transfer of good practices and knowledge. Sustainable. Development of a state-of-the-art Information and Knowledge Management System for ERP as well as other USAID programs (TILO, GILO, NSP) made a major contribution to the system, useful by USAID to track programs and left as a model that the MoE can continue to use and build on. Includes for example, a School Data Bank with information on all schools in the USAID programs. Sustainable. 3.3 Girls’ Improved Learning Outcomes (GILO) This analysis of the Girls’ Improved Learning Outcomes (GILO) is based on implementer quarterly reports to USAID from inception of project (April 2008) to December, 2009 and the 2008 USAID Scope of Work. As the program is half complete, no evaluations of the program have been yet; therefore this analysis is based on self-reported implementation activities and results only. Project Objectives The focus of the GILO project expands on the achievements of the New School Program which closed in mid-2008, using many of the same program strategies and implemented in the same governorates34 but in different schools although sometimes in the same idarra. In picking up decentralization efforts from the ERP 2 when it closed in 2009, GILO includes strategies to assist the MoE in fiscal and administrative decentralization. The basis of the GILO program focuses on improving access, attendance and learning opportunities for girls through sustained assistance to gender-sensitive schools. The project is implemented in underserved communities in the governorates of Minia, Beni Suef, Qena, and Fayoum. The project runs from February 2008 until April 2011 and is funded for $38.2 million. Expected results of implementation of GILO are improved scores on the CAPS, MAP and SCOPE tests which are to be administered yearly over the course of the program. Project Activities and Achievements GILO programming builds on the experiences of recent/on-going USAID and other programs (ERP 1, New School Program, TILO, STEPS II (Canadian International Development Agency), Intel and Microsoft training programs); GILO therefore should reflect best practices derived from these initiatives and reinforces efforts to institutionalize and mainstream sustained quality improvements in education. GILO programming efforts focus on the school and idarra levels although they do offer some support to central decentralization initiatives. 34 With the exception of Qena where NSP was not implemented. Egypt Monitoring and Evaluation Program Aguirre Division of JBS International, Inc. 17 Project Activities by USAID Objective: GILO 1. Improved quality of teaching and learning. Improving Education Quality Train supervisors to promote active learning, and authentic assessment: develop training materials (use of shared materials from ERP, TILO, NSP, and STEAP), teacher coaching, integrate new curricular units. Train teachers and administrators in gender-sensitive active learning, assessment, and standards: training manual: The Student-Friendly Classroom. Promote early grade reading (EGRA) and support Arabic early reading program: baseline collection, piloting, database application. Supply schools with ICT package to promote improved learning outcomes: specifically for use by teachers in classrooms and school administrators. Shared experiences and strategies with TILO. Administer SCOPE and MAP: baseline collection, Year 1 assessment (2009). 2. Increased equitable access to education. Expanding Equitable Access to Education for Girls Identify target governorates, idarras, communities and schools: prioritizing communities by lowest levels of 10-14 year old girls in school, located in contiguous districts and proximate communities, imminent school construction/maintenance by GAEB, motivated and willing staff, and committed and willing communities. Accelerate school enrollments of girls through temporary or multi-grade classrooms. Mobilize community participation to expand access to education for girls: assessment, prepare community education action plan by BOT/CET, school maintenance. Equip/furnish schools to facilitate student-centered active learning: furniture, ICT equipment. 3. Strengthened school governance and management. Strengthened School Management & Governance Train and assist school administrators for improved school management and educational leadership (training curricula and materials shared from NSP, ERP and STEAP projects). Develop Boards of Trustees (BOT) for strengthened school governance: include training of Social Workers. Strengthen BOT capacity: advocacy and promotion of girls’ education access, partnership with school management, participation in school assessment and school planning: preparation for school accreditation. Install and utilize school management information systems: through ICT. Promote opportunities for greater student, parent and community involvement in schools as community resources e.g. health outreach activities. Strengthened Organizational & Institutional Capacity of the GAEB and PPP Provide technical assistance to GAEB and PPP to adopt efficient and effective systems for school construction and maintenance: recommendation of low-cost models and flexible designs. Provide technical assistance to decentralize GAEB roles and functions. Promote community participation in site selection, school construction planning and maintenance to GAEB and PPP. Seek and develop opportunities for additional funding to construct schools and classrooms of new design. Monitoring and Evaluation Complete Performance Monitoring Plan: verification of school, community data to set program indicators and evaluate local capacity for data monitoring, collection and reporting. Collect baseline data for girls’ enrollment, SCOPE and MAP assessments. Ensure effective GILO communication with stakeholders. Results after one year of implementation show that GILO achieved or nearly achieved most of its projected targets with the exception of the number of male administrators and officials trained. The 2009 MAP and SCOPE tests were administered but results are not yet approved for the public therefore are not included here. The CAPS test was not administered in 2009. Egypt Monitoring and Evaluation Program Aguirre Division of JBS International, Inc. 18 GILO SO 22 Targets: Indicators Target 2009 Actual 2009 Number of Administrators/officials trained. M 400 T 500 F 100 M 144 T 266 F 124 Number of learners - Primary schools M 30,900 T 59,500 F 28,600 M 31,596 T 58,952 F 27,356 Number of learners - Secondary schools M 10,900 T 21,800 F 10,900 M 13,991 T 26,173 F 12,183 Number of teachers trained M 2,800 T 3,600 F 800 M 1,695 T 2,868 F 1,173 Number Parent-teacher/school gov’t structures supported 54 32 Percent increase learner achievement (CAPS) Not implemented in 2009 Percent increase teacher behavior (SCOPE) 15% annual improvement Percent increase school management (MAP) 3% annual improvement Source: USAID (2008) GILO Quarterly Report July-Sept 2009. Factor Analysis: Efficiency The proposed funding amount of $38 million for a three year project is the same amount that was awarded to the NSP which ran successfully for nine years. Plus the GILO uses many similar processes, materials, and methods from other USAID projects. The level and scope of implementation seem to be less than the intended funding is able to cover. Efficient? The use of processes, methods, and materials from other USAID projects ensures that these components have already been field–tested and proven successful; therefore GILO can take immediate advantage of best practices in implementation. Efficient Effectiveness The assiduous identification and use of current data from local sources ensures appropriate selection of target populations, assists in data quality assessment, and models data-based decision-making. Effective Sharing and collaboration among other projects already on the ground allows for synchronized approaches, more complete field coverage of targeted populations, unified support of government reform, and reinforces efforts to institutionalize quality improvements. Effective Gender equity in the sector is enhanced by the use of field-tested gender-focused activities and support from GILO into other programs and activities (such as the adoption of the gender equity index into the SCOPE tool). Effective The mobilization of master teachers, idarra supervisors, school administrators, and peer coaches sets up a strong, local, and standardized in-service delivery structure for teacher training. Effective Support to School Management Information Systems will assist in school operations administration, monitoring school quality, and communicating with other stakeholders. Moreover, they can eventually feed right into the national EMIS at the central level once the technology and individual capacity are increased. Effective The trialing and development of the Early Grade Reading program should enable improvement of literacy skills of young children and offer the government a valuable education strategy for early grades. Effective Egypt Monitoring and Evaluation Program Aguirre Division of JBS International, Inc. 19 Sustainability Building of institutional capacity of central government units to work with local branch offices and community groups in provision of education in theory increases decentralization. In practice, central ministry officials fear loss of power and resources; therefore often do not willingly participate in decentralization efforts. Will these initiatives be maintained after project funding is ended? Sustainable? Strong community groups are the force which should drive demand for education reform and improved education outcomes for their children. Their scope of involvement at their local schools should be enlarged beyond participating in school site selection to strengthen their buy-in of school reform. Sustainable The development of an appropriate targeted school improvement plan with the participation of school staff, parents/community, BOTs and idarra officials should meet local education needs and be the common tie which binds these stakeholders into united action. Sustainable 3.4 New School Program (NSP) This analysis of the NSP is based on the implementer’s final project report of September 2009, as well as the final project evaluation in June 200835 and an evaluation by the Egypt Ministry of International Cooperation in February 2006. The evaluations were comprehensive, using both existing quantitative data enhanced by the collection of qualitative data to present a thorough, well-rounded picture of project results. Project Objectives The New School Program was the one of the initial USAID program to actively address decentralization for quality improvement of education at a school-level. The goal of the NSP was to increase school access and enrollment of girls in underserved communities in Minia, Beni Suef, and Fayoum governorates. This was addressed by the following objectives: Increased access to education for girls in targeted areas; Enhanced quality of teaching and learning processes in targeted schools; Increased community participation in girls’ education; Empowered schools, communities and civil society through the use of ICT. Monitoring and evaluation. The project had extensive partnerships with Egyptian government agencies, the private sector, Egyptian and international NGOs and USAID. It began 01/2000, was extended three times and continued until 07/2009. It had total combined funding of $38.5 million. Project Activities and Achievements Through May 2005, the NSP would collaborate with the General Authority for Literacy and Adult Education (GALAE) to deliver literacy programs in the NSP target areas. In October 2004, the NSP started a partnership with a technology foundation to introduce computer-based learning for students, teachers, administrators, and community members thereby necessitating the addition of another objective (the fourth of the five above) while continuing to focus on the initial four objectives. 35 USAID (2009) New School Program Inclusive of the School LINC GDA: Final Report. Egypt Monitoring and Evaluation Program Aguirre Division of JBS International, Inc. 20 Project Activities by USAID Objective: NSP 1. Improved quality of teaching and learning. Improved teaching and learning practices: Implementing National Standards. Principal and Supervisor Training: pre-service training and participation in teacher pre-service training. Teacher Development for pre- and in-service in student-centered active learning approaches and incorporation of instructional technology. Training structure: field-level Education Technical Officers collaborate with governorate-level Education Coordinators linked to central Education Advisors coordinated by Faculties of Education. Supplementary materials: teachers’ guides, student learning kits, school self-assessment tools. Education clusters supported to activate school-based training and evaluation units. Empowered schools, communities, and civil society through the use of ICT: Information and Community Technology Centers established. Teacher, administrator and community IT training. 2. Increased equitable access to education. Increased access to education for girls in target areas: School construction: in collaboration with MOE and GAEB officials and local communities, identify appropriate school sites, land approval, design, and construction of new schools. Furnish and equip schools. Provide school maintenance fee for school up-keep. Multi-grade schools established: based on community needs of girls age 9-14 who have not been to school. Madrasty Activity: Support BOTs to identify school needs. Provide with grants to address needs and facilitate school maintenance. 3. Strengthened school governance and management. Increased community participation in girls’ education: Establishment and training of BOTs and PA through democratic election processes. Establishment and training of Community Education Teams (CET)/BOTs and Parent Associations (PAs) to mobilize support and contributions for girls’ education and plan, implement, and monitor educational activities and service deliver programs. Facilitate development of community education action plans to increase access to education and IT. BOTs and PAs supported to create school task forces: School Solution, Awareness Raising, and Second￾change Education to mobilize and assist communities to solve local education issues especially concerning girls. Monitoring and Evaluation: Monitoring, evaluation and reporting system. Case studies/success stories. Media strategy. Based on the evaluations, the New Schools Program successfully achieved project targets and moreover, made large contributions to girls’ enrollment, school construction, and increased capacity of education personnel.36 Perhaps even more importantly, as a pilot program, the NSP was considered one of the initial education programs to address school-level access and quality issues through decentralization initiatives. 36 Note however, that despite the project focus on increasing girls’ school access, indicators are not disaggregated by sex; therefore the number of girls who were enabled to attend school is not available in the documents reviewed. Egypt Monitoring and Evaluation Program Aguirre Division of JBS International, Inc. 21 NSP Performance Indicators Indicator Target Achieved Students educated 43,000 44,383 Schools constructed 104 98 Multi-grade schools established 190 189 Teachers trained 2,000 2,018 BOTs and PAs established through democratic processes 176 182 School implementing National Standards 120 123 Madras schools renovated 15 15 ICT centers established 98 98 IT training opportunities 1,500 4,293 Source: USAID (2008) Final Evaluation of the New School Program. In 2007, NSP schools participated in the CAPS assessment for students, in the SCOPE assessment of teaches, and in the MAP assessment of school management. In the SCOPE results, NSP teachers scored the lowest when compared to teachers in the ERP and combined ERP/NSP groups. Both the NSP and ERP teachers were in turn outscored consistently by combined ERP/NSP teacher. These same results were found with the MAP; NSP school managers scored higher than comparison groups who did not benefit from program interventions. However, they scored lower than ERP school managers and both groups scored lower than combined ERP/NSP school managers. Factor Analysis: Efficiency Quality learning materials (student kits) were produced and distributed but not well-used or were unused by teachers. Appropriate learning materials in classrooms make a large contribution to education quality; therefore more attention should be given to the integration and training for their use by teachers. Efficient? Charging user fees at technology centers adds support to the centers and increases the understanding of the value of education. Efficient. The use of school clusters as the delivery unit for professional development programs, instructional supervision, and teacher support is an efficient organizational mechanism. Efficient. Effectiveness The construction of schools and classrooms in rural areas with community participation increases attention to issues of education (especially for girls), strengthens community pride and participation in education, and increases enrollment. Effective The practice of providing on-going in-service teacher development is vital to build increased confidence and capacity for provision of quality education; therefore training needs to be offered continuously throughout the year. Effective The use of multi-grade schools in difficult areas and circumstances offers a flexible and targeted schooling option to underserved populations. Education personnel who service these facilities must be well-trained to meet the challenges presented by these unique situations. Effective Sustainability Egypt Monitoring and Evaluation Program Aguirre Division of JBS International, Inc. 22 The development of strong community groups who were the force behind school construction and community mobilization should continue to drive demand for education reform. Their scope of participation should be enlarged in order to increase their capacity and maintain their interest and buy-in for better education service. Sustainable. The development of strong teaching training cadres consisting of MoE professionals across all levels of the government should enable independent continuation of high quality in￾service teacher training. Sustainable. Program implementers must be sensitive to teacher needs and time limitations to attend trainings and be responsible to train others. Overloaded teachers will withdraw from the program thereby diluting program effects and limiting impact of USAID investments. Sustainable? Charging user fees for community use of technology centers increases the demand for technology and supports use of technology for student learning. Sustainable. Enabling an endowment fund for school maintenance through leveraged contributions from communities and Ministry of Education keeps stakeholders involved and adds resources to the system for continued provision. Sustainable The eight-year length of the program allowed for achievement all targets, development of a trusted relationship with stakeholders, and added depth to implementation. Sustainable. 3.5 School Team Excellence Awards Program (STEAP) The evaluation of this project is based on review of USAID Scopes of Work Phase 1 and 2, a quarterly report and final report for 2009, a STEAP results power-point from the implementer, and the Site Visit Guide to Evaluate Egyptian School for Quality. Project Objectives The objective of STEAP was to achieve greater results in student learning achievements in primary and preparatory schools by encouraging high levels of collaboration among stakeholders to implement standards-based school level reform. The focus of the program was on the importance of implementing and achieving targets specified in the National Education Standards (NES) set by the Ministry of Education and laid out in individual School Improvement Plans (SIP) with the view to achieve accreditation. Incentives in the form of monetary awards (a lump sum to winning schools out of which all school personnel received an amount) by the MOE were given to excelling schools at the district, governorate, and national levels. Specific project strategies of the STEAP (by implementation cycle) were to: Raise awareness among school communities about school-based reforms (STEAP I); Emphasize importance of implementing and achieving targets specified by the National Education Standards (STEAP I); Assist the Ministry of Education in implementing two more award cycles based on new standards and improved SIP processes (STEAP II); Conduct periodic site visits during crucial stages of the school improvement cycle to assess degree of progress and offer technical assistance where needed (STEAP II); Obtain private sector support for the school improvement efforts (STEAP II). Egypt Monitoring and Evaluation Program Aguirre Division of JBS International, Inc. 23 The project ran in two cycles: September 2005-2007 (STEAP I) and September 2007-2009 (STEAP II) and the funding amount was $12.9 million. A more complete implementation synopsis can be found Appendix 3.3. Project Activities and Achievements STEAP created great excitement and a sense of competition among the schools and communities at the possibility of being a winning school. There was ‘overwhelming and somewhat unexpected positive community participation’37 which inspired individuals to donate large amounts of money to assist in school improvements. Staffs were brought together, teamwork was encouraged and a positive attitude of students and teachers towards the school was developed. Reports were that the education process at schools was ‘refreshed’. Project Activities by USAID Objective: STEAP 1. Improved Quality of Teaching and Learning Development of information technology: Interactive website developed for developing, recording, and storing school improvement plans for use by MoE officials, school personnel, and trainers. 3. Strengthened School Governance and Management Decentralization: Improvement of the Comprehensive School Evaluation Survey (CSES) as the basis for guidance of school reform and development of school improvement plans. CSES has high reliability ratings. Site Visit Guide to Evaluate Egyptian Schools for Quality developed and printed for use at idarra and higher level for school evaluation. 200 individuals trained as Monitoring Unit for the MoE. 27 governorate project coordinators trained. Partnerships between schools and private sector raised $2.25 million in 2007 and $2.8 million in 2008 for use as incentives for school improvement. A partnership with a local bank distributes the awards at a favorable rate. Development of nationwide management information system: Recording and storing of school improvement plans on website contributes to school data management. Development of school improvement plans (SIP): Training materials improved and distributed. Monitoring and evaluation materials developed which emphasize improved quality. 14,251 school leaders and teachers trained to use Egyptian School Quality Manual to develop SIPs (2007). 13,128 school leaders and teachers trained (2008). 957 trainers trained (2007) 447 new trainers trained (2008). 3,744 schools develop and submit SIPs (2007/08) 2,765 schools develop and submit SIPs (2008/09) During three award cycles, representatives from more than 14,000 schools trained and ultimately 10,550 schools completed all project requirements. Source: USAID (2009) School Team Excellence Award Program II: Final Report Factor Analysis: Efficiency 37 USAID (2009) School Team Excellence Award Program II: Final Report. Egypt Monitoring and Evaluation Program Aguirre Division of JBS International, Inc. 24 The initial implementation plan (to include all primary and preparatory schools in the country) was much too ambitious forcing a severe down-sizing of project schools, number of trainers, and use of different evaluation standards, thereby causing a re-structuring of the program and use of resources. Was there no piloting initially to see how the process would work? Efficient? Partnerships with private sector individuals resulted in large monetary contributions for local school improvement but they should be linked to specific outcomes at the school level. Efficient. Effectiveness The major premise of the program that there would be significant improvements in student learning achievements in a short period of time due to development of a School Improvement Plans and school-wide reforms was unrealistic. Of the six domains in the school improvement plans, student learning increased the least over the years of the project and remained low. At the very least, improved student achievement would be noticeable the year following the initiation of the school plan. Furthermore, no effort was made to ascertain if ultimately student learning actually improved in winning schools over the years as a result of participating in the program. Effective? The use of 20 master trainers (reduced from the original 54) to provide school-level support to a national-level project encompassing 4,000 schools seems insufficient to meet the scope of the project. Effective? Provision of project coordinators (from the government’s quality assurance unit) at the idarra level responded to a reported weakness from the STEAP I cycle and strengthened oversight and provision of knowledgeable contacts at a local level. Trained idarra directors were able to provide schools with guidance and authority on the school improvement cycle and data-based decision-making. Effective. Providing positive incentives to drive school-based reform was effective in engaging many individuals at all levels in the reform process but the system and schedule of incentives needs to be clearly understood and include input from many levels of stakeholders. Effective? The Site Visit Guide to Evaluate Egyptian Schools for Quality, which could play a major role in standardizing evaluation processes nationally, had a couple of major flaws in it concerning data collection, recording, and analysis (See Appendix 3.3 for further discussion). Effective? All manuals and assessment tools for the school improvement process were placed on-line (on a program-developed website) so materials were accessible and transparent. This encouraged independent development of school improvement plans and increased technology capacity. (However sustainability is at risk if the website shuts down after the project closes leaving schools without this resource.) Effective Participating schools received a report card which showed pre- and post-award analysis and could be compared to average idarra and governorate scores, as an added incentive for schools to track their own progress. Effective Sustainability The program became a national public awareness campaign for the National Education Standards, school improvement, and accreditation process. Sustainable. Egypt Monitoring and Evaluation Program Aguirre Division of JBS International, Inc. 25 The project generated great involvement of the community and refreshed the educational process within schools. Sustainable. Resources produced by the program, the Egyptian School Quality Manual, the Comprehensive School Evaluation Survey, and the Site Visit Guide to Evaluate Egyptian Schools for Quality contribute to validity (accuracy and appropriateness) and reliability (consistency and stability) of the school planning process and its evaluation for (eventual) accreditation. Sustainable. Training and involvement in the SIP process at all levels, especially at the idarra and school levels, contributes to building institutional processes and understanding of school reform. Sustainable. The Egyptian government is perhaps not financially able to continue the yearly awards to winning schools over a longer term. Moreover, the one-off nature of the award does not encourage continued effort and focus by schools (winning schools can’t participate again) nor increase equitable incentivization throughout the rest of the system. Sustainable? There seemed to be tremendous amounts of cascade training through layers of trainers down to the school level. How thoroughly did all these trainers learn the material and pass it effectively down to the next level? From the perspective of impact of investment, what has happened to all these trained individuals who are knowledgeable about the school improvement process now that the program has ended? Sustainable? 3.6 Technology for Improved Learning Outcomes (TILO) This analysis of the Technology for Improved Learning Outcomes (TILO) is based on the implementer’s first two annual reports to USAID (Sept 2007-Sept 2009), the first quarterly report of year 3 (Oct-Dec 2009) and an implementer feedback survey report (January 2008) with responses from ministry officials at six managerial levels. Numerous smaller technical documents, for example, training schedules, school selection criteria, and teacher, student, and administrator interview protocols for site selection were also minimally reviewed. Project Objectives The objective of the TILO program is to improve student learning outcomes by up-grading the quality of teaching, learning and school management through the use of technology. The program achieves this by focusing on the four following components: Improve the quality of teaching and learning through technology; Develop public-private partnerships to increase innovation and share responsibility; Build the capacity of the Ministry of Education at all levels for effective management of technology for education; Measure the impact of technology on teaching, learning, and management through effective monitoring and evaluation. The program operates in eleven governorates, Cairo, Giza, 6th of October, Helwan, Fayoum, Alexandria, Beni Suef, Minya, Qena, Assuit, and Aswan in 233 primary schools undergoing School-based Reform, and 85 public experimental Smart Schools. TILO started in September 2007 and is scheduled to run until September 2011 at a funding level of $21.5 million. Activity Activities and Achievements Egypt Monitoring and Evaluation Program Aguirre Division of JBS International, Inc. 26 The TILO program works in two stages: Stage I adds inputs (training and technology resources) to build capacity of schools and MoE staff to include a technology component in education. In Stage II, schools work towards self-management of their own technology abilities through the development of the School Technology Advanced Management Plans (based on the School Improvement Plans). Thus the TILO has built in a plan for sustainability and institutionalization of technology in program schools. Project Activities by USAID Objective: TILO 1. Improved quality of teaching and learning. Improve the Quality of Teaching, Learning and IT Management in Targeted Schools: School Selection Process: (actually supports decentralization: see under #3 below, but is listed in the program plan under this component.) Install Educational technology resource package: equipment. Develop learning objectives and deliver e-content: digital resources e.g. digital learning guides, CDs, DVDs, etc. Develop training strategy/plans: training curriculum and resources. Train MoE idarra supervisors, school supervisors, and school administrators: sustainability planning, motivating and rewarding change, IT skills, leading change and building strong school teams, conducting classroom observation and providing constructive feedback, and school support. Train and support Master Teachers and school supervisors: effective teaching methods, classroom support, IT integration, training of trainers, and teacher interviews. Public-Private Partnerships (PPP): Develop new partnerships: to provide core technology or equipment to schools. Develop private partnerships to share costs, provide IT training, and support management, community involvement, tech options, training and digital resources beyond basics. 3. Strengthened school governance and management. Improve the Quality of Teaching, Learning and IT Management in Targeted Schools: (from #1 ) School Selection Process: according to an application process and input from governorate and idarra levels against a standard list of selection criteria. Capacity for Management of Technology Review of current MoE system, bylaws and decrees relevant to technology management. Design reform-based and demand driven school start-up process. Develop scalable training for technology for pedagogical improvements system. Integrate activities for digital resources/content. Plan for TILO expansion to other schools along with MoE. Monitoring and Evaluation Modify SCOPE tool to include technology indicators. Design and validate TILO data collection tools. Collect, analysis and report on baseline data Administrate TILO data collection tools. Administer SCOPE. After one quarter of implementation in the third year of the program, TILO has achieved 66- 100% of each target indicating that the program seems to be reaching its goals. The program has also been very successful in leveraging private partnerships and since its beginning, has joined efforts with fourteen local and international organizations for an estimated value of more than $1.6 million in technology support. TILO administered the SCOPE tool in 2009 (the information not yet for the public) but does not use the MAP assessment. The CAPS test was not administered in 2009 at all. Egypt Monitoring and Evaluation Program Aguirre Division of JBS International, Inc. 27 TILO SO 22 Targets: Indicators Target 2010 Y3 – Q1 Number of Administrators/officials trained. M 380 T 800 F 420 M 316 T 537 F 222 Number of learners - Primary schools M 87,384 T 188,186 F 100,802 M 64,647 T 129,420 F 64,786 Number of learners - Secondary schools M 9,424 T 22,924 F 13,500 M 13,080 T 24,519 F 11,439 Number of teachers trained M 3,300 T 5,500 F 2,200 M 1,563 T 3,120 F 1,486 Number of Computers delivered to schools Desktop 1,350 Total 1,720 Laptop 370 Desktop 953 Total 1,142 Laptop 189 Number of Textbooks/Teaching￾Learning Materials Books 2,300 Total 2,447 Kits 147 Books 1,969 Total 2,014 Kits 45 Source: USAID (2009) TILO Quarterly Report Year 3, Q1. Factor Analysis: Efficiency TILO benefitted from previous USAID projects’ materials, data, and experiences in their activity implementation allowing them to proceed at a quicker pace than if they had had to do everything on their own (for example, receiving furniture and vehicles from the National Book Program, using school data from STEAP I and II, exploring opportunities to work with the Adult Literacy Program using ERP personnel and investigating ERP experiences with community use of school resources.). Efficient Effectiveness Project school selection was through an application process by individual schools according to set criteria with input from idarra and governorate level authorities so was participative and transparent. MoE officials were pleased with the decentralized nature of the process. Effective. Teacher training is done at the school level using master trainers who are themselves teachers and employ the active student-centered methodologies. This training structure enables teachers to immediately apply new concepts and skills in their classrooms, receive immediate feedback on practical application, and saves time and money on traveling to outside training sites. Master trainers model the same techniques with teachers which should be in use with students. Effective. The use of technology in education is engaging, can support learning, and appears to answer to human resource development needs for the 21st century; however impacts of ICT on student learning are not clear. ICT should not be central to the education process, rather should be used as a tool to add impact and the development of sound teaching pedagogies should remain the priority. Effective? Partnerships between private international technology companies and local Egyptian organizations not only support innovation and add monetary value to the project, but support human resource development, increase the understanding of the value of education, and promote responsibility of all stakeholders in provision of it. Effective and Sustainable. Sustainability Egypt Monitoring and Evaluation Program Aguirre Division of JBS International, Inc. 28 Developing strategies for on-going teacher support at school level (such as peer teacher groups, clustering schools to enable opportunities for professional sharing and exchange across schools, on-line teacher support communities) enables professional sharing and increases knowledge and confidence of teachers to continue to use new methodologies. Sustainable. The thorough involvement of MoE officials at all levels provides leadership, assists to define roles and responsibilities in the reform process, increases government ownership, and aids institutionalization of change in the system. Sustainable. The work at the central level of government (with curriculum development specialists, technology in education experts, and at the Professional Academy for Teachers) encourages responsiveness to education issues, increases ownership, and facilitates standardized practices. Sustainable 4 Review of Instruments Each of the three testing instruments developed by the ERP in collaboration with the MoE will be briefly reviewed here.38 This will provide a more in-depth look at the efforts to measure education reform, expressed in terms of increased student learning, through the development and use of standardized testing instruments. These tests are one of the results of the work of the ERP at the system level to build institutional capacity of the Ministry of Education and leave sustainable mechanisms that will continue to support the government’s education reform effort after the ERP closes in 2010. Results of these tests offer insight to the effectiveness of the major training interventions (for teachers and school administrators) to improve student learning and could be a foundation for standardized tests of performance. 4.1 Critical Thinking, Achievement and Problem-solving (CAPS) Working closely with the National Center for Examinations and Educational Evaluation (NCEEE), the ERP developed the CAPS test to measure students’ critical thinking and problem solving skills, conceptual knowledge, and factual understanding in the subjects of Arabic, Math and Science for grades 4, 8, and 10. The test was developed based on the National Standards and National Curriculum objectives through a rigorous process of curriculum analysis, item writing, piloting, scoring, selection and baseline data collection in 2006. Thereafter the test was administered in 2007 and 2008 in the seven ERP governorates: Alexandria, Aswan, Beni Sweif, Cairo, Fayoum, Minia, and Qena. In addition to reporting results for ERP schools overall, special program results were also included: the Creative Science Education Initiative (CSEI) and Teacher Learning Circles as well as a sample of schools from the New Schools Program. Test results are meant to help parents, teachers, schools, and education officials see changes in individual student learning as a result of system strategies and to guide a continuous planning process that promotes improved education quality and student learning. The CAPS test was not administered n 2009. Student performance in ERP schools was mixed or variable. To illustrate several examples, mean scores decreased from 2006 to 2007 on all CAPS assessments except grade 8 sciences as 38 A more thorough review of each of the tests will be forthcoming as part of the Egypt Monitoring and Evaluation Program. This document makes no effort to analysis validity or reliability of the tests and testing methodologies. Egypt Monitoring and Evaluation Program Aguirre Division of JBS International, Inc. 29 did performance of students in comparison schools, but from 2007 to 2008, mean performance of students in ERP schools increased in Arabic grade 4 but remained unchanged or decreased in math and science. Performance of students in Arabic in ERP and comparison schools was similar in terms of percentages at performance levels in 2006, 2007, and in terms of means scores in 2006 and 2007 but in 2008, students in comparison schools had a significantly higher mean score than students in ERP schools. Performance of both the ERP and comparison groups has increased steadily from 2006 to 2007 to 2008 in grade 8 science. In all three years, in all grades and in all subjects, urban ERP students on average outperformed students in rural ERP schools. Alexandria outperformed other students across governorates and grades at a statistically significant level (except for science grade 4).39 In conclusion, according to the CAPS results, students in ERP schools are not consistently achieving improved scores as a result of USAID reform initiatives. The fact that students in comparison schools also show improved scores and in some cases, higher scores than ERP students, seems to indicate that other factors are working in the system to affect student academic performance exclusive of the ERP initiatives. From a realistic point of view, student progress on achievement tests takes a long time to become apparent and many components, including but not limited to teacher training, improved curricula, availability and quality of learning materials, and school environment, have to be in place and work well before progress is evident. The prospective use of the CAPS as a basis for standardized testing is possible but with the understanding that a) steady improvements in student learning will take time to achieve, b) many components of the teaching-learning process in classrooms will need focused attention for improved scores to become evident (i.e. not just focus on teacher training), and c) it may be difficult to isolate the individual effectiveness of different interventions which have an impact on student learning. 4.2 Management Assessment Protocol (MAP) The Management Assessment Protocol (MAP) is a standards-based approach for evaluating the management system of Egyptian schools based on the National Standards for Education. The purpose of the MAP is to provide the MoE with evidence of improved school management as a result of education reform initiatives. The test also was used to assess if the ERP interventions were promoting the National Standards through training and other means. Following a pilot run in 2006, the MAP was implemented in 2007 and 2008 in the seven ERP governorates of Alexandria, Aswan, Beni Sweif, Cairo, Fayoum, Minia, and Qena, as well as in New School Program schools in three governorates: Fayoum, Beni Sweif, and Mina and in 13 schools in Fayoum, Beni Sweif, and Minia where both ERP and NSP schools worked together. The test evaluates school management in four sub-domains of the Egyptian Standards for Management Excellence: Institutional Culture, Participation, Professionalism, and Change Management. GILO also administered the MAP assessment in 2009 but results are not yet public. Reported results were positive. Overall results for 2008 were higher than those of 2007. ERP schools improved about 23% and combined ERP/NSP school improved about 28% while random comparison schools improved about 16%. Schools whose directors participated in ERP 39 USAID (2008) CAPS Test: 2008 Assessment Report. Egypt Monitoring and Evaluation Program Aguirre Division of JBS International, Inc. 30 professional development training performed significantly better (2.44) that those who received no ERP training (1.66). Female-headed school directors scored higher (2.38) that those headed by males (2.11). Urban schools outperformed rural schools (2.46 compared to 1.95).40 In conclusion, school directors who received ERP training consistently performed better as determined by the MAP than those who received no training; therefore professional development training for school directors would seem to be a good strategy for improved management performance of schools. Extremely noteworthy was the fact that combined ERP and NSP schools had the highest overall mean scores among individual ERP, NSP, and comparison schools suggesting that programs seem to have a substantial cumulative effect on schools. It will be interesting to see how the 2009 MAP scores from GILO compare to these combined program scores, as the GILO makes an effort to implement in schools which have had no previous program support. Ultimately the MAP assessment would seem to have utility as a measure of improved management ability but should be taken into consideration along other assessment techniques such as observed progress of the schools (i.e. attainment of indicators of progress from the School Improvement Plan). 4.3 Standards-based Classroom Observation Protocol for Egypt (SCOPE) The Standards-based Classroom Observation Protocol for Egypt (SCOPE) was designed to assess teacher classroom behaviors using reform-based teaching methods (i.e. active learning methods) outlined in the National Standards for Education in Egypt. Student behaviors that reflect the use of problem solving and critical thinking skills are also measured by the SCOPE. Following a pilot test in 2005, the SCOPE was implemented in all seven ERP governorates in 2006, 2007, and 2008, targeting grade 1-12 teachers and students in ERP schools. The SCOPE test was also administered in GILO and TILO schools in 2009 but results have not yet been made public. Overall, the SCOPE 2008 results were higher than previous years. ERP schools improved about 61% since 2005 and combined ERP/NSP schools scored higher than ERP- and NSP-only schools in each implementation again adding support to the suggestion of the cumulative effects of program interventions. Results indicate that central and governorate-level teacher training has had significant and substantial impact on teachers’ behavior, but school-based training alone had no significant effect; teachers who received training at all three levels had the best performance. Analysis of results suggests that future training be more specific and directed at specific subjects and skills rather than focusing on generic active learning methodologies alone. Overall student gains have not been as great as for teachers but were showing a greater rate of growth in recent tests suggesting that improved teaching methods were finally beginning to show up in improved student learning abilities and practices. Results further suggest that teacher training has been more successful in moving large numbers of teachers out of the lower rating category associated with previous methodologies than in moving large numbers into the very highest category associated with modern teaching methodologies.41 40 Education Reform Program Monitoring and Evaluation Division (2009) Results Summary: 2008 Management Assessment Protocol (MAP) Results. 41 USAID (2008) SCOPE: Standards-based Classroom Observation Protocol for Egypt 2008 Final Report, Part 1: Overall Analysis. Egypt Monitoring and Evaluation Program Aguirre Division of JBS International, Inc. 31 Not surprisingly, results suggest that teachers do better when they receive more training. The need for long-term support to teacher training at various administrative levels is apparent. 2009 SCOPE results for GILO and TILO will provide an interesting perspective on this issue as both programs have significant school-level training interventions. Changing teacher behavior is also difficult as noted that the training in the ERP and NSP programs hasn’t moved many teachers to highest ratings, rather has initiated a move of many of them up the scale from the lowest rating. This would suggest that teachers need on-going training and classroom support to keep them moving up the scale of competency. Also noteworthy is the slow gain in student performance after four years of teacher training interventions indicating that improving student learning is a slow process. Significant is the suggestion that future training should be more content specific (not just generic methodology training). GILO and TILO results should offer some insight to this issue as both programs do more content-specific training. Again results from combined school interventions suggest cumulative program effects. SCOPE also has utility as the foundation for standardized assessment but teacher assessment should not disregard other methods of evaluation including personal interviews and improved student performance on classroom assessments. 5 Synthesis This section will attempt to synthesize how efficiency, effectiveness, and sustainability in implementation have impacted attainment of the six components of decentralization, gender equity, teacher training, community involvement, curriculum development, and policy reform. This synthesis will allow USAID to determine what still needs to be done in the sector, thereby offering guidance to the next education strategy in the country. Supporting information for this section comes from the Six Component Matrix located in Appendix 6 which indicates which programs have addressed what issues in each of the components. Examples of efficiency, effectiveness, and sustainability given as supporting evidence of implementation of each component come from the review of projects in Section 3 of this document. For ease of reading, a list of all of these factors has been compiled into one list by factor in Appendix 5. As examples are given in the text, the programs from which they come from are noted in parenthesis afterwards. 5.1 Decentralization Decentralization has been a major strategy of USAID education assistance and the programs appear to have made major progress to initiate the process. USAID programs have supported decentralization initiatives at all levels of the system. The efforts of ERP 2 at the system and policy level to build a framework for reform have contributed to sustainability by assisting the MoE to define roles and responsibilities for all administrative levels, develop many of the policies which guide decentralization including the development of the National Strategic Plan of Pre-university Education, and develop an effective data-management system. While decentralization often triggers thoughts of bottom-level programming (i.e. school-level), the reality of transitioning centralized government powers to lower levels means building the capacity of the central and middle levels as well. Egypt Monitoring and Evaluation Program Aguirre Division of JBS International, Inc. 32 Several programs have aided in building capacity of individual ministry departments (e.g. the General Authority for Education Building, Professional Academy for Teachers, EMIS database in ERP 1 and GILO, and the Adult Education Authority in ERP 1) which is seen to add to sustainability. TILO capacity building of curriculum development specialists, technology in education specialists and the Professional Teacher’s Academy at the central level in a process which encourages ownership, increases responsiveness to education issues at school-level, and facilitates standardized practices in education reform could have a major impact on sustainability. The strategy of TILO to initially build capacity and ownership of government officials (stage 1) and then transition to a supporting role as the government officials move to a self-managing role (stage 2) would seem to have merit and should continue. Eliminating support to the upper levels in the next strategy might impact the previous investments of USAID as government officials may be reluctant to give up authority and resources of a centralized system and could revert back to former ways if support is not continued. Outside of the ERP 2, the other programs have focused decentralization efforts mostly at the idarra (but sometimes at the governorate) and school/community levels. Building capacity of the governorate level government quality assurance units to provide guidance and oversight of the school improvement cycle is seen as effective and necessary to decentralized education delivery (STEAP). A process to engage school staff, administrators and communities together to make School Improvement Plans (SIP) has been a major strategy in USAID programming to encourage participation in school reform and has been determined to be effective (ERP 1, GILO, NSP, STEAP). The development and facilitation of school-support groups (especially the Boards of Trustees (BOT)) as representatives of the community in the education reform effort is also considered an effective method of increasing community representation in the system. These initiatives are also seen to add to sustainability and directly support the government’s legal framework to address decentralization of the education sector. Other practices that have been deemed effective are the strengthening of the system of local data collection and use in decision-making and putting into place well-planned monitoring and evaluation processes which model good practices and add to institutionalization (as in ERP 2 GILO, and TILO). Adding incentives (in the form of money) to the system has proven effective to engage many individuals at all levels in the reform process but the system and schedule of incentives should be clearly understood by all stakeholders or effectiveness will be affected (STEAP). Adding to sustainability of decentralization efforts has been the development and distribution of materials as in the Egyptian School Quality Manual and Site Visit Guide (from the STEAP) which could offer standardized guides for school improvement planning; they should however be checked for quality before being publically released.42 Placing assessment manuals and tools on￾line increases accessibility and transparency; however sustainability is at risk if the website shuts down after the program closes leaving schools without this resource (STEAP). Involving schools in an application process for program inclusion is considered by the MoE to be participative, transparent and decentralized (TILO). Collaborating with other projects in decentralization is 42 The Site Visit Guide to Evaluate Egyptian School for Quality for example, has major flaws concerning data collection, recording and analysis. See further explanation in Appendix 3.3. Egypt Monitoring and Evaluation Program Aguirre Division of JBS International, Inc. 33 also considered an effective mechanism to improve coverage, unify support of government reform, and reinforces efforts to institutionalize strategies (GILO). Practices where effectiveness or sustainability may be questionable have to do with the apparent waves of training that have taken place at the governorate-idarra junction.43 Although relevant training to stakeholders is a prerequisite for decentralization to happen, one wonders if all the individuals involved were confused as to their roles and responsibilities, if they had overlapping responsibilities and if they received practical guidance in their jobs along with all the training. With so many USAID programs working in the same governorates, do some individuals get redundant training from different programs? If they do, do trainings complement each other rather than being the same? Do programs work together particularly at the idarra juncture so there is cooperation and systematic implementation (ERP 1 and 2)? From the document review the impression one gets is that decentralization via the training activities appears to be moving quickly in some areas which could negatively impact sustainability or effectiveness (ERP1). Another practice where sustainability was questioned was the large yearly awards given to schools which were distributed to individual staff members rather than being directly connected to implementation of the school plan. By their one-off nature, they do not encourage continued efforts by schools nor does the yearly award system improve equitable incentivization of the rest of the system (STEAP). Efficiency of decentralization efforts was affected when programming was initially too broad and implementation activities spread too thinly across many levels, sub-sectors or parts of the country forcing a re-structuring of programming (ERP 1, STEAP). Efficiency of implementation is evident when programs benefit from previous projects’ materials, data, and experiences on the ground allowing for a faster pace of implementation and the use of already field-tested practices (GILO, TILO). The planning and making of School Improvement Plans is the critical tool for schools to address all reform factors: teacher development, community involvement, gender equity, and ultimately, improved student performance. Generating and managing their own budgets based on the needs identified in their School Improvement Plans and complete autonomy in decision-making are prerequisites for schools to have complete control and future success of this process. For successful decentralization, communities need to see results of their participation efforts through improvements and increased returns to education for their children. Two issues where community members will yet need to be involved are curriculum change to meet local needs and input to teacher management. International experience shows the increased effectiveness of education when driven by community need for better employment productivity and life skills.44 Decentralization would thus assist in quality improvement, rectify issues of centralized management of curriculum and teacher deployment, and enable education to address local problems. 5.2 Gender Equity 43 Seen in the ERP 1 between the Social Work Support Team, Social Work Department members, BOT Support Teams and BOTs! 44 Ibid UNDP (2004). Egypt Monitoring and Evaluation Program Aguirre Division of JBS International, Inc. 34 Gender equity has been most specifically addressed by the ERP 1, NSP, and GILO programs as an access issue. The building of schools in rural areas with community participation has been effective in increasing enrollment, strengthen community pride, and increase attention to education issues (especially for girls). The use of multi-grade classrooms in difficult areas offers a flexible and targeted schooling option to underserved populations but education personnel who service these facilities must be well-trained to meet the challenges presented by these unique situations (NSP). Scholarship programs have also been effective in enabling children to access education but sustainability is questionable as in the long-term the government may not be able to continue paying scholarships (ERP 1). To continue progress in this area, the GILO program should be the model for effective gender￾sensitive practices such as the ‘girl-friendly pedagogy and active learning’ and women leadership training. The insertion of these strategies into other programs should be a standard practice (such as the addition of a gender index into the SCOPE test). Increasing girls’ access to education, especially in rural areas where it is an issue, should normally be addressed through the school improvement planning process by schools, communities and local governments; therefore stakeholders need to be aware that this issue needs to be a part of their work. There is no indication in the documentation of how successful programs have been to increase girls’ education access rates overall (i.e. percentage of girls now in school as opposed to program start) which should be part of the monitoring and evaluation process. Beyond those programs mentioned here, other programs did not seem to address the gender issue specifically although at this point, all USAID education programming should be addressing it as status quo and indicators should be included in programming. 5.3 Teacher Training This issue has also received major attention from USAID programming (ERP 2 at a policy level and ERP 1, GILO, NSP, and TILO at a local level) where significant progress has been made. With ERP 2 support, the development of the Teacher’s Cadre and Professional Academy for Teachers is considered effective and aids sustainability and will support the Egyptian government with standardized and structured professional development delivery. All programming involved with teacher training should be following this system and teacher training interventions should contribute to the professional certification process. Several good pilots have been done where the practice of building a structure of teacher support at the school and idarra levels has been identified as effective and adding to sustainability. Of the three teacher training processes trialed, a) involvement of local Faculties of Education and training at all levels (ERP 1), b) the formation of School-based Training and Evaluation Units (SBTEU) at idarra level (ERP 1, NSP), and c), provision of Master Trainers at school/cluster level (TILO), all have their merits and ground-level evaluation will be needed to add more input to the decision of the most effective training strategy. A strong structure can be seen for example in GILO where idarra supervisors, school administrators, master teachers and peer coaches all have identified roles and duties in strong, local, and standardized in-service delivery for teacher training. Egypt Monitoring and Evaluation Program Aguirre Division of JBS International, Inc. 35 Other effective practices in teacher training are the use of school clusters as a training delivery mechanism which provides local service delivery and avails teachers of an opportunity to share professional experiences (The use of school clusters as a delivery mechanism is also an identified efficiency factor (ERP 1, NSP, GILO)). The provision of on-going in-service teacher development at school level is an effective training strategy which should be continued. Training school administrators and supervisors in active learning methodologies which allows them to offer more positive support to teachers and assist in curriculum attainment has also been identified as an effective practice (GILO, TILO, ERP 1). Other effective practices involve the development of school- and idarra-based teacher professional support models such as Learning Resource Centers and Teacher Learning Circles (ERP 1) and on-line peer teacher support communities within and across schools (TILO) which allow professional sharing and support. Practices which may not aid effectiveness are those that did not develop a clear structure of professional development, had to modify the delivery system several times within one program (which could cause confusion among participants), or did not offer teachers support in classrooms along with their theory training (ERP 1). Programs need to be sensitive to teachers’ needs and time limitations to ensure that they are most able to take advantage of the training, especially if they are responsible for training other teachers. Effectiveness is an issue if teachers are trained and then they do not have time to train others. Overloaded teachers will withdraw from programs thereby diluting program effects and limiting impact of investments (NSP). The low percentage of permanent teachers (conversely the high percentage of contract teachers, sometimes up 80% of total numbers) and the high turn-over of teachers in (especially) rural schools was mentioned several times as adversely impacting training efforts and program investments (NSP, ERP 1, and GILO) thereby affecting efficiency and effectiveness. Teachers are being taught the active learning methods, one part of which is authentic assessment of students. Evidence that teachers are actually increasing their use of the new methodologies in classrooms is variable. Previous SCOPE evaluations show that teachers in individual programs perform better than teachers in non-program schools while teachers benefitting from combined programs do better than all other groups (seen in ERP and NSP schools in 2008).45 In another evaluation, teachers could verbally describe the new methods, while in their classrooms they were not yet confident enough to employ them fully.46 This seems to indicate that multiple methods need to be used to evaluate the success of teacher training initiatives: teacher interviews, observations, and student academic testing. A SCOPE analysis further recommends that teacher training should be more content-specific rather than offer general methodologies alone, a process which should also contribute to greater student learning. Anecdotal evidence suggests that some teachers are not yet comfortable with the active learning methods and are not fully committed to using the methodology in their classes.47 These new methodologies are indeed counterpoint to the rest of the education system in which a dense curriculum, centralized teacher deployment mechanisms, low salaries, and the 45 USAID (2008) SCOPE: 2008 Final Report. 46 Ibid USAID (2008). 47 Egypt Ministry of International Cooperation (2006) Project Evaluation Report No. 16: New Schools Program and from an Egyptian consultant via email. 02/09/10. Egypt Monitoring and Evaluation Program Aguirre Division of JBS International, Inc. 36 competitiveness of the national exams continues to drive rote memorization methodologies, diminishes incentives for teachers to improve their professional competencies, and lessens social demand for public school reform. Teacher training needs to be one of a holistic array of strategies and should focus on building strong local structures for in-service delivery (which includes supervision), on-going practical training in classrooms, professional sharing opportunities, and an improved incentive system based on merit. 5.4 Community Development Community development has been a main strategy of USAID programs as part of the decentralization effort. The foundation has been laid for communities to participate with schools and local officials in the school planning process. This has included the formation of Community Education Teams (NSP) which has lead to the formation by government decree of school Boards of Trustees. Examples of community participation have been in school site selection, construction and maintenance (NSP), activity responsibility and implementation (as in adult literacy and early childhood development activities in ERP 1), school improvement planning (ERP 1, GILO, NSP, STEAP), monetary and in-kind donations (ERP 1, GILO, NSP, STEAP), and attention to under-served populations, most notably girls (ERP 1, GILO, NSP). Efforts to increase community participation are considered to increase sustainability in the move to drive demand for education reform and improved outcomes for their children (GILO, NSP). However, in programs where Boards of Trustees have been facilitated, there is no clear understanding if they truly continue to represent the greater community that elected them to the position or if the rest of the community continued to be involved with the schools. The NSP and STEAP programs seemed to have fostered great participation and excitement among community members which is not always apparent in programs which work through Boards of Trustees (GILO). Also, in programs which indicate the community was involved in assisting the development of a school improvement plan, it is not clear exactly what they did, how much they actually contributed, or what their role was in continued support of the implementation of the plan. Communities which contributed to school improvements in the form of money, labor, or in￾kind (ERP 1, GILO, NSP, STEAP) certainly seemed to have been generous although their contributions should be linked to specific outcomes at the school, for example, as scholarships, teacher incentive bonuses (linked to performance), school equipment or repairs. This will enable transparent use of the funds and can be seen as directly targeting items in the school improvement plan. Establishment of endowment funds that leverage resources in the form of money or in-kind from communities and the MoE, encourage long-term involvement and mutual responsibility for school maintenance (NSP). Efforts to increase community use of the school facility (for example, computer labs in NSP, TILO, and GILO) would seem to be an effective and efficient practice and should strengthen their support of school programs and processes. Communities have also been asked to assist in efforts to promote girls’ enrollment and attendance in school (NSP). International experience shows that communities are good partners in community/school mapping exercises to locate out-of-school children as they know where they live and understand their problems on a local level. Egypt Monitoring and Evaluation Program Aguirre Division of JBS International, Inc. 37 To this point, there is not a clear understanding of how much parents and community members actually monitor and evaluate the actions of the school improvement plan. They will need more support to strengthen their understanding of their roles and willingness to take critical action against the issues which need attention. Their efforts to assist in site location and construction of new schools (as in the New School Program) was determined to be an effective practice; they need to be given more responsibilities to participate in school reform in order to keep their interest and buy-in high. The practice of developing private partnerships to support schools is considered to be both effective and aiding sustainability and should be continued (STEAP, TILO). Private businesses and individuals as well as local Egyptian organizations not only offer valuable resources to schools, they support human resource development, local employment and community building, and promote responsibility of all stakeholders to be involved in the provision of quality education. 5.5 Curriculum Development Of the six components, the curriculum has had the least amount of focused attention given to it by USAID. Strategies that target this component seem to do more to enhance the curriculum rather than actually develop it. Content development was addressed by the offering of Life Skills (ERP 1), the current focus on technology and related curricular topic materials of the TILO and GILO projects, and the gender-sensitive materials development and Early Grade reading program of the GILO program. GILO is piloting the Early Grade Reading program which could offer the government an effective strategy for improvement of literacy skills of young children. GILO has also had requests from the grade 4 and grade 7 science and math teachers in two governorates to provide supplemental trainings in these topics in support of the new science and math curricula for these grades which could be an effective way to up-date teachers’ content knowledge in the future. TILO seems to be generating great excitement with the development and provision of digital resources directly supporting national curricular topics, which could enable effective computer-assisted instruction in specific topics. Quality learning kits distributed to schools in ERP 1 could be considered effective but teachers were observed not using them well or at all; therefore efficiency was affected. The use of technology in education is a high priority of the MoE in the belief that student academic performance will improve but the positive impact of ICT in education has not been proven. Moreover, even in the most advanced schools for example in countries of the Organization of Economic Co-operation and Development, ICTs are generally not central to the teaching and learning process; rather positive effects on student achievement may come about when ICTs are used as tools to appropriately complement teachers’ existing pedagogical philosophies.48 Plus, the use of ICTs is seen to be less effective when the goals for their use are not clear or specific; therefore technology in education may not be the panacea the government hopes it to be. The more deliberate strategy should remain on developing sound pedagogical principals in teachers and the use of technology as a support tool. 48 InfoDev (2005) Briefing Paper: Impact of ICT on Learning and Achievement. At www.infodev.org Egypt Monitoring and Evaluation Program Aguirre Division of JBS International, Inc. 38 Realistically, the curriculum seems to be failing many Egyptian children (witness the number of older boys who seem to drop out of school because they are bored with what’s offered in class and the number of graduates who don’t get jobs because they don’t possess the right skills.). The government is currently working to reform the national curriculum to include minimum common learning objectives that allow assessment of student and school performance. Based on this framework, administrators, teachers, and the community should then be able to develop locally-appropriate learning and teaching programs as is promoted in the National Strategic Plan.49 A review of the curricular plan indicates however that it does not yet seem to be operational containing no implementation plan rather is just a rubric of anticipated change.50 A clear implementation guide containing a plan of action and all stakeholders’ contributions seems to be a prerequisite. Revision of the curriculum should include the practice of on-going authentic student assessment rather than using only year-end tests to assess performance. 5.6 Policy Reform Through the facilitation of the ERP 2, a policy framework exists to guide attainment of standardized measures of improved performance of the sector. The ERP 2 facilitated policy and system reform in the following areas: organizational development, teacher professional development (Teacher’s Cadre and Professional Academy for Teachers), strategic planning (National Strategic Plan for Pre-University Education 2007-2012), student performance (CAPS test), M&E and quality assurance, and Education Management Information Systems. In terms of time and degree of structural and policy change made possible by the ERP 2, this effort was considered efficient. Despite the time now needed for implementation and institutionalization, apparently a large part of the framework is in place and should provide the foundation for reform. The National Strategic Plan is in place for system guidance but it is very detailed and does not provide practical tools for application at the school level. It would benefit from development of a scaled set of practical indicators that schools could target on a yearly basis (somewhat as were designed in the STEAP). The main factors limiting change and implementation of the policy framework may indeed be the MoE officials themselves despite being the force behind the push for decentralization. Reasons identified for hesitation to change include lack of commitment and resistance to change, new laws and decrees that do not have accountability and transparency requirements for implementation, lack of vision and capacity at local levels, and lack of resources and knowledge of procedures. Some issues concerning community participation are still not clear leading to misunderstandings and thoughts that Ministry and officials will lose their authority and responsibilities in the education system. Data-informed decision-making at the school level is still hindered by lack of computer technology, knowledge, and skills, lack of technical expertise in data collection, storing and analysis, and few data management system professionals 49 UNDP (2004) Egypt Human Development Report. 50 Egypt Ministry of Education (undated) National Education Indicators (NEIs) for Egypt: Proposed Technical Guide. Egypt Monitoring and Evaluation Program Aguirre Division of JBS International, Inc. 39 at all levels of the MoE.51 Therefore, despite the framework being in place, decentralization efforts will still require focused support by all stakeholders. 6 Conclusions and Recommendation 6.1 Conclusions USAID programming over the last 8-10 years has added much support to the Egyptian government’s vision of improved education for their children. Programming however has been quite broadly focused, addressing many topics and making the task of identifying the main end goal, beyond [the very broad] improved student learning, difficult to discern. What has been progress to date? What percentages of the problems in education have been addressed by USAID reform efforts? Despite continued use of the National Strategic Plan of the MoE as the sector guide, the plan after all has five domains and contains hundreds of indicators; therefore what exactly the USAID programs are aiming to achieve of this plan is difficult to see. In agreement with the Ministry of Education, what does ‘School-based reform’ mean and what are the indicators that it has been achieved? The ERP particularly was huge and seemingly ‘thinly spread’. The more recent programs, GILO and TILO, seem to be more tightly focused according to planning documents but verification of actual implementation will have to come from the field. Major inroads into the decentralization process have been made but again, there appears to be little vision of what the end will be and no indicators that expressly show it. Seemingly education decentralization efforts would be easier to achieve if decentralization was a national initiative so that efforts in education wouldn’t be working in isolation from the rest of national policy. The success of decentralization of course rests on the willingness of the Ministry of Education to make it happen. Laudable efforts have been made on the part of USAID to increase community participation at local schools and the practice should be continued. As the receiver of the service of education, community and parents should see an improvement in their children’s learning and future earning ability. Their demands for better education service could be a major driver of reform efforts. Teacher training has also received major attention from USAID programming and answers an identified need by the MoE of improved teacher quality. While efforts have not been unified across programs, this initiative seems to be having an impact according to the SCOPE results on teacher behavior. The cumulative effects of programming on teacher and administrator behavior have come out strongly in both the SCOPE and MAP; therefore future programming efforts should take program location into consideration. Also coming out quite clearly is the need for teachers to receive on-going training in a supportive environment. The construction of structures of support (i.e. knowledgeable and reinforcing school managers and supervisors at the idarra level) would seem to have merit in effectiveness and sustainability in the sector. Low salaries will continue to negatively affect teacher performance however, and assist to sustain the rampant private tutoring system. 51 Information from this section from: Ginsberg, M. and N. Megahed. (2008) Education Reform Program: Support in the Area of Governance and Management. Egypt Monitoring and Evaluation Program Aguirre Division of JBS International, Inc. 40 Training for community groups, i.e. Community Education Teams and Boards of Trustees and their support groups, was seen to be quite extensive and a confusing, ‘muddy’ process involving many individuals at the idarra level. Is the training redundant? Have these trained individuals been able to continue the use of their skills and knowledge now that programs have ended? If not, then the effectiveness of USAID investments has been seriously impacted. USAID programming has not focused so much on curriculum development. The variable results of the CAPS in the last few years and the poor scores in the last two TIMSS seems to suggest that more work should be done in subject content rather than focus just on delivery. Addition of materials to the sector assists but won’t make great inroads to attainment of the curriculum without a greater unified effort. Technology should be seen to assist in curriculum attainment, not drive it. The development of the CAPs, MAP, and SCOPE assessments built the capacity of the MoE to develop standardized testing tools and their continued use will aid the MoE to evaluate reform progress. The MoE and USAID should keep in mind the length of time and the concerted many-faceted efforts that need attention before student performance shows serious improvement as measured by the CAPS. Curriculum enhancement efforts by USAID should be synchronized with current MoE curriculum revision efforts to add impact and lessen fragmentation. Program locations and target populations responded to identified needs of the government. USAID programming aided many girls to access education but what percentage of the out-of￾school population has been addressed? Certainly rural areas have benefitted well from USAID programming and the great efforts to build schools and provide multi-grade classrooms aided the success of girls’ access initiative. The program literature is showing however, that successful strategies in rural areas don’t always work well in crowded urban schools (for community participation activities also); therefore future programming needs to look more closely at specific strategies and indicators to address specific locations. Evaluation efforts generally seemed to be a little ‘light’; therefore program impacts are difficult to discern. ERP made an excellent database to track all program data and its use by the MoE should be continued. Other programs also developed their own databases. Are they all connected and still in use? The use of data for decision-making is a very effective mechanism and should be a standard operating procedure for programs to model the process and contribute to the national EMIS. In some programs, it appears that the process of making the School Improvement Plan is the end goal without ascertaining that the components that are addressed in the plan, i.e. increased girls’ enrollment, decreased class sizes, improved teacher classroom behavior, etc. are improving. Is the School Improvement Plan truly a working document at school level, or is it just a requirement that schools fulfill for the government? Perhaps more attention on yearly results and the resultant yearly up-dating of the plan is needed. 6.2 Recommendations for Field Evaluation Recommendations made here are meant to guide the field evaluation highlighting areas that seem to be key issues in need of closer inspection. This meta-evaluation was based strictly on a review of documents however and results from the field may diverge drastically from the findings of this report. The following recommendations are made: Egypt Monitoring and Evaluation Program Aguirre Division of JBS International, Inc. 41 1. Teacher training initiatives and structures should be thoroughly reviewed. Teachers should be questioned/observed as to their ability to use new methodologies. Training structures should be in place and continue to be useful to support teachers in classrooms. USAID has put many resources and efforts into teacher training; to protect their investments, these efforts need to be assessed to guide future programming. Moreover, improved education quality is very dependent on improved quality of teachers. 2. As decentralization has also been a main focus of USAID programming, community-level leadership and participation should receive considerable attention from ground-level evaluations. Assessment should ascertain if Boards of Trustees are in place and effectively coordinating with schools to improve performance. Communities should continue to actively participate in the school improvement process also. 3. School leadership, i.e. administrators and supervisors, should be seen to be actively involved in improving quality at their schools. They should be knowledgeable and participative in the process. The School Improvement Plan should be an actively working document, developed with input from all stakeholders and clearly guiding improvement of the education process in schools and communities. 4. The issue of girls’ education attainment should be given close attention. According to the most current education data available for this report, there seems to be an emerging trend of improved enrollment and attainment for girls which should be checked out on the ground. 5. Education leaders on a system level should be assessed as to their willingness to support changes in the lower levels of the system and moreover, should be knowledgeable of current reform efforts and have an opinion as to what still needs to be done to improve education in the country. Egypt Monitoring and Evaluation Program Aguirre Division of JBS International, Inc. 42 Appendices Appendix 1: Scope of Work This document addresses task #1 below. The other tasks are included in the evaluation in its entirety. TITLE: Education Monitoring and Evaluation Program (EMEP) OBJECTIVES: The primary objective of this task order is to provide technical assistance for the evaluation and monitoring of USAID/Egypt’s Education Portfolio. Outcomes include: 1. USAID will be informed of the efficiency, effectiveness and sustainability of its education portfolio. 2. USAID will be informed of suggested steps for the design of its next education strategy. 3. USAID’s monitoring and reporting of its education activities in Egypt will be improved. SCOPE OF WORK: This section defines the tasks included in this evaluation. 1. The contractor will begin by conducting a Meta Evaluation of existing USAID Evaluations as well as a thorough desk review of recent Egypt education sector assessments. The following documents, and others, will be made available: • National Book Program, Final Evaluation, 2008 • New School Program, Final Evaluation, 2008 • Critical-thinking, Achievement, and Problem Solving Skills (CAPS) Test, Reports 2006, 2007, and 2008 • Management Assessment Protocol (MAP) Reports 2007 and 2008 • Standards-Based, Classroom, Observation Protocol for Egypt (SCOPE) Reports 2006, 2007, and 2008 • 2002 USAID sector assessment • World Bank Sector Notes, 2007 • UNESCO Sector Assessment • Other The information gathered from this desk review will be used in tandem with the activity evaluations (see #2 below) to make activity recommendations in the final report. 2. The contractor will use the information from the desk study in the evaluation of six USAID education activities (GILO, TILO, STEAP, ERP, NSP, and EQUIP 2). This evaluation will take a critical look at the efficiency, effectiveness, and sustainability of these activities. It will consider the impact of all USAID education activities in the areas of decentralization, gender equity, teacher training, community involvement, curriculum development, and policy reform. The contractor will submit a report and conduct a presentation on the desk review and portfolio evaluation within four months of arrival in Egypt. Suggested questions include: What did USAID accomplish in these areas? What was successful and effective? What failed? What still needs to be done in these areas of intervention? What was sustainable? 3. The contractor will use existing instruments (SCOPE, MAPS, CAPS) to measure impact of program interventions. The contractor will collect data on an annual basis, analyze data, and report on findings. Egypt Monitoring and Evaluation Program Aguirre Division of JBS International, Inc. 43 Appendix 2: Current Situation of Education in Egypt The strong commitment of the government to sector development has resulted in significant progress. Egypt has met many of the access challenges to education and now is able to provide education to most of its children in basic education, grades 1-9. Almost universal enrollment at this level has been attained. Gender parity as well has nearly been reached and girls make up a total 48% of enrollments in these grades. The quality of education in Egypt remains the major issue in the system today. Successful expansion in basic education enrollments has exacerbated the quality and relevancy of education; it remains low and unequally distributed further compounding educational outcomes across income, geographic, and gender divisions. The generally low academic performance of students, as qualified by for example, the Trends in International Math and Science Survey (TIMSS)52, reveals that the education system is not preparing Egyptian students for the 21st century marketplace. The salient reform issues and their current status will be reviewed here in order to ‘set the stage’ for the remaining challenges in the sector. In this document, the domains of access, equity and quality are used to organize and present these issues, but the lines between them sometimes overlap, often presenting a web of problems which impact each other. Ultimately, the interrelatedness of these issues should guide reform in the sector, the best approach to which should be a holistic array of interventions. Current data are difficult to obtain with the most complete set being available from 2005/2006.53 Even the Ministry of Education notes that their strategic planning is based on current population projections from the last census by the CAPMAS influenced by yearly reported school data. The most recent Ministry of Education data accessible for this report are from 2007/2008. A. Access In Egypt, 90% of students attend public education. About 7% of the student population, grades K-12 are enrolled in private facilities, most of which are in urban areas and cater to the high and middle income student populations. Enrollment in the Al-Azhar system of religious-based education has increased significantly from about 8% in 2001/02 to 13.2% in 2005/06.54 The Egyptian system is one of the largest in the world, containing 16 million students.55 In pre￾university education, at the primary level, gross enrollment (GER)56 stands at 96% and at 52 Trends in International Math and Science Survey (TIMSS), a standardized test of math and science for 4th and 8th graders, is used to compare education achievement on an international basis. The exam tests student knowledge of basic math and science concepts which should be standard curricula content for grade 4 and 8 learners. 53 From World Bank (2005) Analysis of Education Expenditures in Egypt. 54 UNESCO (2008) 55 Egypt Ministry of Education Educational Statistics Academic Year 2007/08. 56 Gross enrollment: The total enrollment (regardless of age) as a percentage of the population in the official age group corresponding to a particular level of education. (World Bank). Egypt Monitoring and Evaluation Program Aguirre Division of JBS International, Inc. 44 preparatory level (grades 7-9), at 93%. The two levels on either side of basic education however, pre-primary on the lower end and secondary school on the upper, have not yet attained full enrollment and represent the initial challenges to the sector. Pre-primary (Ages four to five) Enrollment in early childhood programs (ECE) was about 24% in 2007-2008.57 Pre-primary education provision has been mostly in urban areas (by a three to one ratio) and is about 30% privately provided. The Egyptian government has committed only minor funding to this sub￾sector (less than 2% of the total expenditure on pre-primary education in 2007/08) and has previously encouraged private providers to supply the sub-sector. However, in the latest National Strategic Education Plan, the government outlines plans to re-classify the pre-school program as free and compulsory as is the rest of the system and aims to provide more pre￾school classrooms, especially in rural-remote areas. Secondary (Combined general and vocational) Secondary level (grades 10-12) enrollments are 76% (girls 52%), still significantly lower than that of primary and preparatory levels (See Figure 158). Access at the secondary level is affected by supply and demand side issues. On the demand side, family poverty ‘pulls’ students out of education so that they can contribute to the financial support of their families. On the supply side, the low quality of education at this level acts as a ‘push’ factor for the poor. According to CAPMAS59 on the educational characteristics of the Egyptian Society in 2005/06, academic failure (an aspect of poor quality) constituted 92.5% of the reasons why poor students dropped out of school at this level.60 Source: UNESCO 2008 Current reform efforts to increase secondary enrollments have included extensive in-service training of secondary teachers; provision of secondary schools with infrastructure and equipment needed for the use of new technologies; and modifications of assessment procedures, final exams, and the tracking system at the upper level of education. B. Equity 57 Egypt (2007) National Strategic Plan for Pre-university Education. 58 Data for all the tables can be found in Appendix 1. 59 CAPMAS: Central Authority for Public Mobilization and Statistics. 60 As reported in UNESCO (2008). Egypt Monitoring and Evaluation Program Aguirre Division of JBS International, Inc. 45 In Egypt over the last 20 years, the greatest increases in enrollment and education attainment have been made by children from the most disadvantaged groups, the poor, and those who live in rural areas and Upper Egypt. Increased enrollment has been linked to extensive school construction; as a result, gender, wealth and regional disparities in education have declined over time. Recent reports are that about 99 percent of all villages have access to primary schools.61 However, equity issues continue to surface among diverse population groups. Income Disparities In Egypt, there is strong correlation between poverty and education. Illiteracy rates are more than twice the rate for poor children than among non-poor children. Enrollment rates increase as income increases across all levels of education (Figure 2).62 Source: World Bank 2007 Inequality across income quintiles is lowest for primary enrollment, where only 12 percentage points exists between the richest and poorest quintile (97% and 85%). However, secondary enrollment drops twice as fast as primary enrollments across quintiles and a gap of 30 percentage point exist between the richest and poorest population levels (74% and 44%).63 A 2009 study using 2005 Demographic and Health Survey data found that among children from the wealthiest families, 99 percent of males and females had enrolled in schools and more than 90 percent had stayed through to the end of their secondary education; therefore only the wealthiest quintile of children were approaching universal school completion. Meanwhile, among 19-22 year old females from poor families in rural Upper Egypt, only 50 percent had ever enrolled in school and just 27 percent completed secondary school.64 This situation may be changing however due to the increased focus of education reform efforts in rural areas. In another instance of income inequity, private tutoring, a mainstay of the Egyptian education system, further puts poor students at a disadvantage in accessing education. Formally a remedial activity used for groups of students to improve their school performance, private tutoring now 61 Langsten, R. and T. Hassan (2007) Basic Education Attainment in Egypt: Trends and Determinants. 62 Ibid UNESCO (2008). 63 World Bank (2007) Improving Quality, Equality and efficiency in the education Sector: Fostering a Competent Generation of Youth. 64 Langsten, R. and T. Hassan (2009) Education Transition in Egypt: The Effects of Gender and Wealth. Egypt Monitoring and Evaluation Program Aguirre Division of JBS International, Inc. 46 is most often offered by teachers to individual students either in or out of the school facility. Driven by low teacher salaries, perceived poor quality of in-school instruction, and the competitiveness of the national exams which guarantees entrance to university, an estimated 60% of students at the secondary level pay for private tutoring. The richest quintile of students spend approximately 10 times more than the poorest quintile are able to, thus giving them a greater advantage in accessing more desirable education options.65 Geography and Gender Disparities: Historically, the greatest disparities in enrollment rates were seen in Upper Egypt where more than 50% of Egypt’s poor live. Governorates in this region (as noted below using information from 2003/04) showed both lower rates of enrollment than either Lower Egypt or metropolitan areas (except for Aswan) and in four of the seven governorates, lower rates of enrollment for girls (very distinct especially in Beni Suef and Fayoum). Over the last 15 years, failure to enroll in primary school was highly concentrated in rural Upper Egyptian girls from poor families. Source: World Bank (2005) According to the Ministry of Education however this situation has changed since then as the reported data in 2007/2008 showed that girls comprised 48% of both primary and preparatory enrollments (total primary enrollment: 96% and total preparatory enrollments: 93%).66 Extensive school construction (including one-classroom schools and girl-friendly schools) and gender-focused programming seems to have been successful by increasing girls’ enrollment and 65 World Bank (2007) 66 Egypt Ministry of Education Educational Indicators for All Governorates. 2007-2008. Egypt Monitoring and Evaluation Program Aguirre Division of JBS International, Inc. 47 attainment rates at a faster pace than for boys. However, as of 2005, the MoE reported that despite a high pass rate of the Primary Certificate Exam (94.8 % for girls and 90.2 % for boys), of all students enrolled in first primary grade, 87% from urban governorates and areas from Lower Egypt completed basic education, 82% from rural areas in Lower Egypt and urban areas in Upper Egypt completed, and 73% of students from rural Upper Egypt completed primary education. At the preparatory level, in 2007 the reported total enrollment was 93%, up from 77% reported in 2005 by the MoE.67 At the same time, children from the poorest segment of the population not enrolled in preparatory schools were three times the number of middle- and wealthiest class counterparts. It would seem therefore, that strategies to increase enrollment in grades 7-9 have been successful and the issues in school enrollment appears to have been pushed up to the next level of education although rural students’ difficulties in accessing education continues. The enrollment rate in total secondary education in 2007/08 was 76% (girls: 52%). According to the MoE as of 2005, students living in rural areas accounted for only 24.2% of general secondary education while students from urban areas represented 75.8% of the total number. Using 2005 data, one researcher found that poor females in rural Upper Egypt continued to have the lowest level of education attainment, meaning that they are not getting through to grade 11. The same researcher also found however, that while only 78% of middle class rural upper Egyptian females enter school, once having entered more than 90% of eligible females make each subsequent transition.68 Once enrolled however, girls appear to be quite successful in transitioning through the grades, thereby justifying strategies to address gender and geographic differences. Surprisingly, the wealth group that made the least progress in educational attainment over the last 18 years is poor urban boys, whose secondary school attainment increased by just 3 percentage points over that time. They lagged behind every other group in secondary completion rates except for poor rural girls from Upper Egypt. While the main issue for girls from this region was always failure to enroll in school, dropping out of school played the larger role in education attainment for boys.69 Findings from the Egypt Household Education Survey indicated that the main reasons children may not attend school was a) monetary costs of schooling, 2) parent’s/guardian’s attitudes about schooling (in which formal schooling was not considered important for children or for girls especially) and 3) the opportunity costs of sending children to school thus sacrificing the financial contribution they would make to family income.70 Parents have reported that children, especially boys, leave school early because of lack of interest and poor performance at school. Distance to school may prevent some children especially girls, from attending school. Information from some school projects in rural Upper Egypt report that poor girls living close to schools enroll, as do almost all girls from wealthier families regardless of the distance to 67 Ibid. 68 Ibid Langsten and Hassan (2009). 69 Ibid 70 USAID (2005/06) in UNESCO 2008. Egypt Monitoring and Evaluation Program Aguirre Division of JBS International, Inc. 48 school; therefore school construction in rural areas is an important intervention to address school access.71 Other conditions that may affect students’ school attendance include financial constraints of the government to meet the demand for expanded formal school coverage and school maintenance, high student to teacher ratios, ineffective instructional methods and other dimensions of school quality that limit capacity of the formal system to prepare students with basic skills, and lack of community involvement to monitor government accountability in quality education provision.72 Efforts to decrease the disparities between the poorer and richer segments of the population of children have included rural construction efforts focusing on for example, girl-friendly community schools, one-room schools and multi-grade classrooms. School scholarships particularly for girls seem to have been influential in enabling children to attend school. Clearly though other options such as quality improvements in classrooms need to be concurrently offered in order to keep students interested and attending school. Lastly, there seems to be an extreme need for credible, comprehensive assessments of the current situation of girls’ school enrollments in rural areas and boys in urban areas, to see if and when they encounter difficulties in accessing education and what that means for future policy and program efforts. Academic Disparities The system of tracking students at the secondary level into general and technical vocational education based on the results of the grade 9 exam contributes to system inequity by unfairly dictating subsequent education and future employment opportunities. The approximate 33% of students with high scores in the preparatory end exam are offered general education opportunities which in most cases, translates into guaranteed access to university education and better future employment opportunities. Meanwhile, the remaining 57% of students73 are tracked into lower quality technical vocational education generally with no further opportunities for academic tertiary education. As the education received in the technical education stream has low academic standards and questionable relevance to the labor markets, these students are poorly prepared to enter the job market at school completion.74 This system of academic tracking is quite entrenched, and will need serious reform if secondary education is to produce the skilled workers demanded by today’s market. Efforts of the government to address this issue intend to focus on linking the technical vocational streams closer to the general education stream, strengthening core skills (math, science, language) in all of secondary education, allowing greater flexibility for students to move between the two streams so they can match their interests with appropriate courses of study, and finally, allowing private sector input to influence a more relevant curriculum that meets labor market needs.75 C. Quality 71 Ibid Langsen and Hassan (2009) 72 World Bank (2005) 73 The remaining 10% attended private general secondary schools or Al Azhar education. 74 World Bank (2007) 75 Ibid Ministry of Education (2007). Egypt Monitoring and Evaluation Program Aguirre Division of JBS International, Inc. 49 Educational quality includes myriad variables which singularly and together impact the effectiveness of the teaching and learning process in a school, classroom or educational system.76 Professional Staff Issues of the education workforce have had major impacts on the education system for many years. As of the 2007/08 school year, the Ministry of Education reported a teaching force of 658,431 for all grades, pre-primary to secondary levels. At the same time, the MoE reported a non-teaching work force (administrators, librarians, janitors, etc) of 435,187 in schools across all levels, a ratio of 1: 0.7; therefore a large part of the education budget supports a non￾teaching/administrative force that has no direct bearing on the teaching-learning process in the classroom. Meanwhile, the quality of the teaching staff has been an issue which has direct impact on how much students learn in the classroom. They most often adhere to the rote teaching methodology which does not encourage critical and creative thinking of their students. Their low salaries force senior teachers to move into administrative ranks, leaving the most inexperienced teachers in classrooms. Their low pay also encourages them to participate in private tutoring to supplement their income. Pre-service training is of variable quality although currently there are many Faculty of Education initiatives and reforms to improve the service. Students attend the Faculties of Education because of an almost guaranteed job; therefore they may not have excessive desire to become teachers. Many of them complete pre-service training without having had practical in-classroom experience.77 Furthermore, novice teacher’s areas of specialization and teaching credentials are often not considered when they are hired which means literally for example, that a graduate of the art history faculty could be assigned to teach English. In-service training is guided by the Central Directorate of In-service Training with a branch in each governorate, but is reportedly of doubtful quality. In-service training is provided extensively by donor programs. To improve the quality of the education workforce, a Teachers’ Cadre, established in 2007 aims to standardize the process of professional teacher development, certification, and promotion. The process addresses both pre- and in-service teacher training. The government has been focusing much attention on the development of the Teacher’s Cadre and the Professional Academy of Teaching to ensure a unified and consistent delivery mechanism down to the school level for professional development of teachers and improved classroom teaching and learning processes. The use of reform-based methods of teaching is now espoused as the most effective method to improve student performance.78 As of August 2008, 829,849 teachers, administrators and specialists had sat for Cadre placement tests and achieved a pass rate of 92.45%.79 The Ministry of Education reported that the number of administrators in basic education schools has been reduced by 25% to enable a 76 USAID (2008) Final Evaluation of the New School Program 77 Sheta, T.A. (2007) Novice Teachers: Current Situation and future Directions. 78 In a move away from rote lecture and memorization methods, reform-based methods espouse interactive instruction and learning such as learner- or child-centered teaching, cooperative learning and active learning. 79 USAID (2009) ERP-EQUIP 2: Policy and system Change: Final Report. Egypt Monitoring and Evaluation Program Aguirre Division of JBS International, Inc. 50 more realistic teaching-administrator ratio.80 The implementation of this cadre is expected to reduce the number of teachers who leave teaching in favor of administrative posts in order to improve their financial status and enhance their career. In the meantime, the MoE reported in 2006, that there was a teacher shortfall at the primary level estimated at 86,743 teachers with the largest shortage recorded in Minia and Sohag.81 Improved teacher performance will depend on increased salaries but should be pegged to qualifications and performance and include supportive supervision. International experience shows that no changes in behavior will be politically feasible or otherwise likely to succeed without some significant changes in salaries and enhancement of working conditions for teachers.82 Professional development of school administrators is receiving much attention from the Ministry of Education now as the professional promotion track of the Teacher Cadre is nearing completion. Job descriptions for school principals and deputies are now in place while the process of developing the professional certification for school leadership and revision of the structure of school administration (to re-adjust the number of administrators at schools) are now in reform. Strategies which require school administrators and supervisors to attend in-service trainings on active learning methodologies along with the teachers and capacity development of idarra and governorate level ministry teacher cadres to continue effective teacher training (such as the New School Program83) would seem to have much merit in improving abilities of individuals responsible for student academic achievement. Curriculum The general curriculum seems to be rigid and lengthy.84 According to the National Strategic Education Plan, the main issues of the curriculum relate to the: a) predominance of traditional subject-centered teaching and learning methods; b) predominance of traditional assessment techniques that emphasize memorization; c) poor use of technology in the educational process; and d) the failure of the national curricula to meet the needs of different geographical environments.85 Its relevance to workforce needs may be at least partially to blame for the high unemployment of school graduates due to a mismatch of skills to market needs. Curriculum development has been a recent focus of the government as part of a unified focus (including teacher development, textbooks, technology, and student assessment) on improving student academic performance. As of 2009, the MoE had reformed curricula to reflect ‘new modern’ methods that enhance comprehensive assessment and active-learning through ‘new standards’ for grades 1-4 and 7.86 Examination system 80 Ministry of Education (2007) National Strategic Plan for Pre-university Education, 2007-2012. 81 Ibid. 82 Ibid World Bank (2007). 83 Ibid USAID (2008) NSP: Final Evaluation. 84 Ibid USAID (2008) NSP: Final Evaluation. 85 Ibid Ministry of Education (2007) National Strategic Plan for Pre-university Education, 2007-2012p97-99. 86 Ministry of Education (2009) Strategic Plan Implementation First Report 2007/2008. Egypt Monitoring and Evaluation Program Aguirre Division of JBS International, Inc. 51 The competitiveness to pass the national school-leaving exam at grade 12 is the main driver of the current in-class rote memorization method of ‘learning’ as students who pass are then able to attend university. Those who fail have few options beyond vocational school. Student performance on the TIMSS is low and has not improved between the two participation dates of 2003 and 2007. Typically, students do not learn and are not evaluated in cognitive, problem￾solving, and critical thinking skills which are necessities to increase the competitiveness of Egypt’s workforce in the world market. Student assessment has also been the focus of much attention by the government recently, working hand-in-hand with improved teaching-learning processes in the classroom and curriculum development. Participation in internationally-recognized exams such at the TIMSS is to be continued to allow the government to track its own progress in education reform. Also tests such as the Critical￾thinking, Achievement, and Problem-solving Skills (CAPS) could have utility as a standardized test but should be well-administered over a wide base to ensure reliability, standardization, and acceptance. Classroom Density While the calculated teacher-student ratio (the ratio of number of students to number of teachers on a country-wide basis) at primary level is now 25.6:1 and secondary level 14:1, many classrooms exist particularly in disadvantaged urban areas and rural governorates, where student numbers are high (above 40 or more) per classroom (i.e. class density). Large class sizes are a reflection of unequal distribution of teachers (not enough teachers or teachers unwilling to go to disadvantaged areas) and uneven levels of teaching to non-teaching staff (the more experienced teachers leave teaching to become administrators in better school districts). Source: World Bank (2005) The poorest provinces and populous metropolitan areas such as those seen in Figure 4 have the most crowded classrooms and the fewest and least experienced teachers. Projects from the field report that at rural schools only 15-25% of the teaching staff may be permanent and turn￾over of staff is a frequent occurrence thereby diluting capacity of teachers and project Egypt Monitoring and Evaluation Program Aguirre Division of JBS International, Inc. 52 investments in improving quality in classrooms.87 The Ministry of Education is making a great effort to control class sizes to a country-wide average of 40 (2005 average: 37 in primary, 40 in secondary). 87 USAID (2009), USAID (2008). Egypt Monitoring and Evaluation Program Aguirre Division of JBS International, Inc. 53 Appendix 3: Data for Appendix 1 Figures Figure 1: Gross Enrollments: Primary, Preparatory and Secondary. 1996/97 1998/99 2000/01 2002/03 2005/06 2007/08 Primary 105.1 105.1 104.6 104.7 96 96 Preparatory 83.2 96.2 104.5 103.1 96.9 93 Secondary 59.1 62.8 68.8 75.9 72 76 Source: UNESCO 2008 Figure 2: Enrollment Rates by Income Quintile and Education Level 2004/05 Poorest 2 3 4 Richest Grade 1-9 85.45 90.76 93.9 95.65 97.63 Secondary 43.97 51.09 60.77 67.82 73.07 University 9.67 12.93 19.52 28.56 46.95 Source: World Bank 2007 Figure 3: Net Enrollment rates for Preparatory Education by Selected Governorates and Gender. Metropolitan Upper Egypt Alexandria Cairo Aswan Beni Suef Fayoum Minya Assiut Suhaq Qena Males 86.8 83 92.7 69.8 72.2 82.5 73.2 75.7 73.2 Females 91.6 86.1 93.4 63.2 64.5 69.9 70.2 75.5 73.1 ** USAID governorates are in bold. ** Non-USAID governorates are in blue. Source: World Bank 2005 Lower Egypt Beheira Damieta Ismailia Sharkia Males 71.2 76.3 74.6 79.3 Females 73.9 86.4 77.5 85.9 Egypt Monitoring and Evaluation Program Aguirre Division of JBS International, Inc. 54 Figure 4: Class Sizes: Primary Education by Selected Governorates and Percentage of Children in Classes of 40+ 41-50 51-60 61-90 Assuit 50.9 18 0.9 Beheira 37.6 8.9 0.8 Alexandria 29.1 15 13.9 Giza 34 8.6 4.6 Cairo 26.8 8.6 5.1 Minya 55 11.4 0.3 Qena 42.4 7.1 0.3 Fayoum 61.4 6.7 0 Souhag 44.9 9.9 0.4 Source: World Bank 2005 Egypt Monitoring and Evaluation Program Aguirre Division of JBS International, Inc. 55 Appendix 4: Additional Project Information in Alphabetical Order 1. ERP 1 Original ERP 1 program divisions included the following: 1. Non-formal Education and School Governance Division: promoting out-of-school early childhood development, adolescent life skills, adult learning activities and school governance efforts. It had three components: Integrated Literacy (IL), Early Childhood developmental Life Skills (ECD and LS), School Governance (SG), 2. Educational Quality Division: improving quality of instruction and learning in the formal school system through the use of standards-related training for teachers and educational leaders as well as monitoring tools to measure the quality of school instruction and relevance of the curriculum to ‘life-long learning’ and local employment opportunities, with the following components: Administrative and Leadership Development (ADL), Teacher Professional Development (TPL), Implementation of National Education Standards (SI), School-to-Work systems (STW). 3. Community Organization and Development Division: focusing on three aspects of community participation, a) developing community-based organizations (CDA) to assess and address their own educational needs, b) enabling communities to manage some of their own educational services such as early childhood development, literacy, community schools, scholarships, and school construction, and c) convincing parents of their responsibility to actively engage in the education of their children. It had three components: School construction (SG), Girls’ scholarships (GS), Community Participation (CP). The matrix of ‘Projected Life-of-Project Outputs Anticipated’ noted that the project would identify and work with the following types and numbers of schools in the initial phase, 2004- 2007. In the second phase of the project, the configuration of schools was similar although the number was slightly increased. Also, in the second phase, ERP 1 continued work in six of the seven originally targeted governorates, having had to drop their work in Cairo. Source: USAID (2009) ERP 1: Quarterly Report July￾Sept 2009. 2004- 2007 2007- 2009 Schools identified within all Family of Schools 230 251 Primary 110 129 Multi-grade classrooms 78 Preparatory 70 74 Secondary 23 19 Vocational 23 29 Egypt Monitoring and Evaluation Program Aguirre Division of JBS International, Inc. 56 Project Activities by USAID Objective: ERP 1 Original Program and Outputs, 2004-2007 1. Improved quality of teaching and learning. Teacher Professional Development: a) Preliminary SCOPE analysis indicates teachers are performing better in all domains. b) Groups of well-qualified trainers in several domains available in each governorate and district (outcomes below). Projected Life-of-Project Outputs Anticipated Projected Actual Teachers receiving professional development training 8,154 10,031 Instructional team members trained (total) 1,820 2,513 Teacher trainers No disaggrega ted indicators 1,054 Principals and deputies 432 Supervisors 985 Trainers in Leadership 42 Trainers in Standards 128 Administrator and Leadership Development: a) Principals use SIP to improve education services. b) Trained supervisors support teachers and provide constructive feedback. c) Principals, administrators and supervisors have greater understanding of new teaching methodologies and are able to support teachers better (outcomes above). Standards Implementation: a) Standards developed for Effective Schools and Community Participation domains. b) Standards understood and used at idarra and school level (outcomes above). School-to-work (STW): a) Technical Secondary schools linked with job market and business communities. b) Technical Secondary teachers have enhanced capacity to prepare students for job market. c) Job-readiness of students improved (outcomes below). Projected Life-of-Project Outputs Anticipated Projected Actual School-to-work councils created. (total) 7 56 National level No indicators 1 Governorate level: plans endorsed by governors 7 Idarra: plans endorsed by directors 28 Local level: STW sub-committees formed within BOTs 18 Model Career academies created 2 2 Public-private sector partnerships created to support STW activities 13 No outcomes reported. Industry partners trained in STW agenda 570 Teachers receiving STW professional training and support No indicator 3,510 Technical Schools incorporating STW programs. No indicator 58 2. Increased equitable access to education. Girls Scholarships: a) Community Development Associations (CDA) capable to organize and manage scholarship activity. b) Community committees and BOTs mobilize community resources (financial and human) to support scholarship services (below). Projected Life-of-Project Outputs Anticipated Project ed Actual Girls’ scholarships administered. Primary Preparatory Secondary 120,000 102,222 14,439 3,342 149,000 No Disaggregation Scholarship managers trained. 420 67 Egypt Monitoring and Evaluation Program Aguirre Division of JBS International, Inc. 57 Multi-grade schools: a) Community Development Associations capable to organize and manage activity. b) One Classroom Units increase capacity to provide technical support to one-classroom schools. c) Out-of￾school girls use one-classroom schools as means to re-enter education system (below). Projected Life-of-Project Outputs Anticipated Projected Actual Classrooms constructed 330 42 One-classroom schools supported No indicators noted. 29 Students in all schools. 3,192 CDAs support one-classroom schools 37 OCR Unit receive training and technical assistance 325 School Construction: a) Strengthened capacity of General Authority of Education Buildings to review and implement new school designs, and incorporate community needs (such as community resource rooms) and active learning into school designs. b) Alexandria school construction completed, furnished and turned over to MoE, c) El-Marg school construction completed, furnished and scheduled for turn-over to MoE. Integrated Literacy: Literacy training for women provides them with skills to support their children’s learning processes. In addition, Literacy may increase the number of women participating in BOT elections or other civic activities in the community. Projected Life-of-Project Outputs Anticipated Projected Actual Literacy facilitators trained. 800 335 GALAE literacy supervisors trained. 70 120 Women and adolescent girls directly acquire basic literacy skills. 24,000 13,268 Literacy training for Parent Teacher Councils 700 No outcomes reported. 3. Strengthened school governance and management. School Governance: With MoE Social Work Department enhance capacity of BOTs in school governance and developed model that can be replicated in other idarras and governorates throughout Egypt. Community participation: a) Social Workers (SW) hold BOT elections at all school communities with minimum of support. b) SW Department supervisors directly support school social workers and BOTS in project and non-project idarras. c) Enhanced collaboration between schools and community institutions to support SIPs. d) BOTs increased participation in school self-assessments and school improvement planning: carry out monthly performance reviews, mobilize $77K in cash and $52K in kind contributions from community, private sector, and government authorities at local level. e) Increased collaboration of BOTs, Community Development Associations and MoE to support SIPs in many school communities. Projected Life-of-Project Outputs Anticipated Projected Actual School governance committees operational. 304 245 MoE Social Workers trained (total) 1150 339 SW department supervisors. No disaggregation 90 School Social Workers – direct support to BOTs 245 Life Skills: a) Community Development Associations (CDA) provide and manage boys and girls life skills programs covering topics such as health, first aid, and communication skills. b) A cadre of Life Skills trainers trained at local level to train other men and women to become Life Skills classroom facilitators. Early Childhood Development Education (ECDE): CDA-based facilitators trained in use of Alam Simsim (Sesame Street) to support early childhood education. CDAs also instruct parents how to use Alam Simsim to support development of their small children in the home. Projected Life-of-Project Outputs Projected Actual CDA Committees trained on Life Skills 105 50 CDA Committees trained on ECDE 105 6 Life Skills Centers established and operational. 105 50 Early Childhood Development centers established and operational. Source: USAID (2007) ERP 1: Progress Report Original Program Description 105 No outcomes reported. Egypt Monitoring and Evaluation Program Aguirre Division of JBS International, Inc. 58 Program Activities and Achievements 2005-07 This table attempts to show the difference between the two phases of the ERP 1 program. Without a detailed implementation plan available, Phase 1 activities are not able to be presented in much detail and therefore are taken from the ‘Activities and Achievements’ table above. The Phase 2 activities have been taken from the revised program description given to USAID and is more detailed. Phase 1: 2004-2007 Phase 2: 2007-2009 1. Improved quality of teaching and learning. Teacher Professional Development Trained trainer available governorate level. SCOPE analysis. Teacher Professional Development Build capacity of School-based Training and Evaluation Units (SBTEU). Implementation of curriculum. Active learning, critical thinking. Administrator and Leadership Development: Develop SIPs Trained supervisors. Understanding of active learning. Administrator and Leadership Development: Develop SIPs. Utilize human, financial and other resources to enhance learning. Improve community involvement. Standards Implementation Standards for Effective School and Community participation domains developed. Effective School domain one of main components. Formative Assessment. School-to-work (S-t-W) Support Adult Education Authority 2. Increased equitable access to education. Girls’ Scholarships Multi-grade schools School Construction: Integrated Literacy Girls’ Scholarship Multi-grade Schools School Construction AEA-civil society partnerships Small Grants Mobilize civil society Schools serve needs of community Support NGOs to support schools. 3. Strengthened school governance and management. School Governance School and Community Leadership Community participation BOT elected with guidance from SW Schools/BOTs monitor SIP Community participation BOT elected with guidance from SW. Schools/BOTs monitor SIP Idarra and governorates assist schools/communities to develop SIP Idarra and governorates support professional development. Idarra and governorates support innovation at schools and adoption into system. AEA Branches adopt and institutionalize reform Stakeholders able to generate, analyze and use data for decision-making. Life Skills: Early Childhood Development Education Egypt Monitoring and Evaluation Program Aguirre Division of JBS International, Inc. 59 2. NSP Project Results A description of program results is indicated in this chart. Project Activities by USAID Objective: NSP 1. Improved quality of teaching and learning. Educational Quality: Student Learning Outcomes: On national exams, overall the majority of girls passed and often out￾performed the boys. Total percentage of girls passing in three project governorates was 92.5% (boys 91.8%) in 2006 (combined primary-preparatory) and in 2007, at primary was 96.1% (boys: 95.8%) and at preparatory was 92.5% (boys: 85.7%). On the 2007 CAPS assessment, only 23% of grade 4 students achieved satisfactory (21%) and advanced (2%) levels in Arabic, and in mathematics, 13% of students reached satisfactory (12%) or advanced (1%), indicating that the majority of NSP students are not learning critical thinking and problem-solving skills to a high level. Implementing National Standards: 120 schools implemented national standards as outlined in the National Standards for Education in Egypt (NSEE); however, on the 2007 Standards-based Classroom Observation Protocol for Egypt (SCOPE), NSP schools consistently had the lowest mean scores against ERP schools and combined ERP/NSP schools indicating that in spite of implementing the standards, the depth of understanding and implementation of modern teaching methods may not be employed to their fullest extent. Effects may be cumulative however as noted by the highest scores of the schools which implemented both programs, ERP and NSP. Principal and Supervisor Training: NSP gave 431 principals, 684 supervisors and 163 senior supervisors pre￾service training to increase their knowledge and capacity of the modern methodologies of child-centered learning. This was to develop their role as leaders for learning in their schools and enable them to support the teachers better. Despite the extensive training they received, the evaluation revealed that supervisors largely remained rigid in their outlook and behavior in their accommodation of the new methods and teachers were often taken to task for attempting to use them. Teacher Development: NSP exceeded its target by training 2,018 new teachers. Teachers benefited from continuous training throughout the year, resulting in increased student learning and increased capacity and confidence of teachers to implement new methods. Building up capability of teacher cadres (composed of central, governorate, and school-level MoE officials) resulted in the ability to independently provide teacher trainings by the end of the project. Supplementary materials: NSP provided supplementary materials for classrooms but there was little evidence of their effective use during the evaluation. Information and Community Technology Centers: NSP met its goal for the Technology Integration Activity, by establishing 98 ICT centers to develop and build the capacity of Technology Teams for each ICT center, and offering 4,293 training opportunities to teachers, administrators, and community members on computer skills and/or IT integration in learning, Technology is enthusiastically used and supported by students and teachers in schools and by the community who pay a user fee which further supports technology services. 2. Increased equitable access to education. Access: NSP exceeded its target for student enrollment, consistently increasing yearly enrollments (2001-2007 with exception of 2003) by approximately 10,000 students. Percentage of girls’ enrollment ranged from 80-92% in their schools. Furthermore, 1,879 girls (aged 14 and older) not in school, were able to enroll in 92 newly￾established second-chance education classes. School construction: Ninety-eight new primary and preparatory schools comprising 1,048 classrooms were constructed in Minia, Beni Suef, and Fayoum using a highly-deliberative, community-based approach to site selection, school design and maintenance guided by local education teams and school solution task forces. Egypt Monitoring and Evaluation Program Aguirre Division of JBS International, Inc. 60 Multi-grade schools: 189 multi-grade schools (99% of target) were completed to offer education opportunities for girls, ages 9-14, who had not been to school. The schools were often of poor quality physically but students and teachers alike reported very high levels of satisfaction with the (social) climate in these ‘second-chance’ places of learning and were enthusiastic to have the opportunity to learn. The project also developed and distributed Technology Educational Kits supplying technology training and information for use by teachers in classrooms. While ICT training and use was well-received, there was little evidence the kits were being used in classrooms. 3. Strengthened school governance and management. Community Participation & BOTS/PA: 185 Boards of Trustees (BOTs) and Parents’ Associations (PAs) were established through democratic processes. These groups were the basis for mobilizing community education efforts and took the lead in the school construction efforts including collection of community donations. Source: USAID (2008) Final Evaluation of the New School Program The New Schools Program exceeded four of its six proposed targets as well as met and exceeded its target for the Technology Integration component (added at a later time as a separate activity to the whole project). . Source: USAID (2008) Final Evaluation of the New School Program 3. STEAP Implementation Synopsis The STEAP was a standards-based quality assurance program that employed positive incentives to effect change by establishing a standard of excellence in educational achievement and motivating educators to attain that standard. Decentralization of the education reform process was the objective in order to foster ownership and responsibility for improved education achievement at the local level. The project trained trainers who would then assist schools and the surrounding communities across the nation to write school improvement plans according to the National Education Standards. Over the life of the project, schools were invited yearly to compete for school excellence awards by submission of school improvement plans. Schools were judged on the rate of improvement they achieved according to their plans (evaluated over six domains) and those showing the most improvement in implementation of their plans over one school year won. Incentives in the form of monetary awards by the MOE were given to excelling schools at the district, governorate, and national levels. Private sector partnerships were fostered to Table 1: NSP Performance Indicators Indicator Target Achieved Students educated 43,000 44,197 Schools constructed 104 98 Multi-grade schools established 190 189 Teachers trained 2,000 2,018 BOTs and Pas established through democratic processes 176 185 School implementing National Standards 120 123 Level and number of Winning Schools by Year (STEAP II) Level 2007-08 2008-09 Idarra 673 563 Governorate 75 65 National 5-6 5-6 Egypt Monitoring and Evaluation Program Aguirre Division of JBS International, Inc. 61 encourage monetary and in-kind support of schools from communities. The focus of this project on improvement of standards-based school performance was in preparation for accrediting schools and improving student learning. Additional Results Other reported results are taken from the document narrative (but have no substantiating evidence): ‘Refreshed’ the educational process within schools. Raised level of overall school and students’ performance. Developed positive attitude of students and teachers towards school. Brought faculty closer together and encouraged teamwork. Increased parents’ cooperation with schools and spread spirit of cooperation among school community members. Inspired individual community members to improve school efforts. The project reported that all schools, winning and non-winning, improved their performance rates. The winning schools also significantly out-performed non-winning schools on every domain. The governorates of Aswan and Suez showed consistent higher rates of improvement than other governorates on almost all domains. Reported rates of improvement on the school improvement plans of the STEAP II cycle are presented here: Rate of Improvement on SIP Domains88 2007/08 2008/09 Quality Assurance and Accountability 23% 46% Community Participation and Governance 13% 38% Information Technology and Communication 10% 31% Learning Community and Professional Development 8% N/A Effective Leadership 6% N/A Learners <1% 7% Source: USAID (2009) School Team Excellence Awards Program II: Final Report. Effective - Further Discussion A) In the guide for interviewing stakeholders at all levels, 75% of the interview questions (fully 133 questions) were of the yes/no variety thereby limiting the amount of specific information they could illicit. B) After the interviews, based on the answers received, the evaluator was given the instruction to: (for example: ‘decide how much the school’s students are able to use higher order thinking skills’ without giving more instructions on how to use the data generated to make the decision, and C) After the school visit, the evaluators are to decide if a domain had been achieved, and then write the final evaluation report, without receiving any instructions on what or how to record and use the data generated form survey questions. Appendix 5: Compilation of Efficiency, Effectiveness, and Sustainability Factors from the Six Programs 88 The report is not clear how these rates of improvement were calculated or if they are from the same schools each year or from different schools. Therefore, it is not known if the same schools are improving significantly from year to year or if all schools taken together are doing better collectively at developing and implementing their SIPs. Egypt Monitoring and Evaluation Program Aguirre Division of JBS International, Inc. 62 Efficiency 1. ERP I The ERP started at a wide base of implementation across all levels of the school system focusing on a diverse array of strategies apparently without a common aim. This constrained the depth of change that was able to be attained. Even the strong program focus on teacher development would not have been a common denominator as teachers’ needs (although not necessarily their skills) would be different at each level of the system. The project did not achieve projected targets in several activities in Phase 1. Efficient? The re-structuring every couple of years cost time and money to re-orient everyone in the program and could have confused beneficiaries on the ground level. Efficient? The use of school clusters as the delivery unit for professional development programs, instructional supervision, and teacher support is an efficient organizational mechanism. Efficient 2. ERP II As little information is available on the division of funding between the ERP 1 and 2, little can be ascertained about the efficient use of the money per program. However, in terms of time and degree of structural and policy change made possible by the facilitation of the ERP 2, is would seem that the time was well-spent for the significance of the results produced. Efficient 3. GILO The proposed funding amount of $38 million for a three year project is the same amount that was awarded to the NSP which ran successfully for nine years. Plus the GILO uses many similar processes, materials, and methods from other USAID projects. The level and scope of implementation seem to be less than the intended funding is meant to cover. Efficient? The use of processes, methods, and materials from other USAID projects ensures that these components have already been field–tested and proven successful; therefore GILO can take immediate advantage of best practices in implementation. Efficient 4. NSP Quality learning materials (student kits) were produced and distributed but not well-used or were unused by teachers. Appropriate learning materials in classrooms make a large contribution to education quality; therefore more attention should be given to the integration and training for their use by teachers. Efficient? Charging user fees at technology centers adds support to the centers and increases the understanding of the value of education. Efficient. The use of school clusters as the delivery unit for professional development programs, instructional supervision, and teacher support is an efficient organizational mechanism. Efficient. 5. STEAP The initial implementation plan (to include all primary and preparatory schools in the country) was much too ambitious forcing a severe down-sizing of project schools, number of trainers, Egypt Monitoring and Evaluation Program Aguirre Division of JBS International, Inc. 63 and use of different evaluation standards, thereby causing a re-structuring of the program and use of resources. Was there no piloting initially to see how the process would work? Efficient? Partnerships with private sector individuals resulted in large monetary contributions for local school improvement but they should be linked to specific outcomes at the school level. Efficient. 6. TILO TILO benefitted from previous USAID projects’ materials, data, and experiences in their activity implementation allowing them to proceed at a quicker pace than if they had had to do everything on their own (for example, receiving furniture and vehicles from the National Book Program, using school data from STEAP I and II, exploring opportunities to work with the Adult Literacy Program using ERP personnel and investigating ERP experiences with community use of school resources.). Efficient Effectiveness 1. ERP I The professional development delivery mechanism was modified several times (from a cascade model, to a refined cascade, to direct training model, to collaboration with multiple levels) over the course of the program. Did it cause confusion among stakeholders or add value by being flexible and including many individuals across levels? Effective? The formation of School-based Training and Evaluation Units (SBTEU) to support teacher professional development in school clusters allows for timely, consistent, and on-site training delivery. Effective The development of various Professional Development models (such as Learning Resource Centers and Teacher Learning Circles) and specific coursework (Professional Development: the Road to Quality) targeting educators at all levels supports implementation, sharing among professionals, and strengthens training structures although they are being prepared for national disbursement without any evaluation or empirical evidence of improved teachers’ behavior. Effective? Training school administrators and supervisory staff to become instructional leaders in schools supports teachers’ use of new methods in classrooms. Effective Building capacity at central level for improvements in the national assessment system (national indicators), teacher professional development (Professional Academy for Teachers) and EMIS (databases) is necessary to strengthen the system but should reflect needs at all levels. Effective The scholarship program very successfully enabled many children (150,000) to attend school which is very effective, but the reality is that such an initiative is generally not sustainable by the host government in the longer run. Effective 2. ERP II The use of the National Strategic Plan as the guide for improvement ensures consistency and standardization across the whole system. Effective Recognizing that system change will not happen quickly, the ERP focused on building the need, understanding and tools to support long-term educational reform. Effective System decentralization is difficult, encountering much cultural and political resistance. The current progress, including decrees for fiscal decentralization and devolving more authority to local Boards of Trustees, represents huge steps in a relatively short time and a volatile Egypt Monitoring and Evaluation Program Aguirre Division of JBS International, Inc. 64 environment. However, policy and laws should be made with input from all levels of government to reflect needs and buy-in of lower level stakeholders. Effective? The development of the Professional Academy for Teachers and the Teacher’s Cadre represents a major step in the provision of standardized and structured professional development delivery. Effective Although the two ERPs focused on the same five common strategies, there did not appear to be much coordination of efforts between the two programs especially at the idarra level which seemed to be a logical meeting point of implementation (ERP 1 working up to and ERP 2 working down to the idarra level). Effective? Development of a comprehensive database/information tracking system assists in program monitoring and evaluation, provides a good model for host government use, and allows for efficient tracking of program information. Effective Development of the CAPS, MAP and SCOPE tests provides the host government with model instruments to track progress in school-based reform. Effective 3. GILO The assiduous identification and use of current data from local sources ensures appropriate selection of target populations, assists in data quality assessment, and models data-based decision-making. Effective Sharing and collaboration among other projects already on the ground allows for synchronized approaches, more complete field coverage of targeted populations, unified support of government reform, and reinforces efforts to institutionalize quality improvements. Effective Gender equity in the sector is enhanced by the use of field-tested gender-focused activities and support from GILO (such as the adoption of the gender equity index into the SCOPE tool) into other programs and activities. Effective The mobilization of master teachers, idarra supervisors, school administrators, and peer coaches sets up a strong, local, and standardized in-service delivery structure for teacher training. Effective Support to School Management Information Systems will assist in school operations administration, monitoring school quality, and communicating with other stakeholders. Moreover, they can eventually feed right into the national EMIS at the central level once the technology and individual capacity are increased. Effective The trialing and development of the Early Grade Reading program should enable improvement of literacy skills of young children and offer the government a valuable education strategy for early grades. Effective 4. NSP The construction of schools and classrooms in rural areas with community participation increases attention to issues of education (especially for girls), strengthens community pride and participation in education, and increases enrollment. Effective The practice of providing on-going in-service teacher development is vital to build increased confidence and capacity for provision of quality education; therefore training needs to be offered continuously every year. Effective The use of multi-grade schools in difficult areas and circumstances offers a flexible and targeted schooling option to underserved populations. Education personnel who service these facilities must be well-trained to meet the challenges presented by these unique situations. Effective 5. STEAP Egypt Monitoring and Evaluation Program Aguirre Division of JBS International, Inc. 65 The major premise of the program that there would be significant improvements in student learning achievements in a short period of time due to development of a School Improvement Plans and school-wide reforms is totally unrealistic. Of the six domains in the school improvement plans, student learning increased the least over the years of the project and remained low. At the very least, improved student achievement would be noticeable the year following the initiation of the school plan. Furthermore, no effort was made to ascertain if ultimately student learning actually improved in winning schools over the years as a result of participating in the program. Effective? The use of 20 master trainers (reduced from the original 54) to provide school-level support to a national-level project encompassing 4,000 schools seems insufficient to meet the scope of the project. Effective? Provision of project coordinators (from the government’s quality assurance unit) at the idarra level responded to a reported weakness from the STEAP I cycle and strengthened oversight and provision of knowledgeable contacts at a local level. Trained idarra directors were able to provide schools with guidance and authority on the school improvement cycle and data-based decision-making. Effective. Providing positive incentives to drive school-based reform was effective in engaging many individuals at all levels in the reform process but the system and schedule of incentives needs to be clearly understood and include input from many levels of stakeholders. Effective? Resources produced by the program, the Egyptian School Quality Manual, the Comprehensive School Evaluation Survey, and the Site Visit Guide to Evaluate Egyptian Schools for Quality contribute to validity (accuracy and appropriateness) and reliability (consistency and stability) of the school planning process and its evaluation for (eventual) accreditation. Effective The Site Visit Guide to Evaluate Egyptian Schools for Quality, which could play a major role in standardizing evaluation processes nationally, had a couple of major flaws in it concerning data collection, recording, and analysis (See Appendix 3.3 for further discussion). Effective? All manuals and assessment tools for the school improvement process were placed on-line (on a program-developed website) so materials were accessible and transparent. This encouraged independent development of school improvement plans and increased technology capacity. (However sustainability is at risk if the website shuts down after the project closes leaving schools without this resource.) Effective Participating schools received a report card which showed pre- and post-award analysis and could be compared to average idarra and governorate scores, as an added incentive for schools to track their own progress. Effective 6. TILO Project school selection was through an application process by individual schools according to set criteria with input from idarra and governorate level authorities so was participative and transparent. MoE officials were pleased with the decentralized nature of the process. Effective. Teacher training is done at the school level using master trainers who are themselves teachers and employ the active student-centered methodologies. This training structure enables teachers to immediately apply new concepts and skills in their classrooms, receive immediate feedback on practical application, and saves time and money on traveling to outside training sites. Master trainers model the same techniques with teachers which should be in use with students. Effective. The use of technology in education is engaging, can support learning, and appears to answer to human resource development needs for the 21st century; however impacts of ICT on student learning are not clear. ICT should not be central to the education Egypt Monitoring and Evaluation Program Aguirre Division of JBS International, Inc. 66 process, rather should be used as a tool and the development of sound teaching pedagogies remain the priority. Effective? Developing partnerships between private international technology companies and local Egyptian organizations not only supports innovation and adds monetary value to the project, but supports human resource development, increases the understanding of the value of education, and promotes responsibility of all stakeholders in provision of it. Effective and Sustainable. Sustainability 1. ERP I The increased focus on participation of idarra-level MoE staff in teacher professional development and School Improvement Plans is necessary to build the structure and ensure institutionalizing the process. Sustainable A process to engage school staff, administrators and communities equally together for participation and responsibility for effective school reform creates a sound foundation for continuing attention to quality education. Sustainable The program supported huge amounts of training at all levels, especially of teachers. Was there too much training and too little classroom support for implementation of new methodologies? Sustainable? Decentralization efforts (especially at the idarra and governorate levels) seem to be moving very quickly and are confusing (training for Social Work Support Teams, Social Work Department members, BOT Support Teams, BOTs!). Are these authorities also confused in their understanding of their roles and responsibilities? This process seems to employ/involve many individuals. Are they all necessary or do they have redundant responsibilities? Sustainable? 2. ERP II The framework for reform is in place and even though the actual implementation and achievement of reform will take considerable time, the laws will endure. Sustainable. The initiation of a Leadership Training program and availability of a continued supply of well￾trained staff assists to strengthen the system and perpetuates the transfer of good practices and knowledge. Sustainable. Development of a state-of-the-art Information and Knowledge Management System for ERP as well as other USAID programs (TILO, GILO, NSP) made a major contribution to the system, useful by USAID to track programs and left as a model that the MoE can continue to use and build on. Includes for example, a School Data Bank with information on all schools in the USAID programs. Sustainable. 3. GILO Building of institutional capacity of central government units to work with local branch offices and community groups in provision of education in theory increases decentralization. In practice, central ministry officials fear loss of power and resources; therefore often do not willingly participate in decentralization efforts. Will these initiatives be maintained after project funding is ended? Sustainable? Strong community groups are the force which should drive demand for education reform and improved education outcomes for their children. Their scope of involvement at their local schools should be enlarged beyond participating in school site selection to strengthen their buy￾in of school reform. Sustainable Egypt Monitoring and Evaluation Program Aguirre Division of JBS International, Inc. 67 The development of an appropriate targeted school improvement plan with the participation of school staff, parents/community, BOTs and idarra officials should meet local education needs and be the common tie which binds these stakeholders into united action. Sustainable 4. NSP The development of strong community groups who were the force behind school construction and community mobilization should continue to drive demand for education reform. Their scope of participation should be enlarged in order to increase their capacity and maintain their interest and buy-in for better education service. Sustainable. The development of strong teaching training cadres consisting of MoE professionals across all levels of the government should enable independent continuation of high quality in-service teacher training. Sustainable. Program implementers must be sensitive to teacher needs and time limitations to attend trainings and be responsible to train others. Overloaded teachers will withdraw from the program thereby diluting program effects and limiting impact of USAID investments. Sustainable? Training and allowing communities to use technology centers increases the demand for technology and supports use of technology for student learning. Sustainable. Enabling an endowment fund for school maintenance through leveraged contributions from communities and Ministry of Education keeps stakeholders involved and adds resources to the system for continued support. Sustainable The eight-year length of the program allowed for achievement all targets, development of a trusted relationship with stakeholders, and added depth to implementation. Sustainable. 5. STEAP The program became a national public awareness campaign for the National Education Standards, school improvement, and accreditation process. Sustainable. The project generated great involvement of the community and refreshed the educational process within schools. Sustainable. Training and involvement in the SIP process at all levels, especially at the idarra and school levels, contributes to building institutional processes and understanding of school reform. Sustainable. 6. TILO Developing strategies for on-going teacher support at school level (such as peer teacher groups, clustering schools to enable opportunities for professional sharing and exchange across schools, on-line teacher support communities) enables professional sharing and increases knowledge and confidence of teachers to continue to use new methodologies. Sustainable. The thorough involvement of MoE officials at all levels provides leadership, assists to define roles and responsibilities in the reform process, increases government ownership, and aids institutionalization of change in schools. Sustainable. The work at the central level of government (with curriculum development specialists, technology in education experts, and at the Professional Academy for Teachers) encourages responsiveness to education issues, increases ownership, and facilitates standardized practices. Sustainable. The use of technology in education is engaging, supports learning, and answers to human resource development needs for the 21st century. Sustainable. Egypt Monitoring and Evaluation Program Aguirre Division of JBS International, Inc. 68 Appendix 6: Six Component Matrix Component Matrix by Project Education Reform Program I (EQP I) Education Reform Program II (EQP II) Girls’ Improved Learning Outcomes (GILO) New Schools Program (NSP) School Team Excell. Awards Program (STEAP) Technology for Improved Learning Outcomes (TILO) 1. Decentralization Focus Level: Central (c), Governorate (G), Idarra (I), Communities/School (C/S) G*, I, C/S C, G, I C/I/S G, I, C/S C, G, I, C/S C, I, C/S Community/NGO Strengthening: x x x Training Received (any level) x x x x x x Facilitate school-support groups (e.g. CET, BOT, PTA) x x x Supervisor, School administrator support. x x x x x School Improvement Plan/School standards x x x x x x M&E, Data-informed decision-making x x x x x Teacher deployment/monitoring Fiscal resource generation/management x x MAP administration x x x 2. Gender Equity Girls’ access increased: Scholarships [S], Multi-grade Classrooms [MGC], School Construction [SC] S, MGC, SC MGC MGC, SC Gender-sensitive active learning. x x Women representation/leadership. x x x 3. Teacher Training Active-learning Methodologies x x x x Individual Student Assessment x x Progress towards Professional Dev./Certification In-service Pre-service x x x x x x x x Development of TT Cadres/support system x x x x Identified responsibilities at each level: C, G, I, C/S G, I, C/S I C, G, C/S C, I, C/S SCOPE administration x x x x 4. Community Development School Planning x x x x Monitoring/Evaluation Egypt Monitoring and Evaluation Program Aguirre Division of JBS International, Inc. 69 Donations (monetary, in-kind, labor) x x x x Attention to Under-served Populations x x x Activity Implementation [Literacy, ECDE] Lit., ECD 5. Curriculum Development Student Assessment/CAPS x x x x Content Life Skills Math, EGR, Science, ICT ICT ICT Supplementary Materials x x x 6. Policy Reform Increase capacity of government officials GAEB, SW, AEA C, G, I GAEB, PPP GALAE G, I C, I Organizational Development x Teacher Professional Development x x x Strategic Planning x Decentralization x GAEB Student Performance M&E, Quality Assurance x x EMIS x x Evaluation Guides x Textbooks/Materials x Matrix Notes: All items included under each component were gleaned from programs as well as the from the author’s knowledge of topics. 1. Decentralization: Initials show administrative level: C=central, G= governorate, I= idarra, C/S= community/school. Priority levels of implementation are bolded and in bigger font. 2. Gender Equity: Initials show implementation strategy: S= scholarships, MGC= multi-grade classroom, SC= school construction. 3. Teacher Training: Same initialing as in #1 to show that responsibilities have been identified at that level. 4. Community Development: Lit = literacy activities, ECD= early childhood development. 5. Curriculum Development: EGR= Early Grade Reading, ICT= Information and Communication Technology. 6. Policy Reform: Same initialing as in #1. GAEB= General Authority for Educational Building, SW= Social Work, PPP= Public Private Partnerships, GALAE= General Authority for Literacy and Adult Education. Egypt Monitoring and Evaluation Program Aguirre Division of JBS International, Inc. 70 References Canadian International Development Agency (2009) Education Sector Update: Monthly Report. A. Davider. Cairo. Egypt Ministry of Education (undated) Institutionalizing Decentralization in Pre-university Education in Egypt. Cairo. Egypt Ministry of Education (2007) Integrated Document of Effective School Standards (IDESS). Cairo. Egypt Ministry of Education (undated) National Education Indicators (NEIs) for Egypt: Proposed Technical Guide. Cairo. Egypt Ministry of Education (2007) National Strategic Plan for Pre-university Education, 2007-2012.. Cairo. Egypt Ministry of Education (2007) National Strategic Plan 2007-2012: Towards an Educational Paradigm Shift. Power-point. Cairo. Egypt Ministry of Education (2009) Strategic Plan Implementation First Report 2007/2008. Power-point. Cairo. Ginsberg, M, N. Megahed, M. El Meski and N. Tanaka (draft 2009) Reforming Educational Governance and Management in Egypt: National and International Actors and Dynamics. InfoDev (2005) Briefing Paper: Impact of ICT on Learning and Achievement. www.infodev.org Langsten, R and T. Hassan (2007) Basic Educational Attainment in Egypt: Trends and Determinants. American University in Cairo, Cairo. Langsten, R and T. Hassan (2009) Education Transitions in Egypt: The Effects of Gender and Wealth. American University in Cairo. Sheta, T.A (2007) Novice Teachers: Current Situation and Future Directions. Cairo. UNDP (2004) Egypt Human Development Report: Choosing Decentralization for Good Governance. Institute of National Planning, Cairo. UNESCO (2005) Decentralization of Education in Egypt: County Report at the UNESCO Seminar on “EFA Implementation: Teacher and Resources Management in the Context of Decentralization. Hyderabad, India. UNESCO (2003) Education Reform in Egypt 1996-2003: Achievements and Challenges in the New Century. Cairo. UNESCO (2008) Education Support Strategy: Egypt. Cairo. UNESCO (2005) UNESCO Seminar on EFA Implementation: Teacher and Resource Management in the Context of Decentralization. USAID (2002) Quality Basic Education for All: Strategy Proposal. Aguirre International, Cairo. World Bank (2007) Arab Republic of Egypt: Improving Quality, Equality, and Efficiency in the Education Sector: Fostering a Competent Generation of Youth. Washington, D.C. World Bank (2005) Making Egyptian Education Spending More Effective: Policy Note 2. Washington D.C. World Bank (2005) Public Expenditure Review (PER) Policy Note Number One on the Analysis of Education Expenditures in Egypt. Washington D.C. World Bank (2004) Ten Steps to a Results-based Monitoring and Evaluation System. J.Z. Kusek and R.C. Rist. Washington D.C. By Project Education Reform Program (ERP1)/EQUIP 1 Bernbaum, M. (2007) Education Reform Program: Rapid Assessment Summary. Cairo. Ginsberg, M, N. Megahed, M, A. Abdellah, and A. Zohry. (undated) Active-Learning Pedagogies as a Reform Initiative: The Case of Egypt. Research paper prepared under the Education Quality Improvement Egypt Monitoring and Evaluation Program Aguirre Division of JBS International, Inc. 71 Project 1: School and Classrooms. American Institutes for Research and US Agency for International Development. Washington D.C. USAID (2007) Education Reform Program/EQUIP 1: Progress Report on Original Program Description. June 2004 – July 2007. American Institutes for Research, Washington D.C. USAID (2007) Education Reform Program/EQUIP 1: Revised Program Description. SO 22: Sustained Improvement in Student Learning Outcomes (SILO). American Institutes for Research, Washington D.C. USAID (2009) Education Reform Program/EQUIP 1: Quarterly Report January-March 2009: SO 22: Sustained Improvement in Student Learning Outcomes (SILO). American Institutes for Research, Washington D.C. USAID (2009) Education Reform Program/EQUIP 1: Quarterly Report April-June 2009: SO 22: Sustained Improvement in Student Learning Outcomes (SILO). American Institutes for Research, Washington D.C. USAID (2009) Education Reform Program/EQUIP 1: Quarterly Report July-September 2009: SO 22: Sustained Improvement in Student Learning Outcomes (SILO). American Institutes for Research, Washington D.C. USAID (2009) Education Reform Program/EQUIP 1: Quarterly Report October-December 2009: SO 22: Sustained Improvement in Student Learning Outcomes (SILO). American Institutes for Research, Washington D.C. USAID (2009) Egypt - Education Reform Program Educational Quality Improvement Program 1(EQUIP 1) Technical Application. American Institutes for Research, Washington D.C. Education Reform Program (ERP2)/EQUIP 2 Ginsberg, M. and N. Megahed. (2008) Education Reform Program: Support in the Area of Governance and Management. Cairo. USAID (2009) Education Reform Program - EQUIP 2, Policy and System Change: Final Report 2004-2009. Academy for Educational Development, Cairo. USAID (undated) EQUIP 2 Working Paper: Identifying the Impact of Education Decentralization on the Quality of Education. Girls Improved Learning Outcomes (GILO) Linan-Thompson, S. (2009) Upper Egypt EGRA 2009: Results and Implications. Girls’ Improved Learning Outcomes (GILO). Power-point. Cairo. USAID (2008) Girls Improved Learning Outcomes (GILO) Quarterly Report, Inception to June 2008. Research Triangle International, Cairo. USAID (2008) Girls Improved Learning Outcomes (GILO) Quarterly Report, July-September 2008. Research Triangle International, Cairo. USAID (2008) Girls Improved Learning Outcomes (GILO) Quarterly Report, October-December 2008. Research Triangle International, Cairo. USAID (2008) Girls Improved Learning Outcomes (GILO) Quarterly Report, January-March 2009. Research Triangle International, Cairo. USAID (2008) Girls Improved Learning Outcomes (GILO) Quarterly Report, April-June 2009. Research Triangle International, Cairo. USAID (2008) Girls Improved Learning Outcomes (GILO) Quarterly Report, July-September 2009. Research Triangle International, Cairo. USAID (2008) Girls Improved Learning Outcomes (GILO) Quarterly Report, October-December 2009. Research Triangle International, Cairo. USAID (2008) Scope of Work: Modification #03: Expanding Equitable Access to Education for Girls. Cairo. Egypt Monitoring and Evaluation Program Aguirre Division of JBS International, Inc. 72 New School Program (NSP) Egypt Ministry of International Cooperation (2006) Project Evaluation Report No. 16: New School Program. Cairo. USAID (2009) New School Program, Inclusive of the School LINC GDA: Final Report. American Institutes for Research, Washington, D.C. USAID (2003) New Schools Program: Mid-term Evaluation. Aguirre International, Washington, D.C. USAID (2008) New Schools Program: Final Evaluation. American Institutes for Research. Washington, D.C. School Team Excellence Awards Program (STEAP) Creative Associate International (undated) STEAP Results. Power-point. Washington, D.C. Schaeffner, M, A.K.A. Badran, and A. El-Baz (2008) Site Visit Guide to Evaluate Egyptian Schools for Quality. Cairo. USAID (2008) Scope of Work: School Team Excellence Awards Program (STEAP). Cairo. USAID (2008) Scope of Work: School Team Excellence Awards Program (STEAP) Phase 2. Cairo. USAID (2009) School Teams Excellence Awards Program II: Quarterly Report April-June 2009. Creative Associate International, Cairo. USAID (2009) School Team Excellence Awards Program II: Final Report. Development Associates, Cairo. Technology for Improved Learning Outcomes (TILO) USAID (2009) Feedback Survey Report on TILO. Creative Associate International, Cairo. USAID (2008) Technology for Improved Learning (TILO) Annual Report 1 September 2007-September 2008. Creative Associate International, Cairo. USAID (2009) Technology for Improved Learning (TILO) Annual Report 2 October-September 2009, FY09-Q8. Creative Associate International, Cairo. USAID (2009) Technology for Improved Learning (TILO) Quarterly Report Year 3, Q1 October-December 2009. Creative Associate International, Cairo. USAID (2010) TILO: Summary of Private Sector Leverage. Creative Associate International, Cairo. USAID (2010) Technology for Improved Learning Outcomes: Armada Retreat. Power-point. By Instrument Critical-thinking, Achievement, and Problem-solving (CAPS) USAID (2008) CAPS Test: 2008 Assessment Report. National Center for Examinations and Education Evaluation. Cairo Management Assessment Protocol (MAP) USAID (draft 2009) 2008 Management Assessment Protocol (MAP) Results Summary. Monitoring and Evaluation Division and ERP, Cairo. Zohry, A. (2008) MAP Data Analysis Report. Cairo. Standards-based Classroom Observation Protocol for Egypt (SCOPE) USAID (2008) SCOPE: 2008 Final Report. Monitoring and Evaluation Division and ERP. Cairo