July 20, 2010 This publication was produced for review by the United States Agency for International Development by the Aguirre Division of JBS International, Inc. (Frank Method, Tesmesgen Ayele, Cameron Bonner, Nancy Horn, Ayele Meshesha, Tefera Talore Abiche) under Task Order 27 of the Global Evaluation and Monitoring (GEM) II BPA, EDH-E-00-08-00003-00. IMPACT ASSESSMENT OF USAID’S EDUCATION PROGRAM IN ETHIOPIA 1994-2009 Impact Assessment of USAID’s Education Program in Ethiopia 1994-2009 July 20, 2010 Frank Method, Team Leader Tesmesgen Ayele Cameron Bonner Nancy Horn Ayele Meshesha Tefera Talore Abiche This assessment was conducted by the Aguirre Division of JBS International under Task Order 27 of USAID’s Global Evaluation and Monitoring (GEM) II BPA, EDH-E-27-08-00003-00. The authors’ views expressed in this publication do not necessarily reflect the views of the United States Agency for International Development or the United States Government. Map of Ethiopia Source: National Geographic Impact Evaluation of USAID/Ethiopia’s Education Activities i Table of Contents Table of Contents ........................................................................................................................... i Acknowledgements ...................................................................................................................... iii Acronyms ...................................................................................................................................... iv Executive Summary ...................................................................................................................... 1 Overall Findings ....................................................................................................................................... 1 Achievements of the USAID program .................................................................................................................................. 3 Challenges over the past years ............................................................................................................................................... 4 Current challenges: ................................................................................................................................................................... 4 Trends Affecting USAID Strategy Options Over the Next Five to Ten Years ................................ 4 Main Recommendations ......................................................................................................................... 5 Summary of Recommendations ............................................................................................................ 5 Recommendations on Modalities ........................................................................................................................................... 6 1. Introduction .............................................................................................................................. 7 2. Educational Background – A Brief Overview ........................................................................ 8 3. Methodology ........................................................................................................................... 11 4. Impacts on Access and Equity ................................................................................................ 12 4.1 Impacts on Expansion of Access .................................................................................................... 12 4.2 Gender Equity (Female Enrollment and Completion) .............................................................. 15 4.2.1 Improving Access and Equity for Girl Students ..................................................................................................... 15 4.2.2 Improving Access and Equity for Prospective Female Teachers ........................................................................ 17 4.2.3 Current Capacities and Needs for Further Capacity Building ............................................................................ 18 4.3 Impact on Equitable Access for the Hard-to-Reach and Underserved .................................... 19 4.3.1 Support for Pastoralists ............................................................................................................................................... 20 4.3.2 Support for OVC .......................................................................................................................................................... 20 4.3.1 Support for Women’s Literacy and Livelihoods Efforts ....................................................................................... 22 4.3.4 Needs for Further Capacity Building ........................................................................................................................ 23 5. Impacts on Quality ................................................................................................................. 25 5.1 Impacts on Learning and Teaching .............................................................................................. 25 5.2 Teacher Preparation and Support ............................................................................................... 28 5.2.1 Teacher Training ........................................................................................................................................................... 28 5.2.2 Support for Supervisors .............................................................................................................................................. 32 5.2.3 Impact of Turnover and Implementation Delays ................................................................................................... 32 5.2.4 Current Capacities and Needs for Further Capacity Building ............................................................................ 33 5.3 Materials Development ................................................................................................................. 34 5.4 Assessment and Monitoring/Reporting Capacities ..................................................................... 36 6. Educational Management Capacity Building ....................................................................... 40 6.1 Administrative and Management Capacities – The Administration of Education ................. 40 6.2. Community Participation and School Governance ................................................................... 41 6.2.1 PTAs and Kebele Education & Training Boards ..................................................................................................... 42 6.2.2 School Incentive Awards (SIAs) ................................................................................................................................. 42 6.2.3 School Partnerships ...................................................................................................................................................... 43 6.2.4 Current Capacities ....................................................................................................................................................... 43 6.2.4 Needs for Further Capacity-Building ....................................................................................................................... 44 ii Impact Evaluation of USAID/Ethiopia’s Education Activities 7. Health Impacts Affecting Education Results ........................................................................ 46 Summary .................................................................................................................................................................................... 46 Current Capacities .................................................................................................................................................................. 47 Need for Further Capacity-Building .................................................................................................................................... 47 8. Summaries and Recommendations ....................................................................................... 48 8.1 Summary of Findings and Observations ...................................................................................... 48 8.1.1 Individuals ....................................................................................................................................................................... 48 8.1.2 Institutions ..................................................................................................................................................................... 48 8.1.3 Policies/systems ............................................................................................................................................................. 50 8.1.4 Coordination/modalities ............................................................................................................................................. 51 8.2 Constraints to Deeper, Broader or More Sustainable Impacts ................................................ 52 8.3 Scenarios for the Future ................................................................................................................ 53 8.4 Key Recommendations .................................................................................................................. 56 Priority Programmatic Recommendations ......................................................................................................................... 56 Recommendations on Implementation ............................................................................................................................... 57 8.5 Summary Responses to SOW Questions .................................................................................... 57 ANNEXES .................................................................................................................................... 65 Annex A. Scope of Work ...................................................................................................................... 65 Annex B. Selected List of Documents Consulted ............................................................................. 78 Annex C. Organizations and Individuals Interviewed ...................................................................... 81 Annex D. Summary of USAID Project Assistance to Education, 1994 -2014 ............................... 86 Annex E. GEQIP Parameters .............................................................................................................. 90 Annex F. Statistical Data ..................................................................................................................... 92 1. Access and Equity Indicators (School age population, GER, NER and Children out of school) ....................... 92 2. Key Quality Indicators on the Provision of Primary Education (Active Learning Methods, Learners’ Assessment Results, Survival and Completion Rates)...................................................................................................... 95 Tables in Text 4.1 Enrollments Over Time 4.2 Promotion and Dropout Rate for the Neediest OVC supported by CASCAID for 2000 EC Academic Year 4.3 Promotion and Dropout Rate in Percentage for All Students Enrolled In CASCAID Schools 2000 EC Academic Year 4.4 Dropout Rate by Region and Sex 2007/8 4.5 Repetition Rates in 1,800 CSPP Schools (Baseline Survey) 5.1 Mean Scores for Urban and Rural Pupils: National Learning Assessment of Grade Four Students, 2008 5.2 Ability of Students to Read at Grade Level, From AEI Assessment 2009 Impact Evaluation of USAID/Ethiopia’s Education Activities iii Acknowledgements Many people contributed to the successful completion of this assessment of USAID assistance to the children of Ethiopia, most of whom are identified in the Appendix listing our Schedule of Meetings. Special thanks are owed to several people: State Minister Fuad Ibrahim and Solomon Shiferaw, Abebe Defersha, and Tizazu Asare were among the senior MOE officials who were generous with their time and information. The members of the USAID Basic Education Team: Allyson Wainer and Tesfaye Kelemework, Chief and Deputy Chief, respectively, Befekadu Gebretsadik, Demissie Legesse, and Assefa Berhane for their solid assistance with preparation, thoughtful guidance on our workplan and detailed comments on earlier drafts of this document. Special thanks goes to Aberra Makonnen, former Chief of Basic Education, whose perspectives and insights from his long history of involvement with USAID assistance were invaluable. We appreciate the candor and willingness to share from the staff of the implementing partners at their Washington offices and at their national, regional, and local offices in Ethiopia. Ed Allan at JBS International provided solid home office support. Finally, in addition to the many unnamed individuals who contributed to our understanding of basic education in Ethiopia, we recognize and commend the efforts of the many educators – including the school leaders, members of PTAs and GEACs, as well as the administrators and officials – whose commitment and professionalism contributed to the remarkable progress Ethiopia has made over the past 15 years, as well as to the success of the projects USAID has funded. iv Impact Evaluation of USAID/Ethiopia’s Education Activities Acronyms AAMU Alabama Agricultural and Mechanical University ABE Alternative Basic Education ABEC Alternative Basic Education Center AED Academy for Educational Development AEI African Education Initiative AFL Adult Functional Literacy AGSP Ambassador’s Girls’ Scholarship Program AIR American Institutes for Research AL Active Learning ANFAE Adult and Non-Formal Education Association in Ethiopia ARV Anti Retro-Virals BEP Basic Education Program BESO I Basic Education System Overhaul BESO II Basic Education Strategic Objective BPR Business Process Re-engineering CA Continuous Assessment CASCAID Communities and Schools for Children Affected by HIV/AIDS CBO Community-Based Organization CBTEP Capacity Building in Teacher Education Project CC Cluster Center CCCU Cluster Center Coordinating Unit CDA Complementary Drought Assistance CE Center of Excellence CGPP Community-Government Partnership Program CHEMIS Community-based Health and Education Management Information System COP Chief of Party CPD Continuous Professional Development CRC Cluster Resource Center CSAP Community/Schools Activities Program CSI Child Status Index CSO Civil Society Organizations CSPP Community-Schools Partnership Program CTE Center of Training/Teaching Excellence DfID Department for International Development EFA Education for All EGRA Early Grades Reading Assessment EMPDE Educational Materials Production and Distribution Enterprise ESDP Education Sector Development Program ETEP Ethiopian Teacher Education Portal EETPS Ethiopian Education and Training Policy Study ELIP English Language Improvement Program EMIS Educational Management Information System EQUIP II Educational Quality Improvement Program: Educational Policy, Systems, and Management ETNLA Ethiopian Third National Learning Assessment FCA Formative Continuous Assessment FGM Female Genital Mutilation FHI Family Health International Impact Evaluation of USAID/Ethiopia’s Education Activities v FTI Fast Track Initiative GEQIP General Education Quality Improvement Program GFDRE Government of the Federal Democratic Republic of Ethiopia GAC Girls’ Advisory Committee GEAC Girls’ Education Advisory Committee GEM Girls Empowerment and Management Project GER Gross Enrollment Rate HAPCO HIV/AIDS Prevention and Control Office HERQA Higher Education Relevance and Quality Agency HIV/AIDS Human Immunodeficiency Virus/Acquired Immunodeficiency Syndrome IFESH International Foundation for Education and Self-Help IGA Income Generating Activity IP Implementing Partner IQPEP Improving the Quality of Primary Education Program IR Intermediate Result IRC Instructional Resource Center IRI Interactive Radio Instruction JICA Japanese International Assistance Agency KETB Kebele Education and Training Board LAN Local Area Network LEAP Livelihood Enhancement for Agro-Pastoralists and Pastoralists LIMS Library Information Management System M&E Monitoring & Evaluation MAP Management & Administration Program MDG Millennium Development Goals MERA Monitoring, Evaluation, Research and Analysis MIS Management Information System MOE Ministry of Education MOH Ministry of Health MSI Management Systems International NCATE National Council for the Accreditation of Teacher Education NGO Non-governmental Organization NLA National Learning Assessment OVC Orphans and Vulnerable Children PC3 Positive Change: Children, Communities and Care PDI Positive Deviance Inquiry PEPFAR President’s Emergency Plan for AIDS Relief PLI Pastoral Livelihoods Initiative PMC Planning and Management Component PMIS Personnel Management Information System PMP Performance Monitoring Plan PSCAP Public Sector Capacity Building Program PTA Parent-Teacher Association RSEB Regional State Education Bureau RIMS Registrar Information Management System SAP School-Age Population Save Save the Children SCOPSO School-Community Partnerships Serving Orphans and Vulnerable Children SDA School Development Agents SDU Staff Development Unit vi Impact Evaluation of USAID/Ethiopia’s Education Activities SES Social-Economic Status SGP School Grants Program S/I Supervisor/Inspector SIA School Incentive/Improvement Award SIP School Improvement Program SMC School Management Committee SMT Senior Management Team SNNPR Southern Nations, Nationalities, and Peoples Region SO Strategic Objective SRIMS Student Records Information Management System TA Technical Assistance TALULAR Teaching and Learning Using Locally Available Resources TDA Tigray Development Association TDP Teacher Development Program TEACH Transforming Education for Adults and Children in the Hinterlands TEI Teacher Education Institution TELL Teach English for Life Learning TESO Teacher Education Systems Overhaul TLMP Textbook and Learning Materials Program TOT Training of Trainers TT Teacher Training TTC Teacher Training College TWG Technical Working Group UNESCO United National Educational Scientific & Cultural Organization UNICEF United Nations Children’s Fund USAID United States Agency for International Development USAID/W United States Agency for International Development/Washington USG United States Government VSO Voluntary Service Overseas WASH Water, Sanitation and Health WB World Bank WCB Woreda Capacity Building WCRC Woreda Cluster Resource Center WEO Woreda Education Office WFP World Food Program WL/WLI World Learning/World Learning International Impact Evaluation of USAID/Ethiopia’s Education Activities 1 Executive Summary The Education Impact Assessment team was tasked with reviewing USAID assistance to education in Ethiopia since it began in 1994 with the Basic Education System Overhaul program, BESO 1, through the current program. The assessment focused on the impacts of this assistance on 1) education access, equity and quality, and 2) on the management systems and institutional capacities to plan, assess and implement improvement strategies. The assessment focused on identifying lessons learned, the sustainability of the initiatives, systems and innovations supported by USAID, the degree to which continuing support may be needed and used effectively, and on recommendations for future program and project support. A team of three U.S. specialists and three senior Ethiopian specialists worked in Ethiopia from April 14, 2010 through May 6, 2010, preceded by a week of desk study and meetings with USAID and implementing partners in Washington, DC (AED, SAVE, World Learning, and PACT). Following team work planning and review of plans with USAID, the first full week in Ethiopia was used for meetings with all key MOE Departments, implementing partner project teams and key individuals with background knowledge of USAID assistance programs and MOE initiatives over the previous 15 years. The second week the team travelled to six regions (Amhara, SNNPR, Somali, Oromia, Harari and Dire Dawa) for site visits and consultations with RSEBs, TTCs, woredas, schools and resource centers. The remaining time in Ethiopia was used for follow-up meetings, drafting of an initial report, and briefing of USAID. Field activities were followed by several weeks of further analysis and report preparation. Overall Findings The overall assessment is that USAID assistance has been key to Ethiopia’s education progress over the past 15 years, both in terms of the specific impacts on management systems, quality improvement and institutional capacity building and in terms of the abilities of Ethiopians to undertake essential functions of needs assessment, systems analysis, policy development, strategic planning and coordination with decreasing reliance on external technical assistance for these core functions. During this period, Ethiopia has experienced explosive growth in education capacity (in many ways unprecedented in any country), a major decentralization requiring institutional restructuring calling for new capacities at the regional and local levels and a strong commitment to full enrollment of girls, vulnerable children and children in rural and pastoralist areas. Ethiopia also experienced numerous policy shifts on curriculum, language, teacher qualifications and pedagogic support systems. The fact that USAID has been a strategic partner throughout this period and has been able to work with good coordination in most respects is recognized and appreciated at all levels of Ethiopia’s education system. The team found it remarkable that key people from the MOE to the Regional and woreda levels to teachers, supervisors and community leaders encountered during field visits knew the specifics of USAID assistance as well as the actual acronyms for the various activities. Thus, the first overall finding is that USAID has earned unique credibility as a reliable partner willing to support a variety of specific short-term project tasks and small-scale innovations addressing current needs. At the same time it has provided longer-term support for systems improvement and institutional strengthening, some of which take considerable time to have full impact on education outcomes. The ability to support longer-term objectives and to continue support long enough for the changes to have impact, has been a unique strength of the USAID program approach and a comparative advantage for USAID 2 Impact Evaluation of USAID/Ethiopia’s Education Activities The report is organized around ten themes, plus recommendations. Major findings are: • USAID activities have had a substantial impact on primary education enrollment capacities, access and equity. The quantitative achievements have been mainly the result of Ethiopian investments. The impacts of USAID assistance have been: mainly indirect through strengthening of institutional capacities, management and support systems rather than direct inputs to expand school capacity. • USAID has had a substantial impact on equity through support for development of Girls Education Advisory Committees (GEAC) at schools, support grants for Orphans and Vulnerable Children (OVC) and other children at particular risk, incentive grants supporting PTAs and school-community partnerships and support for alternative education approaches in pastoralist and hard-to-reach areas. • Gross enrollment has grown from 22% (18% for girls) in 1994/95 to 94.2% (90.7% for girls) in 2008/09. The enrollment of girls has expanded in all regions and now approaches parity in most areas; dropout rates are decreasing and completion rates increasing, particularly for the first cycle. There remain significant differences by region, particularly in Afar and Somali. • USAID has provided substantial support for the expansion and qualitative improvement of Alternative Basic Education Centers (ABEC) for pastoralist and hard-to-reach areas as well as for out of school children and adults in pastoralist and hard-to-reach areas. ABECs are now part of Ministry of Education policy for all regions. • The development of school clusters has created more effective structures for supporting school management, providing pedagogic support and facilitating exchanges among the schools and teachers in the clusters. USAID has collaborated with UNICEF and others in developing the cluster models and the supervisory, training and pedagogic support systems for the clusters. • The school grants programs and related training for school and community leaders have led to greatly increased parent and community participation and engagement with the schools, with substantial community mobilization of additional resources. Active PTAs and other school￾community committees are taking responsibility for ensuring all children are in school, including especially poor children and OVCs. These participation and mobilization activities are continuing, and appear likely to continue, beyond the period of USAID grants to the committees. • As part of the school mobilization activities USAID launched the development of Girls Education Advisory Committees (GEACs) and Girls Clubs. The GEACs and the PTAs help advocate to get girls enrolled, provide mentoring, tutoring and other encouragement to girls in school and send delegations to the households to find out why a child is not enrolled or has stopped attending. These have become an integral part of most schools, particularly in the districts where USAID activities have concentrated and are now being adopted (without further USAID assistance) in other districts and schools across Ethiopia. • USAID activities have contributed both directly and indirectly to the improvement of learning outcomes. – Indirectly, USAID supported the National Learning Assessments (2000, 2004 and 2008) which have provided important insights into student achievement levels and the problems associated with low achievement, providing an objective basis for development of improvement plans by the MOE and RSEBs. – Directly, there has been considerable in-service training and workshop support for teachers, emphasizing active learning, continuous assessment and more child-centered approaches. – Projects have provided English language texts, supplementary materials for mathematics and science, and training for teachers and supervisors on their effective use. – There also has been support for cluster resource centers, support for teacher upgrading programs and support for improvement of pre-service teacher training (including e-learning modules to help improve subject knowledge of students at TTCs). Impact Evaluation of USAID/Ethiopia’s Education Activities 3 • Program areas in which the assessment team did not observe progress include the establishment of “Centers of Excellence” at selected TTCs. This requires a more in-depth assessment and design review, going beyond the scope and time limitations of the assessment team. Some TTCs appear reasonably well equipped, e.g. the Assella and Hawassa TTCs have well developed TALULAR centers, as well as computer centers, libraries and model classrooms. However, the team did not observe at Jijiga TTC or hear in interviews any sense that the targeted TTCs are being looked to as models by other TTCs. A partial exception is the publishing facility at Debre Berhan. • The literacy program under TEACH reached a significant numbers of adults (50,297 according to project reports) but little is known about the level of literacy achieved and about the impact of the functional literacy project on actual behaviors and outcomes. Achievements of the USAID program Programmatic Achievements: • English language texts for grades 1, 6, 7, 8, with grade 2, 3 and 4 still underway • Support to OVCs • Support to ABECs in pastoralist and hard-to-reach areas • National Learning Assessments – 2000, 2004, 2008 • School incentive grants, supporting school self-assessment and planning and local resource mobilization, with grants used for school improvements • MA-level training for about 200 staff in key positions • E-learning materials for teacher upgrading and self-study at TTCs in mathematics, science and geography • Supplementary teaching materials for schools in mathematics and science • EMIS capacity building, training to improve quality of data and data use, computer training at all levels and development of a range of MIS tools for educational management, e.g., Personnel Management Information Systems (PMIS) for use at RSEBs and woredas, and a student record information system for use at TTCs. • Improved financial management for the schools and management and planning capacities at MOE, regional, woreda levels Policy Achievements: • The ABEC model is now supported as MOE policy for all regions • GEACs or their equivalents are now required at most schools and, in reporting, gender must be disaggregated in all reports and data sets • The school cluster model for in-service teacher training is now utilized in all regions • Local public-private partnerships now exist at most schools and are encouraged by the Ministry and RSEBs • Attention to full inclusion is growing, though not yet affirmed as policy, for OVCs and other vulnerable children, for some special needs and for children in pastoralist areas Indirect Achievements: • Transparency and accountability has improved at all levels, with objective data increasingly available and accessible, current and used for planning and oversight. • Though USAID support for addressing health constraints to learning and school participation has been limited, there is growing coordination by schools with health services and health screening 4 Impact Evaluation of USAID/Ethiopia’s Education Activities • Monitoring, evaluation and reporting systems appear adequate, even somewhat excessive in terms of the volume of data generated, reported and published. • Schools are increasingly seen by communities as “their” schools and many communities are now active in mobilizing resources for schools and participating in school planning and oversight. • Partially as a result of the 2004 National Learning Assessment, MOE planning began for the multi￾donor supported GEQIP. • The competence of senior staff in core function areas in the MOE appears quite high, leading to increasing confidence in strategic planning and coordination of implementation plans. Challenges over the past years • Scale of Ethiopia (distances and population size) necessitating targeting of selected regions, woredas and schools. • Initially, lack of well-trained and experienced Ethiopian staff in key MOE, RSEB, woreda, and school leadership positions, necessitating both substantial training support and substantial numbers of external technical assistance personnel. • Policy decisions on rapid expansion necessitated early emphasis on capacity building and limited the impact of activities addressing quality. • As the system has expanded and decentralized there has been substantial turnover and mobility of key staff, limiting the impacts of training and institutional strengthening. Current challenges: • Growing demand for secondary access • Need to improve content knowledge of teachers – math, science, English • The remaining quantitative challenges for Ethiopia include: – lowering class sizes – improving the physical environment of schools (adequate classrooms, water) • Continuing need for scalable, effective models for pastoralists and other hard-to-reach children. • Addressing the needs of OVCs and other children with special needs, particularly those who cannot participate easily in regular schools due to vision, hearing and mobility limitations. • Further work is needed to improve ABEC quality and support systems, including “second chance” options for older youth and functional literacy/skills training for adults... Trends Affecting USAID Strategy Options Over the Next Five to Ten Years • English language competence will be a need for teachers at all levels. • There will be increasing demand for access at the secondary level and problems maintaining quality as the numbers at secondary level increase. • Along with increased demand for secondary and tertiary education, there will be increasing need for attention to workforce development. • There is likely to be at least one major curriculum review and revision exercise within five years, with consequent need for textbook revision and alignment. • The supply of diploma-level teachers will be increasingly adequate in number if not in quality of training and motivation and some consideration is likely for raising qualifications to degree level at least for upper primary (Cycle 2). • The electricity grid and reliability can be expected to improve, along with cell phone and other media density and improved connectivity. It will be increasingly feasible media and IT-supported education as well as for training and pedagogic support, network use, and sharing of information. Impact Evaluation of USAID/Ethiopia’s Education Activities 5 • The primary education system is likely to approach full enrollment capacity within the next 5 years, or sooner, but there will remain significant numbers of children who cannot access the schools or cannot participate fully or successfully. • There is likely to be increasing interest and demand from parents for more early childhood and pre￾school support. • The need for adult literacy and other adult education will continue to be large; and the relative disadvantage of those who have less than primary school will increase. • The ABECs are likely to have expanded and become more fully integrated with the school clusters and support systems from the woredas. – Somewhat unique, it would be useful for some of this documentation to be available for possible adaptation in other countries. • Finally, as all of the above trends play out, supervisors, planners and managers at all levels will have increasing responsibilities for a more complex set of education tasks. Though further training and technical support may be needed and welcomed at other levels, the key link in the “value chain” connecting national and regional policies and program funding to the schools and classrooms will be the woredas. Main Recommendations Summary of Recommendations • The first priority is institutional capacity building at TTCs – capacities for pre-service training – capacities to provide pedagogic support to cluster schools and woreda supervisors plus additional upgrading programs for teachers in the schools – An in-depth review of the Center of Excellence models, roles and support strategy is recommended. • The second priority for institutional capacity building is support for the woredas. Woredas will have increasing responsibilities and will need further strengthening in providing training support. – Cluster resource centers need to be managed better to provide more active and interactive support for teachers and school leaders – WEOs and school supervisors need training on how to provide better mentoring and pedagogic support to schools. • USAID will need to continue to respond at some level to the needs for education opportunities in pastoralist areas and for other hard-to-reach children, as well as for OVCs and for some aspects of special needs. – The team does not see a good option for large scale USAID support for quantitative expansion but recommends continued support for experimentation and support for Ethiopian efforts to develop policies and strategies addressing these needs. – We assume that USAID may want to continue grants programs for OVCs and possibly for other specific needs, but the team makes no recommendation as to the scale or targeting of such programs. • USAID should continue support for efforts to improve and strengthen support for the ABECs, but should not provide direct support for expansion, except possibly on a targeted basis in the pastoralist areas. – Support might include experimentation with mobile libraries, with materials development and use of media, with improved training and support for facilitators – Better monitoring and more assessment of learning outcomes are needed. – Such activities should not be limited to ABECs in pastoralist areas but should include rural and isolated communities in all regions. 6 Impact Evaluation of USAID/Ethiopia’s Education Activities • The adult literacy program provided in ABECs is still small scale and will require sustained research and experimentation with other approaches (possibly greater use of media, more formal “second chance” education options for dropouts and additional skill and content areas of interest to adults) and assessment of results before larger scale initiatives can be justified. • The priority tasks for curriculum and materials improvement are the improved alignment of textbook and other materials development with teacher training methodologies and with the realities of instruction and learning in typical schools. This does not require support for materials production and distribution of basic texts, which will be adequately covered by other funding mechanisms – USAID support should include supplementary materials, teaching guides and support for feedback from the schools and exchange of experience among the TTCs, among regions and among the units responsible for curriculum development, materials development and teacher training. – The priority subject for pedagogic and material support should be English language. – Teacher training is needed to support early grade reading in all languages. – Resources permitting, USAID should consider support for mathematics and science both for primary level and for lower secondary level. – The mode of assistance should be mainly training and other technical support for the respective curriculum development and materials development units and selected TTCs, with any USAID partners in support roles integrated as fully as possible with the Ethiopian implementing units and institutions. • USAID should continue to engage with the MOE and RSEBs as to the major and unpredictable turnover among teachers, principals, and administrators. Recommendations on Modalities • Focus on assessment and documentation of existing innovations and improved practices as a higher priority than support for additional innovations, pilots or experimentation. • If additional experimentation is supported it should be supported in very close collaboration with TTCs or other institutions as part of an Ethiopian process and local problem-solving rather than as activities identified with external implementing project partners. • The early programs, BESO1 and BESO2, had substantially more flexibility and ability to respond to emerging needs than do current project modalities. Implementing strategies and procurement mechanisms should anticipate the need for periodic reviews and revisions of priorities. • The current program appears well managed, but has so many different actors, implementing partners and accountability/management systems it is confusing to the Ministry of Education, complex to coordinate, and difficult to assess. It is desirable to work toward having a smaller number of program umbrellas (not more than three) under which a variety of subcomponents can be managed, coordinated and monitored/assessed for comparative impact. • Consider options for cross-sectoral programming, integrating education and health SOs and perhaps other SOs. Impact Evaluation of USAID/Ethiopia’s Education Activities 7 1. Introduction USAID’s long-term commitment to improving access, equity, and quality in Ethiopia’s primary schools and its educational system was launched in 1994 with Non-Project Assistance which became the Basic Education Systems Overhaul (BESO) initiatives, and has continued through the awarding of some 14 projects to a range of service providers. Projects were designed in close collaboration with the Ministry of Education. The current assessment sought to identify the impacts of this assistance on education quality, enrollment and equity, and management and institutional capacities for the period 1994 to the present. The assessment focused on identifying lessons learned, the potential of the innovative practices introduced, and the degree to which continuing support may be needed and used effectively. As a result of the findings and recommendations presented in this report, it is anticipated that USAID will have sufficient input to create a strategic plan for its education portfolio and design future projects and programs in which USAID has achieved a comparative advantage. The report is organized in the following manner: Section 1 is this introduction. Section 2 presents a brief overview of the recent history of education in Ethiopia as well as a chart presenting a brief summary of the education and education-related USAID-funded projects that partners implemented to support access, equity and quality. Section 3 outlines the methodology used to conduct this assessment. Section 4 addresses the specific issue of access through three subsections: enrollment impacts overall, enrollment increases leading to gender equity, and enrollment increased achieved for hard￾to-reach populations. Section 5 addresses issues of quality in four subsections: overall impacts, teacher preparation and support, materials development, and assessment, monitoring and reporting capacities. Section 6 addresses impacts of capacity building in two sections: administrative and management capacities, and community participation and school governance. Section 7 addresses some of the school-based health initiatives that have been implemented in schools in providing a “bundle” of inputs needed to create a healthy enabling environment for children to learn. Section 8 presents summaries leading to the team’s recommendations in five sub-sections: first, a summary of findings and a discussion of constraints; next, an analysis of probable trends and needs and possible USAID responses; then recommendations. The body of the report ends with a table responding to each of the SOW questions. 8 Impact Evaluation of USAID/Ethiopia’s Education Activities 2. Educational Background – A Brief Overview The United States has been supporting education improvement in Ethiopia since the early 1950s with the exception of the period of the Derg, 1974-1991.1 During the time of the Derg, the educational system declined severely, schools became unusable due to lack of repair, community support for schools was either low or non-existent, no significant decisions were made to improve the quality of instruction, and textbook and materials content and development were severely flawed. Almost a full generation of students and teachers were inadequately trained. In 1992-93, USAID, in close consultation with the newly invigorated Ministry of Education (MOE), conducted a comprehensive assessment to identify the most critical educational needs, determine strategies for achieving and sustaining improvements, and design a robust program for U.S. Government (USG) support. The study revealed that the starting place for reform should be primary schooling, and that isolated inputs or interventions in this sub-sector would be ineffective. No less than a transformation of the entire primary system, from the top down and bottom up, was needed. It was also concluded that while the system desperately needed to accommodate vast numbers of students, the MOE along with other donor efforts was capable of managing the physical aspects of this expansion. What needed attention during this rapid growth period were quality and equity, areas that USAID decided were to be its principal focus. As anticipated, since 1994, and particularly following the promulgation of free primary education in 1997/98 – under the first five-year Education Sector Development Plan (ESDP 1) – enrollment of Ethiopia’s children in primary school grew exponentially. This “education for all” (EFA) approach created a demand for more schools and more teachers that the MOE could not meet on its own. From a low gross enrollment rate (GER) of 20 percent in 1992/93, to 62 percent in 2001/02, and over 96 percent in 2008/09, this achievement has been nothing short of remarkable. Nevertheless, expansion in GER put serious strains on quality, particularly the supply of quality- boosting inputs, including well-trained teachers. The Constitution of Ethiopia, approved in 1993, decentralized administrative and policy functions to the regions, including administration and policy for education. In turn, this required MOE policy changes and restructuring as well as development of new education administration, planning, management and support capacities in the regions. The decentralization created nine Regional State Education Bureaus (RSEB) reflecting a regional structure generally based along dominant ethnic lines (Amhara, Oromia, SNNPR, Tigray, Afar, Benishangul Gumuz, Somali, Harari and Gambella), and two administrative regions (Addis Ababa and Dire Dawa). Each of the regions and administrative areas has its own education structure, although all are based on the woreda (district) system. That made educational development a more complex proposition, but it also facilitated necessary adjustments to the specific needs and contexts of the different regions. Each RSEB has adopted and has each of its constituent schools follow the national curriculum, but is allowed flexibility to modify it to accommodate topics of local interest. However, at both the regional and woreda levels, decentralization requirements resulted in a severe lack of institutional, human and financial resources to address the multiple language issues, poor communication and transportation infrastructure, and insufficient planning and management capacity to collect and analyze data needed to inform policy decisions. Each region has its own educational calendar, language, and implementation 1 Unless otherwise noted, dates make use of the Gregorian (European) calendar. Impact Evaluation of USAID/Ethiopia’s Education Activities 9 procedures. In addition, each region teaches the first cycle of primary school in the dominant mother tongue (generally grades 1-4), with Amharic and English taught as subjects, and with English becoming the language of instruction at any point between grades 5 and 9. The system is also decentralized financially. The Federal MOFED transfers funds (as a block grant, but it is known as regional transfer) to regions through BOFEDs. The regional governments allocate to the different sector bureaus including education and determine the woreda transfer. MOE doesn’t give funds to RSEBs. The system is also decentralized financially. The Federal MOFED transfers funds (as a block grant, but it is known as regional transfer) to regions through BOFEDs. The regional governments allocate to the different sector bureaus including education and determine the woreda transfer. When the Education for All (EFA) and then the Millennium Development Goals (MDG) were promulgated, they were strongly embraced by the Government of the Federal Republic of Ethiopia (GFDRE), and the MOE committed itself to meeting them. Consequently, under the Second Education Sector Development Plan (ESDP II), the GFDRE pursued five goals for primary education: 1. Increased access to educational opportunities at the primary level to achieve universal primary education by the year 2015 2. Improved quality of education 3. Enhanced efficiency in the use of resources 4. Increased equity between males and females, among regions, and between rural and urban areas 5. Increased relevance of education to students to bring about behavioral changes that promote development. The successful implementation of these goals and plans required close cooperation between the MOE and regional state education bureaus and a range of stakeholders, including donors and implementing partners as well as parents, local communities, religious organizations, and the like, all of whom were called upon to assume significant roles in the finance and management of schools. As the system expanded and developed, largely with Ethiopian resources and leadership, USAID and other donors were called upon to help strengthen the planning and management capacities. Capacities for teacher training and curriculum/materials development improved along with the ability to monitor and evaluate educational programs. Ethiopia developed improved data systems and a series of National Learning Assessments (NLA) began in 2000. Teacher training improved, communities became more involved in their local schools, and more books became available to children in all of the local languages adopted by the RSEBs. However, despite all of these improvements, several issues, in addition to the problems of financing and managing rapidly expanding access, remained: 1. how to increase girls’ enrollment and retention; 2. how to address the learning needs of hard-to-reach populations, such as pastoralists and those living in more remote locations not served by government schools; 3. how to improve educational quality broadly, with special attention to effective teacher preparation under a policy that requires only a 10th grade education for acceptance into Teacher Training Colleges (TTCs); 4. how to upgrade and update the existing primary teacher corps; 5. how to decrease the drop-out rate and make education more relevant to children and their families; 6. how to improve the planning and management systems in the Regional State Education Bureaus (RSEBs) and woredas; and 10 Impact Evaluation of USAID/Ethiopia’s Education Activities 7. how to improve the management of schools through strengthening the PTAs and school leadership teams. In response to these growing demands and based in large part on the initiatives supported by USAID since 1994, in 2007 the MOE developed the General Education Quality Improvement Program (GEQIP), which presented five pillars of activity: • Teacher Improvement/Teacher Development Program (TDP) • Curriculum, Textbooks and Assessment • Management Administration Program (MAP) • School Improvement Program • School Grants Program. Annex D contains an overview of the projects through which USAID has provided support to basic education in Ethiopia over the two decades 1994 to 2014. See Annex E for a summary of the GEQIP parameters. Impact Evaluation of USAID/Ethiopia’s Education Activities 11 3. Methodology A six-member team of U.S. and Ethiopian consultants conducted this assessment over a period of four weeks (three weeks in Ethiopia) between mid-April and mid-May 2010. It was structured principally as a meta-analysis of past evaluation and reporting documents, combined with interviews with USAID staff, MOE staff at all levels, staff of implementing partners, and project beneficiaries and site visits. The assessment was not designed for – nor did time permit – the collection of original data except in the form of structured interviews with key informants. Following orientation meetings with headquarters staff of Washington-based partner organizations, and task clarification meetings with USAID/W and USAID/Ethiopia staff, the US contingent traveled to Addis Ababa, where, joined by the three Ethiopian consultants, additional meetings were held. These included sessions with USAID/Ethiopia Basic Education Office staff and the Chiefs of Party for the Addis-based implementers of current USAID supported education programs. Follow-up meetings were then held with key staff of the implementing partners. Team members also met with the co-chair of the Education Donors Technical Working Group to learn of the activities of the major donors to the sector, as well as to get general perceptions of how USAID’s programs are perceived by others and fit into the broader set of interests of the GOE. Members also conducted structured interviews with officials from the Ministry of Education and Ministry of Finance and Economic Development (MOFED). During the second week, members traveled in sub-teams of two for five days to six regions (Amhara, SNNPR, Somali, Oromia, Harari, and Dire Dawa) to witness evidence of impact first-hand. Interviews were conducted with a sample of project beneficiaries, focusing principally on the post-BESO 1 period. Visits were made to RSEBs (5), WEOs (6), TTCs (5), linkage and cluster primary schools (14), Alternative Basic Education Centers (ABECs) (1), and a woman’s literacy & self-help project site. We interviewed as many probable beneficiaries as were available (though the election campaign duties of high-level regional officials made access to many of them impossible). At these sites and through interviews the team assessed impact following a structured set of inquiry topics. Included among our interviewees were school principals and teachers, representatives of PTAs and Girls’ Education Advisory Committees (GEACs), ABEC facilitators, woreda and regional education officials, staff and leaders of TTCs, and others knowledgeable of or impacted by USAID activities. During the assessment, the team referenced over 100 documents provided by USAID, the implementing partner organizations, and Ethiopian educators. From this information gathering, we discussed our findings individually, and where consensus was reached, formed our joint assessments of impact, findings and recommendations. Using ten general impact theme areas (access, equity, quality, teachers, materials, assessment and monitoring, administration/management, community participation and health-related education activities) the team sorted and collated its data. Taxonomy of types and levels of impact was constructed and framed into an impact matrix to use as a tool for organizing our data, observations and findings. The matrix helped us to classify impact types by the ten themes. This process led to the findings and recommendations that are embodied in this report. Documents found to be of particular importance to the team are listed in Annex B, Selected Readings. Annex C lists the names and positions of individuals contacted. Unfortunately, the list of informants is not complete, for there were instances where the focus groups were large or the conversations were not planned. 12 Impact Evaluation of USAID/Ethiopia’s Education Activities 4. Impacts on Access and Equity Access and equity, especially for girls, are very closely interrelated and often have impacts that extend far beyond the classroom. The clearest example of this is the impact of Girls Education Advisory Committees, whose mandate is generally not confined to the school premises. In addition to, or as part of, improving the quality and indeed the possibility of education for girls, the GEACs typically look at curbing harmful traditional practices, especially early marriage and forced marriage, and in community after community they have served as a fulcrum to engage community leaders and civil authorities, such as the police, to enforce existing laws on issues such as early marriage, forced marriage, rape, and abduction. However, since the objective of this section is to discuss the impacts of USAID assistance on enrollment, it should be read in close conjunction with Section 6 on capacity building per se. 4.1 Impacts on Expansion of Access In this sub-section, we discuss impacts on expansion of access, in general and the specific sub-sets of impact on improving gender equity and impact on access for the hard-to-reach and the underserved. Summary Since 1994 USAID/Ethiopia’s principal focus has been on quality and equity concerns. Nonetheless, there have been significant impacts on access and retention resulting from targeted program activities. Directly attributable to USAID efforts, the various community-school granting schemes have demonstrated positive impacts on the number of children attending schools in the supported communities and their persistence to higher grades. Dropouts have lessened and females are attending in higher numbers and remaining longer. Schools in these communities are more strongly managed and overseen by PTAs, Kebele Education & Training Boards (KETBs) and WEOs as a result of USAID efforts, making the school a more child-friendly and parent connected institution. The pioneering efforts of Pact in establishing ABECs for children who are difficult to reach with formal schools have brought education to many more children and literacy to many of their parents. National policies on PTA roles and responsibilities, GEAC recognition and composition, and ABEC acceptance as Cycle-1 primary-equivalent education – each one influenced by the work of USAID partners – have also contributed positively to access, and on a scale that extends far beyond the reach of USAID support. Background USAID’s sector review of Ethiopia’s education system in 1992/93 concluded that access to primary schools would increase rapidly through the new government’s own efforts and with the assistance of other donors. Thus, from the earliest days of BESO 1, the emphasis of USAID assistance to basic education was never the expansion of access per se, although it was acknowledged that any effort that improved quality of instruction and learning would directly lead to expansions in enrollments and retention. The predicted expansion did indeed occur: in the early 1990s; gross enrollments rates were estimated at 22%, while the most recent data (2008) indicate a GER of 94%. USAID saw its role during this period as being principally to assist the GOE apply a basic set of quality measures and inputs while expansion was occurring, and to lay the groundwork for the time when quality would become a first￾order priority for primary school improvement. USAID’s attention to quality, however, wasn’t absolute. Impact Evaluation of USAID/Ethiopia’s Education Activities 13 Exceptions to its main strategy started during BESO2, when activities were developed to reach distant and hard-to-reach populations as well as children who were marginalized by gender, societal status and diseases such as HIV/AIDS. In this context, impacts on access increases were mainly indirect consequences of USAID’s efforts, with the important exception of several specialized attempts to reach the underserved. Current Capacities/Impacts From the beginning, most USAID assistance has had expanding enrollments as a major element, and the impacts are evident, as documented in various evaluation documents and anecdotal reports. First and foremost, individuals have been positively impacted in several ways. As a direct result of USAID-funded grants to communities, new classrooms have been built and furnished and older ones renovated, creating an increase in capacity for children to attend school. The dialogue that accompanied these community-initiated projects dealt with issues of non-attendance and persistence, with particular attention given to girls. Also, more children have been attracted to the grant-aided schools in locations where attention was given to making schools more child-friendly through activities such as making the physical facilities cleaner, safer and more healthful. Additionally, the construction of Alternate Basic Education Centers (ABECs) provided new opportunities for children to gain a first-cycle equivalent primary education. At the school level, positive impacts on access and gender equity have occurred related to PTAs and KETBs, which through training have an improved understanding of the value of schooling and better oversight of school operations. Similarly, GEACs and Girls’ Clubs, started under the USAID Community Schools Activities Project and spread to other non-assisted locations, both spontaneously and by national policy, have sensitized members of school communities in particular and stakeholders in the broader community to factors that deter a community’s girls from participating in education at the community’s schools. In many of these schools, School Incentive Awards helped members of the school’s community to create improved, healthier environments, e.g., access to drinking water, separate latrines for girls, provision of school materials, leading to more child-friendly facilities and helping to encourage children at particular risk to attend school. Other institutions have also been strengthened in ways that promote school access. Many Woreda Education Offices, through their direct involvement with the school-grants programs, are now better able to understand and act on constraints to access and equity. Regional State Education Bureaus have conducted school mapping exercises, which coupled with improved planning methods, have helped these bureaus to identify underserved areas and respond with appropriately sited schools. Teacher training institutions have been strengthened by enrolling more female students, preparing gender related materials and establishing discussion forums on the problems of female students and teachers, thus having an effect on female enrollments at school level. The formal recognition of ABECs and establishment of GEACs demonstrate that USAID efforts that support improved access have also impacted policies of the MOE. In the case of ABECs, early pilot efforts followed by study tours abroad and BESO-sponsored MOE-based research directly led to a major policy shift that established the legitimacy and equivalency of this alternate route to a Cycle-1 primary education. Similarly for the GEACs, early USAID support ended up creating national impact. Arising from the pioneering community-based work of World Learning International and Save the Children, in which constraints to girls’ access and persistence featured prominently in parent-school dialogue sessions, the locally-driven spontaneous creation of similar committees to deal with gender￾specific barriers led to a rapid spread of GEAC or equivalent throughout SNNPR and eventually the 14 Impact Evaluation of USAID/Ethiopia’s Education Activities promulgation of regional policies in, e.g., Oromia mandating GEACs and a national policy encouraging nation-wide adoption of the approach. Remaining needs for further capacity building to expand access: • Communities can still benefit and enrollments still need to expand in several regions, especially the emerging ones such as Somali and Afar. At the end of the current Community-School Partnership Program (CSPP), many communities will still be untouched by interventions like those used by the CSPP and its predecessors, which have had major impacts for stimulating enrollments and improving persistence, particularly for girls and OVC.2 In addition, schools and communities continue to need help with educational assessment, planning as much as or even more than actual grant or in-kind inputs. • As CSPP-type programs continue, there is a need for more study in the durability and sustainability of the community interventions as well as to see if spontaneous spread is occurring in unsupported communities, as has been reported anecdotally. USAID deserves to be proud of its pioneering work in connecting communities with their schools, as well as for the influential role it played (along with others) in getting the MOE to include school-based improvement plans and school block grants into GEQIP. However, caution is in order as the community support packages become more defined and prescriptive by virtue of USAID earmarked funding sources, to ensure that community voice and thus ownership in the intervention is not compromised. Similarly, with implementation of GEQIP’s school-based efforts, it will be important to monitor whether the intensified government involvement improves or detracts from the quality and/or frequency of community-school interactions. • ABECs have proven themselves to be credible alternatives to formal Cycle 1 primary schools. The program deserves to be expanded, and USAID should play a key role in its continued strengthening. The phenomenon of some ABECs being viewed by communities as only transitional means to establishment of regular Cycle 1 schools needs to be studied. The consequences of this development could have important impacts on enrollments, cost, flexibility and access for adults. The alternate nature of the ungraded instructional delivery system has financial, personal and learning advantages for many students over the formal school model. These advantages should not be lost in the rush to “formalize” the school. • Though over-age enrollment appears to be moderating, it is common to see children in grade 1 who are 9, 10 or older. This can be seen as a positive sign that primary education is now a normative expectation for all children and those not previously enrolled are now doing so. However, it makes age-appropriate pedagogy difficult and probably accounts for a significant amount of dropout and non-completion, as a child entering at age 10 would be at least 18 before completing primary school. 2 Because most of the USAID support comes from PEPFAR, in the USAID context the term “OVC” generally refers to Orphans and Vulnerable Children affected by HIV/AIDS. However, as a concomitant to the USAID￾assisted activities, many schools and communities have taken advantage of the approaches and capacity-building to address the needs of children who are vulnerable as the result of HIV/AIDS to assist children who have been orphaned or who are particularly vulnerable as the result of other illness or accident and/or severe poverty. Impact Evaluation of USAID/Ethiopia’s Education Activities 15 4.2 Gender Equity (Female Enrollment and Completion) Summary Each USAID-funded project has included components focusing on increasing girls’ enrollment and retention at the primary school level. From BESO I to the current CSPP and School-Community Partnership Serving HIV/AIDS-Affected OVCs (SCOPSO) projects, each identified a strategy to address the needs of girls – whether establishing GEACs or training administrators at woredas – a component considered the under-enrollment and underachievement of girls. In primary schools, attention was focused on the girl child through the workings of the GEACs. At TTCs, the focus was on the increased opportunity for young women to become successful teachers and role models for their future girl students. 4.2.1 Improving Access and Equity for Girl Students Historical Context. Review of the Statistical Abstracts for the years 2005/2006-2008/2009 shows an escalating rate of enrollment for boys and girls, with a dramatic increase – particularly for girls – beginning in 2004/2005. 4.1 Enrollments Over Time Year GER NER Gender Disparities in percentage points (GER) Male Female Total Male Female Total National 1990/91 22.0 16 22.0 6.0 2001/02 71.7 51.2 61.6 59.0 45.2 52.2 20.5 2002/03 74.6 53.8 64.4 60.6 47.2 54.0 20.8 2003/04 77.4 59.1 68.4 62.9 51.8 57.4 18.3 2004/05 88.0 71.5 79.8 73.2 63.6 68.5 16.5 2005/06 98.6 83.9 91.3 81.7 73.2 77.5 14.7 2006/07 98.0 85.1 91.7 82.6 75.5 79.1 12.9 2007/08 100.5 90.5 95.6 86.0 80.7 83.4 10.0 2008/09 97.6 90.7 94.2 84.6 81.3 83.0 6.9 Unfortunately, it is not possible to obtain disaggregated enrollment and retention statistics for schools that have benefited from USAID support from the MOE Statistical Abstracts. However, by presenting a few of these statistics and comparing them with those in project documents, we can see a pattern emerge on the differences between USAID-supported and overall statistics on enrollment, retention, and repetition. The 2007/8 MOE Statistical Abstract presents the following data: • Survival rates for both girls and boys through grade 5 rose from 44.2% in 2000/1 to 56.8% 2006/7, with girls’ survival rates higher (59.4%) than boys’ (54.7%). • Primary completion rates for grade 8 increased from an overall rate of 18.7% in 2001/2 to 44.7% in 2007/8. The completion rate for girls rose from 13.7% to 39.9% for the same years. 16 Impact Evaluation of USAID/Ethiopia’s Education Activities • Repetition rates for boys increased from 5.9% in 2002/3 to 6.6% in 2006/7, but repetition rates for girls decreased from 7.7% in 2002/3 to 5.7% in 2006/7, with the combined rate decreasing from 6.7% to 6.2%. Repetition rates are highest in grades 1, 5, 7, and 8. In each grade the rates for girls are slightly lower than for boys.3 • Dropout rates for boys are declining, from 19.6 for boys in 2002/3 to 13.1% in 2006/7 and for girls from 18.5% to 11.6% in 2006/7. Dropout rates are highest in grades 1, 5 and 8 • Though there is some improvement in the emerging regions, only 28% of the total estimated 575,000 girls of primary school-age in Afar and Somali are presently in schools. Net enrollment rates for these two regions are 24% and 32% respectively, compared to the national NER of 83% (MOE Education Statistics Annual Abstract 2008/09). • Further, girls’ retention rates are lower. Within the overall context of improving access to education for pastoralists, particular attention should be paid to approaches that can improve enrollment and retention, including inter alia the feasibility of establishing and maintaining GEACs, Girls Clubs and other support mechanisms. Key Interventions. In schools, the key innovation is the introduction of Girls’ Education Advisory Committees (GEACs). Beginning with BESO I and II, GEACS were established training was provided to teachers, parents, and other school stakeholders on how to identify and address girls’ needs. These committees have been established at several thousand schools and do the following: • House-to-house visits, which encourage parents to enroll and keep girls in school, often providing suggestions as to how parents can make this possible. • Tutorial classes for female students. • Provision of school supplies, clothing, and hygiene supplies for girls from poor families, often through the establishment of school-based income-generating activities • Provision of counseling to prevent dropout, deterrence of sexual harassment and bullying, gender education, and awards for high achieving female students • Organization of meetings with parents and community leaders about traditional customs such as polygamy, female genital mutilation, early marriage, abduction, rape and other factors affecting the safety and education of girls. • Rescue of female students from abduction attempts; support to continue their education • Advocacy to bring teachers and others who abused female students to trial • Organization of woreda level meetings to discuss problems/challenges of girls' education and to develop remedial actions for better school participation by girls • Guidance on HIV/AIDS prevention and control, avoiding stigma and discrimination against people who are HIV positive Under the overall guidance of the PTA, GEACs are effective in promoting girls’ education and protecting girls from harm.4 The establishment of GEACs has become a nationally accepted support mechanism 3 At grade 5, children from Cycle 1 transfer to Cycle 2 and, in many regions, it is the grade at which English becomes the medium of instruction. Repetition of grade 8 may indicate a “no pass” grade in school leaving exams. 4 Some schools have established Girls’ Clubs in addition to or in lieu of a GEAC, with both operating on similar issues. Girls’ Clubs, however, only involve school-based adults as advisors rather than the entire community, and unlike GEACs, Girls’ Clubs are not typically part of a school’s formal administrative structure. Impact Evaluation of USAID/Ethiopia’s Education Activities 17 for girls at primary level and, in some cases, has had a ripple effect in secondary schools. These activities helped to produce the following results: • a steady increase in girls’ enrollment and retention such that in some schools the number of girls in school exceeds that of boys;5 • the reduction of early marriage and marriage abductions and an increase in prosecution; • greater academic success due to the provision of academic and esteem-building support; • the mobilization of the community on behalf of girls and women generally. As noted above, in emerging regions such as Afar and Somali, girls’ enrollment and retention rates lag significantly. Within the overall context of improving access to education for pastoralists, particular attention should be paid to approaches that can improve enrollment and retention, including inter alia the feasibility of establishing and maintaining GEAC. 4.2.2 Improving Access and Equity for Prospective Female Teachers Current Situation. According to the Office of Women’s Affairs at the MOE, the number of teachers, both male and female has increased substantially in both absolute and relative terms. In 1990/91, there were approximately 50,000 male and 18,000 female teachers. By 2007/8 there were approximately 160,000 male and 95,000 female teachers. In terms of qualifications, the gender gap is now very small. In Cycle 1 with 95.6 % of female teachers qualified at the certificate level or higher in 2007/8 compared with 97% of male teachers. At cycle 2, 72.5% of the female teachers are qualified at the diploma level as compared with 64.1% of male teachers. The upgrading program requires that in five years’ time all Certificate holders must earn their Diplomas. The SNNPR RSEB head reported that there had been an 80% increase in the number of women going for their diplomas, either in the evenings or during the summers. Many of those who already had earned their diplomas were now earning their first degree. Upgrading programs will provide teachers with more content knowledge as well as enhance their understanding of all of the USAID teaching/learning innovations that have become a part of the TTC curriculum to produce quality teachers. Data on the current enrollment of students at TTCs indicate there is still room for improvement in enrolling females. At one TTC in SNNPR, there were 1,691 students enrolled, of whom 674 (40%) were female. There were 81 staff, of whom only 6 were female (7.4%). When the selection for new students is made at the woreda level, limited attention is paid to creating a gender balance. In former years, the TTC recruitment process favored women with the first 50% of students accepted being women, and then the second 50% more competitive. In other regions such as Amhara, TTC students are now selected by an entrance exam rather than by woreda quotas and the TTC has substantial policy discretion to give some preference to female students. A continuing problem is that many girls fail the 10th grade leaving exam, which makes them ineligible to take the TTC entrance exam, thus creating a lack of women in the pipeline to become teachers. At TTCs, initiatives to create a more welcoming environment and to increase the number of female teachers include: 5 According to the MOE 2007/8 Statistical Abstract, overall total enrollment is 8.3 million boys and 7.2 million girls. 18 Impact Evaluation of USAID/Ethiopia’s Education Activities • Gender Clubs –incentive funding to hold meetings and discuss common problems • Forums and Networks – connected female students in all TTCs and female teachers in their respective regions • Modules and Courses – gender sensitization modules and courses were produced and, using teacher guides, implemented in various courses. However, it appears that insufficient focus has been placed on the recruitment of female faculty. A few of the major factors inhibiting women’s roles in education, according to Women’s Affairs at the MOE, are: • Lack of commitment to a flexible model of educational provision • Unsupportive school culture (e.g., facilities, environment, attitudes) • Irrelevant curricula to girls’ lives (e.g., life skills training) According to MSI’s 2007 evaluation of BESO II, in Addis Ababa the number of female school principals increased from 65 to 95 in 2007; in Afar from 2 to 6 in 2006; in Amhara from 433 to 499 in 2006; in Harari from 15 to 22 in 2006; in Somali from 4 to 5 in 2007, in SNNPR from 115 to 201; in Tigray from 30 to 65 in 2007; in Dire Dawa from 5 to 12 in 2007, for a total increase during this time period of from 669 to 905 or 26.1%. Despite these efforts, there is still a gap in the number of female teachers and the number of female administrators. At the woreda and RSEB levels, there is still a severe lack of professional women in positions of leadership. While the Business Process Re-engineering may have made advancing women into leadership positions at different educational structural levels a priority, there was little mention of this when interviewing RSEB, woreda, and school heads. The number of women in administrative and educational leadership positions is increasing, but very slowly. At the TTCs, creating women’s networks and generating gender-sensitive teaching materials have not yet produced the desired outcomes of increased women faculty. Participation in Girls’ Clubs may have had an individual impact, but institutionally, there have been very few improvements for women. 4.2.3 Current Capacities and Needs for Further Capacity Building GEACs in some areas have the capability to identify a broad range of issues – both inside and outside the school that hurt girls’ ability to benefit from education, and to resolve them with parent/community assistance. GEACs provide academic tutoring, life skills development, training in personal hygiene and sanitation, and in HIV/AIDS. In Jigjiga and other locations, GEACs and PTAs go to parents to try to convince them to allow their daughters to return to school, even though they may have been promised for an impending marriage. Girls’ Club advisors at TTCs elicit input from female students on both social and academic problems. At primary schools, GEAC members, in most cases women (but in some schools, men because there are no women staff), provide or arrange for special tutoring for those in academic need. The USAID-introduced innovation of GEACs has now been adopted by the MOE and has become a “must have” at every primary school. However, the MOE has not yet had the opportunity to provide training to establish GEACs at all schools. With significant turnover in school administrators, many school leaders have not been fully introduced to the idea and in schools where leadership has changed, some of the GEACs have become moribund. At one school in SNNPR, the GEAC leader was promoted to the woreda, and with her departure no staff member took up her responsibilities. Hence, Impact Evaluation of USAID/Ethiopia’s Education Activities 19 when we interviewed the new Vice Principal, he knew nothing about GEACs. Continuous awareness raising and capacity building is required to meet the needs created by the high turnover. It is critical for girls’ access, retention and survival that training be provided on GEACs at all schools. At TTCs there is still a great need to create female-friendly environments by hiring more female faculty, establishing Girls’ Clubs, and developing professional networking as well as aggressive leadership to address issues of discrimination and sexual harassment. In our interviews at TTCs, male faculty did not address this issue at all. To create a more gender-balanced cadre of teachers, each TTC should develop a plan as to how they will work with the RSEBs and woredas to increase the number of female students as well as the number of female faculty and administrators. RSEBs and woredas should themselves develop a plan that will recruit more women administrators in leadership positions and thus remove the glass ceiling. To continue a focus on girls’ education, representatives of Women’s Affairs in each region and woreda should be encouraged to develop an action plan to work with schools so that leaders at all levels will become more gender sensitive. Special efforts will need to be developed to encourage significantly greater participation by pastoralist girls in education. 4.3 Impact on Equitable Access for the Hard-to-Reach and Underserved Summary Some 80 million people live in Ethiopia; 16.5 million of them are school-age children, and 49% of them are girls. There are 12-15 million pastoralists, the largest pastoral population in Africa (Pastoralist Development Forum, 2006). Though Ethiopian pastoralists represent a sizeable portion of the population, they are underserved and live without access to basic health, education and other social services. Four million children (not all school age) are considered OVC, who no longer have the tangible and intangible supports their parents would give them. While there was (and is) a great desire and willingness across Ethiopian society to help OVC, this was accompanied by frustration and discouragement at not knowing how to do so effectively. And large numbers of adults are non-literate women in sore need of livelihoods skills. Various efforts to reach the more remote and underserved segments of Ethiopian society have been implemented by USAID-funded partners over the past eight years. In some cases, this expansion of access to particularly disadvantaged groups was a sub-set of interventions to improve access; for example, through CGPP children (and their parents and grandparents) in many communities in Benishangul-Gumuz gained access to education for the first time ever. In this section, however, we discuss projects whose key focus was on reaching the hard-to-reach and the underserved. Their impacts on the educational opportunities for pastoralist populations, for OVCs, and for non-literate adult women have been impressive, but relatively small against the total needs of the country. This notwithstanding, USAID has supported models for boosting educational access for these populations that have been shown to be effective. For pastoralists and various other groups not reached by formal schools, 531 Alternative Basic Education Centers (ABECs) sponsored through TEACH 1 have provided primary (Cycle 1) equivalent education to over 150,000 children and literacy training to over 50,000 adults 20 Impact Evaluation of USAID/Ethiopia’s Education Activities received literacy training. The MOE has formally recognized the equivalency of the ABEC model of delivering education and has adopted it as a national policy. For OVC, World Learning and Save the Children have taken the approaches originally developed under the BESO I Community-Schools Activities Program in SNNPR to promote community support of the schools and has incorporated the “lessons learned” from other PC3 partners in areas such as food, health, and psychosocial support, and has helped school stakeholders in hundreds of communities to develop and institutionalize replicable community-driven approaches to support OVCs. Because of PEPFAR funding and the limited amount of funding available for each school, USAID’s efforts could directly target only children affected by HIV/AIDS and only a portion of them. However, schools could and do make use of the approaches to provide support to many children who have been made particularly vulnerable due to other causes, such as malaria. 4.3.1 Support for Pastoralists Data on the total number of school-age pastoralist and semi-pastoralist children are limited. The two regions where data are available, Afar and Somali, report 1.34 million school-age children, of whom only 30% of them have access to primary education, mostly through the ABECs. Additionally, though quantitative data is not available, a sizable number of school-age children reside in the Borena and Guji Zones of Oromia and the South Omo Zone of SNNPR. As noted under 4.1.1 above, only 28% of the total estimated 575,000 girls of primary school-age in Afar and Somali are presently enrolled in schools. Net enrollment rates for these two regions are quite low, 24% and 32% respectively, compared to the national NER of 83%. The TEACH 1 project (the first of its kind) began implementation in 2004 by PACT-Ethiopia and its 27 Ethiopian partner non￾governmental organizations (NGO). The project operated in eight regions of Ethiopia and helped to establish over 500 ABE centers, and introduced in them education and literacy programs that resulted in increased educational access for more than 150,000 children and 50,000 adults in disadvantaged parts of the country. It also improved the capacity of 63 woreda education offices to enable them to manage the new ABE system of education and literacy programs. WEOs, RSEBs and the Jigjiga TTC, especially the Center of Excellence for Pastoralist Education, have also been institutionally strengthened in ways that expand access for the hard to reach populations, particularly pastoralists. These institutions are now better able to understand and manage constraints to access and equity, and to offer solutions to the ABE centers and adult functional literacy programs. Although USAID is not the only implementer of ABECs, it is certainly one of the most influential. The formal recognition of ABECs as an equivalent for Cycle 1 primary education and the inclusion of this modality in GEQIP demonstrate MOE policy-related impacts that have resulted from USAID-funded efforts. 4.3.2 Support for OVC In 2004, USAID launched PC3 (Positive Change: Children, Communities and Care Program of Support for Orphans and Vulnerable Children Affected by HIV/AIDS) in 2004 to address the multifaceted problems of OVCs in Ethiopia. The project was implemented by SAVE/USA and its international and Impact Evaluation of USAID/Ethiopia’s Education Activities 21 local partners, namely WLI, FHI, CARE, World Vision, and 35 local NGOs and 575 Community-Based Organizations (CBOs). In 2006, World Learning began the two-year CASCAID (Communities and Schools for Children Affected by HIV/AIDS) project which used the schools as the community focal point for delivery of multiple services and included many schools which had not previously benefitted from BESO. CASCAID has been succeeded by School Community Partnerships Serving OVCs (SCOPSO), which will work with 400 primary schools. The following two tables indicate how promotion and dropout rates changed as a result of the implementation of the CASCAID program targeting support for OVC, a sub-set of disadvantaged children, including girls. The first table from the CASCAID Final Technical Report, 2009 (GC) presents the data for the neediest OVC; the second table presents the data for all students in the same schools. 4.2 Promotion and Dropout Rate for the Neediest OVC supported by CASCAID for 2000 EC Academic Year 4.3 Promotion and Dropout Rate in Percentage for All Students Enrolled In CASCAID Schools 2000 EC Academic Year Region Rate of Dropout Rate of Promotion Rate of Not Promoted M F T M F T M F T Amhara 0.08 0.07 0.08 0.68 0.70 0.69 0.07 0.08 0.07 Oromia 0.11 0.09 0.10 0.69 0.71 0.70 0.05 0.06 0.05 SNNPR 0.12 0.09 0.11 0.70 0.72 0.71 0.06 0.07 0.07 Total 0.11 0.08 0.10 0.69 0.71 0.70 0.06 0.07 0.06 In each region, the dropout rates are higher and the rate of promotion lower for all students than it is for those OVC receiving USAID support to keep children, especially girls in school. While these statistics do not correlate directly with those presented in the annual MOE abstracts, they illustrate a connection between project interventions and survival rates, especially for OVC. In compliance with PEPFAR requirements, in 2008-2009 Save conducted a baseline for the CSPP project to be able to track changes in access, retention, and survival and impacts as a result of interventions made. The CSPP baseline survey provided the following dropout information for all 1,800 schools covered in the project, by region: Region Rate of Dropout Rate of Promotion Rate of Not Promoted M F T M F T M F T Amhara 0.04 0.03 0.03 0.82 0.81 0.82 0.05 0.08 0.06 Oromia 0.01 0.02 0.02 0.88 0.84 0.86 0.03 0.05 0.04 SNNPR 0.00 0.01 0.01 0.85 0.81 0.83 0.09 0.09 0.09 Total 0.02 0.02 0.02 0.85 0.82 0.84 0.06 0.07 0.06 22 Impact Evaluation of USAID/Ethiopia’s Education Activities 4.4 Dropout Rate by Region and Sex 2007/8 Region Dropout Rate Boys Girls Total Tigray 12.2 8.2 10.2 Afar 24.1 20.8 22.7 Amhara 16.9 13.9 15.4 Oromia 20.0 17.5 18.8 Somali 19.8 25.9 22.1 Benishangul Gumuz 18.8 18.9 18.8 SNNPR 21.1 18.9 20.1 Gambella 20.4 20.9 20.6 Total 19.0 16.4 17.7 The highest dropout rates occur for these schools at first grade (boys 27.1%, girls 24.1%, together 26%). The dropout rates decline steadily until grade 5, at which time they spike, decline for grade 6, and then spike again for grade 7. The baseline also indicates that the highest repetition rates for each grade for all target schools in all regions is for boys, as follows: 4.5 Repetition Rates in 1,800 CSPP Schools (Baseline Survey) Grade Percentage of Total Population Repeating Percentage of Boys Repeating Percentage of Girls Repeating 1 7.4 52.8 46.2 2 6.4 53.3 46.7 3 6.2 53.5 46.5 4 7.4 50.1 49 5 9 52.7 47.3 6 6.3 55.5 44.5 7 8.6 53.6 46.4 It will be important to conduct a panel study on many of the variables included in this study so that project impacts on educational access, retention and promotion can be tracked. By requiring projects to track this information at each intervention school would demonstrate the types of impacts the interventions are having. 4.3.1 Support for Women’s Literacy and Livelihoods Efforts The Ethiopia WORTH project represented an adaptation of a highly successful model that PACT piloted in Nepal. At present more than 9,000 adult women organized in 400 groups/centers in hard-to-reach environments in 15 woredas in Amhara, SNNPR and Oromia have benefited from Literacy Lead Saving and Credit- WORTH project in which literacy materials focus on livelihood topics so that as a woman learns to read, she also learns how to operate a business. However, an evaluation of the WORTH pilot project showed mixed results. After three years of participation, only 33.9% of members were able to read. For those who did not achieve literacy, many women relied on their daughters to read the lessons to them. In the group of 23 observed outside of Zwai in Oromia, all could write their names in the Amharic syllabary, but only two indicated that they continued their reading (one read the Bible and the other read signboards outside of offices). Impact Evaluation of USAID/Ethiopia’s Education Activities 23 Nonetheless, there was a major impact for the women in the increased incomes they obtained from putting the business lessons into practice, which gave them the funds to send their children to school. Ability to generate more sustainable incomes improved household relationships, increased household assets, enabled the women to grow in community esteem and participate in wider community events. These impacts notwithstanding, many women are still seeking outside assistance for increased capital for their savings and lending activities and group businesses as the amounts they are able to save as a group are inadequate to meet their borrowing needs. 4.3.4 Needs for Further Capacity Building In terms of educational access, marginalized groups, principally pastoralists, agro-pastoralists, OVCs and other children residing in the most difficult to reach areas of the country represent very large numbers of children who remain disproportionately disadvantaged. Children in pastoralist and other remote communities. These represent well over 1 million children, with the number increasing greatly over the next decade. One key step, provision of official recognition to ABECs as equivalent to formal schools, has already been institutionalized. To address ongoing needs, the following are required: • The ABEC and/or other alternative model for pastoralists’ children and harder to reach should be strengthened with recruitment of additional facilitators, production of text books and provision of more training for facilitators and supervisors. Any direct expansion should be limited to targeted areas. • Organizing ABECs in clusters could play critical role in maximizing the benefits and intended results by creating more effective structures for supporting ABEC management, providing pedagogical support and facilitating exchanges among the ABECs and facilitators within the clusters. Though the distances between ABECs will preclude the physical interactions and use of cluster resource centers available to the regular school clusters, it still is desirable to plan, assess and support the ABECs in a given area as a structured system rather than as merely a collection of independent and isolated entities. • Closing gender disparities remains a challenging, but important need. USAID should: (a) help the MOE and regional education authorities6 to develop and implement social marketing campaigns to help pastoralist parents gain an appreciation of the benefits of education for girls; (b) use the leverage of positive impacts brought by CSPP to bring more pastoral girls (especially in Afar and Somali) to ABECs and maximize internal efficiency observed in TEACH-I; (c) encourage regional education authorities to provide additional support for girls and work with education providers to implement GEAC or functional equivalent. • The Center of Excellence for Pastoralist Education at the TTC/Jigjiga COE needs organizational, operational and material support, both to improve its own program of facilitator training as well as to serve as a model for other regions as they address the education needs of their pastoralist populations. 6 Because of the very weak capacity of the education system as a whole in Afar and Somali, especially relative to the need, we are using the term “regional education authorities” as opposed to RSEB so as to suggest that any available resource be used. 24 Impact Evaluation of USAID/Ethiopia’s Education Activities Support for OVC. While USAID-assisted efforts have definitely improved the quality of life for thousands of school-age OVC and the ability of service providers to address their educational and other needs, nonetheless there remain many more highly vulnerable children who lack effective access to education. The assessment team assumes USAID will continue support for OVCs at some level, but makes no recommendation as to the scale of the continuing USAID support programs, which will be determined largely by budget availability in any case. Literacy and Livelihoods for Adult Women. While the WORTH program has had some success in improving livelihoods for struggling women, it has had only minor success in improving effective literacy. As an education intervention, therefore, it is hard to justify, although it can be justified on other grounds. Impact Evaluation of USAID/Ethiopia’s Education Activities 25 5. Impacts on Quality 5.1 Impacts on Learning and Teaching Summary The systems change that USAID brought about through joint planning with the MOE created an awareness of quality education. Prior to these interventions, the MOE was more concerned with expansion or access than quality. The MOE was of course aware of the need for attention to quality before the USAID interventions. However, the MOE was not giving proper emphasis to quality of education. USAID support for the National Learning Assessments (2000, 2004 and 2008), provided important insights into student achievement levels and the problems associated with low achievement. The analysis of the assessment data, and the subsequent dialogue and policy reviews within the MOE, influenced the MOE to focus more sharply on quality of education, which resulted in the development of GEQIP USAID activities have contributed both directly and indirectly to the improvement of learning outcomes. Directly, there has been considerable in-service training and workshop support for teachers, emphasizing active learning, continuous assessment and more child-centered approaches as well as providing supplementary materials for mathematics and science, English language materials and training for teachers and supervisors on their effective use. There has also has been support for pedagogic resource centers at school clusters, support for teacher upgrading programs (including e-learning modules for pre-service students) and support for improvement of pre-service teacher training and the establishment of “Centers of Excellence” at selected TTCs. USAID has contributed to the development of alternative approaches, including materials and training for facilitators, for the ABECs which are now being implemented and expanded nationwide as Ministry policy. On a more limited basis there also has been support for small-scale pilots supporting adult education and literacy for pastoralists. Though there is some statistical and analytic evidence (e.g., the 2009 AEI Assessment ) and the judgments of key individuals interviewed and site visits by the team) that improved practices are beginning to take hold and becoming generalized (more active learning, better classroom management, more child-friendly environments, better trained teachers with more in-service support), the inescapable fact is that measured learning achievement remains unacceptably low, with significant differences by region. In fact, the measured learning levels have decreased over the period 2000-2008, with particularly low scores for early grade reading, for mathematics and English language competence. There are a number of methodology issues with respect to the National Learning Assessments, which the team did not have time to assess in any detail. For example, the results are aggregated on a regional basis and thus mask the existence of better-performing schools; the sample for the Grade 4 assessments do not yet include the ABECs so there is no basis for comparison of learning in the ABECs with that in regular schools (other than the grade 4 leaving exam); the sampling for the National Learning Assessments is such that it is impossible to infer, let alone to determine, whether any USAID interventions have had any impact on any learning; and there is some suggestion that as the assessments have been refined in each round they have gotten somewhat harder. 26 Impact Evaluation of USAID/Ethiopia’s Education Activities In any case, the main explanation for lack of measured improvement in average achievement levels is the explosive growth of enrollment over the past 15 years which has led to very crowded classrooms, expanding numbers of poorly resourced schools, reliance on poorly qualified and inadequately supported teachers, and a variety of problems of inadequate textbook supply and lack of pedagogic resources. To some extent, the improvements in survival and completion rates discussed in Section 4 can be taken as a proxy for improving quality, at least to the extent that the grade 8 leaving exam measures quality of learning and children and parents perceive their chances of passing. Current situation There continue to be large numbers of over-crowded, poorly-resourced schools without sufficient qualified teachers or adequate pedagogic materials and support. However, there are some encouraging trends. The numbers and proportions of diploma-qualified teachers are going up and qualified female teachers now outnumber male teachers by a factor of 2:1 in many schools. Textbook supply is improving and will improve further with the large textbook program now beginning under the GEQIP with World Bank/Fast Track Initiative funding. Girls’ enrollment, persistence and completion rates are going up and now approach parity in most schools and in the aggregate. From the Third National Learning Assessment, measured achievement rates for girls are essentially the same as for boys in most subjects and most regions, with the boys’ scores for both genders showing only small variance by region (composite scores at grade 4 ranging from 33.8 in Somali to 47.3 in Amhara). Similarly, there is little difference between rural and urban achievement scores, with rural areas having slightly higher scores in about half the subjects and with little variation across the regions (mean composite scores ranging from 33.5 in Somali to 46.1 in Amhara). The mean scores nationwide for all subjects are higher for rural areas in all subjects. 5.1 Mean Scores for Urban and Rural Pupils National Learning Assessment of Grade Four Students, 2008 Subject Location rural/urban Mean Mathematics Rural 40.3 Urban 40.2 Reading Rural 45.1 Urban 42.3 English Rural 37.3 Urban 35.5 Env. Science Rural 43.3 Urban 41.7 Composite Rural 41.7 Urban 41.6 Performance in the first cycle appears better, and the practices of active learning appear more widespread in the first cycle (1-4) than in the second cycle (5-8). It is reasonable to expect that improved practices will become more widespread and institutionally supported, that textbook supply and pedagogic support will continue to improve, and that the measured learning outcomes will show some improvement by the time of the next National Learning Assessment, planned for 2011. National capacities for supporting quality improvements have been strengthened, in substantial part as an impact of USAID technical assistance and perhaps more importantly as a result of close collaboration Impact Evaluation of USAID/Ethiopia’s Education Activities 27 in assessing needs and joint planning. Similarly, there has been substantial improvement in planning, policy development and management/monitoring at the Regional State Education Bureau levels. The woreda level is the key link in the support chain leading to improved quality. In addition to their management functions, they are the location for most school and cluster supervisors and play a role in organizing much of the in-service training. Woredas are responsible for the administration of cluster centers located in nearby primary schools at which staff of TTCs provide training on a range of topics to cluster and satellite schools. They also are responsible for much of the data collection and reporting, but have little analytic capacity. USAID activities have provided much training support, management system support and ICT support, but have not been able to do so for all woredas, and many remain very weak. USAID should consider continuing to target direct support for WEO, especially in woredas that have had relatively little direct assistance from USAID activities. The bulk of support for WEO training should probably be provided indirectly, through capacity building of the relevant MOE and RSEB EMIS and planning units. Support for the WEO should also go to strengthening the effective capacity of their resource centers. Remaining Needs and Opportunities for Impact on Quality Analysis of the National Learning Assessment (NLA) data confirms the importance of teacher guides and basic text materials, along with good school management, scheduling and in-service support for teachers. Other less tangible factors are school/community variables affecting the expectations of parents, the motivation of teachers and student self-perspectives and expectations for achievement. Thus, in addition to continuing efforts to improve the availability of key inputs, continued support for principals in management and school leadership, continued efforts to encourage community engagement and support and continued oversight, monitoring and reporting on quality factors at the school are needed. The cluster resource centers (CRC) at the woreda level (as observed during team site visits) in some areas, e.g. Amhara, are little more than distribution warehouses for books and other materials. They do not appear to function as pedagogic support centers with the ability to facilitate teacher in-service training, work-shopping and local materials production. The CRC do appear to be more appropriately developed and used in other regions, e.g. SNNPR and Tigray. Similarly, there is great variation in the pedagogic resource centers at the schools, ranging from weak and under-resourced to model centers in which teachers create their leaning aids, conduct unit planning, and share problems and experiences. Strengthening such resource and pedagogic centers appears necessary for support of quality improvement at the school, cluster and classroom levels. Strengthening should be understood as making them more active and interactive and more responsive to assessed needs of the teachers, school leaders and students (not just stocked with more materials and equipment that is underused). There is not yet much data on the pedagogic effectiveness of the ABECs providing alternative first cycle education in rural areas. An impact study conducted for PACT found that facilitation in many centers was weak and that during the current TEACH 2 project, this shortcoming will be addressed. The primary education facilitators at the ABECs are severely under resourced and the MOE’s assumption of the TEACH 1 schools (transforming ABECs into formal primary schools) has put a burden on the RSEBs to provide all the support materials that other schools receive. These students are not included in the National Student Assessment samples. The appropriate measure of success will be the numbers and proportions continuing to the second cycle. At least in the sample of ABECs supported by USAID, there should be measures of learning achievement, including reading, and perhaps early mathematics, science and English. 28 Impact Evaluation of USAID/Ethiopia’s Education Activities Similarly, there is not yet an objective measure of the effectiveness of the small adult education and literacy pilot activities in hard-to-reach and pastoralist areas. The team judgment is that the adult literacy impacts are very limited, though the impact of these programs on women being able to organize self-help saving organizations and start microenterprises may be significant. In any case, these activities are still very small (about 9,000 women participating), and an assessment of the impact will require more time and larger scale activities. 5.2 Teacher Preparation and Support 5.2.1 Teacher Training The many impacts of USAID’s teacher education program (both pre- and in-service) can be found throughout the educational system. One TTC head said that a paradigm shift has occurred in teacher training – moving from a more didactic to a more learner-centered approach, and conducting in-service training using a cluster model - that impacts the entire country. New pedagogical models have been introduced that will have an impact on current and future generations. Some of these changes include: 1) adoption and expansion of the cluster, in-service training model (launched by UNICEF and adapted and expanded by USAID) by MOE; 2) adoption and expansion of the linkage school model to provide practicum experiences for TTC students and for TTCs to provide pedagogical and material support; and 3) a national policy on teaching active learning and continuous assessment at TTCs and at in-service workshops. To accommodate these changes at TTCs, restructuring has taken place, School Development Units (SDU) have been established to plan future departmental activities, and relationships with woreda-based superintendents have been established to manage the operations of the cluster center schools. Other impacts of USAID programs at TTCs include improved planning enabled by the contribution of computers and different software packages, ability to teach students how to make learning aids using local materials (TALULAR), improvement of subject matter knowledge through the use of self-learning modules in science, math, and geography, better student records made possible by the development of a database for use by the Registrar and staff, and libraries at TTCs, and access to more learning resources through the Cluster Resource Centers (CRC). Influenced by USAID programs and as the result of the need for greater subject matter and pedagogical knowledge by teachers in training, TTCs are now implementing a five-year upgrading program, calling for replacement of the current one-year certificate with a three-year diploma. According to the SNNPR RSEB, 80% of female teachers are already in the process of working for diplomas. Delivery mechanisms include summer school, evening school, and distance education. TTCs are also providing a Special Higher Level Diploma program in School Administration to increase the management capacity of principals. Those who wish to undertake a higher degree program in this area can do so at Addis Ababa University. According to the MOE’s Teacher Development Program, over 50,000 teachers have personally benefited from the updating, in-service training programs offered by TTCs through cluster centers and linkage schools, and over 42,000 teachers have been upgrading their diplomas to degrees. Pre-Service Teacher Training Moving from one year of teacher training for 10th grade school leavers (certificate program) to three years (diploma program) will provide more time for students to acquire more in-depth knowledge and be able to practice active learning, continuous assessment, creation and use of learning aids, lesson planning, and classroom management (materials all generated by USAID). Using the linkage schools (some of which are Cluster Centers) as locations for practicums – a system that has been in place for Impact Evaluation of USAID/Ethiopia’s Education Activities 29 practicums since before USAID interventions - will provide students the opportunity to put into practice what they learned at the TTC, especially if linkage schools are provided with resources teachers can access as needed. When TTC faculty conduct observations, feedback they provide on all of the above topics and others included in the curriculum should enhance the professional development of future teachers. One USAID input not being fully utilized either at the TTC or during practicums is videotaping of student-teachers in the classroom and using the videotapes for constructive criticism. Each TTC is to have a media center, equipped with a camcorder and the means to view a recording. Although media center managers were trained during BESO II, many managers transferred to other locations and the media centers fell into disuse. At Hawassa TTC, the media manager had never videotaped a teacher or student-teacher in a classroom; rather, he had videotaped a meeting. The proper use of this equipment in building the competency of current and future teachers should be strengthened. In-Service Teaching In an effort to provide Continuous Professional Development (CPD) to teachers, TTCs are providing in￾service training in all of the above-mentioned topics (as well as others) at Cluster Resource Centers (CRC) (led by woreda-based Supervisors, and coordinated by the Cluster Center Coordinating Units (CCCU) at the TTCs). Resources at the CRCs vary. At some, there are libraries, TALULAR centers, supplementary learning materials, computers, Internet, and the like. At others, very few of these are in evidence. BESO II provided most of these resources to their 22 target TTCs, but since the project ended in 2007, many resources are now underutilized, dated or in poor repair . Since the Supervisors were appointed as CRC Coordinators after the end of BESO II, they have not had adequate training on the management of the CRC, the identification of teacher learning needs, or on the substance of what is being taught during in-service programs. Consequently, the CRCs are not managed very well and are not as attractive a resource for cluster and satellite teachers as they might be. While CPD is an appropriate effort for USAID to make to improve the quality of teachers, the opportunity created by CRCs is not being maximized. In-service workshops are provided at CRCs, but for many, this is all that is offered. Once teachers are equipped with the skills entailed in each lesson, they are expected to go to their schools, cascade the training, and then put into practice what they have learned. However, there is no support for teachers to provide lessons to their fellow teachers, and there is no substantive follow-up support by the Supervisor as they are inadequately resourced to visit satellite schools. If the emphasis on CPD is truly professional development, then more must be done to enhance the professionalism of teaching. Each CRC must be fully equipped with materials that will engage teachers and motivate them to want to learn more. For instance, the TTC media person could come to cluster/satellite schools to videotape “best practice” for teachers to watch, and these videos could be available at each CRC. If linked through the Internet, teachers could be mentored by other teachers in Ethiopia or outside of the country to receive encouragement and to view “best practice” in other countries. Videotapes and CDs should be made available for viewing and learning. Curricula could be developed around certain model presentations on how learning can be achieved. Supervisors would then be resident on-site, be given a different type of job description (including school follow-up), and then equipped to perform their new jobs. 30 Impact Evaluation of USAID/Ethiopia’s Education Activities English Language Training, Language Education and Early Reading Programs English language training falls into two categories: 1) that for students to increase fluency; and 2) that for teachers in training on how to teach English in all primary grades. While student fluency increases as a by-product of all of the materials and training a student receives as a result of USAID-produced materials, the ability to teach English as a second language has not yet been addressed. This gap requires serious attention. Two areas must be explored: • The training required on the use of the English language textbooks produced by the USAID￾supported TLMP project for grades 6, 7, 8, 1, and (soon to be) 2, 3, and 4. The USAID-funded TELL Project was designed to provide in-service training to teachers of appropriate grades on how to use these materials. (One English teacher said that the textbooks for grades 6, 7, and 8 are very difficult and more training on their use is needed.) However, no TTC offers these classes to their current students so that they may become familiar with the texts and the Teachers’ Guides before using them in the primary school classroom. • The pedagogy of teaching English as a second language differs from that of teaching English as a first language/mother tongue. In the former, it is presumed that learners have mastered the alphabet, have developed letter/sound correlation, and that reading/decoding and comprehension skills have been developed in the first language. In the latter, nothing is presumed and instruction begins with sound/letter correlation, and/or reading readiness. When team members observed classrooms in the mother tongue during the 2009 AEI evaluation, they found that reading skills per se were not being taught in any language. It appears that when a student starts learning English, a method of “sight reading” and/or recitation/memorization of what has been written on the board constitute the primary learning methods. The lack of application of appropriate pedagogy, as well as the lack of sufficient textbooks for all children, led to the following observations concerning the ability to read in English, based on reading passages taken out of the TLMP textbooks for grades 6, 7, and 8. (The 8th grade passage was used with students in grades 8-12.) 5.2 Ability of Students to Read at Grade Level From AEI Assessment 2009 Grade Level Read Fairly Fluently No./% Read with Considerable Difficulty No./% Cannot Read No./% Total 6 25/45.5 24/43.6 6/10.9 55 7 31/38.3 41/50.6 9/11.1 81 8 38/49.4 30/39.0 9/11.7 77 9 18/40.9 21/47.7 5/11.4 44 10 15/60 9/36 ¼ 25 11 8/100 8 12 32/94.1 2/5.9 34 These findings are indicative of many points that must be taken into account in the design of English Language Teacher Training: 1) English language fluency of teachers in training; 2) design of a curriculum for ESL; 3) pedagogy for ESL; 4) development of appropriate texts and supplementary materials that use ESL methodologies; 5) teaching of reading skills in the mother tongue. The last point constitutes the link between ESL and acquiring early reading skills in the mother tongue. The many languages used in teaching children how to read in each region requires pedagogy that links sound/letter/symbol recognition, so that these same skills can be transferred when acquiring English. Impact Evaluation of USAID/Ethiopia’s Education Activities 31 Attention to a phonics-type approach for each language would help develop these skills. Once sound recognition is established, then children can begin learning how to write words, understanding the meaning of each word/vocabulary term acquired. In some schools, this is what happens, but it does not happen consistently in each school, and when English is introduced, children have a difficult time making any sense of letters and then words. Hence, when a child reaches 5th grade, when classes are taught in English in many regions, children have not been adequately prepared to learn from the materials in English used in teaching all subjects. At the pre-service level, early reading in the mother tongue must be taught as a forerunner of being able to read in English and to function in the English language on all MOE-set examinations. Courses in how to teach reading in the mother tongue as well as English as a second language must also be taught during in-service workshops so as to equip teachers at all primary levels with the ability to deal effectively with the many language issues extant in the primary school system. TTC Centers of Excellence USAID has supported certain TTCs as Centers of Excellence (CE). The rational for the establishment of these CEs was to develop very specific competencies and centers of learning that could service not only their own faculty and students, but also the entire network of TTCs. • Assella TTC – TALULAR. Established as a CE in the creation of teaching and learning materials using locally available resources, Assella TTC (Oromia) had a well-developed resource center with tools and materials donated by the project (located in its own building). The building also had a resource room filled with the different materials developed that could be checked out. Students are taught how to make the different resources, and then, when conducting their practicums, they share the resources with the schools in which they work. As a CE, the Assella TTC held a national workshop on TALULAR, and then was tasked by the RSEB to hold workshops for administrators, cluster centers, and teachers in the Oromia region. While visiting the Hosanna TTC, there was no evidence of a TALULAR Center • Jigjiga TTC – ABEC Facilitator Training. Established in the Somali Region, where there is a significant number of pastoralists and hard-to-reach populations, Jigjiga has developed the methodology for training ABEC facilitators, both for primary education and adult literacy. When members of the team visited the CE, they learned that more than 6,000 facilitators were currently undergoing or had undergone facilitator training. • Debre Berhan TTC – Local Publishing Facility. At this CE, located in Amhara Region, a publishing facility has been established to print and distribute curricular materials and textbooks. We learned that when there was a shortfall of English language textooks, many regions had more reproduced in Debre Berhan. While it was intended that each CE would serve as a resource for the broader network of TTCs, in effect they have largely addressed the relevant needs for the region of their location only. IQPEP has on its workplan the establishment of 12 more CEs; however, if current and future CEs are to become national resources, more work on networking and capacity building needs to occur beyond the regions. Annual TTC network meetings need to be scheduled for longer than two days, and more sharing of the types of resources that can be accessed needs to take place. Site visits should be scheduled so that TTCs from other regions can become familiar with what is available to them. Resources to establish similar, although probably slightly scaled down, CEs at each TTC should be made available. 32 Impact Evaluation of USAID/Ethiopia’s Education Activities 5.2.2 Support for Supervisors When supervisors were interviewed last year for the AEI evaluation, none was responsible for a CRC; their duties and responsibilities were traditionally those of a supervisor/inspector going to different schools to observe teachers and to provide advice. (Most of the 20+ supervisors interviewed said that they were supposed to be in schools most of the time, but budget shortfalls prevented them from leaving their offices in the woredas.) Since last year, supervisors have been given the responsibility of coordinating training with the TTCs at the CRCs. However, lack of a full capacity-building program for supervisors has prevented them from becoming resources for professional development at the centers. Moreover, after in-service training has been offered to cluster and satellite school teachers, supervisors do not necessarily visit the teachers to observe how they have put to use what they learned or to assist satellite teachers in cascading the training to their colleagues. 5.2.3 Impact of Turnover and Implementation Delays USAID considered the range of competencies, material support, and curriculum that were needed to produce the basics of quality teaching. Consequently, the projects implemented had many sub￾objectives. Due to the way each of these components was introduced, many TTCs, at times, were not able to see the whole. Turnover of trained staff at TTCs created large gaps in the ability of the TTC to implement the entire program, either on the level of their own development or the development of students. Moreover, primary school teaching staff was also mobile. Those who took part in only one or two training workshops knew more than they did before, but did not know all they needed to successfully implement what was designed. For participation to be complete, either at the TTC or primary school level, a commitment to finish the course of trainings might have placed greater importance on continued attendance. However, teachers at all levels move and are moved without taking ongoing commitments into account. It will be important for any future project for USAID to insist on completion of training before educators are transferred. TTC principals interviewed were not fully cognizant of all of the components of BESO I and II and EQUIP II because many were new and many had not been trained.7 While programs at the TTCs and higher levels of education focus on school management-related topics, subject-matter curriculum is not generally an integral part of that training for principals. Consequently, new administrators have not been fully taught about current classroom practices, as implemented by USAID. The only training available on changes in classroom teaching and learning was provided by BESO II and EQUIP II, both of which came to an end, the latter in 2009. It is critical for purposes of sustainability that whatever is taught to student-teachers at TTCs and to teachers at in-service training also be taught, albeit in a somewhat abbreviated manner, to principals and other administrators so that they also are continuously learning and are equipped to manage the many different innovations being introduced in their schools. This also includes training on GEACs, PTAs, and community mobilization. Turnover has created major obstacles to sustainability. It is unrealistic to expect that a project will provide training over and over again to teachers and administrators who leave or who are transferred. While the argument can be made that those who have been trained might still be “in the system,” it is also unrealistic to believe that they will transfer these skills to whatever position they take up next. A responsibility system must be implemented to ensure sustainability, and this requires an institutional exit 7 The regional director for the CSPP program in Gambela said that in the last three months, 60% of principals have been replaced. Impact Evaluation of USAID/Ethiopia’s Education Activities 33 strategy for TTCs in which TTC responsibilities are spelled out for when the project ends. The RSEB must also be held accountable as they are largely responsible for mid-year transfers. The time gap between BESO II/EQUIP II and IQPEP created a drop in enthusiasm at the TTCs, and the complexity of mobility has meant that much trained person power was lost. Moreover, without an exit/ handover strategy to the MOE, many TTCs simply “cooled down.” With the advent of IQPEP, we were told that the TTCs are “heating up” again. The handover to the MOE of the BESO and EQUIP innovations was not as seamless as desired. Lags in the provision of budgets by the MOE meant that staff may have been ready to move forward but could not due to the lack of receipt of a budget allocation. Moreover, BESO training covered 22 TTCs, and when the project was handed over, 30 TTCs existed. The problems entailed in making a focused program national need further attention. 5.2.4 Current Capacities and Needs for Further Capacity Building Current Capacities • All TTCs are now linked through a network that meets annually for experience sharing and mutual learning • Linkage programs create the opportunity for TTCs to provide vital technical and professional development to schools, including highly regarded training programs and accompanying materials • School teachers obtain access to college facilities, including computers, resource materials, and TALULAR supplies • Schools are provided highly regarded support in school management, participate in experience sharing visits and are provided with materials for their pedagogical centers (although limited) • The capacity of TTCs in person power, organization of training programs, and preparation of technical materials is improving • The linkage program supports student teachers and TTC instructors to get a firsthand experience of schools, which increases the understanding of primary schools, their teaching and learning needs, and their management • Due to the models presented in cluster center and linkage schools, many primary schools now have pedagogical centers in which school teams meet, create plans, and develop curriculum Needs for Further Capacity Building While evidence was found of the impacts of teacher training, work on capacity building remains. • Each TTC should develop a strategic plan to take into account the adopted innovations in teacher training, and to reconsider the subject-matter knowledge needed for a Cycle 1 vs. a Cycle 2 teacher with appropriate curriculum revisions made. • The Media Center at each TTC should videotape student-teachers at their practicum’s, which can be used to provide constructive feedback to students; the Media Center should also stock each CRC with videotapes of “best practice” of local and other teachers • Cluster Resource Centers require appropriate management and resourcing to be attractive to teachers • Supervisors of CRCs should be equipped with management and professional development skills to manage the CRC, conduct follow-up supervision observations, and to provide supplemental support to teachers. Supervisors must also be present to support satellite school teachers as they cascade what they have learned to their school colleagues. 34 Impact Evaluation of USAID/Ethiopia’s Education Activities • English language teaching must be conducted at TTCs to improve fluency of students, and for students to be able to teach English as a second language. The MOE already has an English Language Improvement Program (ELIP) and any USAID assistance will need to be developed within the ELIP framework. • Pre-service and in-services courses on the methodologies of teaching reading in the mother tongue must be developed and implemented in each Region. With children learning decoding and comprehension schools early, they will be able to more easily acquire English as a second language. • TELL must be able to implement a pre-service course at all TTCs on the use of all the TLMP￾produced English language textbooks. • CEs should be more deeply networked and serve as national resources for all TTCs. • The biggest issue that still must be addressed is how to help the MOE scale up and implement nationally all the innovations launched by BESO I and II and EQUIP II and adopted by the MOE in a limited number of TTCs and schools. Support for the MOE in creating Human Resource systems not only to provide the man power needed in the growing number of schools, but also to provide the on-going training needed for those who transfer in. Policies on transfer must be developed and enforced to maintain a stable teaching and administrative cadre at one location for at least three years. • The professionalization of the teaching profession must be addressed, not only with in-service training (that adds to the work a teacher must do) and adding a diploma to their credentials (and, hopefully, to their salaries), but also to the respect and position teachers hold in the community. Unless salaries are upgraded substantially and an HR system is developed based on qualifications and years of service, the problems of personal choice mobility will persist and the innovations launched by USAID and adopted by the MOE will not be sustainable. 5.3 Materials Development Summary At the beginning of USAID assistance in 1994, textbooks, while flawed, were available in most subjects in most schools, although not in anywhere near adequate numbers. However, currently most schools, both those in relatively urban areas and particularly those in more rural and isolated communities have few text materials and essentially no supplementary materials of any kind. Many factors have contributed to this: the curriculum reforms that started in 1995; the rapid expansion of enrollment in increased numbers of schools, many in areas where Amharic is not the local language; decentralization of policy and administrative responsibilities to the regions; and limited finance. USAID assistance has contributed to the availability of instructional materials in three main ways: the development of English language text materials for grades 1, 6, 7, 8, with grade 2, 3 and 4 still underway, for distribution to all schools in Ethiopia; production of supplementary materials in mathematics and science; and training for curriculum development and textbook production staff of the MOE. Materials also have been developed for the Alternative Basic Education and for adult literacy initiatives, as well as workshops to encourage development of locally produced materials – Teaching and Learning Using Locally Available Resources (TALULAR) activities. School committees, with grants and raining support under the CGPP and CSPP programs have in many cases raised funds locally and used the grants for purchase texts and other materials. Impact Evaluation of USAID/Ethiopia’s Education Activities 35 In addition to materials for use in primary classrooms, a variety of training and learning materials have been produced by several USAID-supported projects. For example, E-learning materials have been developed for pre-service teacher training (85 modules, including modules on science, math and geography). Each lesson focuses on one topic and serves as model lesson plans for the user. Each lesson states learning objectives, provides creative learning experiences, and self-assessment tools on what has been learned. Under BESO II, 173,000 self instructional kits for teachers and 152,000 supplementary reading materials for students were developed, printed and distributed.. Additionally, AED tested the use of interactive radio instruction (IRI) through BESO, and Education Development Center piloted the Interactive Radio Instruction for Somalis (IRIS) project, which has been adapted and expanded in Somalia. Manuals have been developed on community mobilization that are now being shared throughout the country; 16 modules for delivering health care lessons to primary schools students have been developed; capacity building materials for GEACs and PTAs have also been developed and are available for reproduction and dissemination to schools that are adopting these school support mechanisms. More than 17 planning and management modules were jointly created by USAID education staff and AED/BESO II and EQUIP II project staff to build the capacity of RSEBs, woredas, and school principals. With the recent BPR exercise, it is likely that the relevance of these modules will have to be reconsidered. BESO II and EQUIP II generated approximately eight MIS systems for use in tracking teachers (PMIS), recording student records at TTCs, tracking materials, and the like. Most of these software packages are being used at RSEBs, woredas, TTCs, and primary schools. Limitations on materials development impact are: the lack of direct involvement with curriculum development affecting the content of text materials other than for English, with texts for most subjects remaining very dense with difficult vocabulary; logistic problems with the distribution of materials; and inadequate research and feedback to iterative materials development on the instructional effectiveness of materials as used in typical classrooms by teachers with limited training on their use. Current capacities The availability of text materials is improving, but varies greatly by region, e.g. Amhara reports 1:1 text availability in all subjects except Ethics/Civics, but other regions, e.g. SNNPR, report less availability and problems supplying texts in the local languages. During team site visits to schools in the regions, the team observed uneven supply of texts with much of the problem attributed to transport difficulties and lack of inventory controls, e.g., texts were available in resource centers but not delivered in a timely manner to schools. The World Bank, under the FTI mechanism and in coordination with GEQIP priorities, is planning a program to supply all schools in all subjects (including English language) on a 1:1 basis. Supplementary materials remain very limited, both in subject areas and in terms of supplementary readers, libraries and exemplar materials at the regional, TTC and woreda/cluster resource centers. Most supplementary materials observed in classrooms are wall charts and similar graphic materials. Few manipulables and demonstration kits or other teaching aids were in evidence. There is little use of basic media (videos, audio recordings on CDs or other media replay devices). The CRC are mainly locations for workshops, without much resource material or supplies/equipment for locally produced materials – other than 36 Impact Evaluation of USAID/Ethiopia’s Education Activities printers/copiers for printed materials. Materials distribution systems operating through the respective resource centers lack adequate inventory controls and have problems with transport in distributing texts and other materials efficiently to all schools. The three resource centers that the team saw in Amhara seemed to function only as warehouses and distribution centers. Remaining needs for capacity building • There is a substantial need for supplementary materials for all subjects. Particularly needed are supplementary reading materials in Amharic and other languages, as well as in English: stories, topics of interest to children, perhaps songs and poetry, perhaps student-generated materials. Careful attention is needed to the level of language and vocabulary and to appropriate illustrations. • Resource center materials (reference materials, sample lessons, teachers’ guides, etc.) are needed for the resource centers at woredas and TTCs and for the pedagogic centers at school clusters. Some improvements in physical capacity and ICT support (printers, copiers) and other materials reproduction equipment at schools, particularly school cluster centers, would be useful. • TTCs need a richer variety of resource materials for use by students and by lecturers. Curriculum guides, demonstration kits, reference materials are needed, along with improved access to on-line resources. • Logistic supply systems need attention, particularly with respect to inventory controls and to transport of materials to woredas and schools. This may require some changes in the locations and the staff support for resource centers, preferably relying more on the woredas as distribution points. • Much more use of basic or “small” media would be appropriate, e.g., CDs, DVDs, MP3 media. It is unrealistic to expect on-line resources to be available at most schools any time in the near future, though connectivity should be improving at most TTCs and possibly at more of the resource centers. • Both during the team’s work in-country and subsequently, we were not able to get a meeting with Ato Demissew at EMA to learn about plans and prospects for educational technology, including broadcast. 5.4 Assessment and Monitoring/Reporting Capacities Summary At the beginning of USAID assistance in the mid-1990s, Ethiopia had a national reporting system with administrative data from schools reported through the regions and aggregated nationally. Data reporting was vertical with little feedback to or use by the regions, woredas or other local education institutions such as TTCs. Data reporting was typically two years behind, or more, and as a result this data was used mainly for administrative purposes (budget accountability) and internal as well as international reporting. There also were school leaving examinations at grades 8 and 12 which were used to inform decisions on promotion to higher levels of education. The capacity for using such data for quality assessment and reform strategies is reported to have been very limited. USAID activities have contributed to capacity development in the form of impacts of training on individuals at the MOE, Regional and woreda levels; impacts on institutions and systems, particularly improved management at the MOE, RSEB and woreda levels; and impacts on the use of data and information systems for planning and budgeting at the MOE and Regional levels, and to some extent for the Teacher Training Colleges. Monitoring, evaluation and reporting systems appear adequate, even in some cases somewhat excessive in terms of the volume of data generated, reported and published. Though such capacity building is an indirect strategy for improving learning in the schools, over time Impact Evaluation of USAID/Ethiopia’s Education Activities 37 these improvements in individual, institutional and systems capacities have an impact on the quality, equity and effectiveness of education. Such capacity building has been key to the progress Ethiopia has made, and it is doubtful that a more direct assistance strategy focusing on inputs would have had as substantial and as sustainable an impact. Similarly, at the school level, assistance has focused on capacity building in self-monitoring, planning, mobilization and reporting. Principals were also trained on data collection and reporting of school-level information. USAID-supported projects provided PTAs and/or OVC Support Committees with School Incentive Awards (SIAs), which were used to leverage community contributions, made in the form of cash, supplies, and/or services. The PTAs were then responsible to the school and the broader community for reporting how funds were obtained and how they were used. Members of GEACs kept track of support to girls, and PEPFAR-assisted schools kept detailed records on OVC, generally both on OVC receiving support from the USAID activity and on other OVC deriving benefits from the school. The various grant programs effected significant positive changes in the school support systems, community participation and oversight. Current Capacities There is now much evidence of current data available to and used by woredas and schools, e.g. in team visits to schools and woredas, data was readily available, current and in detail, both in reporting documents and records maintained by the institution head and in posters and charts on the walls. Annual abstracts are prepared at the regional level and aggregated nationally. The national Annual Abstract for 2008 is available in hard copy; the abstract for 2009 is available on-line. There have been three National Learning Assessments (2000, 2004, and 2008) to date for grades 4 and 8. The assessments for 2000 and 2004 were conducted with substantial technical help and training from AED under BESO I and BESO II, with additional support through the Higher Education Relevance and Quality Agency (HERQA). . The Assessment for 2008 was carried out primarily by the General Education and Quality Assurance and Examinations Agency with support by USAID for Ethiopian project staff but with no expatriate technical staff. The next National Assessment, scheduled for 2011, will include Grades 10 and 12 and will be carried out entirely by MOE and Regional staff. This is a significant success for USAID support of capacity building, both in the technical quality of the National Learning Assessments and in the growth of Ethiopian capacities to continue such assessments. Further, the Assessments, beginning with the second Assessment in 2004 provided an objective basis for joint assessment and planning by the Ministry and funders, which then led to the GEQIP program now underway with pooled funds. Other partners, including USAID, have their own coordination mechanisms with the MOE. EMIS capacities have improved both in terms of the quality and level of detail of data and in terms of the analysis of trends and monitoring of progress (or lack of) toward the Education Sector Development Plans (currently ESDP 4 under review) objectives and criteria by the Planning and Budgeting Department of the MOE for budget forecasting and strategic planning. It is less clear how effectively such data are used at the regional levels, and the national MOE has only limited ability to effect strategic planning decisions at the regional level. Further, it is unclear to what extent the TTCs rely on such national data for planning of teacher training programs and capacities. In most cases they appear to make planning and budget decisions based on data obtained directly from the schools and budget guidance from the RSEBs. 38 Impact Evaluation of USAID/Ethiopia’s Education Activities The key level for quality oversight, supervision and support of the school clusters is the woreda level. The woredas appear at present to function mainly as data collectors and aggregators, with little analytic or strategic planning capacity. Two information systems appear to be improving and in use for management and planning purposes. The Personnel Management Information Systems (PMIS) is used at the Regional and woreda levels, and the TTCs (e.g., Gondar Teacher Training College) report that the Student Record Management System is now a valuable tool for internal management and monitoring of individual student status. Remaining needs for capacity building • Though initial training and capacity building has contributed to capacity development at each level (MOE, RSEBs, woredas, school administrators), there remains a need for continuous training and workshops. There is considerable turnover of key staff and need for training of new staff. There will be continuing refinement of the software packages and information collection systems, and there is need for workshops to facilitate capture of lessons and problems in the collection, analysis and use of data. Such workshops and training can help to facilitate transfer of such experience to other schools, woredas, RSEBs, TTCs and other institutional users. • In addition to more formal, structured and/or certificated training, more of the capacity building should be on a workshop basis, at the cluster centers, woredas and TTCs, both to encourage more peer-to-peer exchanges and networking and to reinforce the roles of the cluster centers, woreda and TTC linkage and resource centers as nodes for professional exchange and development. • Though use of data for management and supervision purposes appears to be improving, analytic capacities and processes appear inadequate for effective use of the detailed data at the woreda and school levels as well as at the TTCs. • Much of the needed analysis requires research capacities beyond the current or likely capacities of the management units at each of the institutional levels. For example, there is no mechanism for researching the relationships between the inputs and participation rates for the Annual Abstracts and the findings of the National Student Assessments. Similarly, there does not appear to be much qualitative research on the ways textbooks and other materials are used, the pedagogic and learning difficulties of such materials, or of how such feedback is used for iterative improvement of materials, teacher training and pedagogic support. • The planned Early Grade Reading Assessment appears likely to be a significant addition to such analytic and research tools. An issue to be addressed is where such assessment activity will reside within the Ministry and/or other institutions and how this will be used for curriculum planning, materials development, teacher training and other pedagogic improvements. • The ability of woredas, TTCs and to some extent RSEBs to make more effective use of data, and to facilitate collection and reporting, is limited by ICT capacities and little or no connectivity. The team is skeptical on the feasibility of establishing and sustaining adequate ICT (computers, printers, copiers) at all woredas, but recommends that the costs and feasibility of doing so be investigated It is not necessary, or particularly useful, for USAID to attempt to fill all such gaps directly. First consideration should be given to local resource mobilization, including contributions by communities and other funding partners, and to MOE, RSEB and woreda budget allocations for continued support, technical/logistic as well as budgetary. An appropriate role for USAID may be support for such capacity assessment, infrastructure planning and cost projections. • A research activity contributing to improved monitoring, assessment and analysis of access, persistence and completion patterns would be a large sample longitudinal study of sufficient size to facilitate tracking of students by gender, region/locality, SES, age and other factors, with some expected relationship to persistence and completion rates. It would be very beneficial if sampling were such as to incorporate meaningful samples of schools which have benefitted from the various Impact Evaluation of USAID/Ethiopia’s Education Activities 39 USAID-assisted activities. Data collection could start at grades 1, 5 and 8 (or 9) and be collected on an annual basis, resulting in a cohort analysis for each cycle within four years. Such a longitudinal study would require an initial cohort sample of 2,000 to 2,500 students plus annual surveys of status and annual cohort sampling of similar size to the initial sample. Over four years, and hopefully continuing, this would provide very important insight to the variables affecting student persistence and completion as well as the changes in such rates over time. 40 Impact Evaluation of USAID/Ethiopia’s Education Activities 6. Educational Management Capacity Building There have been two policy-level impacts that can be traced to USAID support, both embodied in GEQIP. Within this national program to improve system quality, the MOE has recognized and adopted the approaches initiated by BESO to improve planning, budgeting, resource allocation, and M&E to build capacity at national, regional, woreda and Kebele Education & Training Board (KETB) levels, including improved access to and availability of data. Also, GEQIP’s provision of school-level grants is an outgrowth of the School Incentive Awards (SIA) activities pioneered by USAID through its various projects to strengthen community support for the schools. 6.1 Administrative and Management Capacities – The Administration of Education Summary In 1994, when BESO 1 started, regional education bureaus and zonal and woreda offices had undefined roles, responsibilities and authorities for managing the educational institutions under their charge. Staffing positions did not articulate well with the new decentralized structures imposed by the central government. Some staff were redundant, others unprepared for their new functions. Administrative processes were inappropriate, out-dated or non-existent. Computer-based management information systems were a dream of the future. These problems were especially acute at the woreda level. At schools, principals and head teachers had not received any specialized management or leadership training; they were governing their institutions based on intuition and patterns of behavior modeled or imposed during the repressive Derg regime. Parents and local leaders were uninterested in and disconnected from the affairs of the schools in their communities. Compared to that period, today’s decentralized system is functioning at reasonable levels of competency. There are additional and continuing needs that require addressing and support, but basic processes are in place, supported by management information systems, well-defined administrative procedures and a clear set of responsibilities from MOE to regions, and on down to woredas and schools. Improvements to the administration and management of the educational system have been assisted by AED through its part of BESO 1 and BESO 2 plus EQUIP 2 and, currently, IQPEP. The strengthening of community support for the schools has been effected by World Learning, Save the Children, and Tigray Development Association through their respective parts of BESO 1 and BESO 2 plus CSPP and activities in support of OVC. In many cases, of course, the impacts of these respective projects have interacted and complemented one another. These efforts are paying off in more substantive interactions between school and woreda, brought about by better trained supervisors and head teachers; better prepared and more timely woreda and regional plans; and improved financial tracking and budgeting operations. Communities are also better able to engage in and implement school improvement projects, something that was unknown fifteen years ago. Current Capacities BESO-supported training has improved the management and administrative skills of thousands of education sector employees at RSEBs, WEOs and schools. Capacity-building in strategic planning, program management and financial and budgeting procedures has strengthened RSEBs and WEOs and is reported to have resulted in better plans, implementation and accountability systems at these institutions. Further, WEO supervisory functions have changed to become more supportive of student-centered innovations and encouraging of more effective teacher Impact Evaluation of USAID/Ethiopia’s Education Activities 41 practices. Development of and training on several types of MIS packages, particularly the PMIS, have resulted at regional level in better, more timely reporting and in some cases more informed decision￾making. To a lesser degree these gains are also evident at some WEOs. As discussed in other sections, monitoring and evaluation functions of the RSEBs and WEOs have been better defined and performed, while at the national level, the General Education and Quality Assurance and Examinations Agency, the institution responsible for the National Learning Assessments, has had its capacity strengthened with assistance from BESO 1 and 2 to the extent that the Third Assessment was carried out in 2008 independently of any other outside technical assistance (though with USAID financial assistance and the involvement of Ethiopian staff under USAID-supported projects), Very critically, the delivery of education at schools has benefitted from improved school leadership and supervision, realized through various BESO supported training efforts, along with the provision of school operational guidelines (e.g., modules in leadership, supervision, financial management, student monitoring, etc.). Remaining needs for capacity building • Woreda capacity building is key and needs continuing support. Staff turnover is a serious problem facing the WEOs and RSEBs, but there are early signs that some workforce stability is returning. In the meantime, continued training of staff is needed and methods developed to facilitate skill transfer to newly appointed employees. In particular, school principals need a well-defined process for handing responsibilities over to their replacements. • Awareness is growing of the information that is useful at woreda and school levels, but more capacity is needed on how to use MIS for management and decision-making purposes, not just reporting. More attention is particularly needed to make more staff competent in the use and potential of the PMIS system. • Management systems at woreda and regional offices could be further improved with better local area networks and more reliable connectivity infrastructure. 6.2. Community Participation and School Governance Historically, especially in rural schools, it was very unlikely that a parent or community member would visit the school except when there was a problem with their children’s behavior and in Ethiopia, as in a great many developed and developing nations, members of the public – lay people – were discouraged from involvement (or “interference”) with the schools, which were in any event under tight controls from central government. Since the beginning of USAID-funded projects, parents and members of the community not only “drop in” for a visit to the school, but also exercise bona fide ownership of the school. PTAs have learned to exercise control over the use of assets and to play a bona fide advisory role to school administrators and teachers. Consequently, PTAs and other community stakeholders in over 6,900 communities are now working in concert to improve the quality of their schools’ physical and socio-cultural environment, to provide oversight to the school, and to mobilize the community in support of various school-based initiatives. These stakeholders now also have abilities to plan, implement and account for development activities, not just for self-directed projects that address activities traditionally considered “education” (e.g., classrooms, school supplies, tutoring) but also for activities in other sectors, e.g., health, water and sanitation, that have direct impacts on the ability of children to benefit from schooling. This significantly enhanced community involvement in school management and oversight has been made possible by the training given to PTAs, GEACs and KETBs primarily through USAID projects (CSAP, 42 Impact Evaluation of USAID/Ethiopia’s Education Activities CGPP/SCOPE, CSPP, PC3, CASCAID, SCOPSO) that combine community-oriented technical assistance provided primarily by experienced professionals who have local roots and School Incentive Awards. 6.2.1 PTAs and Kebele Education & Training Boards Before the USAID-funded projects, PTAs and KETBs8 were only symbolic; now they are fully functioning owners of most schools. They now have the ability to develop and implement activities to improve the school’s general environment for learning and to make it significantly more possible for children at particular risk, e.g., girls, OVC, highly impoverished, to benefit from education. MOE policies have determined that all schools should have PTAs and should be trained in planning, finance, and related topics. USAID took on the task of training PTAs to identify and prioritize school-related problems that can be addressed within their likely resources, develop plans to address the problems, manage finances, write proposals to obtain their own funding, and manage implementation of the plans. As a result, PTAs have shown the community how it can help itself and community members can see how their own efforts and initiative have directly made positive contributions to the quality of education that their children receive. In a number of instances, this newfound capacity to plan and implement projects and activities has gone well beyond the school to include, for example, community-oriented public works projects. The strategies of mobilization are uniquely Ethiopian. PTA members identify local opinion leaders to take the lead in mobilizing the community. With these well-respected people guiding the process, their interest inspires others to participate in school support and so the community assumes ownership of the school. These same lessons in community mobilization have now been adopted in many secondary schools. Mobilization strategies have also been utilized in the development of ABECs and the recruitment of students. 6.2.2 School Incentive Awards (SIAs) Apart from its major contributions to the quality of teaching and learning, the greatest impact produced by USAID has been found in the mobilization of communities through the granting of School Incentive Awards (SIA). SIAs have resulted in significantly greater feelings of school ownership and community commitment to the education of their children. Communities have been capacitated to plan, execute, and evaluate the physical, instructional support, and sociocultural interventions made possible by the SIA. When the SIAs were no longer available through USAID funding (whether because the particular project had come to an end or because the community had already taken advantage of all the SIA for which it was eligible), many communities continued their commitment by raising funds, conducting other building projects, and, in general, maintaining their pattern of supporting the school. Of particular note, a number of communities have shared what they have learned as mentors to other schools, including high schools. 8 KETB – Kebele Education and Training Board, or equivalent of community school committee. Strictly speaking, KETBs and PTAs are separate bodies, although when a kebele has just one school, the two bodies typically have almost identical memberships. Impact Evaluation of USAID/Ethiopia’s Education Activities 43 Community mobilization around the receipt of SIAs by primary school PTAs has launched a sea change in the way schools do business. By USAID’s providing small amounts of money (typically a maximum of 12,000 birr, or about $1,400) to schools, the entire community was mobilized to support the school in the construction of new classrooms and - in some cases - entire schools, pedagogical centers, libraries and other structural changes. Those projects that sought to create partnerships between communities and schools established a trend that has long-term ripple effects throughout the country. With increased classrooms and schools, plus improvements such as the construction of latrines and water supplies, and the ability to provide educational and support materials as the result of the establishment of income-generating activities (e.g., gardens, shops, rental of water pumps), the projects have had a profound effect on the increase in enrollments and retention. Over the course of USAID support, from BESO I and II, to PC3, CASCAID and CSPP, the SIA grant has been the main catalyst for community mobilization on behalf of the school and vulnerable populations within the school (i.e., OVC in PC3, CASCAID and CSPP). Moving from initial large grants, which were too large for most communities to emulate on their own, in BESO 1 to smaller grants which are much more realistic for even poor communities to use as models today, USAID has illustrated how relatively small amounts of funding can typically leverage at least a ten-fold return in interest, concern, and contributions. Many principals and PTAs indicated that the project has given them a “spoonful” of resources, but they themselves have provided a “shovelful” in return. Now, instead of just providing access to schools, the activities of the PTAs and the communities have provided access to quality education. The bar has been set higher than previously in terms of what the community expects the school to provide and how schools and communities can work together to achieve school-related goals. 6.2.3 School Partnerships As a result of USAID projects, many partnerships have been established between the health and education sectors, schools and communities, schools and the justice system, and a range of community linkages in support of OVC. By creating strategic partnerships with various government units, OVC are both more protected and more provisioned with the goods and services they need to remain in school. The health/ education partnership, in particular, has made possible weekly classes in health, sanitation, and nutrition. 6.2.4 Current Capacities PTAs have the capacity to be more self-sustaining in raising funds and making decisions affecting the quality of the school. They have gained the trust of the community by creating plans, having the community approve and financially support them, and by reporting back to them on how funds were expended. It is this transparency in financial reporting that has played a vital role in the success of USAID-facilitated activities to promote community mobilization for education. PTAs regularly invite groups of community members to come to the school and inspect the improvements that have been made. Communities have been “permanently” mobilized to support different groups of children in need and to provide the school what it needs to create an enabling environment for quality learning. Community members who are linked to Diaspora Ethiopians have enlisted their support and that of school alumni to contribute to education in their home areas. Students play a role in community mobilization by communicating school-based needs identified by PTAs and community members to their parents, relatives and friends. This is particularly the case for girls who are representatives on their school’s GEAC. Consequently, students have increased in their 44 Impact Evaluation of USAID/Ethiopia’s Education Activities awareness of the needs of the school and work as a school body to contribute to the alleviation of these problems. This awareness and participation will have long-term ripple effects as the children grow into adulthood. Woredas have created a linkage between education and health through the establishment of local education/health networks that meet regularly to address the health needs of children in school. The systems of partnerships emanating with the school and moving upward in the structure, e.g., health, have created significant benefit to the children. Project such as CSPP, which seeks to improve health, nutrition, and sanitation standards through the creation of partnerships between schools and other government agencies, should be continued and serve as a model for other multi-faceted endeavors that improve the learning environment and learning for children. 6.2.4 Needs for Further Capacity-Building The USAID-launched SIA has been adapted and adopted by the MOE as the School Grants Program. While the current grants are largely being used to meet shortfalls in budgetary allocations for school needs, as assessed by the PTA/school committees. Future grants at additional schools will require similar guidelines, monitoring, and capacity on the part of the PTA and school administrators to plan and implement each grant. Given the success USAID has had in building the capacity of PTAs and their related committees in planning, implementing, monitoring and reporting on the use of the grants, USAID should assist the MOE by sharing the lessons learned and providing administrative training in support to successfully implement this grant program. Community mobilization has been very successful in many communities and is beginning to show results in emerging regions. However, greater attention needs to be focused on addressing the needs of these regions and on how communities can be mobilized to support primary schools. Continued awareness raising of PTAs and their affiliated committees about different challenges students face is necessary to help communities to better understand the needs of different children, e.g., students who have physical, learning, and/or developmental disabilities, children who go to school hungry, victims of abuse. Community awareness on the needs of the most vulnerable must be introduced to overcome cultural stigma and to define courses of action to support these children. Further strengthening of partnerships between schools and the legal system, including the police, is necessary to prevent any wrongdoing against children and to protect them. Protection and prevention must be extended to include child endangerment, child sexual exploitation, cultural practices harmful to children (e.g., female genital mutilation, early marriage, abduction, and the like), and labor practices that prevent children from going to school. There are examples of successful partnerships with health services at a number of schools and this could be expanded to address the many needs of the "whole child" at the schools This same model of alliance development could create further opportunities for children. For instance, an alliance with sub-sectors of economic growth could create learning opportunities for 1) environmental science and eco-friendly income generating activities, 2) skill building in improved animal care and cultivation for children and parents, 3) life skills training in money management and income generation, etc. To enhance this process, a “Making Cents” or “Junior Achievement” type program could be developed that would arm young people with skills to innovate and practice different types of income generating activities (both of these approaches are being used in several developing countries, especially in rural areas where income generation has been limited to cropping or animal products). USAID should investigate the streams of Impact Evaluation of USAID/Ethiopia’s Education Activities 45 funding available for Economic Growth activities at schools and at ABECs so as to lay the groundwork for the development of further cross-SO strategic alliances. The relevance of the curriculum for children’s lives as well as the increasing needs for children to be income generators (at the expense of being in school) calls for an innovative approach to meeting these needs. The approach might be used in ABECs or formal schools. While we advocated above for consolidation of innovations, this approach has already been successfully piloted in government schools in Alaba and bears further investigation for future USAID programming. Community awareness on adult illiteracy could also be addressed through community mobilization strategies. While the school is the “logical” place for functional literacy courses to be taught, a more appropriate approach might be the mobilization of other community institutions (such as churches, businesses, marketplace management committees, etc.) to address the literacy needs of adults. USAID might play a role in such non-formal education endeavors by providing capacity building to the NFE department in establishing guidelines in the development of functional literacy materials. USAID might also work with community organizations to mobilize them to take on the role of providing literacy training. 46 Impact Evaluation of USAID/Ethiopia’s Education Activities 7. Health Impacts Affecting Education Results Summary USAID’s involvement with the impact of health on education began with the BESO Community-School Activities Program – because thirst interferes with a child’s ability to learn, a number of SNNPR schools used their School Incentive Awards to bring drinking water to the school. Additionally, from the beginning of BESO, schools have created a more favorable environment for girls by building separate latrines for girls and boys, building modesty rooms for girls who have their menses, and providing sanitary inputs for cleanliness. With BESO 2, USAID began addressing a broader range of health-related issues that have a direct impact on children’s learning. The SIAs provided under BESO 2 CGPP and SCOPE enabled many more communities, in more regions, to provide their children with drinking water, and the Complementary Drought Assistance activity added to the three CGPP cooperative agreements (World Learning, SAVE/US, and TDA) provided food, water, and other material support that helped to keep children (and their family members) in drought-affected villages in school and learning. Through the Kokeb Kebele Initiative (SNNPR) /Model Kebele Initiative (Amhara), which World Learning began in 2005 in tandem with John Snow’s Essential Services for Health in Ethiopia (ESHE) project, USAID explored the synergies of closely linking the delivery of education and the delivery of health services in rural communities. This CGPP activity, which was an adaptation of USAID/Madagascar’s Champion Community project, established close active linkages between staff of community health posts and students and staff of schools, the drilling of wells in a number of communities sited where they would be reasonably convenient for the schools, and the implementation of WASH hygiene activities at participating schools. These activities have been expanded and explicitly incorporated into the ongoing CSPP. Through local school/health care extension partnerships and School Health Committees, children received immunizations against certain preventable diseases, and learned how to prevent others by participating in weekly health information classes making use of 16 modules developed by the CSPP project. Inter-sectoral linkages such as these create synergies whose impacts go beyond those specified in reporting documents. In addition to the health needs common to all children, OVC have particular health concerns. Since 2004, USAID has been working to improve the ability of OVC to benefit from education and to thrive, beginning with PC3, continuing with CASCAID (begun in 2006) and now with the ongoing SCOPSO project. Typically using the school as a child-friendly community focal point for helping the child at risk, these projects have been expanding the use of modalities developed under BESO to mobilize community members to provide “whole needs” support for OVC (e.g., learning, psychosocial support, health, nutrition, safety). With the increased care and attention provided to OVC, many perform very well academically, pass school leaving exams, and are successful in secondary school. In interviews with two principals, we learned that PTAs and communities will not allow one child to remain at home; they will find a way to support the OVC so they can continue in school. From accounts of school leaders, PTAs, and other community members, this concern and action on behalf of OVC and children in need will continue after the project interventions have ended. Several long-term income generating activities have been established to ensure that the flow of funds continues to support OVC and that the project idea is sustained although SIAs and other support will end. Impact Evaluation of USAID/Ethiopia’s Education Activities 47 Current Capacities As a consequence of PEPFAR and other USAID-funded projects, and in support of addressing the needs of the “whole” child, schools are better able to address the WASH (water, sanitation, and health) needs of students. Children understand the relationship between maintaining good health and personal/environmental sanitation and act accordingly. On a broader institutional level, networking between health and education at the kebele and woreda levels helps ensure that the health care needs of children are addressed in schools. Projects have produced many different types of learning materials and manuals which enable various stakeholders to address the needs of OVC, which also contribute to helping community members better address the needs of all children. Indeed, education and the benefits that going to school can bring a child at particular risk opens the “Standards of Services for OVC in Ethiopia,” which was the original template for OVC quality assurance for other nations. Students are not passive observers of all of these changes. They participate in maintaining the cleanliness of the school environment, share the lessons they learned in health classes with their families, and actively apply what they have learned. Need for Further Capacity-Building Difficulties in implementing the WASH approach can be found in the insufficient budget that was provided to bring water to all the target schools. In explaining the project to the community, expectations were raised among school and community members that cannot be met with the current budget. It is anticipated that this will be rectified after the upcoming CSPP mid-term evaluation. Restoration of children’s health by providing them with medicines to address parasites and micronutrient deficiencies is a key element of the health program. However, not all targeted schools have participated in this. Although programmatically each project targeting health is considering the “whole” child, synergies between and among projects have not been achieved, nor has full integration at the SO level within the mission occurred owing to different reporting practices required for PEPFAR and mainline USAID funding. More attention needs to be paid to creating these synergies implemented through cooperative agreements (for flexibility) if other sector partnerships will be created in the future. Local CBOs need further strengthening and training on planning, management and community mobilization so that local institutions, in addition to PTAs and their affiliate committees, can continue to address the needs of OVCs and other children in need. 48 Impact Evaluation of USAID/Ethiopia’s Education Activities 8. Summaries and Recommendations We begin this section with a summary of our findings and observations and discussion of constraints to deeper, broader or more sustainable impacts. We then provide our projections of future circumstances and of possible USAID responses. We conclude this section with a statement of our key recommendations as to programmatic and implementation approaches for USAID to consider. N.B. In all of the findings, observations and recommendations it is important to keep in mind that most of the ongoing education improvement will be carried out by Ethiopians with Ethiopian resources operating through the relevant MOE and RSEB systems. In the past 15 years, USAID has been supporting primary education in Ethiopia in defined areas of interventions. It implemented few activities in less than 20% of the Ethiopian primary school. The number of USAID supported ABECs was also about 16% of the total number ABECs in Ethiopia. In other words, it is expected that the Ethiopian government will continue to be the main driver of further education expansion and investment and donors such as USAID will be most useful in capacity building, technical assistance , training and other support roles. 8.1 Summary of Findings and Observations 8.1.1 Individuals • School principals need more support as instructional leaders; they play a key role in engaging the staff and the community in analyzing and acting upon problems within schools. • Some teachers have adopted and put into practice what they have learned about student-centered, active learning, continuous assessment, group learning, and managing large classrooms. Teachers/school staff are at least using the vocabulary of the new pedagogy. This was observed directly in team visits to a limited number of schools, including schools assisted by USAID projects, but there still are many schools with traditional teacher-centered, text-dependent and authoritarian practices, reflecting low motivation, inadequate training and pedagogic support, large classes, inadequate school management and supervision among other factors. – Over 50,000 teachers have personally benefited from the updating, in-service training programs offered by TTCs through cluster centers and linkage schools, and over 42,000 teachers have been upgrading their diplomas to degrees in activities assisted by USAID. – The total number of teacher that were trained during BESO II and EQUIP II were over 106,000. • As discussed below, high turnover of teachers and administrators, which can take place at any time of the year, continues to cause disruption to education at the schools affected, specifically including disruption to USAID-assisted activities, which are particularly susceptible because they are not national or even regional in scope. Institutional memory is lost, there is little if any planning for transition, and not infrequently activity materials themselves go missing. This is separate, of course, from disruption to the lives of the individual educators involved. 8.1.2 Institutions • Woreda capacities are key to the effective functioning of a decentralized education system. USAID-supported training and technical assistance for woreda heads and other staff have helped improve management systems, planning, IT capacities and computer skills, EMIS data collection and data management and support for school clusters. The woredas appear now to have good Impact Evaluation of USAID/Ethiopia’s Education Activities 49 financial management practices and all grant funds appear carefully managed with full accountability. However, the woredas need continuing support to ensure that all new staff receive training in planning, management, EMIS, and supervision. • The impact of school grant programs (and related training on assessment, mobilization and management practices) on community engagement and support has been significant. Specifically, PTAs/KETBs have been strengthened, community involvement has increased, and school￾community partnerships are active. School teams and PTAs are becoming adept at assessing school needs, and mobilizing local resources to address the needs. There appears to have been a significant shift from seeing the schools as belonging to government to seeing them as belonging to the community. School and community committees provide a variety of support services from basic food, clothing and other needs to mentoring support in school for highly vulnerable children. Despite these advances, communities and PTAs still need assistance with assessment, planning and related topics to support the work of the school. • The various incentive awards provided to school communities have been used well and have leveraged much more from the community than what they were given. – Grant awards have been used for the purchase of classroom furniture, construction of separate latrines for girls, improved classrooms, fences, pedagogic centers for teachers, libraries, provision of potable water, and income-generating activities to provide amenities and needed support supplies. – Earlier community grants program worked well in creating capacity, community participation and ownership – later grants program more a menu of inputs. • Though USAID may wish to consider similar awards for schools in additional areas as catalysts for community engagement and support, given GEQIP there does not appear to be a need for continued grant support of inputs for the schools previously or currently supported by USAID programs, where the precedents have been set and the activities will continue with community mobilization of resources. The cluster school model has merit for in‐service teacher training and support, but questions remain about cluster‐satellite school interaction, use of resource centers, provision of supplementary learning materials, and support by woreda supervisors. • The quality and utilization of Cluster Resource Centers varies, from mainly storage and distribution centers, often locked, to well-equipped centers with a variety of resources and meeting/work space used regularly by teachers from cluster and satellite schools. Cluster Resource Centers appear key to strategies of pedagogic improvement and support for teachers. However, any further support for such centers should be for much more active and interactive models with a greater variety of resource materials. CRC should be designed to facilitate active involvement of teachers and school teams in assessing their own support needs. – In the schools visited, there was little evidence of manipulables or other materials to support teaching and learning in the classroom. Most materials were in the form of wall charts and other printed materials. – In many instances, innovations are being spread by active mentoring by members of PTAs/KETBs of schools receiving USAID assistance to their colleagues in neighboring schools. In at least some instances, primary schools that have received assistance through SIAs have been getting neighboring schools to band together to push for the establishment of a secondary school to serve the broader community. Such mentoring and exchange activities might be facilitated by better documentation of key innovations, e.g. video documentation of GEACs and their roles in communities. 50 Impact Evaluation of USAID/Ethiopia’s Education Activities • Computers were provided to many woredas and TTCs. However, concerns exist about appropriate use, Internet connectivity, development of LANs, intermittent electricity, and recurrent costs. To date, such connectivity and computer capacities are used mainly for management (including data management) and planning purposes. They should also be used significantly more for pedagogic support, distance training and access to teaching resources. • GEACs and Girls Clubs have made a substantial impact on the enrollment and persistence of girls. Where strong leadership is lacking in the PTAs and by principals (often due to turnover), GEACs are likely to atrophy. On the other hand, with strong leadership many of the GEACS have expanded their mission to include mobilization and support systems for both boys and girls. • High turnover rates and turmoil in the personnel system have caused the following problems: loss of institutional memory, loss of trained staff, loss of training materials, and the existence of unfilled positions, all leading to inefficient administration of the system from the regional level down to the school. • there are early signs that the high turnover rates are decreasing. • In any case, other than through dialogue with the MOE, there is no obvious option for slowing the rate of turnover other than to assume that training will continue to be needed and perhaps to encourage the practice of “handover” notes capturing lessons learned and plans underway as part of the transition for new appointees. • To some extent, innovations introduced in one school are being spread informally to other schools through transfer of trained teachers and school principals. 8.1.3 Policies/systems • USAID impact have helped Ethiopia accomplish revolutionary changes in education capacities– not so much in the sense of being radical in pedagogy or in ideas – but in the sense of helping Ethiopia develop capacities to plan, manage and otherwise accomplish unprecedented expansion over the past 10-15 years along with improving equity and strategies for improving instructional quality and outcomes. Through its community-strengthening activities, USAID’s impact has also been revolutionary in changing the mindsets of parents and other community members as to their relationships to the schools their children attend and in creating a capacity for positive change and raising the bar for attitudes and expectations of potentials from the community on up. An important lesson from the USAID education program experience in Ethiopia is that the impacts have been as much, if not more, through capacity building and institutional systems strengthening as through direct inputs to schools. Particularly significant improvements include: – Parent involvement, local ownership, ongoing local support systems for schools. – Strengthened capacities for planning, management and assessment/monitoring at all levels from MOE to RSEBs and woredas to schools and school clusters – Active learning, continuous assessment, management of large classrooms etc – Strategies supporting gender equity and full participation, including GEACs and gender￾disaggregated monitoring of education variables at all levels. – Support for the ABECs, leading to national policy support for ABECs, selected TTCs, selected woredas – ripple effect on other TTCs, schools, woredas • A national policy on the establishment of GEACs was promulgated on the basis of their role in reducing dropouts and improving the quality of life for girls. USAID can take substantial credit for the development and institutionalization of the GEACs, which have had a substantial impact on the enrollment and persistence of girls in primary schools and are now morphing into support systems for all children. • The success of the provision of incentive grants has influenced the development of the school block grants which are a part of the GEQIP and School Improvement Programs. Impact Evaluation of USAID/Ethiopia’s Education Activities 51 • USAID support for the expansion and improvement of the ABECs has contributed to the current national policy supporting ABECs in all regions. They began prior to USAID support, but USAID support has had a substantial impact on refinement of the alternative (accelerated learning) approach, the training of facilitators and the policy affirming the ABECs as a strategic part of the national education sector development plans. There does not appear to be a priority need for further USAID support of ABEC expansion, except possibly for limited and targeted experimentation with improved delivery systems in the pastoralist areas. The current ABEC model appears to be functioning reasonably well in providing an alternative first cycle primary education, but the other possibilities for addressing second chance education needs of older youth and a wider range of adult education needs will require a somewhat different model and increased support. An important continuing need is for performance assessment of learning outcomes and equivalencies and for transfer of lessons learned on effective pedagogic innovations. 8.1.4 Coordination/modalities • Program planning and development appears to be done very collaboratively and jointly, with the head of the MOE planning department now chairing the USAID Technical Working Group. • However, coordination and communication during implementation has some problems, with the project teams having a higher profile than the respective MOE units and with initiatives frequently seen as “the AED project” or the “World Learning” program. • Further, the sustainability of some subproject activities is in question, as they are more activities of the USAID-supported projects than program initiatives by the relevant Ministry and regional education units. For example, under BESO, EQUIP and now IQPEP, AED has initiated and implemented efforts for computerized resource labs, computerization of library holdings, linkage schools and a variety of training and support programs which are not yet integrated and institutionalized at the MOE, regional or woreda levels. • Momentum is lost when there are extended gaps between one project and its successor. • USAID partners don’t seem fully aware of what each other is doing. More opportunities for inter￾project workshopping and exchange of experience, including with MOE and RSEB counterparts, beyond what takes place at the Technical Working Group meetings, appears needed. • Trust and collegial working relationships between USAID and the MOE appear good. In interviews at the MOE and in the regions, the team had strongly positive impressions of the level of professional exchange and detailed knowledge of USAID-assisted programs and generally positive opinion of RSEB and others is that USAID is working within Ethiopian framework and upholding its commitments/promises. The problems most frequently mentioned are: – Historically, there has not been adequate information sharing as to proposed USAID-sponsored activities, especially when specific communities had been identified. – USAID policy not to participate in pooled funds arrangements, though generally understood, entails additional coordination of joint program planning and implementation. – Project teams operate more as extensions of the MOE and RSEB units than as integral parts of the units, with several key people commenting that they didn’t know in sufficient detail what exactly the project staff are doing and don’t have much if any supervisory relationship over their activity. • The PACT/TEACH activities in remote pastoralist areas have many local NGOs as partners implementing parts of the activity. TEACH works within the strategy and priorities of the MOE; and appears to be managed effectively but does not yet have a well-coordinated program able to monitor results and share experience and practices across its many partners. 52 Impact Evaluation of USAID/Ethiopia’s Education Activities 8.2 Constraints to Deeper, Broader or More Sustainable Impacts • The fit of AED in the Ministry of Education is not optimal for the integration of IQPEP activities into MOE priorities and plans. Similarly at the regions, the roles of the AED regional coordinators could be expanded to provide more technical input into the challenges faced by regions, woredas and TTCs. However, this should not become a parallel system to that of the RSEBs, WEOs and TTCs. One option is to place more key technical staff directly in the ministry or RSEBs and TTCs, rather than as project staff working in parallel to these units. There may be policy reasons for Ministry or regional limitation on such assignments and USAID would still need a management mechanism for planning and support of such personnel, but movement toward more integrated support and assistance appears needed, given the maturing capacities at senior levels and the desire of Ethiopian officials to integrate activities within their planning and implementation systems. • The problem of staff turnover at all levels of the system is causing inefficiencies in the capacity building efforts of the various USAID partners. – USAID/Ethiopia’s program has multiple implementing partners, local NGO subcontractors and subgrantees as well as a significant number of additional program and project activities under AEI, PEPFAR and other mechanisms. The program appears fragmented and difficult to manage and coordinate. However, there actually are somewhat fewer implementing partners than existed in earlier years under BESO 1 and BESO 2. What appears to have changed is more the specificity of the contracts/grants implementation objectives and performance management plans. – More consideration is needed of ways to build in design reviews and adjustments as necessary of performance plans from the beginning of implementation. Given the likely changes in assessed needs and the pace of institutional capacity building in general plus possible external changes in the overall environment (e.g., political changes, activities by other donors), it should be assumed that corrections and adjustments will be needed as the situation evolves over any 3-5 year period and an explicit schedule for “course correction,” which could include inter alia revision of indicators, by the end of, e.g., the second year of implementation should be built into project design. – This is particularly needed for USAID strategies focused on long-term capacity building and institutional strengthening rather than shorter-term strategies or tactics that provide direct inputs such as texts or other materials. • The practice of deliberately avoiding multiple external assistance interventions in a single location prevents synergies from being realized. The MOE and RSEB prefer to spread assistance to multiple communities, which limits the opportunity for overlap, so this constraint is unlikely to change. A partial solution is to focus USAID efforts on capacity building at the RSEBs, TTCs and woredas to develop and implement support activities rather than targeting USAID activities on selected schools. This could include any further support for school grants as well as for training and the provision of other inputs. • Data, analysis, information, and lessons learned are not being optimally used by education planners and decision-makers at the regional and woreda levels, although they appear to be used effectively at the national level. In some cases the data sets seem too voluminous and detailed for any anticipated use at woreda levels. An option is to explore ways to generate more disaggregated data sets at local levels, available in real time (i.e., current planning year rather than only for one or two years previous). Even though less refined or useful for comparisons across regions, such snapshots and school/cluster data can be very useful for woreda and regional planning purposes and possibly for comparative assessments among schools and school clusters. Impact Evaluation of USAID/Ethiopia’s Education Activities 53 8.3 Scenarios for the Future Over the next five to ten years (2015 and beyond), several trends are likely, each with implications for USAID strategies and program priorities. • English language competence will be a need for teachers at all levels. The priority focus for USAID should be improving the standards of English in the TTCs – Language competence of the teachers (and lecturers) – Pedagogic competence in the teaching of English language – Competence in teaching subject matter in English • There will be increasing demand for access at the secondary level and problems maintaining quality as the numbers at secondary level increase. – The options for USAID assistance will be essentially the same as for primary education over the past decade, namely assistance with planning and management, monitoring and assessment, in￾service training and pedagogic support, and possibly some involvement with curriculum and materials development. – Pre-service training for secondary teachers will be a challenge, as this will require working with/through the universities and would require a more substantial engagement with university planning, restructuring and policy than USAID may wish to undertake. . – No substantial USAID support for quantitative expansion of secondary capacities is recommended; – However, USAID may wish to focus on the specific problems of rural primary completers, particularly girls, in accessing lower secondary. There is a need for “safe schools” approaches that incorporate modalities such as GEACs among others, and there is a need for sustained experimentation to develop smaller-scale lower secondary models appropriate for smaller communities. – Such schools may require some integration of content, cross-training of teachers, increased use of media to augment teacher content knowledge and probably alternative approaches to school management (e.g. a principal teacher or head teacher rather than a non-teaching principal/administrator). • Along with increased demand for secondary and tertiary education, there will be increasing need for attention to workforce development. – The assessment team did not have opportunity or time to assess the current capacities for technical training and makes no recommendation for USAID involvement with the existing technical training Institutes and centers. This would require a separate, focused assessment. – A possible area for USAID initiative would be to help Ethiopia explore variants of the U.S. community college systems, particularly the mechanisms for linking the community colleges to employers through jointly developed training programs, advisory bodies, internships/practicums, incubators and other outreach, extension and support systems for employers and entrepreneurs. The community college model(s) in general and the linkage systems in particular are areas of U.S. comparative advantage. • There is likely to be at least one major curriculum review and revision exercise within five years, with consequent need for textbook revision and alignment with teacher training and support systems. USAID will have opportunities to assist through capacity building and training and possibly through additional TA with curriculum improvements for English, mathematics and science. As noted above, there will be greater need to address workforce development issues. • The supply of diploma-level teachers will be increasingly adequate in number if not in quality of training and motivation. – There will be continuing need for in-service training and pedagogic support. 54 Impact Evaluation of USAID/Ethiopia’s Education Activities – There also is likely to be policy consideration of raising the qualifications for primary teachers, particularly Cycle 2 subject teachers, to degree level. If/when this policy decision is taken, there will be a need to add degree programs (both B.A./B.Ed. and possibly M.A./M.Ed.) to at least some of the TTCs, with consequent need for restructuring, quality assurance and degree equivalency. – The United States has many best practices models and USAID, possibly in partnership with the U.S. Department of Education, could play an influential role working with and through U.S. accrediting bodies such as NCATE (formerly known as the National Council for Accreditation of Teacher Education) and other specialized accrediting bodies for teacher colleges on quality assurance systems, standards development and accreditation processes. • The electricity grid and reliability can be expected to improve, along with cell phone and other media density and improved connectivity. It will become increasingly realistic and cost-effective for all TTCs, at least some of the woredas and cluster resource centers, and an increasing number of schools (both at primary and secondary levels) to have internet access and the capability to make effective use of media. – USAID may not have the resources and/or may not prioritize a role assisting with communication and ICT infrastructure per se, but USAID could play a very influential role in supporting efforts to make more effective use of these capacities, particularly for pedagogic support, for teacher training and for dissemination/transfer of experience and e-training. – A specific possibility would be development of a robust interactive portal serving the TTCs and resource centers on-line with links to the respective curriculum and materials development and teacher training units of the MOE and RSEBs. • The primary education system is likely to approach full enrollment capacity within the next five years, but there will remain significant numbers of children who cannot access the schools or cannot participate fully. In addition to the pastoralists, children in small or remote communities without schools, OVCs and other poor and vulnerable children, there will continue to be significant numbers of children with limited mobility, vision and hearing limitations and other special needs which limit their access and participation. USAID can best assist in addressing these needs through: – Training for specialized teachers – Assessment – both screening and individual assessment at schools and quantification of the incidence of physical learning limitations – Exploration of feasible ways of mitigating such physical limitations – transport, self-study materials, assistive technologies and media and other means. – Collaboration with health services on health screening and on de-worming and other health problems. • There is likely to be increasing interest and demand from parents for more early childhood and pre￾school support, with increasing support of such programs by the private schools and private sector generally. The entry age for primary school is age 7, which is later than in most countries. Thus, there are two types of opportunity and challenge. – For the early childhood programs (ages 3-4 or 3-5), it may be best for USAID to leave this largely to the private sector and NGOs (local and international) with perhaps some small grant support and support for training programs. – For pre-school or kindergarten programs for ages 5-6 or just age 6, these are likely to be added to an increasing number of primary schools. They can be cost-effective both in terms of readiness for school and early development of skills/behaviors and in terms of normalizing the age cohort so that problems of over-age students are reduced. – For both forms of early childhood education, much can be done relatively cost-effectively (compared with school-based approaches) through community-based and home-based programs. Training and support of facilitators, parents and other care givers can be combined Impact Evaluation of USAID/Ethiopia’s Education Activities 55 with other programs such as adult and community functional education, health/nutrition programs, leadership development and community organization development. • The need for adult literacy and other adult education will continue to be large and the relative disadvantage of those who have not completed at least primary school will increase. – There is no obvious breakthrough technology or delivery system to reach these populations effectively and cost-effectively. – USAID is advised not to support large campaign-style approach and to favor instead a variety of smaller-scale initiatives supported by local NGOs and tailored to specific adult and community education and economic needs. – Two options for larger scale USAID-supported initiatives are: • increased use of mass media such as instructional modules on CDs, MP3 mode and possibly increased use of radio and television distribution systems, and: • some form of “second chance” education support for those who drop out or for various reasons (marriage, work, health) and are not able to complete either primary or secondary education. In theory, adult literacy is an area where KETB can play an important role, and the increased availability of electricity at schools, including in many rural areas, will increase the possibility of using schools as adult learning centers. – The team was skeptical about the feasibility of scaling up the WORTH activities as a way of achieving larger scale literacy impacts, but does agree that these activities appear to be useful in supporting women in self-help and microenterprise/savings activities. • The ABECs are likely to have expanded and become more fully integrated with the school clusters and support systems from the woredas. – Further USAID support for quantitative expansion or promotion of the ABEC model probably will not be needed. – Continued support for performance assessment, feedback to the training of facilitators and the ongoing development of materials and pedagogic support will be useful and appropriate for USAID. – As the ABECs become more normalized as ongoing parts of the primary school system, the status of the facilitators may need to change and their training and support may need to be similar to the diploma-level teachers for the regular schools, i.e. training for the ABEC teachers likely will become a specialty at the diploma level, rather than a separate training program with lower status or equivalency. As this begins to become policy, USAID should consider integrating such specialist diploma training with any other support it provides to TTCs and to the improvement of primary teacher training generally. This should not be limited to any one TTC such as the Center of Excellence for pastoralists at Jigjiga. – As the number of ABECs increases and as experience grows in the different regions, it should be possible to identify differences in the performance of ABECs and to identify differences in practice or materials or other pedagogic factors accounting for the differences. USAID could play a useful role in helping the MOE identify such differences and to develop strategies for documenting and disseminating such practices for possible application and adaptation elsewhere in Ethiopia. In fact, given that the ABECs are somewhat unique, it would be useful for some of this documentation to be available for possible adaptation in other countries. • Finally, as all of the above trends play out, supervisors, planners and managers at all levels will have increasing responsibilities for a more complex set of education tasks. Though further training and technical support may be needed and welcomed at other levels, the key link in the “value chain” connecting national and regional policies and program funding to the schools and classrooms will be the woredas. – Ongoing, continuous training through workshops and other means for woreda leaders, supervisors and other key woreda staff should continue. 56 Impact Evaluation of USAID/Ethiopia’s Education Activities – Rather than continued USAID project support for such training and support activities, USAID might (in collaboration with the MOE, RSEBs and in coordination with other funders and partners) develop a strategy for support of institutional capacities for such training in each region. Some form of Institute of Education Administration might be appropriate. UNESCO’s Institute for Capacity Building in Africa may be able to help develop relevant models. 8.4 Key Recommendations Priority Programmatic Recommendations • The first priority is institutional capacity building at TTCs, both in terms of their capacities for pre￾service training and in terms of their capacities to provide pedagogic support to cluster schools, woredas, additional upgrading programs and in-service training for teachers in the schools, and other roles supporting research, innovation and iterative improvement of instructional capacities. • The second priority for institutional capacity building, closely linked to the above, is support for the woredas. Woredas need further strengthening in management, planning, EMIS, and research capacities. Cluster Resource Centers need to be managed better and supervisors must have their capacities built to provide mentoring and pedagogic support to schools and coordinate in-service training and workshops at the cluster level. • USAID will need to continue to respond at some level to the needs for improving education opportunities in pastoralist areas and for other hard-to-reach children as well as for OVCs and children with special needs. Approaches which link schools with other formal and informal community stakeholders, such as those used for OVC, can be refined and expanded to develop policies and strategies addressing the needs of other children who are hard to reach. USAID may want to continue grants programs for OVCs and other specific needs, but the team makes no recommendation as to the scale or targeting of such programs. – USAID should continue support for efforts to improve and strengthen support for the ABECs, but should not provide direct support for expansion, except possibly on a targeted basis in the pastoralist areas. Support might include experimentation with mobile libraries, with materials development and use of media, with improved training and support for facilitators and with monitoring and assessment of learning outcomes. Such activities should not be limited to ABECs in pastoralist areas but should include rural and isolated communities in all regions. – The adult literacy program provided in ABECs is still small scale and will require sustained research and experimentation with other approaches and assessment of results before larger scale initiatives can be justified. • The priority tasks for systems improvement are the improved alignment of textbook development and other materials with teacher training methodologies and with the realities of instruction and learning in typical schools. This does not require support for materials production and distribution of basic texts, but should include supplementary materials, teaching guides and support for feedback from the schools and exchange of experience among the TTCs, among regions and among the units responsible for curriculum development, materials development and teacher training. • The priority subject for continued pedagogic and material support should be English language. Additional priorities for support are early grade reading in all languages. Resources permitting, USAID can provide useful support for mathematics and science, both early concepts for cycle 1 and for subject teaching and learning in cycle 2, and possibly for lower secondary. • As a matter of significant priority, USAID should engage more extensively with the MOE as to the major and unpredictable turnover among teachers, principals, and administrators. USAID has made very substantial investments in working with educators and community stakeholders become more effective advocates and implementers of learning, and these educators and stakeholders have themselves often made substantial commitments in their time. Further, projects rarely have much Impact Evaluation of USAID/Ethiopia’s Education Activities 57 opportunity to train new personnel. When rapid turnover takes trained educators away from their work, the investment of the American taxpayer, and the commitment of the educators and community members, gets dissipated. Recommendations on Implementation Three recommendations on modalities of assistance are: • Focus more on assessment and documentation of existing innovations and improved practices as well as supporting for additional innovations, pilots or experimentation, especially on Adult literacy, ABECs, OVC, Early child education and Secondary education in relation to the overall improvement of quality education in Ethiopia. Consider supporting some of this new experimentation in very close collaboration with TTCs or other local institutions as part of an Ethiopian process of local problem-solving rather than as activities identified with external implementing project partners. • The early assistance programs BESO1 and BESO2 had substantially more flexibility and ability to respond to emerging needs than do current project modalities. Implementing strategies and procurement mechanisms should anticipate the need for periodic reviews and revisions of priorities. • The current portfolio has many good components but has so many different actors and accountability/management systems that it is confusing to the MOE, difficult to coordinate, and burdensome to USAID in terms of management. It is desirable to work toward having a smaller number of program umbrellas (not more than three) under which a variety of subcomponents can be managed and coordinated. 8.5 Summary Responses to SOW Questions Objective 1: Determine which changes in educational access, quality and equity have taken place in USAID-supported teacher training colleges, primary schools, and local, regional and national education offices. Determine if and how these changes have had an impact on the development of the education sector in Ethiopia 1.1. Is there evidence to show that improvements in the quality of teaching, girls’ equity, and the general learning￾teaching atmosphere have taken place in USAID-supported schools? Is so, has the knowledge transferred into the system? Project reports indicate that Pre- and In-service training on active learning, continuous assessment, managing large classrooms, etc., are all in evidence in the classrooms of USAID-assisted schools, but not to the extent desired nationwide. Girls’ equity has almost reached parity, and dropout rates for girls are now less than for boys. The system has not yet fully come to terms with all the USAID-introduced innovations and more assistance will be needed by the MOE to scale-up these activities nationally, especially in emerging Regions. 1.1. Is there evidence that quality and equity in education have been enhanced per the following indicators: • Active learning and continuous assessment in primary schools are used effectively. • The survival rate of children to grades 5 and 8 has increased in USAID assisted schools. • The learning achievement of children has improved in USAID-supported The most recent survey evidence is from the AEI Assessment in 2009. Active Learning – 100% of teachers observed (58) were using AL; Continuous Assessment – 88.7% of teachers observed (62) were using CA Average % of teachers using active learning methods in AED cluster primary schools during 2003/4- 2005/6 improved from 54.12 % to 70.26 % (see Appendix on Selected Statistics). The percentage of children in USAID-supporting schools surviving to grade five is six points higher than the national average, while the percentage surviving to 58 Impact Evaluation of USAID/Ethiopia’s Education Activities schools. • The efficiency of regional education bureaus and woreda education offices has improved in planning and managing primary education. • Parent-teacher associations (PTAs) in USAID supported schools are empowered and are effectively supporting and managing their schools. • Children and adults have received quality basic alternative education that could lay the foundation for their future education. grade eight is thirteen points higher (see appendix on GER and NER). It increased from 40.6% at grade 5 & 25.0 % at grade 8 in 2001/2 (base year) to 52.3 % and 36.3% in grade 5 and grade 8 respectively in 2004/5 (see Appendix on Selected Statistics). The dropout rate for children at CASCAID schools in Amhara is 0.08% while that of the region is 15.4%; for Oromia it is 0.10 for CASCAID schools and 18.8% for the region; in SNNPR it is 0.11% for CASCAID schools, and 20.1% for the region (CASCAID Final Technical Report and CSPP Baseline Study). Promotion rates for OVC in CASCAID schools ranges between 82% and 86%, while the overall regional rates range from 69% to 71%. RSEBs and WEOs use data for planning, projecting teacher needs to determine TTC intake numbers, track transfers. Mobility/turnover remains a problem with respect to planning and management capacity building... PTAs at USAID-supported schools are actively engaged in mobilizing the community in support of schools and in support of the girl child. Many have made commitments to support schools and children in need even without SIA awards. When taking the NLA, ABE students performed better than other students achieving scores as much as 24 points higher than other students with girls performing just as well as boys. This resulted in the transfer to government schools for 5th grade of 28,778 students in 2007 and 2008. It is not clear how many adults have acquired literacy skills as a result of their participation in literacy classes. 1.3. Have innovations introduced through USAID support been sustained by the government, communities and schools, and/or become a part of the national education policy in Ethiopia? Innovations in teacher education have been adopted by MOE and are now being utilized (cluster/linkage model initially developed by UNICEF but then expanded by USAID; active learning, continuous assessment, etc., are all now a part of the TTC curriculum for pre-and in￾service training. A revised version of the SIA has been adopted by the MOE in the School Block Grants. The establishment of GEACs has been adopted by the MOE for all primary schools. Community mobilization in support of schools has been adopted by many non￾USAID schools. 1.4. Which programs or innovations failed to produce intended results or to be sustained and why? Increases in female faculty at TTCs – not a priority. Sustainability of BESO I and II, and EQUIP II innovations as training stopped and participants “cooled off” during the time between the end of the projects and the implementation by MOE. GEAC sustainability in light of transfers and principals not being aware of these groups. Impact Evaluation of USAID/Ethiopia’s Education Activities 59 Objective 2: Identify the specific inputs and processes that contributed to changes achieved by USAID programs, including planning, design, management, coordination, Monitoring, etc. 2.1. How well did the design of programs address their sustainability? Were programs successful in achieving sustainability? • Some activities have led to MOE policy changes and nationally-scaled programs (TTC gender￾based incentives, ABECs, GEACs, SIPs, NLA) and are thus being sustained. • The community grants programs (CSAP, CSGP, and CGPP) generally have been sustained in the sense that parental/community involvement with schools issues has continued beyond the life of the financial incentives, the primary intent of these efforts. • The TTC program improvement interventions have proved to be harder for the institutions to sustain beyond USAID funding (Resource & IT Centers). • The “gap” caused by the break between BESO2 and IQPEP was often cited as the reason certain early innovations were not continued during AED’s hiatus (RSEB & WEO capacity building, TTC Resource/IT Centers). • The State Minister for General Education and Dire Dawa REB head both stressed the need for more attention to transition planning and sustainability in USAID partner programs. 2.2. Were USAID education interventions designed to coordinate and collaborate with the larger, overall needs in the education sector? If so, has this impacted or hindered the success of the programs? • USAID has since BESO1 worked closely and collaboratively with the MOE and RSEBs. Its systems approach has given USAID a good understanding and strategic view of overall sector needs. • Generally, coordination of efforts has been moderately good, but not optimal due to GOE interest in spreading external assistance equitably rather than for gaining for synergistic multipliers. 2.3. How effective and efficient were the mechanisms of implementation of USAID programs, including structures and organization, planning, implementation modalities, resource flow? • BESO1 NPA was the right modality at the early stage of sector revitalization. Good progress was made during 3 years of reform based budgetary support. • Using local and international NGOs for Implementation of the various community-based grant programs was correct for the period given the weak government capacities and lack of policies regarding school governance, oversight and support. The local capacities for managing such grant programs are now significantly improved and the need for external NGO management is now less necessary and less appropriate. • There is MOE (State Minister) concern that the AED operation and staff resources could be better integrated into the Ministry’s workload. 2.4. How effective has the relationship between USAID, implementing partners and host • The BCCC (in BESO1) and TWG forums have proven effective and valued by all partners for 60 Impact Evaluation of USAID/Ethiopia’s Education Activities government institutions been in the planning, managing and monitoring of USAID programs? How has this contributed to the larger picture of education sector development? planning, monitoring and troubleshooting sector progress and problems. • This regular and close proximity to decision￾makers has put USAID in a unique position to engage early with and influence sector priorities and plans. 2.5. How could USAID improve its efficiency and effectiveness in the next five years of programming? • USAID needs to explore with MOE and REBs how to more fully integrate its efforts with those of the government, and to work more deliberately toward sustainability. Several officials expressed concern in this area. • Over the years, USAID assistance has become more activity- and less system-focused. For future programming closer attention is needed to the human, institutional and budgetary constraints to achieving lasting, systemic change. 2.6. How effective have the processes of needs identification and program design been to ensure the participation of key stakeholders and success of programs? • USAID interventions have been based on solid needs identification and collaborative programming involving the key stakeholders and partners. • However, in USAID’s movement toward a program of assistance with national scope and reach, the ability to tailor efforts to the specific needs of (older, stronger, younger, weaker) institutions has been reduced, and with it the potential for systemic and sustainable change. Objective 3: Assess current challenges of the education sector in Ethiopia. Address the current structure, capacity, policy environment, and organization. 3.1. What are the key factors affecting (positively or negatively) successful development of the education sector including equitable access, quality, and education that is relevant to the social and economic development of Ethiopia? Factors such as: capacity; reliance on donor assistance; transparency; planning; governance and management and allocation of resources? • Ethiopia initially emphasized quantitative expansion, with attention to geographic distribution and gender equity. • Initial effects included a sharp drop in instructional quality, inadequate classrooms, texts and qualified teachers • With USAID assistance, planning and management capacities have improved at all levels • availability of qualified teachers, texts/materials and active learning pedagogies improving, but quality still low 3.2. What are the key challenges to successful enrollment, retention, completion of male and female students at the primary level? • Grade repetition and dropout rates are declining; survival to grade 5 and completion rates at grade 8 are increasing in all regions. • Participation rates for girls are near parity; completion rates for girls are improving but lag rates for boys due to relatively recent improvement of access and survival rates • The main challenges are addressing the hard-to￾reach in rural and pastoral areas and emerging regions 3.3. What are the challenges to quality that are keeping primary completers from learning to read and from possessing the basic • The Early Grade Reading Assessment (EGRA) should inform this question. • Challenges include improved pedagogic training Impact Evaluation of USAID/Ethiopia’s Education Activities 61 competency skills necessary to benefit from and continue with their education? for teachers and increased availability of reading materials (not just texts) in local languages and at appropriate levels of language use. • English language capability will be an increasing challenge for teachers and students at all levels, but particularly for Cycle 2 (grades 5-8) 3.4. What challenges do teacher’s face in becoming qualified and in improving the learning of their students? • Though the availability of teachers with diploma qualifications is increasing, there remain large numbers of certificate-qualified teachers. TTCs have a variety of upgrading programs, with some 20,000 in a combination of distance education and evening classes. 3.5. What are the largest challenges in underserved regions in Ethiopia, mainly Afar and Somali? • Sparsely populated areas requires small schools at primary and/or ABECs and secondary levels • Distance between schools and distance from woreda and regional centers limits pedagogic support and creates logistic (transport) constraints. • Lack of electricity limits use of media Objective 4: Based on the conclusions drawn from addressing objectives 1 – 3, identify USAID’s historical and present comparative advantages in the education sector and provide recommendations on future USAID strategy to support the development of the Ethiopian education sector. 4.1. Based on historical evidence and a current analysis of the education sector, where does USAID’s comparative advantage lie? How should USAID focus its resources in order to have the greatest impact on the sector? • USAID has a good record for collaboration and joint planning and for being willing to invest in long-term capacity building rather than short-term direct inputs • USAID has not been substantially involved in curriculum development to date, but has comparative advantage in the fields of English, Mathematics and Science. The comparative advantage lies not so much in subject knowledge but in the practices of aligning curriculum with materials with teacher preparation and pedagogic support and in utilizing assessment and research tools to inform iterative development. • Assessment, monitoring and research • USAID mobilizes communities very well in support of the school and vulnerable children • USAID creates strategic alliances well (e.g., health and education) in the interests of supporting the whole child • USAID creates interest in and support of girls • USAID has comparative expertise in the use of data, research and assessment tools and processes for education decision-making 4.2. How can USAID’s resources be best used • Continue to focus on capacity building at all 62 Impact Evaluation of USAID/Ethiopia’s Education Activities to improve the quality of basic education in Ethiopia? levels • Strengthen TTCs, teacher training methodologies and outreach/ pedagogic support roles for woreda and cluster resource centers • Focus on English language, early mathematics and early reading and science. • Enrich cluster resource centers with supplementary materials, demonstration materials, kits and media • Further development of the ABEC models, with substantially more assessment and monitoring, support of innovations and additional materials and support strategies • Create other strategic alliances (e.g., for livelihood development and/or environmental science/agriculture) to address the learning needs of children 4.3. What has USAID been doing that has worked, and what has not? Looking to the future, what types of specific programming will be the most effective? • Long-term, sustained capacity building to strengthen systems and institutions and to train key people to the Master’s level. • Small grant support to school/ community committees and PTAs have helped shift the “ownership” of the school, increased local participation and mobilization of resources and led to increased school and community attention to OVCs and other children in need and to the enrolment and continued participation of girls. • As Ethiopian capacities have improved (e.g. National Learning Assessment, EMIS, planning) USAID has continued support but reduced the role of USAID contractors. • Note: more can be done to reduce the visibility of USAID implementing partners and to support the Ethiopian units more directly. For example, if small grants programs such as CSPP are continued, the woredas should be able to administer them without substantial contractor help. • Beyond continued support for monitoring and assessment activities, USAID should look for opportunities to build research capacities – possibly including institutional linkages/twinning, support for a semi￾autonomous institute of education, advanced training for key researchers and support for specific research activities such as longitudinal students and ongoing diagnostic work on the factors limiting early reading, Impact Evaluation of USAID/Ethiopia’s Education Activities 63 the incidence and epidemiology of special needs and similar topics requiring sustained inquiry. • Support for MOE scale-up of all USAID innovations Impact Evaluation of USAID/Ethiopia’s Education Activities 65 ANNEXES Annex A. Scope of Work Annex B. Selected List of Documents Consulted Annex C. Organizations and Individuals Interviewed Annex D. Summary of USAID Project Assistance to Education, 1994 -2014 Annex E. GEQIP Parameters Annex F. Statistical Data EDH-E-00-08-00003-00 EDH-E-27-08-00003-00 Attachment 1 SCOPE OF WORK IMPACT EVALUATION OF THE UNITED STATES AGENCY FOR INTERNATIONAL DEVELOPMENT (USAID’s) EDUCATION PROGRAM IN ETHIOPIA 1994 - 2009 I. SUMMARY The United States Agency for International Development (USAID) in Ethiopia seeks the services of a Contractor for the purpose of conducting an impact evaluation of USAID￾supported education programs in Ethiopia. The evaluation will enable USAID to analyze the overall changes in enhancing the quality and equity of primary education in Ethiopia over the life of the program, with an emphasis on the BESO II project which began in 2002. The overall purpose of this evaluation is to provide USAID Ethiopia with a comprehensive analysis of the impact of USAID’s education activities and recommendations for future program design. The evaluation will review the design, implementation and monitoring of the major components of USAID’s education programs. USAID Ethiopia will use the results and recommendations of the evaluation to ensure that future planning and implementation decisions are efficient, effective, and relevant to education sector development and quality enhancement in Ethiopia. The main objective of the assessment is to determine which changes in educational access, quality and equity have taken place in USAID-supported primary schools, teacher training colleges, and local, regional and national education offices and to determine if and how these changes have had an impact on the development of the education sector in Ethiopia. The objective can be achieved by (1) Identifying the specific inputs and processes that contributed to changes achieved by USAID programs, including planning, design, management, coordination, monitoring, etc. (2) Assessing the current challenges of the education sector in Ethiopia and describe the current structure, capacity, policy environment, and organization of the sector and (3) Based on the conclusions drawn from activities one and two, identify USAID’s historical and present comparative advantages in the education sector and provide recommendations on future USAID strategy to support the further development of the Ethiopian education sector. II. BACKGROUND Ethiopia, with a population of nearly eighty million and an annual growth rate of 2.51 percent, continues to struggle to provide quality education to its citizenry. USAID has provided over $180 million to enhance the quality of primary education in Ethiopia in the last fifteen years. Many of the innovations introduced by USAID’s support in teacher education, community strengthening and local governance, planning and management, and delivery of alternative basic education, have been adopted and institutionalized by the Ministry of Education (MOE). USAID has worked closely with the MOE, educational administrators, researchers, curriculum developers, teacher trainers, classroom instructors, non-governmental organizations (NGOs), private sector companies, and communities, parents, and students throughout the country. For the past 15 years, Page 18 EDH-E-00-08-00003-00 EDH-E-27-08-00003-00 USAID’s largest program has been implemented by Academy for Educational Development (AED). AED, with USAID’s guidance, has been working closely with the MOE in the curriculum, planning, and teacher education departments. USAID Ethiopia’s education program currently has six major implementing partners. The education office also convenes a Technical Working Group quarterly consisting of the federal MOE, the Ministry of Finance and Economic Development (MOFED), all Regional State Education Bureau Heads, and all implementing partners. The purpose of these partnerships is to support Ethiopia to provide quality basic education equitably to all children. In 1994, the United States Agency for International Development, in close consultation with the Ministry of Education, conducted a comprehensive joint assessment to refine their understanding of educational needs, determine strategies for achieving and sustaining improvements in access, quality, and equity, and streamline U.S. Government’s support. The study revealed that isolated inputs or interventions would be ineffective in Ethiopia. It was determined that USAID programs in education must contribute to a larger effort to transform the entire system. The decentralization of the education system to regional education bureaus, though necessary, increased the complexity of the challenge. Responsibility for delivering educational services was placed at the regional and sub-regional levels, where a severe lack of institutional, human, and financial resources was compounded by language issues and very poor communication and transportation infrastructures. Planning and management capacity to collect and analyze data needed for informed policy decisions was severely lacking. In 1995, USAID Ethiopia launched the Basic Education System Overhaul (BESO). BESO started in two regional states, Tigray and the Southern Nation Nationalities and People’s Region (SNNPR), and in 2002, extended its coverage to all regions. BESO, and its successor, the Basic Education Program (BEP), addressed some of the education sector’s greatest needs. The strategic objective of ‘Use and Provision of Quality Primary Education Services Enhanced’ remained the same as the issue of quality continued to be a major challenge of the system. The four Sub-Intermediate Results that contribute to the strategic objective are the following: a) Community participation in the management and delivery of primary education services strengthened: Community participation includes receiving social services as well as managing those services at the community level. Involving the community in the management and delivery of primary education is essential in Ethiopia to attain the intermediate result in terms of access, equity and quality. The attainment of the SO also requires that health and education activities are well coordinated both at the federal and grassroots levels, and that they share strategies and mechanisms for implementation, especially on community capacity building and provision of services. Activities include: strengthening the capacity of communities, promoting gender equity, raising awareness on HIV/AIDS, reproductive health and family planning, promoting community outreach in support of education, and assisting schools’ income generation Page 19 EDH-E-00-08-00003-00 EDH-E-27-08-00003-00 activities. Through our Community Schools Partnership Program, USAID has provided school incentive grants to 5,100 primary schools to help them improve their learning environment. USAID has also provided capacity building to the Parent-Teacher Associations of these schools to enable them to mange the grants. Currently, USAID targets 1,800 primary schools in the most disadvantage parts of the country with emphasis on water, health and sanitation. The MOE has recently rolled out a nationwide school incentive grant, modeled after this USAID program. b) Planning, management and monitoring and evaluation for delivery of primary education services strengthened: Currently, the responsibility for managing primary and secondary education lies at the district level, as a result of the decentralization program of the Ethiopia Government, which began ten years ago. Unfortunately, capacity remains inadequate at all levels of governance, especially at the district level. USAID capacity building support is being provided nationwide to the Federal Ministry of Education, regional education bureaus and district education offices. Activities include: conducting capacity need assessments and training, providing custom manuals and equipment, improving monitoring and evaluation mechanisms to track the effectiveness of the education system, improving educational planning and management, and strengthening community mobilization mechanisms from the school to federal Ministry of education levels. Tens of thousands of educators at all levels have been trained in planning and managing educational development at all levels of the system in the last fifteen years. c) Quality of primary education improved: Quality has been a major problem in the education system as a result of inadequate educational inputs and poor quality of the teaching-learning process. USAID focuses on improving the major educational inputs and helping change the teaching learning process. Activities include: strengthening resource centers in teacher education institutions (TEIs), improving pre-service teacher training, developing support systems for women teacher trainees, improving teacher in-service training through the “cluster” model, enhancing active learning methodology in TEIs and primary schools, and increasing the availability of supplementary educational materials in critical areas such as HIV/AIDS, nutrition, gender, family planning, and the environment. USAID has been providing support to all of the teacher education institutions in Ethiopia and school-based in-service teacher training in 1,200 elementary schools to improve the quality of teaching in primary schools. To date, over 80,000 teachers and school directors have been trained through the school-based in-service teacher training program. As part of helping the capacity building efforts of the Ministry and the Regional State Education Bureaus (RSEBs), USAID has trained about 150 volunteer teachers in the last twelve years through the International Foundation for Education and Self-Help (IFESH). The volunteers have been very effective in the teacher education institutions, with some leading the Higher Education Diploma Program of the Ministry. Page 20 EDH-E-00-08-00003-00 EDH-E-27-08-00003-00 The other major effort of USAID to help improve quality has been the support provided to strengthen English language teaching and learning. The Textbook and Learning Materials Program (TLMP) helped in developing English language textbooks for grades 1, 6, 7, and 8 through Alabama A&M University in coordination with the Ministry of Education and Regional curriculum experts and selected pilot schools. Through the TLMP about 3.3 million textbooks of the newly developed textbooks have been printed and distributed to schools around the country. In the last year, 20,000 English language teachers from grades 6 – 8 were trained in teaching the newly developed English textbooks. d) Equitable primary education service strengthened: Low levels of funding and implementation capacity are major problems for expanding primary education in Ethiopia. In addition to this, a number of socio-economic and cultural factors inhibit the equitable access to primary education. The rigidity of the formal school system tends to reduce interest in formal education amongst rural and pastoralist populations. Location (distance from urban and semi-urban places), religion, and culture are other factors that keep children away from schools. Therefore, taking schools closer to the children through alternative basic education centers (ABECs), introducing a curriculum that meets the interest of the learners, and using flexible approaches to teaching that respond to communities’ economic and cultural realities can play a pivotal role in achieving universal primary education. These centers also serve as development centers for the community, where functional adult literacy on population and family planning, the environment, HIV/AIDS, and agricultural extension information and other skills are imparted. Adult literacy programs will also be integrated to other programs such as a micro-finance schemes to improve livelihoods of family, and thereby to reduce income insecurity. The non-formal basic education support of USAID earmarked a substantial amount of resource to establish Alternative Basic Education Centers around the country for five years. Five hundred fifty Alternative Basic Education Centers (ABECs) for children and adults have been built and are functional with an enrollment nearing 120,000 through the Transforming Education for Adults and Children in the Hinterland (TEACH) implemented by PACT. With PEPFAR funding, USAID initiated a new program to enhance access to and retention in primary education for orphans affected by HIV/AIDS. The program has been providing education, health and other comprehensive services, including food, to 5,000 children from 100 primary schools in three regions. In another effort to enhance access to girls, USAID has been implementing the Ambassador’s Girls Scholarship Program (AGSP,) providing scholarship to 1,380 secondary school girls. Currently 970 girls are benefiting from the program. Page 21 EDH-E-00-08-00003-00 EDH-E-27-08-00003-00 III. STATEMENT OF WORK The evaluation will address the following questions through a desk review, research, field research, interviews and dialogue with USAID, Ethiopian education officials, implementing partners, other donors and stakeholders as relevant and described in the methodology of this SOW: Objective 1: Determine which changes in educational access, quality and equity have taken place in USAID-supported teacher training colleges, primary schools, and local, regional and national education offices. Determine if and how these changes have had an impact on the development of the education sector in Ethiopia. 1.1. Is there evidence to show that improvements in the quality of teaching, girls’ equity, and the general learning-teaching atmosphere have taken place in USAID-supported schools? Is so, has the knowledge transferred into the system? 1.2. Is there evidence that quality and equity in education have been enhanced per the following indicators:  Active learning and continuous assessment in primary schools are used effectively.  The survival rate of children to grades 5 and 8 has increased in USAID assisted schools.  The learning achievement of children has improved in USAID-supported schools.  The efficiency of regional education bureaus and woreda education offices has improved in planning and managing primary education.  Parent-teacher associations (PTAs) in USAID supported schools are empowered and are effectively supporting and managing their schools.  Children and adults have received quality basic alternative education that could lay the foundation for their future education. 1.3. Have innovations introduced through USAID support been sustained by the government, communities and schools, and/or become a part of the national education policy in Ethiopia? 1.4. Which programs or innovations failed to produce intended results or to be sustained and why? Objective 2: Identify the specific inputs and processes that contributed to changes achieved by USAID programs, including planning, design, management coordination, monitoring, etc. 2.1. How effective have the processes of needs identification and program design been to ensure the participation of key stakeholders and success of programs? Page 22 EDH-E-00-08-00003-00 EDH-E-27-08-00003-00 2.2. How well did the design of programs address their sustainability? Were programs successful in achieving sustainability? 2.3. Were USAID education interventions designed to coordinate and collaborate with the larger, overall needs in the education sector? If so, has this impacted or hindered the success of the programs? 2.4. How effective and efficient were the mechanisms of implementation of USAID programs, including structures and organization, planning, implementation modalities, resource flow? 2.5. How effective has the relationship between USAID, implementing partners and host government institutions been in the planning, managing and monitoring of USAID programs? How has this contributed to the larger picture of education sector development? 2.6. How could USAID improve its efficiency and effectiveness in the next five years of programming? Objective 3: Assess current challenges of the education sector in Ethiopia. Address the current structure, capacity, policy environment, and organization. 3.1. What are the key factors affecting (positively or negatively) successful development of the education sector including equitable access, quality, and education that is relevant to the social and economic development of Ethiopia? Factors such as: capacity; reliance on donor assistance; transparency; planning; governance and management and allocation of resources? 3.2. What are the key challenges to successful enrollment, retention, completion of male and female students at the primary level? 3.3. What are the challenges to quality that are keeping primary completers from learning to read and from possessing the basic competency skills necessary to benefit from and continue with their education? 3.4. What challenges do teacher’s face in becoming qualified and in improving the learning of their students? 3.5. What are the largest challenges in underserved regions in Ethiopia, mainly Afar and Somali? Objective 4: Based on the conclusions drawn from addressing objectives 1 – 3, identify USAID’s historical and present comparative advantages in the education sector and provide recommendations on future USAID strategy to support the development of the Ethiopian education sector. 4.1. Based on historical evidence and a current analysis of the education sector, where does USAID’s comparative advantage lie? How should USAID focus it’s resources in order to have the greatest impact on the sector? Page 23 EDH-E-00-08-00003-00 EDH-E-27-08-00003-00 4.2. How can USAID’s resources be best used to improve the quality of basic education in Ethiopia? 4.3. What has USAID been doing that has worked, and what has not? Looking to the future, what types of specific programming will be the most effective? IV. OVERALL METHODOLOGY, KEY TASKS, DELIVERABLES, TIMING, AND LEVEL OF EFFORT Overall Methodology: The assessment team, in collaboration with USAID/Ethiopia, will finalize the overall methodology for the assessment. However, USAID/Ethiopia expects that the Assessment Team will, at a minimum, carry out the key tasks listed below. Key Tasks: Task 1: Draft description of overall methodology and conduct a desk review The desk review will serve to answer many of the assessment questions and provide background information. After one or more telephone meetings with USAID/Ethiopia, the Team Leader and other team members shall draft a description of the overall methodology/ methodological framework and work plan to answer the assessment questions and submit to USAID/Ethiopia within one week after beginning desk review. Within two weeks after receiving the draft methodology and work plan, USAID/Ethiopia will provide feedback to the Team Leader. For the desk review, the assessment Team Leader will provide USAID/Ethiopia with a recommendation on how to carry out the desk review included in the submission of the draft methodology. USAID Ethiopia will ensure that the documentation is available to the team prior to the agreed-upon dates for this task. Task 1 deliverable and date: 1. draft description of overall methodology and work plan, within one week after beginning desk review Task 1 timing and level of effort: to be proposed, approximately 2 weeks depending on overall level of effort Task 2: Finalize overall methodology and schedule and conduct in-country field visits When the assessment team arrives in-country, they will finalize the overall methodology for the assessment. While the desk review will answer many of the assessment questions, the in-country field visits will serve to (1) validate preliminary conclusions from the desk review and (2) answer questions that could not be answered from the desk review. The field visits will also be an opportunity to gain an understanding of the basic education sector context in Ethiopia. The field visits shall include interviewing USAID staff, implementing partners and project beneficiaries, host government counterparts at appropriate levels, basic education sector Page 24 EDH-E-00-08-00003-00 EDH-E-27-08-00003-00 development partners, sector experts and researchers who have conducted research on basic education issues in Ethiopia, and other stakeholder groups. Table 1 (below) contains a list of the types of stakeholder groups to be interviewed and illustrative examples of who to include from those groups. The assessment team is encouraged to identify and visit additional relevant Ethiopian organizations and groups, both formal and informal, as identified from the desk review and relevant to answering the assessment questions. USAID Ethiopia will provide the assessment team with input and guidance in setting up a schedule of interviews and site visits, but the responsibility for the schedule resides with the Contractor. The draft schedule must be ready for review and discussion at the initial in-country assessment team meeting with USAID Ethiopia and finalized within one week of arrival in Ethiopia. Table 1: List of types of stakeholder groups to be interviewed (not exhaustive) Type of stakeholder group Illustrative examples USAID project beneficiaries School principals, teachers, teacher training college deans and trainers, students, parents, PTAs, Alternative education center management committees, Girl’s Education Advisory Committees, etc. Host government counterparts and other relevant government officials Ministry of Education, Regional State Education Bureaus, Ministry of Finance and Economic Development, Teacher Training Colleges, relevant World Education Offices Basic education sector development partners and other donors providing assistance in the sector World Bank, DFID, JICA, UNICEF, and other members of the Education Technical Working Group (ETWG), etc. Sector experts and researchers Academicians, other research institutions Task 2 deliverables and dates: 1. final description of overall methodology, within one week after arrival in Ethiopia, 2. Schedule of field visits, within one week after arrival in Ethiopia. Task 2 timing and level of effort: approximate 5 weeks Task 3: Debrief and draft executive summary The team will debrief, in the form of a presentation, USAID Ethiopia staff on the preliminary conclusions and recommendations of the assessment. This will occur three days before departing Ethiopia. The team will also provide a draft executive summary document of the assessment report prior to the presentation. The executive summary is to contain a clear, concise summary of the critical elements of the report, including the main findings, conclusions and recommendations. During the debrief meeting/presentation, USAID Ethiopia staff will provide feedback on the preliminary conclusions and recommendations and the draft executive summary. This feedback will be incorporated into the draft assessment report. Page 25 EDH-E-00-08-00003-00 EDH-E-27-08-00003-00 Task 3 deliverables and dates: 3. presentation of key findings and preliminary conclusions and recommendations, three days before departure from Ethiopia. 4. draft executive summary, prior to debrief for discussion and comment during debrief meeting Task 3 timing and level of effort: level of effort included in total time in Ethiopia; presentation to be completed 3 days before departure from Ethiopia and draft executive summary to be submitted prior to presentation. Task 4: Assessment report The team will prepare an assessment report in English. The draft assessment report will be submitted to USAID Ethiopia for review and feedback on departure from Ethiopia. The final assessment report will be submitted two weeks after receiving and incorporating comments from USAID Ethiopia. USAID Ethiopia Contracting Officer’s Technical Representative (COTR) or Agreement Officer’s Technical Representative (AOTR) will be responsible for review and approval of the final document. The following sections shall be included in the document: a. Executive Summary – (3 to 5 pages) containing a clear, concise summary of the critical elements of the report, including the main findings, conclusions and recommendations. b. Table of Contents c. Assessment findings and conclusions (no more than 30 pages): discusses the major findings and the related issues and questions raised. This section shall include:  Assessment objectives and questions;  Evidence/findings of the study concerning the assessment questions;  Conclusions drawn from the findings (including lessons learned) d. Future program recommendations (no more than 15 pages), based on findings and conclusion of desk review, field visits, and stakeholder’s workshop. Recommendations should take into account an annual education budget of approximately $20 million from 2010 and onwards and provide rationalization of program focus. e. Appendices, including:  Assessment Scope Of Work;  description of team composition and assessment methodology (1 page maximum);  list of documents consulted and of individuals and agencies interviewed; and  More detailed description of methodological or technical issues and limitations, as appropriate. Contractor shall be responsible for providing the final assessment report to USAID Ethiopia electronically (in Microsoft Word 2003 or newer) and in hard copy (10 bound copies). The Contractor shall also provide an electronic copy of the final assessment report to DEX, the database of the USAID Development Experience Clearinghouse (DEC), in accordance with typical USAID/Washington requirements. Page 26 EDH-E-00-08-00003-00 EDH-E-27-08-00003-00 Task 4 deliverables and dates: 5. draft assessment report, before departure from Ethiopia, 6. final assessment report, two weeks after receiving and incorporating comments from USAID Ethiopia. Task 4 timing and level of effort: Overall timing and level of effort: The assessment team will spend approximately six weeks carrying out activities related to this Scope of Work: approximately two weeks on the desk review and preparation of overall methodology; approximately five weeks in Ethiopia carrying out the field visits and preparing the draft assessment report; and approximately one week after departing Ethiopia preparing the final assessment report. Not all members of the assessment team will necessarily be engaged in the various tasks on a full-time basis. V. TEAM COMPOSITION AND QUALIFICATIONS Team Members: The assessment team should comprise, as described below: 1) U.S.: a Team Leader and two Senior Team Members; plus miscellaneous backup from the contractor’s office; and 2) Local: two Evaluation Specialists and a Logistics Coordinator; plus Interpreters and Drivers, as needed. In addition, USAID Ethiopia Education Office personnel, USAID/Washington Education Office personnel, and/or a representative from the GoE Ministry of Education may accompany the team in meetings and on field visits at their own expense. The Contractor shall hire the team members and shall assure that the overall composition of the team includes the required mix of expertise and experience necessary to carry out the tasks described in the SOW. Team Leader. The Team Leader must have: Relevant Education Master’s degree. Demonstrated knowledge of Basic Education issues and applied social research, stakeholder facilitator, and critical thinking skills. At least 15 years’ experience in designing or implementing Basic Education programs in developing countries, and significant experience in conducting sector-wide assessments, evaluations, and/or contextual studies. Familiarity with USAID and particularly its basic education policy guidance is desirable. He/she must be fluent in English and have excellent analytical, writing, presentation/ facilitation, and management skills. Professional experience in Africa required; in Ethiopia strongly desired. Two Senior Team Members. The two Senior Team Members must each have: Relevant Master’s degree. Demonstrated knowledge of the Basic Education sector in Ethiopia. At least 12 years’ experience in implementation of Basic Education programs in developing countries, and significant experience in conducting sector-wide assessments. Familiarity with USAID and particularly its basic education policy guidance is desirable. He/she must be fluent in English and have excellent analytical, writing and presentation/ facilitation skills. Professional experience in Ethiopia or East Africa desired. Two Ethiopian Education/Specialists. The two Ethiopian Education/Evaluation Specialists must each have: Relevant Bachelor’s degree. Page 27 EDH-E-00-08-00003-00 EDH-E-27-08-00003-00 Five to seven years of experience in the Basic Education sector in Ethiopia, preferably with a mix of technical skills related to pre-primary, primary, non-formal education, and teacher education and education program assessment and design skills and experience. In-depth understanding of the socio-political and cultural context in Ethiopia or the East Africa region. Familiarity with USAID and other donor assistance in the Basic Education area is desired. At least one team member should be fluent in written and spoken Amharic. All team members must have very good written and spoken English. Logistics Coordinator: The Logistics Coordinator will organize and coordinate meetings, make hotel reservations, organize transportation, etc. S/he must have: 1. Secondary school diploma. Some university-level education desirable. 2. At least 3-5 years of work experience. Interpreters and Drivers, as needed. Drivers and other in-country assistance, such as sign language and/or local language interpreters, are to be hired, as needed and as determined by the Team Leader. Local language interpreters: at least secondary school diploma and at least 1-3 years of work experience. Some university-level education is desirable. Drivers: must be licensed and vehicles must be properly insured. VI. SCHEDULE AND LOGISTICS The assessment effort should commence no later than April 7 with the two-week research task. The team should arrive in Addis Ababa, Ethiopia, and be prepared to begin work no later than the end of February or beginning of March. The team is required to meet with USAID at three points during the assessment. First, an initial planning meeting, no later than the second day the team is in country, will establish clear expectations about the assessment outcomes in addition to covering the goals, schedule, and methodology to be utilized. At the mid-way point, the second meeting will brief on progress of the assessment. Approximately 3 days prior to departure, the team will make a final oral presentation to USAID/Ethiopia on key findings and preliminary conclusions and recommendations and submit a draft executive summary prior to this presentation for discussion and comment. VII. SPECIAL PROVISIONS Duty Post Addis Ababa, Ethiopia Access to Classified Information The Contractor shall not have access to any Government classified material. Logistical Support The Contractor is responsible for providing all logistical support. Office space shall not be provided by USAID. The Contractor will be responsible for providing office supplies, Page 28 EDH-E-00-08-00003-00 EDH-E-27-08-00003-00 Page 29 equipment, computers, copiers, printers, etc. Translation services and transportation (to/from Ethiopia and in-country) are the responsibility of the contractor. Supervision The team will work under the direct supervision of the USAID/Ethiopia Chief of the Basic Education Services Office, with the assistance of project COTR/AOTRs, as appropriate. Performance Period The assessment will be carried out over approximately nine weeks, beginning in April 2010 or when the contract is finalized. A six-day workweek is authorized. No premium or overtime pay are authorized under this contract. Deliverables/Final Products Deliverables and final products belong to USAID, not to the consultants or Contractor, and use of any material in the documents is expressly prohibited. 78 Impact Evaluation of USAID/Ethiopia’s Education Activities Annex B. Selected List of Documents Consulted Academy for Educational Development. N.d. BESO II. Teacher’s Module 1. How to Help Students Learn More Effectively Using Active Learning Methods. Grades 1 to 4 (Revised). Academy for Educational Development. N.d. BESO II. Teacher’s Module 2. Continuous Assessment in the Primary Schools. Grades 1 to 4 (Revised). Academy for Educational Development. N.d. BESO II. Teacher’s Module 3. Continuous Assessment: Remediation and Enrichment. Grades 1 to 4 (Revised) Academy for Educational Development. N.d. BESO II. Teacher’s Module 4. Gender Issues. Grades 1 to 4 (Revised Academy for Educational Development. N.d. BESO II. Teacher’s Module 5a. Curriculum Integration. Grades 1-2 (Revised). Academy for Educational Development. N.d. BESO II. Teacher’s Module 5b. Curriculum Integration. Grades 3-4 (Revised). Academy for Educational Development. N.d. BESO II. Teacher’s Module 6. How to Manage a Large Class to Promote Active Learning: Some Tips. Grades 1 to 4 (Revised). Academy for Educational Development. 2005. Basic Education Program. PMP Indicators Report for Option Year One (2004/05). Academy for Educational Development. 2006. Basic Education Program. PMP Indicators Report for Option Year Two (2005/06). Academy for Educational Development. 2007. Basic Education Program. PMP Indicators Report for Option Year Three (2006/07). Academy for Educational Development. 2006. Basic Education Program. Teacher’s Module 1. Active Learning: A Practical Guide. Grades 5 to 8 (Revised). Academy for Educational Development. 2006. Basic Education Program. Teacher’s Module 2A. Sample Active Learning Activities for Grades 5-6 (Revised). Academy for Educational Development. 2006. Basic Education Program. Teacher’s Module 2B. Sample Learning Activities for Grads 7 to 8 (Revised). Academy for Educational Development. 2006. Basic Education Program. Teacher’s Module 3. Continuous Assessment and How to Use It Grades 5 to 8 (Revised). Academy for Educational Development. 2006. Basic Education Program. Teacher’s Module 4. Gender Issues for Second Cycle Teachers Grades 5 to 8 (Revised). Addis Ababa: USAID, Basic Education Program. Academy for Educational Development. 2006. Basic Education Program. Teacher’s Module 5. HIV/AIDS Education: Activities for the Classroom or School Club Grades 5 to 8 (Revised). Impact Evaluation of USAID/Ethiopia’s Education Activities 79 Academy for Educational Development. 2006. Basic Education Program. Teacher’s Module 6. Large Class Management: Tips and Guidelines Grades 5 to 8 (Revised). Academy for Educational Development. 2007. Basic Education Program August 2002-December 2007. Final Report. Volume II: Full Report. Academy for Educational Development. 2008. USAID-AED/EQUIP II Project. Building the Capacity of Primary Education Managers and Educators. First Quarter Report, January – March. Academy for Educational Development. 2008. USAID-AED/EQUIP II Project. Building the Capacity of Primary Education Managers and Educators. Quarterly Report, April-June. Academy for Educational Development. 2009. USAID/Improving the Quality of Primary Education Program in Ethiopia (IQPEP). Five Year Implementation Plan August 2009 – July 2014. American Institutes for Research. 2009a. USAID-AIR Teach English for Life Learning (TELL). Annual Performance Report December 2008 – September 2009. American Institutes for Research. 2009b. USAID-AIR Teach English for Life Learning (TELL). Classroom Observation Activity. Data Analysis Report. Berke, Berhanu. 2009. PMP Assessment Report of Woreda Capacity Building, Personnel Management Information System, Kebele Education and Training Board and Primary School Principals’ Training. Addis Ababa: USAID-AED/EQUIP II Project. Ethiopia, Federal Democratic Republic of, Ministry of Education. 2005. Education Sector Development Program III (ESDP III) 2005-2006 – 2010-2011. (1998 EFY – 2002 EFY). Program Action Plan (PAP). Final Draft. Ethiopia, Federal Democratic Republic of, Ministry of Education. 2007. General Education Quality Improvement Program (GEQIP) 2008-2009 – 2012-2013 (2001 – 2005 E.C.) Program Document (Draft). Ethiopia, Federal Democratic Republic of, Ministry of Education. Education Sector Development Program Planning and Policy Analysis Department. 2008 . Educational Statistics Annual Abstract. 1999 E.C. (2006-2007 G.C.) Ethiopia, Federal Democratic Republic of, Ministry of Education. Education Sector Development Program Planning and Policy Analysis Department. 2009. Educational Statistics Annual Abstract. 2000 E.C. - 2007-2008 G.C. Garcia, Marito and Andrew Sunil Rajkumar. 2008. Achieving Better Service Delivery through Decentralization in Ethiopia. World Bank Working Paper 131. Washington, DC: World Bank. Juarez & Associates. 2005. Mid-Term Evaluation, BESO II, Community-Government Partnership Program (CGPP). International Foundation for Education and Self Help. 2007. Assessment of the Impact of the Capacity Building in Teacher Education Project (CBTEP). 80 Impact Evaluation of USAID/Ethiopia’s Education Activities Management Systems International. 2007. Basic Education Strategic Objective (BESO) II – Basic Education Program. Final Evaluation. Washington, DC: MSI. Pact Ethiopia. 2008. Transforming Education for Adults and Children in the Hinterland (TEACH). Alternative Basic Education Level I. Students’ Assessment Study Report. Addis Ababa: Pact Ethiopia. Pact Ethiopia. 2009. Transforming Education for Adults and Children in the Hinterland (TEACH). Alternative Basic Education Level I. Final Report October 2004 – September 2009. Addis Ababa: Pact Ethiopia. Save the Children US. 2009. USAID/Positive Change: Children, Communities and Care (PC3). Annual Report April 1, 2008-March 31, 2009. Addis Ababa: USAID. Save the Children US. 2009. USAID/Community-School Partnership Program in Education and Health (CSPP). Annual Report. 13 February 2008 – 30 June 2009. Addis Ababa: USAID. UNESCO Institute for Capacity Building in Africa (ICBA). 2005. In the Framework of the UNESCO Teacher Training Initiative in Sub-Saharan Africa. Country Report. Ethiopia. Paris: UNESCO. Aguirre Division of JBS International, 2009. Evaluation of the President’s African Education Initiative, Ethiopia. Prepared by the USAID Basic Education Program. 2007. Active Learning Practices and Behaviors in USAID/Basic Education Program Supported Primary Cluster Schools and Teacher Training Institutions (2003/04 – 2006/07). Addis Ababa: USAID. USAID Basic Education Program. 2007. Effectiveness of the USAID School-College Linkage Program in Preservice and Inservice Teacher Development Programs in Ethiopia. Addis Ababa: USAID. World Bank. 2005. Education in Ethiopia. Strengthening the Foundation for Sustainable Progress. Washington, DC: World Bank. World Learning. 2004. USAID BESO II - CGPP. Complementary Drought Assistance Program. Closeout Report. World Learning. 2009. School Community Partnership Serving Orphans and other Vulnerable Children (OVC) Affected by HIV/AIDS (SCOSPO). Synopsis. World Learning. 2009. Communities and Schools For Children Affected By HIV/AIDS (CASCAID). Final Technical Report. Impact Evaluation of USAID/Ethiopia’s Education Activities 81 Annex C. Organizations and Individuals Interviewed Washington, D.C. U.S. Agency for International Development Allyson Wainer, Chief, Education Office, USAID/Ethiopia Catherine Powell Miles, Education Advisor, USAID/Africa Bureau Christopher N. Steel, Education Officer, USAID/EGAT/ED Academy for Educational Development Brenda Arrington, Vice-President and Director, Africa Education Programs Elizabeth Leu, Senior Education Advisor, Global Education Center Ken Rhodes, Africa Education Programs PACT Lisbeth Loughran, Vice President, Strategic Management Leonardo Hosh, Programs Director Save the Children, U.S. Margaret Schuler, Associate Vice-President, HIV/AIDS and former Ethiopia Country Director Eric Eversmann, Senior Director, Basic Education World Learning International Development Programs Carlos Sosa, Director of Education, WLID Ezra Simon, Senior Technical Specialist, Child and Youth Programs Gilliam McClelland, Program Officer, Education and HIV/AIDS Programs In-Country Project Schedule with People and Organizations Interviewed Site visit teams: Oromia and SNNPR: Tefera Talore and Nancy Horn Amhara: Ayele Meshesha and Frank Method Somali and Dire Dawa: Ron Bonner and Temesgen Ayele Date Organization Attendees/Titles Purpose 4/16 Chiefs of Party for USAID implementing partners Ed Graybill, COP IQPEP, AED Mamo Mengesha, Country Representative, IFESH Barbara Greenwood, COP CSPP, Save Desta, TEACH Project, PACT Asmelech, TDA Jeremy Cook, COP TELL, AIR Tahir Gero, CSPP, WL Orientation; initial discussion of impact; schedule individual organizational meetings 4/17 USAID Ato Aberra Makonnen, former Chief, BEP, History of USAID 82 Impact Evaluation of USAID/Ethiopia’s Education Activities USAID involvement in basic education; receive advice on field research 4/19 USAID Alysson Wainer, Chief, BES Tesfaye Kelemework, Deputy Chief, BES Demissie Legesse, Project Management Specialist, ABEC Assefa Berhane, Project Management Specialist Review evaluation task order 4/19 MOE, Department of Planning Solomon Shiferaw, Planning & Resource Mobilization Management Process Owner Impact of USAID programs, GEQIP, EDSP III and IV, and related policy concerns 4/20 AED: Ed Graybill, COP Wro Asnakech G/Selassie, Gender Manager Workeye Tegegnu, Deputy COP for Teacher Education Muletu Keffelen, Deputy COP for Operations Bahru Shikur, National Coordinator, Pre￾Service Bonsa Bayisa, National Coordinator, In-Service Kassaw Ali, Deputy COP for Planning BESO and EQUIP impacts and IQPEP work plan 4/20 AIR Jeremy Koch, COP TELL project, and use of TLMP textbooks and teachers’ guides 4/20 PACT Leslie Mitchell, COP Dereje, M&E Zewdu, Director, Education Wzo. Makda, WORTH TEACH I & 2, WORTH, and different strategies and challenges in implementing ABE 4/21 World Learning Wzo. Adanech Kebede, Director, Operations Feleke Desta Tahir Gero CGPP, CSPP, PC3, and CASCAID projects 4/21 IFESH Mamo Mengesha, Country Representative Wro. Makdas, Assistant IFESH activities over the past 10 years 4/22 Save the Children Barbara Greenwood, COP, CSPP Wro Fellekech Bharu, Community Partnership and Gender Advisor Kassaye Yimer, Capacity Building Advisor Getahun Teka, SHN Advisor CGPP, CSPP, and other projects 4/22 Donor Working Group Marianne Kujala-Gracia, new co-chair of DWG Functions and operations of DWG 4/22 Ministry of Finance and Economic Development (MOFED) Demelesh Magersa, Consultant Asnakech Teferra, Bilateral Cooperation Senior Expert MOE financing of education 4/22 Gender Management Process Owner, MOE Wro. Asmaru Berihun Long-term desired outcomes of gender￾related USAID programs 4/23 Teacher Development Program, Oromia RSEB Mamo Bayele, Teacher Development Changes in TE as a result of USAID programs 4/23 TLMP, MOE Tizazu Asare, former Head, MOE, Planning, current leader TLMP Historical development of USAID-supported program in MOE 4/23 National Assessment of Student Achievement (NASA), MOE Tamiru Zerihun and Ato Zewdu Changes in student achievement over time 4/23 MOE Ato Fuad Ibrahim, State Minister Specific impacts of USAID Impact Evaluation of USAID/Ethiopia’s Education Activities 83 programs on the MOE and its operations at Federal and Regional levels 4/23 Teacher Education, MOE Abebe Defersha, Process Owner, Teacher Education and Leadership Training Eshete Asfaw, former Head of TE, Solomon, Senior Expert, TE Impacts of USAID programs on TE 4/23 Tigray Development Association Asmalash Assefa, Deputy Executive Director Tsehaye Fikadu, In-Service Training Specialist Impacts of USAID programs on TDA and beneficiaries 4/23 Adult Education, MOE Tekele Alemu, Expert in Adult Education Colin Holunby, VSO, NFE Advisor Future planning for ABECs and functional literacy programs 4/25 PACT - Zwai environs, WORTH Group Wro Makdas from Pact; 23 WORTH members WORTH program 4/26 WL, Awassa Regional State Education Bureau, SNNPR, Awassa Edget Fana Primary School (first cycle), Awassa Misrak Bar Primary School (first cycle), Awassa Berhanu Manalew, Regional Coordinator, Hassan Abdu Bashir, Head, RSEB Abebe Fayisa, Principal Dessalegn Yuta, Principal Local impacts of WL projects. USAID programs in the region and their impacts on the RSEB, the woreda, TEIs and schools. Impacts of CGPP and PC3 on school, community and enhancing the enabling environment for girls and OVC. 4/26 Amhara RSEB Debre Berhan TTC (informal meeting in Bahir Dar) Woreda in Bahir Dar World Learning, Bahir Dar Teferra Feyissa, Deputy RSEB – process owner for teachers, supervisors, core processes, inservice & preservice Mamuy GebreHiwot, Dean, Debre Berhan TTC Gashaw Admassu, Woreda Education Office Melak Haileleul and Eshete Mengistu Impact of all USAID programs in region: AED programs at TTC; AED and other programs on WEO; Overall WL programs of in Amhara. 4/27 Abela Lida Primary School (first and second cycle) Shebbedino Woreda Education Office, Leku, SNNPR Leku Primary School (Cluster Center and Linkage School) Belay Qaabeto, Principal Asemera Ammenu, Director; Zineba Chasa, Vice Director Asemera Ammenu, Woreda Director; Principal, Leku School Impact of CGPP on girls’ enrollment Impact of EQUIP II training on woreda operations; cluster centers, and teacher training; observe use of PMIS Observe cluster resource center, especially 84 Impact Evaluation of USAID/Ethiopia’s Education Activities Arsi Negelle Woreda, Oromia Toomam Hamid, Vice Director of Woreda and Process Owner of Teacher Development TALULAR; briefly discuss linkage with TTC Impact of BESO II and EQUIP II training on how woreda operates; PMIS challenges 4/27 Sertse Dingle Primary School, Bahir Dar. Woreta Woreda. Woreta elementary ABEC between Bahir Dar and Gondar. Dudmegen Primary School. ABEC in Woreta District. Woldimagne Tarekegne, Principal Emebet Debase and Malkam Guade, Teachers Daniel Sebsibe, WEO Getnet Alemu, ABEC Facilitator Elizabeth Misalew, Principal Impact of USAID programs on principals, teachers, schools and woredas. Operation and success of ABECs. 4/27 BESO/AED Somali Region. Jijiga Woreda Education Office. Somali Region TTC. Dudhidy Lower Primary School, Jigjiga Woreda Karamara Lower Primary School, Jigjiga Woreda Sheik Abduselam Lower Primary School, Gursum Woreda Mohammed Roble, Regional Team Leader and Teacher Ed Expert, AED Girma Besha, WEO Director Ahways Mohammed, Director, and senior staff (group of six) Impact of USAID programs at the region, woreda, TTC and primary school levels. 4/28 Gondar TTC. Checheha Primary School, Gondar Tsadiku Yohannes Primary School Senay Zegaye, Dean, TTC Manene Assefa, Principal Shimeles Birre, Principal AED programs implemented at TTC. Impact of projects at primary school. 4/28 Somali Region TTC. Wilwaal Primary School, Jigjiga. AED Dire Dawa. Dire Dawa RSEB Ato Kamal, Principal Yeshitula Menylshewa, AED IQPEP Regional Coordinator Riyad Hasheem, Director and Ato Wondifraw, Curriculum Head Impact of USAID programs at TTC, primary schools, RSEB and with current IQPEP regional coordinator. 4/29 Masbira Primary School, CSPP and Cluster Center, near Hossanna Girma Bekele Primary School, Hossanna Town, CASCAID World Learning Takele Sumoru, Principal Argenesh Abicho, English Teacher; Amarech Tsawere, Vice Director; Tamerat Koraje, Unit Leader, grades 5-8 Ato Wolde, Zonal Coordinator Impacts of WL’s implementation of CSPP. Impacts of WL’s implementation of CASCAID Overall impacts of CGPP and CSPP in the zone 4/29 Kossoye Primary School, Gondar Gondar Woreda Dessie Abate, Principal Abaligne Mulu, Director Impact of USAID programs at primary schools and woreda. Overview and impact of Impact Evaluation of USAID/Ethiopia’s Education Activities 85 WL, Gondar. Makaunent Yehuwala, Regional Coordinator WL projects in region. 4/29 Jole Belina Cluster School, Dire Dawa Area. Abuker & Drie Xiara Primary Schools, Harari. Model-I and Gai Medrassa Primary Schools, Harari. Harrar TTC Principals and Directors of all schools noted. Impact of USAID programs at all primary schools and TTC. 4/30 Hossanna TTC Omour Doud, Acting Vice Dean Ato Fikre, Math Instructor and Linkage School Math advisor ICT Instructor Practicum Director Impacts of AED programs at the TEIs, at cluster centers, and at the teacher and principal level. 4/30 Dire Dawa RSEB Staff of ICT Unit Impacts of AED IT assistance at RSEB. 5/6 USAID Tom Staal, Mission Director Nancy Estes, Deputy Mission Director Allyson Wainer, Chief, BES Tesfaye Kelemework, Deputy Chief, BES Demissie Legesse, Project Management Specialist Kevin Smith, Program Officer Befekadu Gebretsadek, Project Management Specialist Kibru Mamusha, M&E Specialist Assefa Berhane, Project Management Specialist Presentation of team findings and recommendations to USAID 86 Impact Evaluation of USAID/Ethiopia’s Education Activities Annex D. Summary of USAID Project Assistance to Education, 1994 -2014 Years Project Title Implementing Partners Implementation Regions Beneficiaries Components/Goals 1994- 2001 Basic Education System Overhaul (BESO I) AED WL TDA SNNPR Tigray • School Mgmt. Cmtes. (SMC) • TTCs • RSEBs • Woredas • Community/School Activities Program (CSAP) • Pre-service Teacher Training • Community-government partnerships • Strengthening the decentralized management and administration, increasing the efficiency of educational financing • Strengthening systems for managing personnel • Curriculum development and instructional materials • Monitoring and evaluation • Improving the school environment and strengthening teacher-learner support systems. 2002- 2003 Complementary Drought Assistance (CDA) WL TDA 6 drought-affected Regions 361 schools, 153,000 children • School Feeding • School Supplies dissemination • Maintaining school stability 2002- 2007 Basic Education Strategic Objective/Community￾Government Partnership Program (BESO II/CGPP) AED WL TDA Save BESO All 9 regions, Addis Ababa and Dire Dawa CGPP • Save, 1500 schools, Afar, Gambella, Oromia, and Somali • TDA,400 schools in Tigray • WL, 1800 schools in Amhara, Benishangul￾Gumuz, SNNPR BESO • 17 TTCs • RSEBs • Woredas CGPP • 3700 schools • PTA • GEAC • KETB BESO • Pre-Service Teacher Training • In-Service Teacher Training • School Leadership Training • Supplementary Media Development and Training • Socially Relevant Materials • Capacity Building of the MOE and RSEBs • Capacity Building of Staff and Development of the Personnel Management, Planning and Monitoring, Evaluation and Information Systems • Monitoring, Evaluating, Reporting, Analysis (MERA) CGPP • School Improvement Awards (SIA) • Community Mobilization • Training of PTAs • Establishment of GEACs and improvement of girls’ education enrollment and retention Impact Evaluation of USAID/Ethiopia’s Education Activities 87 • Strengthening KETBs • School Development Agents (SDA) to mentor community leaders 2003- 2010 Capacity Building in Teacher Education (CBTEP) (a volunteer program) IFESH All Regions and Cities • TTCs • MOE Depts. • Improve English language teaching • Provide assistance to TTCs and the MOE in improving the quality of primary education • Improve HIV/AIDS education at TTCs and in surrounding communities • Provide English language books, reference materials and other resource materials for schools and resource centers. 2004- 2009 (2006- 2008) Transforming Education for Children and Adults in the Hinterlands (TEACH 1) WORTH (Pilot Project under TEACH 1) PACT and 27 local partners PACT and several local CBOS Hard-to-reach children and adults in 8 Regions Selected Regions Out of school children; adults 9,000 women • ABE – Alternative Basic Education (condensing Cycle 1 curriculum for children to prepare them to enter grade 5 in regular schools) • AFL – Adult Functional Literacy • Woreda Capacity Building to manage ABE and AFL • Development of functional literacy and income generating skills through the use of materials focusing on income generation • Development of self-help groups to generate savings and group lending 2004- 2009 Positive Change: Children, Communities and Care (PC3) (Program of Support for OVC) Save WL FHI CARE World Vision 35 local NGOs 550 local CBOs Schools in Addis Ababa, Amhara, Afar, Benshangul-Gumuz, Dire Dawa, Oromia and SNNPR • OVCs • Local NGOs • Local CBOs • Increased availability, quality and consistency of community-based support services for OVC and families affected by HIV and AIDS • Improved capacity of Ethiopian Civil Society Organizations (CSO) to plan, implement, monitor, evaluate, manage and report on OVC programs. services • more supportive environments for OVC and their households developed through strengthened coordination, networking and advocacy 2005- 2006 Kokeb Kebele Project/ Model Kebele Initiatives WL Pathfinder ESHI 20 kebeles in 5 woreda in SNNPR and a similar number in Amhara • Health care providers • Kebeles • Schools To develop kebele-based education-health care provider partnerships in schools to: • Improve effective access to and linkages across schools and community health centers • Provide immunizations in selected kebeles • Provide water supply to selected kebeles 88 Impact Evaluation of USAID/Ethiopia’s Education Activities • Provide family planning education • Reduce girls drop-outs from primary school • Increase the number of household pit latrines • Establish hand washing facilities in all households and schools after using the toilet • Introduce environmental and personal hygiene • Raise awareness of HIV/AIDS 2005- 2009 President’s African Education Initiative (AEI) AED (TT) AED (AGSP) and 7 local partners Alabama A&M University (TLMP) All Regions and Cities TT • TTC • Woredas • RSEBs AGSP 1000 girls grades 7 to 12 TLMP Teachers and students in grades 6, 7, 8 TT Goals same as AED portion of BESO II AGSP Provide scholarships, material support, and housing/ feeding to 1000 at-risk girls to continue their education through high school TLMP In a collaborative manner with the MOE Curriculum Department, develop English language textbooks for grades 6, 7, 8 (and 1) for all schools in Ethiopia 2006- 2008 Communities and Schools for Children Affected by HIV/AIDS (CASCAID) WL Amhara Oromia SNNPR 100 primary schools 5,000 OVC • Increase access to primary education for OVC • Provide essential educational services to enhance success • Strengthen school/community groups to identify and support OVC • Facilitating access to additional support services for OVC 2008- 2009 Building the Capacity of Primary Education Managers and Educators (EQUIP II) AED All regions and cities Woredas KETBs Principals TTCs Linkage Schools • Capacity building in planning, management and financial management • Development of and training on several types of MIS packages • Capacity building for curriculum developers, especially in physics and math • Training on continuous assessment • Capacity building on monitoring and evaluation 2008- 2011 Community-School Partnership Program (CSPP) Save WL TDA Afar, 100 Amhara, 400 Benishangul￾Gumuz 80 Gambela 70 Oromia 550 Somali 150 SNNPR 300 1800 primary schools • Capacity building of PTAs, GEACs, KETBs, and woredas in support of health, sanitation, and school-based water development • Providing SIAs to PTAs for school improvement • Facilitating the growth of school gardens for Impact Evaluation of USAID/Ethiopia’s Education Activities 89 Tigray 150 income generation in support of OVC • Through health extension workers, de-worming and HIV/AIDS awareness raising, and sanitation practices • Constructing separate latrines for boys and girls 2008; 2009- 2011 Teaching English for Lifelong Learning (TELL) AIR All regions and cities Teachers in grades 1, 2, 3, 4, 6, 7, 8 • Provide workshops on the use of TLMP-produced English language books and Teachers’ Guides 2009- 2012 School –Community partnership serving HIV/AIDS affected OVCs (SCOPSO) WL All Regions and Administrative Cities 40,000 OVC in 400 primary schools • Provision of quality, comprehensive services to 40,000 HIV affected or infected OVC with a focus on increased enrollment, retention and academic performance • Strengthen school – community partnership in 400 primary schools to enhance their capacity to plan and manage OVC support programs. 2009- 2013 Transforming Education for Children and Adults in the Hinterlands (TEACH 2) PACT and 27 local partners Children and adults primarily in pastoralist areas in 8 Regions Out of school children; adults • ABE – Alternative Basic Education (condensing Cycle 1 curriculum for children to prepare them to enter grade 5 in regular schools) in remote areas where there are no government schools • AFL – Adult Functional Literacy • Woreda Capacity Building to manage ABE and AFL • Strengthening facilitator training 2009- 2014 Improving the Quality of Primary Education Program (IQPEP) AED All regions and cities TTCs Woredas RSEBs • Curriculum and textbook assessment • Pre- and In-service teacher development • School improvement • Management and Administration (MAP) at the RSEB, woreda, KETB and principal levels • Expand computer capability • Program coordination and monitoring and evaluation • Early reading assessments • Conduct 9 different research projects • Support National Learning Assessments 90 Impact Evaluation of USAID/Ethiopia’s Education Activities Annex E. GEQIP Parameters 1. Teacher Improvement/Teacher Development Program (TDP): “To increase the supply of effective teachers, trainers and facilitators.” Planned outputs are: • PP1: Teacher Educators: “Knowledge, expertise, motivation and conduct of teacher educators improved.” • PP2: Pre service and Selection: “Teacher skills enhanced via extended duration and quality of Pre￾Service teacher training, improved selection procedures and Practicum” • PP3: In Service: “Pedagogical knowledge and capacity of teachers improved within a formal structure of career-long development” • PP4: Teacher Education System: “Organization and management system for teacher education improved.” • PP5: ELQIP: “Quality of English language teaching and teaching of core subjects in English improved” • PP6: Non-Formal Education/ABE Training: “The quality and reach of Non Formal Education will be improved and the supply of effective facilitators will be increased” • PP7: Special Needs Training: “Training teachers in SNE screening and implementing, strengthening SNE Pre-Service and In-Service Teacher Training and provision of SNE Education Materials to TEIs and Cluster Resource Centres • PP8: Early Childhood Development: “Production of high quality materials and the training of sufficient numbers of trainers and ECD teachers” TDP I specifically addressed the need for gender balance via the creation of targets for numbers of female teachers trained in the Higher Diploma Program. However, under GEQUIP, TDP II gender issues and targets will be more comprehensively addressed through all priority programs 2. Curriculum, Textbooks and Assessment • To develop a curriculum which is responsive to the economic and democratic needs of the country, international economic conditions, which promotes gender equality, and which is of a better quality • To ensure that the core subjects are the basis for effective teaching and learning in other subjects • To improve the teaching and evaluation methods of teachers, and to produce high quality teaching aids • To improve the development, printing and distribution of teaching and learning materials • To improve inspection, student assessment and examinations systems 3. Management Administration Program (MAP) • To effectively and efficiently deliver education services to the end user through decentralized planning, management and administration • To establish open, transparent, low cost and productive management and administration systems • To establish systems to facilitate effective horizontal and vertical communications within Government and with stakeholders Planned outputs are: Impact Evaluation of USAID/Ethiopia’s Education Activities 91 • PP1: Education Planning, Finance, Monitoring & Evaluation: to build capacity for regional and woreda level planning and budget analysis, and to strengthen systems for resource allocation and transfer. • PP2: School Planning and Management: to build capacity in school planning, management and monitoring with a strong emphasis on community participation and accountability mechanisms. The Leadership and Management Program (LAMP) which was started under TDP will be further developed and expanded to build the capacity of school principals and supervisors in planning and management (leading to a certificate and diploma qualification). • PP3: Institutional and Organizational Development: Business Process Reengineering (BPR) to develop a more service-oriented and outcome focused approach; the ongoing revision of the Woreda Program Implementation Manual (PIM); improvement of physical IT capacity at woreda level and training support. • PP4: Human Resource Management: To strengthen the implementation of HRM procedures for improved management, performance and accountability through the further development and rolling out of the Personnel Management Information System (PMIS) at the centre, regions and woredas. A pilot program will be established in selected departments, RSEB’s and WEO’s to increase the proportion of women in management and leadership positions to enhance gender equity in leadership. 4. School Improvement Program • To improve student learning achievement outcomes • To ensure democratic, participatory, open and accountable school management and administration • To facilitate greater freedom in school management and administration through decentralization of responsibilities 5. School Grants Program The MOE Blue Book outlines the levels of grant that schools should receive as 10 Birr per year for every Grade 1-4 child that is enrolled in school, 15 Birr per year for every Grade 5-8 child, 20 Birr per year for every Grade 9-10 child and 50 Birr per year for every Grade 11-12 child. The purpose of this grant is to cover schools operating costs and augment non-salary expenditures. Pooled Fund To meet the budgetary requirements of achieving the goals developed within each of these pillars, MOE established a “pooled fund” mechanism to which any/all donors could contribute. Like UNICEF and JICA, USAID does not contribute to the pooled fund. With GEQIP as the overall framework, ESDP III was developed to give specific direction to educational development beyond the progress made under ESDP II. ESDP IV is currently in draft form and seeks to further improve all elements identified under GEQIP, with additional foci on early childhood education, increasing reading ability, and improving English language skills. Under this guiding document, the GFDRE seeks to reach the Millennium Development Goals (MDG) and to create a vibrant interactive learning system in which all children will have the opportunity to learn how to become productive citizens. 92 Impact Evaluation of USAID/Ethiopia’s Education Activities Annex F. Statistical Data 1. Access and Equity Indicators (School age population, GER, NER and Children out of school) Table 1: School Age (7-14) Population at National level by year Year Male Female Total 2004/5 3,948,118 3,859,474 7,807,592 3,318,612 3,216,157 6,534,768 Total 7,266,730 7,075,631 14,342,360 2005/6 4,122,872 4,039,205 8,162,077 3,345,841 3,245,241 6,591,082 Total 7,468,713 7,284,446 14,753,159 2006/7 4,217,224 4,132,570 8,349,794 3,518,339 3,422,060 6,940,399 Total 7,735,563 7,554,630 15,290,193 2007/8 4,273,895 4,122,037 8,395,932 3,892,967 3,761,176 7,654,143 Total 8,166,862 7,883,213 16,050,075 2008/9 4,395,458 4,240,066 8,635,523 4,002,323 3,867,785 7,870,108 Total 8,397,781 8,107,851 16,505,631 Source: MoE (2005/6—2008/9) Statistical Abstract Impact Evaluation of USAID/Ethiopia’s Education Activities 93 Table 2: School Age Children within the School and Outside the School Years # of School age Children Children(age 7-14) in Schools Children (7-14) out of Schools in Number in % in Number in % 2004/05 14,342,360 11,448,641 79.8 4,500,000 20.2 2005/06 14,753,159 13,474,674 91.3 1,278,485 8.7 2006/07 15,290,193 14,014,276 91.7 1,275,917 8.3 2007/08 16,050,075 15,340,788 95.6 709,287 4.4 2008/09 16,506,040 15,549,524 94.2 956,516 5.8 MoE (2005/6-2008/9) Statistical Abstracts Table 3: School Age Children within the School and Outside the School by Sex Year In the Primary School system Out of the Primary School System Male Female Total Male % Female % Total % 2004/05 2,944,340 2,320,347 5,259,683 4,322,389.9 48 4,755,283.7 52 9,077,674 63 2005/06 7,364,151 6,111,650 13,469,634 104562.0 8 1,172,795.8 92 1,277,358 9 2006/07 4,132,880 3,516,817 7,656,761 3,602,683.5 47 4,037,812.9 53 7,640,496 50 2007/08 3,535,931 3,096,964 6,635,021 4,630,931.3 49 4,786,248.7 51 9,417,180 59 2008/09 7,549,909 6,852,049 14,403,362 8,47871.5 40 1,255,801.6 60 2,103,673 13 Note: Calculated as per MoE (2005/6—2008/9) Statistical Abstracts 94 Impact Evaluation of USAID/Ethiopia’s Education Activities Table 4: Net Enrollment Rate (NER) at Primary level (1-8) by Region Regions NER at primary (1-8) Male (%) Female (%) Total (%) Tigray 95.6 98.1 96.9 Afar 25.3 23.2 24.4 Amhara 101.4 103.1 102.2 Oromia 80.9 74.8 77.9 Somale 33.3 29.4 31.6 Benshangul 97.0 80.1 88.6 SNNP 94.3 84.5 89.4 Gambella 80.2 69.7 75.2 Harari 100.2 83.6 91.9 Addis Ababa 78.2 74.4 76.1 Dire Dawa 76.5 70.2 73.4 Total 84.6 81.3 83.0 MoE (2008/9) Statistical Abstract Impact Evaluation of USAID/Ethiopia’s Education Activities 95 2. Key Quality Indicators on the Provision of Primary Education (Active Learning Methods, Learners’ Assessment Results, Survival and Completion Rates) Table 1: Average percentage of teaching in using Active Learning Methods in AED clusters primary schools Plan vs actual results (2003/4 −2005/6) Year Projected/Planned (A) Actual (B) Grade-1 Grade-4 Grade-8 Total Average % Grade-1 Grade-4 Grade-8 Total Average % 2003/4 (1st Impact year) 30% 30% 28% 29.3 53% 53% 56% 53.9 2004/5(2nd Impact year) 60% 59% 62% 60.3 61% 66% 56% 61.0 2005/6(3rd Impact year) 70% 69% 72% 70.3 73% 70% 68% 70.6 AED (2005/6) PMP Indicator Report submitted to USAID Table 2. Summary of Learners Assessment Test Scores in Formal Primary Schools (FPS) and Pact-ABECs No. Subjects NLA-1st cycle Primary ABE-LA for Level -1 Mean Score SD Mean Score SD 1 Mathematics 40.3 16.98 64.1 23.6 2 Reading 43.9 20.39 59.2 25.5 3 English 36.5 15.73 52.4 23.7 4 Env. Science 42.6 16.35 60.5 25.4 5 Composite 40.9 11.77 59.0 16.6 NLA (2008) for grade 4 completers and ABE_LA (2008) for Level-I Students 96 Impact Evaluation of USAID/Ethiopia’s Education Activities Table 3: National Survival Rate at Grade 5 Year Boys Girls Total 2000/1 44.5 43.7 44.2 2001/2 41.8 39.0 40.6 2002/3 37.7 40.3 38.8 2003/4 51.7 55.6 53.4 2004/5 57.2 61.9 59.3 2005/6 55.2 58.0 56.4 2006/7 54.7 59.4 56.8 2007/8 45.8 53.8 49.2 MoE (2005/6—2008/9) Statistical Abstracts Table 4: National Primary Completion Rate by Gender Year Grade 5 Grade 8 Boys % Girls % Total % Boys % Girls % Total % 2001/2 52.2 31.5 42.0 23.5 13.7 18.7 2002/3 56.9 36.4 46.8 30.2 16.5 23.5 2003/4 60.0 42.2 51.3 34.9 19.1 27.1 2004/5 65.2 49.5 57.4 42.1 26.3 34.3 2005/6 69.2 56.0 62.7 50.1 32.9 41.7 2006/7 71.6 61.6 66.6 51.3 36.9 44.2 2007/8 71.7 67.0 69.4 49.4 39.9 44.7 2008/9 79.4 78.4 78.9 48.4 40.5 43.6 MoE (2005/6—2008/9) Statistical Abstracts Impact Evaluation of USAID/Ethiopia’s Education Activities 97 Table 5: SAP, Enrollment, GER, # of students graduated from ABECs and joined the formal Schools in Pact Catchments * Since data on dropouts and repeaters number is not available), it is not possible to show the total (2005—2009) USAID-PACT TEACH Project Records Table 6: School Survival Rates at National level (Gov., NGOs + USAID assisted) & in USAID cluster schools at grade 5 by gender MoE (2004/5). Statistical Abstracts and AED (2005/6) PMP Indicator Report submitted to USAID Year School age population (SAP) Enrolled in ABECS Transferred to formal School Total GER 2005 63,421 34,936 0 34,936 55.1 2006 102,052 65,801 0 65,801 64.5 2007 104,725 87,042 0 87,042 83.1 2008 107,469 83,185 13,156 96,341 77.4 2009 110,285 77,931 15,622 93,553 70.7 Total 487,952 * 28,778 377,673 71.5 Year In all Schools (Including Non-USAID supported) In USAID Supported Schools Grade 5 Grade 5 Grade 8 Boys % Girls % Total % Boys % Girls % Total % Boys % Girls % Total % 2001/2 41.8 39.0 40.6 41.8 39.0 40.6 25.8 39.0 25.0 2002/3 37.7 40.3 38.8 39.0 35.2 34.2 23.4 19.0 21.7 2003/4 51.7 55.6 53.4 58.2 61.3 59.6 46.7 52.6 49.2 2004/5 57.2 61.9 59.3 50.1 54.9 52.3 33.7 39.8 36.3