USAID / ZAMBIA STRATEGIC OBJECTIVE TWO CLOSE OUT REPORT SO Name: Improved Quality of Basic Education for More School-Aged Children SO Number: 6110002 Approval Date: September 15, 2000 Geographical Area: Zambia Total Cost: $17,267,831.00 Access to Basic Education Child Survival 8,891,339.00 Development Assistance 8,376,492.00 _____________ Total USAID Funding $ 17,267,831.00 Principal Implementing Partners Creative Associates International Education Development Center Academy for Education Development American Institutes for Research UNICEF Macro International Background SO2 was USAID’s initial involvement in the education sector in Zambia. In late 1998, USAID participated with the Government of Zambia (GRZ) and its cooperating partners in appraising and refining Zambia’s plans for a Basic Education Sub-Sector Investment Program (BESSIP). Through the implementation of BESSIP in 1999, the Government of Zambia began to address many problems within the education system that had seriously deteriorated during the past twenty years due to economic decline, the lack of resources and institutional inefficiencies. Problems included undertrained teachers, lack of teacher incentives low attendance of girls in school, poor health and nutrition of school children, HIV/AIDS, increasing numbers of school-aged orphans and inability of families to pay school fees as well as a lack of textbooks, desks and sanitation facilities. Under a Limited Scope Grant Agreement (LSGA) followed by the Strategic Objective Agreement (SOAG) for Strategic Objective 2 (SO2), USAID catalyzed and strengthened Ministry of Education (MOE) support for several activities whose main purpose was to improve quality of basic education for more school-aged children. The S02 SOAG was USAID's principal instrument for obligating funds for education activities and through which USAID provided assistance to support basic education in Zambia. Activities conducted under SO2 were in the following areas · School Health and Nutrition (SHN) · HIV/AIDS prevention and mitigation · School and community sensitization and mobilization · Gender and Equality (support for vulnerable children). · Education Management Information System (EMIS) development · Interactive Radio Instruction (IRI) for out of school children · Institutional Development · Assumptions: The key assumptions during this period were that the GRZ and other donor support would continue through USAID’s and the MOE’s strategic plan period. Decentralization and restructuring would continue to be a part of the overall Government of Zambia reform program. USAID / Zambia Close Out Report 6110002 2 The Ministry would have adequate levels of staffing to implement its BESSIP activities, especially in the Planning Unit. The mitigation of the effects of HIV/AIDS would continue to be a part of the overall Government of Zambia strategy. · Most of these assumptions were valid. Donor support remained constant throughout this period. Decentralization, however, was not taken up seriously by the MOE or donors. Little was accomplished in this area. The Ministry had sufficient number of personnel to meet basic requirements but the skill level required to make significant impact were lacking. The Planning Unit continues to struggle to meet basic requirements and is overwhelmed with demands. HIV/AIDS continued to have a devastating impact on the MOE workforce but until recently the MOE HIV/AIDS response has been superficial. The implementation of HIV interventions were few in number and not effective. Summary of overall impact at SO level and IR level The achievements of SO2 have provided a sound basis for continuation and expansion of USAID support to the MOE. The follow on SO6 includes the same fundamental portfolio that was started within SO2 period; i.e., Interactive Radio Instruction Program, School Health, HIV/AIDS prevention and Education Management Information System (EMIS). Each of these specific interventions, however have been significantly expanded and new components have been added. For example, the original EMIS program has been rolled into the Leader With Associates (LWA) EQUIP 2 program that now includes policy formulation, decentralization, institutional capacity development, HIV/AIDS workplace program and student s’ continuous assessment intervention. The EQUIP2 program as a result the SO2 ground work remains to be directly linked and embedded within the MOE headquarters structure. The overall achievements of SO2 were positive and established productive working relationship with the Ministry o Education, a consequence USAID remains in a strong position to support the MOE implement its strategic plans. In addition, USAID is a solid contribution to the harmonization efforts among donors. IR1: Improved participation of girls and other vulnerable children. 2000 Good progress was made in 2000 in the delivery of interventions of the USAID-supported, UNICEF￾implemented program for the Advancement of Girls’ Education PAGE). From 1999 and 2000, targeted PAGE schools increased overall enrollment by 12.6% with a slight increase in the percentage of girls enrolled (from 47.6% to 47.9%). Learning environments of the 42-targeted PAGE schools were improved through the provision of anti-HIV/AIDS materials, education kits and trained teachers to use them, and guidelines and teacher training on creating girl-friendly schools. Thirty-five schools have improved water supply through provision of new boreholes, reaching over 10,500 pupils. Water, sanitation and hygiene education (WASHE) community committees were trained to maintain boreholes/hand pumps, build latrines, and prevent malaria and HIV/AIDS. Expectations were exceeded in USAID’s new interactive Radio Learning Center (IRLC) activity, providing access to grade 1 basic education to over 1,200 children, as compared to the target of 800. 2001 During FY2001, USAID support for the interactive radio instruction (IRI) program expanded from 21 IRI centers with a total enrollment of 1,254 pupils in two provinces in 2000 to a total of 169 centers (target 171) in all 9 provinces with a total enrollment of 7,782 in 2001. The number of children enrolled in USAID-supported basic education schools and IRI centers dramatically increased from 12,565 pupils in 63 basic education sites in 2000 (40 government schools, 2 community schools, and 21 IRI centers) to 37,140 pupils enrolled in 256 basic education sites in 2001 (85 government schools, 2 community schools, and 169 IRI centers). USAID / Zambia Close Out Report 6110002 3 An assessment of Grade 1 learners from a sample of 405 pupils in 30 IRI centers (18% of 169 centers) indicated that pupils learned from the lessons. The literacy skills assessment, based on the MOE’s Grade 1 curriculum, indicated a mean score performance of pupils of 56.5% as compared to a target of 47% and the conventional primary school standard set for “passing” of 50% or better. USAID assisted the MOE to develop its first HIV/AIDS strategic plan, which led to its 2001 HIV/AIDS annual plan. The EMIS strategy incorporated the development of district and school-level HIV/AIDS data to help planners understand how the educational system was affected in order to address these effects. The teacher-training component of SHN taught life skills so teachers could transfer important attitudes, knowledge and skills to their pupils regarding HIV/AIDS. The community mobilization effort integrated HIV/AIDS information to ensure that parents and community leaders consider what HIV/AIDS is, how it affects life in Zambia, and measures to prevent their children, especially girls, from the disease. In addition, the IRI program provides basic education to out-of-school AIDS orphans and other vulnerable children who have no access to government schools. 2002 In 2002, the USAID basic education framework within the new Country Strategic Plan 2004-2010 evolved to : Improved quality of learning environments. A part of this IR was Community Sensitization and Mobilization Campaign (CSMC): Communities in 25 school catchment areas in 5 districts were sensitized during CSMC research and verification activities. That resulted in forty zonal (sub-district) multi-sector teams consisting of 5,818 teachers, community leaders, PTA members, and villagers (from 138 villages) being sensitized and mobilized to take action concerning girls’ education and HIV/AIDS. 203 provincial, district, and zonal officials were trained in gender and equity issues in education, HIV/AIDS awareness, and community mobilization. This effort led to the development of 65 zonal and 139 community-level action plans to address local issues related to getting more girls in school and mitigating the effects of HIV/AIDS on children. Interactive Radio Instruction Program (IRI): The IRI program to deliver basic education to out-of-school children developed 200 Grade 3 lessons and over 130 of 200 Grade 4 lessons in CY2002, and broadcast lessons for Grade 1 through Grade 4 pupils. The quality of programs improved as a result of the installation of digital production equipment and the successful training of technicians who, initially, were not computer literate. The digital equipment helped improve recording quality and editing precision and efficiency of radio program production. The number of IRI centers throughout Zambia rose from 169 to 369, with a total of 11,498 children enrolled (5,746 girls; 5,752 boys). Program for the Advancement of Girls Education (PAGE): The first education activity of the Mission, ended in December 2002. Enrollment of girls in the PAGE schools increased between 1999 and 2001 by 14 percent while the boys’ enrollments increased by 18 percent. Girls comprised 49% of the children enrolled in USAID-supported schools, which is indicative of the over-all proportion of girls in primary schools.) 2003 The national school census indicated that the primary school population (7 to 13 years) in CY2003 was 1,808,090, an increase of more than 76,500 students compared to the previous year. Girls’ gross enrollment ratio has remained steady at 85% compared to 91% of boys. Overall, more students entered school. USAID support to primary schools expanded dramatically, reaching 212,624 students in CY2003, compared to 55,156 in CY2002.This increase could be attributed to successful community sensitization programs that promote primary education, particularly for girls. The Net Admission Rate in these schools for grade one, was however, less than expected with a slight decrease from 33% in CY2002 to 31% in CY2003. Thirty eight thousand students (45% girls) completed grade seven in the schools supported by USAID, a completion rate of 63%. This was below the national completion rate of 71%, but a significant achievement considering the programs were operating in remote, hard-to-serve areas. USAID / Zambia Close Out Report 6110002 4 2004 The number of IRI centers increased from 21 in FY01 to 647 in FY04, with a corresponding increase in enrolment from 1,250 children in FY01 to 38,513 children (19,101 girls; 19,412 boys) in FY04. Of the 647 centers, 188 are community schools using IRI methodology. This increase in enrolment means that more than 39,000 children who don’t have access to government schools are receiving an education. Community Sensitization and Mobilization Campaign (CSMC) expanded to all 661 basic schools and 67 community schools in the Southern Province, reaching a total of 116,275 students (56,590 females, 59,685 males). A total of 5,818 teachers, community leaders, PTA members and villagers (from 138 villages) have been sensitized and mobilized to take action. Two hundred and three (203) provincial, district and zonal officials were trained in gender and equity issues, in education, HIV/AIDS awareness, and in community mobilization techniques. Sixty-five (65) zonal and 139 community action plans were developed to address issues related to getting more girls in school and mitigating the effects of HIV/AIDS on education. Over 12,000 children have benefited. IR2: Increased school-based health and nutrition (SHN) interventions to support pupil learning. 2000 SHN intervention pilot test begun in Eastern Province. FY 2000 was the baseline year for this activity 2001 was the year the pilot testing of the school health and nutrition interventions commenced. Progress had been made in laying the foundation for the pilot test activity, including the development of multi￾sectoral agreements between the MOE and the Ministries of Health and of Community Development and Social Services; completion of a SHN situation analysis; orientation of personnel of government departments in Eastern Province, selection and site visits of the pilot test schools, and development of pupil health cards, drafted cognitive assessment items, and other measures for monitoring and evaluation the intervention’s success. 2001 USAID made significant progress in assisting the MOE to introduce SHN interventions in 40 schools in Eastern Province and set up a pilot SHN delivery system in 20 of those 40 schools for the treatment of worms and provision of micronutrients. Preliminary analysis of SHN baseline data collected on pupils in the 20 pilot schools indicated that half of the surveyed schools had a greater than 50% positive rate for urinary schistosomiasis, while considerably more than half of the schools have a greater than 50% positive rate for other worm infestation. Japan has partnered with USAID for the first time in education for this activity by funding the MOE’s purchase of all the de-worming and micronutrient supplies required for the USAID supported delivery system. 2002 The School Health and Nutrition activity developed cost-effective but rigorous instruments and protocols to conduct a baseline survey of health and nutrition of school going children. This included height poles that allow teachers to determine appropriate dosages of tablets to give to pupils based on their height, a simple questionnaire that enabled children to self-diagnose whether they have bilharzia, and a cognitive assessment instrument (CAI) to detect the link between health interventions and children’s cognitive function. A follow-up post-intervention survey was carried out six months later. 2003 In the Eastern Province 110 schools implemented health and nutrition interventions (compared to 50 in CY2002 and a target of 60). These health interventions implemented by partner organizations focused on teacher managed de-worming efforts and eradication of bilharzia infestations plus the provision of vitamins. The results of these efforts not only indicated improved health of the children but through USAID / Zambia Close Out Report 6110002 5 specifically designed tests, demonstrated significant improvements in the cognitive performance of the students. These results reinforced the value of cost-effective school managed health interventions. In CY2002 a total of 20,596 students (10,119 girls; 10,477 boys) participated in the second phase of this health program. In CY2003, the program reached a total of 44,267 primary school children (21,333 girls; 22,934 boys), which is more than double the target of 21,000. This increase was due to the acceptance of the program and its rapid expansion of coverage at the request of the MOE. Teachers, community health workers and representatives from the District Health Management Teams, for the first time, are routinely worked together to maintain proper health records; monitor the provision of the drugs; and educate parents. 2004 Teacher-managed de-worming of students, eradication of bilharzia infestation plus the provision of vitamins have been the primary focus of the USAID supported partner organization. These low-cost, school-managed interventions proved popular with communities, school personnel and students. In FY04 a total of 306,815 primary school students (151,267 girls 155,554 boys) from 555 schools participated in the school heath program, over five times the planned target of 140 schools and 56,000 students. In FY04, the Schistosomiasis Control Initiative (SCI) was also initiated in the Southern Province in conjunction with the community sensitization and School Health program. The SCI program began in 60 schools in three districts reaching 14,666 community members on how to prevent and treat schistosomiasis. The SHN component has reached nearly 306,815 children in 555 schools over five times the planned target of 140 schools and 56, 000 students. A survey conducted in Eastern Province among the targeted schools show SHN interventions have had positive impact: hookworm prevalence among school children was reduced from 55% to 8%, and bilharzias prevalence was reduced by half from 48% to 24%. IR3: Improved information for education decision making 2000 USAID, as a main donor for the EMIS activity, helped the MOE analyze report findings from a backlog of school census data, resulting in “Trends in Primary Education 1996-1999” and showing that primary education enrolment increase had stagnated at around 1%, though school-age population increased by 3.1% per year. Other EMIS achievements included a situation analysis, development of a two-year EMIS plan, and commencement of the MOE’s EMIS re-engineering process. 2001 USAID’s support for Zambia’s National Census in 2000, combined with the short term technical assistance provided to the MOE for its education management information system (EMIS), led to the establishment of an EMIS platform and a significant improvement in the accuracy of education data available to education planners for making better-informed decisions. Unlike previous years when Basic Education Sub-Sector Investment Program (BESSIP) partners, including the World Bank, disputed MOE education statistics reports, BESSIP partners at this year’s annual review accepted the report without requiring modifications. The national census in 2000 school-age population figures revealed that the primary school population (the 7-13 year age range) had been underestimated by about 250,000 and therefore the past gross and net enrollment rates from the MOE were artificially high. The most recent school enrollment number is 1,589,544 pupils for 2000. Boys’ enrollment still exceeds that of girls but the gender gap has slowly decreased to 7.4%, as compared to an 8.8% gap in 1996. USAID / Zambia Close Out Report 6110002 6 2002 For the second year, the Education Management Information Systems (EMIS) activity exceeded its target of “this year’s school census data this year” with four months to spare. In addition, a statistical trends report was published by the end of the 2002 calendar year. Improved EMIS management in the Planning Unit ensured printing and distribution of annual school census forms and coordination of training of more than 4,432 provincial, district and school level staff to complete the forms. The Planning Unit also distributed the 2003 annual school census questionnaire at the end of 2002 in anticipation of the 2003 exercise. Additionally, a computer training facility with 20 workstations is near completion. As part of the move toward decentralization, it has been established to train provincial and district planning staff on the EMIS system and to place good information at the fingertips of decision-makers in District Education Boards, district education offices, schools and communities. The Education Data Survey, an amplified module of the Zambia Demographic Health Survey covered 4,245 households, 3,975 parents/guardians and 8,027 children of primary school age. It confirmed the net enrolment rates reported by the MOE; identified monetary reasons as a primary factor behind low attendance, dropout and absenteeism; and reported that parent/guardians overwhelmingly recognized the importance of schooling for both boys and girls and were generally satisfied with the schooling they received. It also found that parent/guardians were more supportive of including AIDS education in the basic education curriculum than sex education, but would generally support both if introduced in the upper grades. 2003 USAID was uniquely positioned, for the past two years, in providing technical assistance to the MOE to establish a management information system. The achievements had been impressive and for the first time ever the MOE had reliable electronic data on the entire national school system. In addition required data was being submitted in a timely manner making it possible for MOE planners to coordinate their efforts with the Ministry of Finance and National Planning and others. All the collaborating donors supporting the education sector were utilizing the data generated to allocate their funds. A significant achievement was the establis hment of a computer network in the central MOE offices, for the first time linking all six of the key departments. In CY2002, the MOE headquarters was the only unit considered even remotely capable of generating meaningful reports on the status of the education system. In CY2003 the targets for education management units producing Annual School Census reports and managing data effectively were met; this included reporting at the national level, four reports at the provincial level and 19 reports at the district level. A feasibility study was completed on how to expand the network throughout all entire nine provinces in the country. 2004 The following was achieved in 2004: · EMIS units were able to produce periodical reports, 19 of the targeted 47 districts are able to produce regular education reports and only four of the provinces out of the targeted six can provide summary reports. · Improved EMIS management ensured timely printing and distribution of annual school census forms. More than 4,432 provincial, district and school level staff have been trained. Statistical trend reports are now being published regularly and by the end of calendar year .Annual school census statistical report were produced on time (4 months earlier than targeted time) HIV/AIDS Interventions USAID began planning the implementation of an HIV/AIDS workplace program for MOE employees late in 2004. This initiative, funded by The President’s Emergency Plan for AIDS Relief (PEPFAR) was designed to increase the staff acceptance and use of Voluntary Counseling and Testing (VCT) services and USAID / Zambia Close Out Report 6110002 7 Anti-Retroviral Therapy (ART). USAID maintained the Ambassador Girls' Scholarship Program (AGSP) and took the initiative to include PEPFAR funds to support school level HIV/AIDS interventions and expanded the provision of scholarships. At present the USAID Africa Education Initiative (AEI) funds support 693 orphaned children (620 girls) in grades eight to twelve. Significant changes in the Results Framework during the life of the SO. Changes in the original Results Framework included (i) the intermediate result planned for HIV/AIDS was dropped as it was premature given the development of program activities (ii) the performance indicator on the Net Admission Rates were dropped because data was only available at the national level and not the district level. Summary of activities used to achieve the SO and their major outputs. Programme for the Advance of Girls Education PAGE aimed at creating a girl-friendly environment to promote the greater participation (access, persistence and achievement) of both girls and women (as teachers) in education. PAGE provided training and resources for; the use of appropriate pedagogy to promote girl’ learning outcomes; the involvement of parents in providing a good education for their children, especially girls; the promotion of leadership skills among girls; the provision of safe water (boreholes) and latrines; and advocacy for girls and women. USAID supported PAGE in 4 districts in 42 schools in Southern Province. By 2002, Southern Province’s education office had expanded PAGE to 40 schools in all 4 districts. Interactive Radio Instruction Program “Learning at Taonga Market” (LTM) The purpose of this activity was to increase the access of out of school children to basic education. Initially, technical assistance was provided to develop and deliver Grade 1 IRI lessons for out of school children in selected communities and schools around Lusaka. Based on the positive result of this pilot activity, the “Learning at Taonga Market” program was expanded to include more grades (grade 1 through 5) and is now broadcast nationwide. Community based Interactive Radio Learning Centers, staffed by volunteer mentors, serve as the delivery point. The findings of various studies indicate that there is a high demand for the LTM program, particularly in the rural and disadvantaged communities. The quality of the IRI lessons and program has improved as a result of the installation of digital production equipment and successful training of technicians at Education Broadcasting Services (EBS). The irregular status of the IRLC mentors poses the greatest threat to the program. Communities Supporting Health, Nutrition, Gender and Equity Education in Schools Project (CHANGES) The CHANGES Project comprised two inter-related and complementary components: (i) the School Health and Nutrition program and (ii) the Community Sensitization and Mobilization Campaign. The purpose of the School Health and Nutrition Activity (SHN) was to improve pupils' health and nutritional status in order to increase their learning capacity and performance in school. Working with three ministries (the MOE, the Ministry of Health, and the Ministry of Community Development and Social Services) at the central, provincial, district and zonal levels initially in Eastern Province and later Southern Province, CHANGES helped the MOE establish an SHN delivery system that: provides micro-nutrient and de-worming treatments to students and teachers; disseminates information and education on health, nutrition, water and sanitation, HIV/ AIDS, and life skills; and raises community awareness on these issues and mobilizes them to action through a variety of innovative Information, Education and Communication (IEC) methods (including community theatre). The SHN Activity developed an SHN information system, including instruments and protocols to conduct baseline surveys of the health and nutritional status of school-going children and to assess the effects of its interventions. The purpose of the Community Sensitization and Mobilization Campaign (CSMC) was to increase access to basic education, especially for girls, orphans, and other vulnerable children (OVCs), by promoting the active participation of communities in the education of their children and raising awareness of issues that USAID / Zambia Close Out Report 6110002 8 can threaten educational participation, such as SHN and HIV/AIDS. Support was provided to build capacity of MOE personnel and community partners (e.g. local NGOs and CBOs) to work with communities so that they develop the skills, techniques and communication channels to identify and address their own educational and schooling needs. Multi-sectoral teams have been established to train communities in planning, budgeting, and implementing action plans to improve education. Education Management Information System (EMIS) and Competency Testing The purpose of the activity was to assist the MOE to develop an education management information system (EMIS), build its capacity to operate (i.e. collect, enter, analyze and report data) the EMIS effectively, and to use the EMIS data for diagnostic, planning, and management purposes. Support has been provided to help the MOE re-design its annual school census instrument, train head teachers in its use, establish a computer training lab, and train key EMIS personnel at MOE Headquarters and in all districts in two provinces. Assistance was also provided to support the Zambian Examinations Council in the development of competency tests to measure pupil learning in grade four. The USAID supported programme enhanced MOE’s information technology infrastructure by establishing access to the Internet. It allowed for the first time use of e-mail and provided the means for effective information access and sharing. Additionally, the program introduced the Education Automated Statistical Information System Toolkit, a system that enables the MOE to collect, manage, disseminate, and use Zambia’s educational data for effective decision-making and planning. 2002 Zambia DHS Education Survey (ZDES) The 2002 Zambia DHS EdData Survey (ZDES), a nationally representative sample was designed to provide information on education among children 6-14 in order to inform the development, planning, monitoring and evaluation in Zambia. It complements the school data collected by the Ministry of Education. The survey focuses on the factors influencing household decisions about schooling. It provides information on; adult education attainment; children’s characteristics; rates of school attendance and absenteeism and the factors that cause them; household expenditures on schooling and other contributions; and parent/guardian perceptions about schooling. The ZDES also provided anthropometrics and basic literacy and numeracy data on school-aged children. The ZDES is linked to the Zambia DHS and the MOE’s school census database (through a school coding system), so that greater socio-economic, as well as school, details can be developed about each household. Prospects for long-term sustainability of impact and principal threats to sustainability. The prospects for sustainability are good for several reasons. The EMIS, SHN and HIV/AIDS activities initiated under S02 are featured in the Ministry of Education Strategic Plan (MOESP), are being institutionalised and mainstreamed into MOE structures and operations, and will be expanded nation-wide. The innovative "Learning at Taonga Market" Interactive Radio program for the out-of-school children will be expanded, and the IRI technology and methods will be applied to formal classroom instruction, teacher development, and community mobilization activities. In addition, it appears that the MOESP, including the USAID-introduced intervention, is likely to receive sufficient funding. The GRZ has pledged to meet sector-funding levels required for HIPC debt concessions. Several cooperating partners have pledged budgetary support to the MOE through a Sector-Wide Approach (SWAp) mechanism, which will help the MOE to implement the MOESP during the transition period. Under its new education S06, USAlD will continue to support these activities, focusing on strengthening managerial and technical capacity. Although economic growth prospects for Zambia are generally positive, the macro economic picture is threatened by structural weaknesses and corruption, and could have adverse implications for sector financing. Ambitious decentralisation and restructuring plans may pose a danger in terms of management delays thereby affecting the efficiency of the MOE programs. USAID / Zambia Close Out Report 6110002 9 All the SO2 education programs were designed to fit within the Ministry of Education (MOE) strategic framework. Each program was implemented in collaboration with specific Government counterparts at MOE Headquarters, provincial or school level. It was essential for the USAID supported programs to secure “buy in” and support from the Government counterparts. The majority of the education activities were managed through the MOE network and relied on the MOE contributions. Despite these efforts to ingrain the USAID programs within the MOE systems there remained a tendency for the Government to view USAID’s support as time bound within a project framework. Frequently, USAID supported programs are more flexible, better planned and move faster than the Ministry can absorb, as a consequence the Ministry develops a dependence on the USAID supported programs. USAID activities tend to fill gaps within the Government systems and the Government though is appreciative is frequently not capable of absorbing new responsibilities quickly and requires the continuing support. Lessons Learned In the area of performance monitoring, the following was learned: The Performance Monitoring Plan required a great deal of effort to develop and finalise, but was useful in that it helped the SO2 team to manage its resources more efficiently by providing the indicators, instruments and information to track progress and signal problems in a timely manner. It also proved useful as a communication tool in appraising the MOE and other CPs about progress and problems. In the area of education sector reform support, the following was learned: The Sector Wide Approach (SWAp), as developed in Zambia, has been positive to USAID. The approach requires that the MOE develop a defined and detailed Strategic Plan, which articulates goals, strategy, priorities, approaches, and indicators. This serves multiple purposes. It enables USDAID and other CPs to better understand the MOE's goals, challenges and successes; its provides the parameters and framework that USAID can use to define its support; it helps harmonize and coordinate donor input; and it ensures MOE ownership and, to various degrees, institutionalisation. The SWAp has promoted greater collaboration and communication among CPs. The degree of success achieved by (Basic Education Sub-Sector Investment Program) BESSIP is unlikely to have occurred, if at least some CPs were not prepared to contribute to the funding "pool". The debate about SWAps has been a lengthy one and consumed a great deal of donor time, but has yielded positive results. The MOE has adapted several practical management procedures, such as putting in place joint (i.e. MOE-CP) committees or task forces for planning, budgeting, management, and review. Directorate of Planning and Information plays a critical role in the MOE's sector reform program, as it is responsible for policy development, planning, information systems, special crosscutting issues, and assessment of both implementation progress and the validity of the overall strategy. At the same time, this Directorate has often seemed to be the most understaffed. Although similar situations have been reported in other countries, it is important that the lacunae in this key area be filled. USAID hopes that with the restructuring and decentralization underway at the MOE, this problem will be remedied. Under S06, USAID will provide support to strengthen this Directorate and its units. Performance Indicators · Because the process of developing performance indicators was consultative and iterative, it was labor-intensive and lengthy. The result, however, was a set of indicators reflecting shared goals and targets, and a thorough understanding by all parties of what the indicators were meant to signify. USAID / Zambia Close Out Report 6110002 10 · In the initial stages of USAID assistance to the education sector, the collection of baseline data was not as systematic as desired because the MOE had not yet established an EMIS. With USAID support under S02, the MOE developed a functional EMIS, which serves not only the MOE but also the CPs, including USAID. The MOE's EMIS facilitates USAID's tracking and reporting, and reduces the challenge of obtaining accurate and timely information. · Performance indicators should be tracked over a number of years in order to capture incremental changes, as well as allow the data collection and reporting systems to develop. · The performance indicators are a useful management tool, providing proof of progress (or not), ensuring consistency in assessing and reporting progress, and providing a foundation for strategy and program evaluation. · The timely delivery of EMIS Annual School Census Reports is a good measure of the extent to which the MOE has developed and producing quality data to inform its planning decisions. · Data on Net Enrolment has not been disputed but is available thus far only at the national level without being disaggregated according to districts, thereby making it difficult to isolate target areas. · Data on Gross Enrolment both for formal and non-formal schooling has proved reliable, is based on EMIS Reports, and is easier to disaggregate. · Obtaining information on IRLCs has been problematic because they have not been included in the EMIS Annual School Census exercise; reflecting in part the MOE's unresolved view of their status. Consequently, USAID through its projects has had to assume responsibility for collecting this data. · Although the performance indicators are less conventional for education, the data obtained for the SHN activities has proved reliable, because of USAID's emphasis on monitoring and evaluation in the initial design of CHANGES Project. · Several special studies and evaluations, commissioned by USAID have made notable contributions to the Performance Monitoring Plan. · The assessment study of the pilot IRI program found that significant learning gains were being made among children attending the non-formal IRLCs, thus leading to USAID's decision to expand the program and laying the foundation in terms of methodology for future evaluation and performance indicators relating to the IRI activity. · The summative evaluation of USAID's support of PAGE revealed positive changes in girls' education due to the intervention. Prior to the USAID-supported study, it was difficult to identify the effects of the PAGE interventions. · Criterion-referenced testing has been a useful tool in building USAID's confidence that its interventions are having positive impacts on children's learning. If USAID had not insisted on having its projects develop the tests with the MOE and use them annually, MOE and USAID could not have shown the benefits of its innovative programs (such as L I'M). · The CHANGES District Profile Reports contain good perspective of the status of education at grass root levels, although their value has not yet been sufficiently exploited · The ZDES provided a view of education and schooling from the household perspective, focusing on educational demand, rather than educational supply (as done by the EMIS and annual school census). It also pioneered the development of survey instruments to collect data on basic literacy and numeracy and anthropometry of school-aged children, which are now being used in other countries. In fact, for the first time in Zambia, data was obtained about the nutritional status of school-aged children, in terms of stunting wasting and underweight measures. Evaluations and Special Studies · Dr Y. A. Chondoka, "Needs Assessment Survey on Radio Programmes for Out of School Youths, Ministry of Education, Zambia 2000. · Kathleen Letshabo, "Evaluation Strategy for Interactive Radio Instruction Pilot Project for Out-of School Children and Youth," July/August 2000. · Education Development Center, "Interactive Radio in Zambia for Out-of-School Audiences," October 15, 2000. USAID / Zambia Close Out Report 6110002 11 · Education Development Center Inc., "October 2000 Evaluation of the Interactive Radio Pilot for Out of School Audiences," October 2000. · Alice K. Siachitema et al, "Evaluation of Results of USAID Support to the Program for the Advancement of Girls Education (PAGE)," October 2000. · Richard Siacewena, "Consultant's Report Verifying and Elaborating on the Findings of EDC Pilot Test Evaluation," December 2000. · Ministry of Education, "Report on an EMIS Situational Analysis & Statistical Support," December 2000 and revised January 2001. · Ministry of Education, "School Health and Nutrition Situation Analysis Conducted in Eastern Province," June 2001. · Ministry of Education, " Basic Education Sub-Sector Investment Programme (BESSIP) 2000 Programme Performance Indicators," November 30, 2001. · Ministry of Education, "Interactive Radio Instruction for Out-Of-School and Youth in Zambia: An Evaluation Report for Grade 1 Learners," December 2001. · Ministry of Education, "Learning Achievement at Grade Two Level in the IRI Centres: Report on the Comprehensive Study of the Basic Competencies in English 3rd Mathematics in the IRI Centres and GRZ Schools," 2001. · Diane R. Garavaglia et al, "Needs Assessment Analysis of Zambia's Basic Competence Testing Program: Observations and Recommendations," AIR, May 2002. · William Nance et al, "Education Non-Project Assistance Feasibility Study-US AID/Zambia," L T Associates Inc., July 2002. · Paul Hebert et al, "Review and Analysis of Zambia's Education Sector," LT Associates Inc, August 2002 · Richard Siaciwena, et al, "Report on the Rapid Appraisal of the MOE's Interactive Radio Instruction (IRI) Basic Education Delivery System, August 2002. · USAID/Zambia, "FY 2003 Annual Report," December 30, 2002 · USAID/Zambia, “Performance Monitoring Plan," revised 2002. · M. R. Malyenkuku et al, "Education Management Information System (EMIS) and Related Activities: Community Data Collection and Use Consultancy Report," June 2003 · Glory 1.T. Makwati et al, "Mid-term Assessments of USAID's. Support for Zambia's Education Management Information System (EMIS) and Related Activities," March 2003. · Central Statistical Office and ORC Macro, "2002 Zambia DHS EdData Survey," April 2003. Instrument Closeout Reports Instrument close out are still in process of final closure by RCO - most awaiting final financial reports. Technical reports have been received including Cost Share reports for EDC COAGS. Key contacts Frederick Henning, Ed.D Education Advisor USAID/Zambia Email: fhenning@usaid.org Phone 260-1-254303/6 Partners Creative Associates International 5301 Wisconsin Avenue NW Suite 700 Washington DC 20015 USA USAID / Zambia Close Out Report 6110002 12 Education Development Center 55 Chapel Street Newton, MA 02458-1060 USA Academy for Education Development 1825 Connecticut Avenue Washington DC 20009 USA American Institute for Research 100 Thomas Jefferson, NW Washington DC 20007 USA Macro International Inc. 11758 Beltsville Drive Calverton, Maryland 20705-3119 USA United Nations Children’s Fund (UNICEF) Alick Nkhata Road P.O Box 33610 Lusaka