USAID/Malawi Education Decentralization Support Activity (EDSA) Mid-Term Evaluation July 2011 This publication was produced for review by the United States Agency for International Development. It was prepared by Tracy Brunette, Joseph Chimombo, Grace Chiuye and Tom Tilson through Social Impact Inc. and Management Systems International. USAID/Malawi EDUCATION DECENTRALIZATION SUPPORT ACTIVITY (EDSA) Mid-Term Evaluation DISCLAIMER The authors’ views expressed in this publication do not necessarily reflect the views of the United States Agency for International Development or the United States Government. This document is available in printed or online versions. Documents are also made available through the Development Experience Clearing House (www.dec.org). Additional information can be obtained from Social Impact, Inc. 2300 Clarendon Boulevard Arlington, VA, 22201 Tel: (703) 465-1884 Fax: (703) 465-1888 info@socialimpact.com This document was submitted by Social Impact, Inc., [with Management Systems International], to the United States Agency for International Development under USAID Contract No. RAN-I￾00-09-00019-00. Education Decentralization Support Activity (EDSA) Mid-Term Evaluation i ACKNOWLEDGMENTS The authors would like to thank each and every individual that provided perspective and insight on EDSA and on general education decentralization and reform efforts in Malawi. We hope that we were able to convey their wisdom in this report and further hope that this report will lead, in some way, to improvements to education in Malawi. The team would like to acknowledge the key MoEST stakeholders at the Central Ministry for providing crucial guidance in informing the future direction of EDSA activities. We are especially indebted to the EDSA staff for opening up all of their records, providing us with documents and being available for countless interviews. The development partners also deserve special thanks for their time and support, especially for their invaluable comments during the draft dissemination workshop. The team also wishes to acknowledge the wonderful cooperation and support it received from the district level staff at the education offices and the assemblies: the DEMs, PEAs, CPEAs, Desk Officers, DEMIS Officers and District Assembly staff all deserve special thanks for their input and logistical support during the field work. At the school and community level, the team is especially indebted to the head teachers, teachers, community leaders, chairpersons and committees representatives for sparing their time to talk with us during our visits. We wish to express our gratitude to learners in the bursary scheme for providing a glimpse of their world during interviews. We learned so much from the stakeholders on the ground. The team is also grateful for the support and guidance provided by Chikondi Maleta and the USAID/Malawi education team. We appreciated the insightful comments provided at various stages and feel confident that we were able to provide useful information with relevance beyond EDSA. Education Decentralization Support Activity (EDSA) Mid-Term Evaluation ii ACRONYMS AED Academy for Education Development CIDA Canadian International Development Agency CPD Continuing Professional Development CPEA Coordinating Primary Education Advisor CSO Civil Society Organizations DA District Assembly DCBT District Capacity Building Team DEM District Education Manager DEMIS District Education Management Information System DfID Department for International Development DIAS Department of Inspection and Supervisory Services DOPP Department of Director of Public Procurement DSS Direct Support to Schools DST Decision Support Tool EDSA Education Decentralization Support Activity EGRA Early Grade Reading Assessment EMIS Education Management Information System ESIP Education Sector Implementation Plan FTI Fast Track Initiative GIZ Deutsche Gesellschaft für Internationale Zusammenarbeit GoM Government of Malawi HIV/AIDS Human Immunodeficiency Virus/Acquired Immune Deficiency Syndrome INSET In-Service Education and Training IRI Interactive Radio Instruction JICA Japanese International Cooperation Agency M&R Maintenance and Rehabilitation MoEST Ministry of Education, Science and Technology MoF Ministry of Finance MoGCCD Ministry of Gender, Children, and Community Development MoLGRD Ministry of Local Government and Rural Development MSI Management Systems International NSCPMPS National Strategy on Community Participation and Management of Primary Schools NESP National Education Sector Plan ODL Open and Distance Learning OVC Orphans and Vulnerable Children PCAR Primary Curriculum and Assessment Reform PEA Primary Education Advisor PSIP Primary School Improvement Program PSSP Primary School Support Project Education Decentralization Support Activity (EDSA) Mid-Term Evaluation iii PTA Parent Teacher Association PTR Pupil Teacher Ratio RFTOP Request for Task Order Proposals RTI Research Triangle Institute SAC School Assessment Chart SI Social Impact SIP School Improvement Plan SMC School Management Committee SWAp Sector-Wide Approach TA Technical Assistance TDC Teacher Development Center T’LIPO Teachers Living Positively with HIV/AIDS UNICEF United Nations Children’s Fund USAID United States Agency for International Development WB World Bank WFP World Food Program YONECO Youth Net and Counseling ZIP Zonal Improvement Plan Education Decentralization Support Activity (EDSA) Mid-Term Evaluation iv CONTENTS I. EXECUTIVE SUMMARY .............................................................................................1 II. DESCRIPTION OF THE PROJECT.........................................................................8 III. EVALUATION PURPOSE........................................................................................ 11 IV. METHODOLOGY........................................................................................................ 12 Review of extant data and documents ...................................................................................................... 12 Direct observation of processes and systems .......................................................................................... 12 Key informant interviews ............................................................................................................................. 12 Case study of the links between EDSA activities and school- and student-level outcomes ....... 13 Sample size and coverage ............................................................................................................................. 14 Gender considerations .................................................................................................................................. 14 V. FINDINGS, CONCLUSIONS AND RECOMMENDATIONS .................... 14 CENTRAL LEVEL POLICY AND STRATEGY ARTICULATION, INTERPRETATION AND IMPLEMENTATION................................................................ 15 Articulation and Implementation of National Policy ............................................................................ 15 EDSA as pilot for PSIP ................................................................................................................................ 16 EDSA support to the SWAp ....................................................................................................................... 17 Monitoring and Evaluation .......................................................................................................................... 18 Clarification of Roles and Responsibilities and Establishment of Linkages .................................... 19 Recommendations .......................................................................................................................................... 19 DISTRICT CAPACITY BUILDING............................................................................................ 20 District Level Training .................................................................................................................................. 21 Training Materials ........................................................................................................................................... 23 Data for Decision Making ............................................................................................................................ 23 District Links to Schools and Central Ministry ...................................................................................... 24 The Devolution Guidelines ......................................................................................................................... 25 Recommendations .......................................................................................................................................... 25 COMMUNITY PARTICIPATION AND SCHOOL IMPROVEMENT PLANNING... 26 Training for School Communities ............................................................................................................. 26 School Improvement Planning ................................................................................................................... 27 Community Participation ............................................................................................................................. 28 Recommendations .......................................................................................................................................... 30 OVCS, BURSARIES AND COUNSELING................................................................................ 31 Bursaries for OVCs ....................................................................................................................................... 31 Counseling ........................................................................................................................................................ 34 Recommendations .......................................................................................................................................... 35 EFFECTIVENESS OF THE PROJECT DESIGN.................................................................. 35 EDSA Focus on Education Sector ............................................................................................................ 36 EDSA Links to PCAR .................................................................................................................................. 37 APPENDIX A. SCOPE OF WORK................................................................................ 39 APPENDIX B. PERSONS CONTACTED................................................................... 46 APPENDIX C. REFERENCES.......................................................................................... 50 APPENDIX D. DISTRICTS VISITED BY ZONES AND SCHOOLS................ 53 Education Decentralization Support Activity (EDSA) Mid-Term Evaluation 1 I. EXECUTIVE SUMMARY The Education Decentralization Support Activity (EDSA) aims to assist the Ministry of Education Science and Technology (MoEST) and other key stakeholder ministries and development partners in making decentralization a functional concept within the education sector in Malawi. More specifically, EDSA provides assistance to the MoEST to strengthen the decentralization process at the central, district and school levels. Funded through USAID/Malawi at $12.5 million over three years, EDSA started in February 2009 and is slated to end in February 2012. The program is funded through USAID’s EQUIP 2 mechanism and the implementing partners are the Academy for Educational Development (AED) and Research Triangle Institute (RTI). Along with the Central Ministry assistance, EDSA is supporting decentralization efforts in six districts (Blantyre Urban, Dedza, Dowa, Mangochi, Mulanje and Nkhata Bay). Furthermore, EDSA activities and innovations piloted in these six districts are being replicated in six additional districts (Blantyre rural, Mzimba North, Ntcheu, Salima, Thyolo and Zomba) as part of the MoEST’s Primary School Improvement Program (PSIP). The MoEST plans to extend PSIP to all 34 districts by 2014. PURPOSE OF THE EVALUATION A mid-term evaluation of EDSA was requested to inform USAID/Malawi of the program’s performance and to provide guidance on areas that require strengthening. More broadly, the evaluation was expected to assess the extent to which EDSA has achieved expected results and succeeded in assisting the MoEST and other key stakeholders in making decentralization a functional concept in education in Malawi. More specifically, the evaluation was to assess the effectiveness of the project design and to inform the future direction of EDSA support for related activities. Recommendations focus on (1) priority activities that EDSA might undertake as well as what EDSA-like assistance will be needed to help ensure sustainability after EDSA comes to an end, and (2) lessons that would guide MoEST implementation of EDSA-supported activities in the future. METHODOLOGY Methods utilized for this evaluation include the review of extant data, direct observation of processes and systems and key informant interviews. Interviews and direct observation of outcomes were conducted in three of the six EDSA districts (Dedza, Mulanje and Nkhata Bay) and in one comparison district (Ntchisi). Four schools were visited in each of the three EDSA districts and an additional four were visited in the comparison district, for a total of 16 schools. Interviews were conducted with key stakeholders in Lilongwe and the four districts as well as with head teachers, school counselors, community members and orphans and vulnerable children (OVC) bursary recipients. A case study in one district (Mulanje) was also undertaken to get more detailed information about links between EDSA activities and school- and student-level outcomes. Two workshops were held to elicit feedback from key stakeholders on preliminary findings and recommendations. Education Decentralization Support Activity (EDSA) Mid-Term Evaluation 2 FINDINGS, CONCLUSIONS, AND RECOMMENDATIONS Overall, EDSA has achieved expected results in most major programmatic areas. In the six EDSA Phase 1 districts, primary education advisors (PEAs) have trained school community members in all 1,084 school communities (a total of 5,420 individuals), all of which have developed School Improvement Plans (SIPs). All schools in the six districts were able to access EDSA funds directly through newly established school bank accounts1 . With EDSA assistance, districts as well as school- and community-level stakeholders were sensitized in major MoEST policies: the National Education Sector Plan (NESP), the Education Sector Implementation Plan (ESIP), and the National Strategy on Community Participation in the Management of Primary Schools (NCPMPS). The Decision Support Tool (DST) is in use in the six districts and District Education Managers (DEMs) believe it is helping them use data to inform crucial management decisions. Finally, school communities in a sub-set of zones were given grants to support OVCs, and all school communities were able to manage OVC bursary funds and provide much needed school and personal items to 8,584 primary and secondary school boys and girls. Underlying these accomplishments are systems that EDSA assisted in developing to help ensure that these activities continue after the project comes to an end. All of the EDSA￾supported activities mentioned above (with the exception of the bursaries) are intended to be carried out by the MoEST as part of the Primary School Improvement Program (PSIP). In fact, PSIP activities are already being “scaled up” to another six districts and the plan is to have them in every district nation-wide in the next four years. Currently, 2,229 schools (45 percent of the public schools in Malawi) are in PSIP districts and participate in the program. Bearing in mind the overall accomplishments of EDSA, it is crucial to look at specific activity areas for lessons and future direction. This report considered EDSA implementation efforts and achievements in the following areas: Central Level Policy Articulation and Implementation; District Level Capacity Building; Community Participation and School Improvement Planning; OVCs, Bursaries and Counseling and the Effectiveness of the Project Design. This report is organized by the aforementioned areas and presents findings, conclusions and recommendations within each of these sections. Due to overlap, some recommendations may be found in more than one section as they relate to more than one activity area. Central Level Policy and Strategy Articulation, Interpretation and Implementation EDSA was successful in assisting the MoEST in finalizing major policy documents (NESP, ESIP), implementing these policies at the school level and using its activities in the six districts as a piloting and launching pad for the Ministry’s own efforts in decentralizing education and increasing community participation through the PSIP. EDSA was instrumental in increasing awareness of national level policy documents at the district and sub-district levels. All district officials interviewed were familiar with the documents, as were teachers and community members in the evaluation schools. Some aspects of the documents themselves, however, may need revisiting. The NSCPMPS places 1 A recent development is that the schools were also successful in accessing Ministry funds through the same system. Education Decentralization Support Activity (EDSA) Mid-Term Evaluation 3 extremely high expectations on the communities, and roles and responsibilities of actors at the various levels of the system are still not well understood. This may be due to a combination of unrealistic expectations and lack of clarity in the documents. The MoEST is transitioning to take on leadership of PSIP activities and several key internal mechanisms have been put in place to ensure institutionalization. These include allocated government funding for PSIP and the allocation of office space in the MoEST. Other key elements not yet finalized include the identification of a national PSIP coordinator and national training team. EDSA needs to ensure key civil service staff members have been identified for these roles before coming to an end2 . Though MoEST intends to roll out PSIP nationwide, there is an expressed need (by the MoEST and donors) for continued external technical assistance throughout the duration of EDSA and beyond. PSIP rollout would seriously slow down without ongoing technical support. The current plan for PSIP to be “housed” in the Directorate of Basic Education (DBE) may not take into account the appropriate functional roles and responsibilities of the DBE or other ministry offices. Despite the fact that USAID does not provide direct budgetary support to the “pooled” education sector fund (unlike DFID and GIZ), MoEST and most partners felt that EDSA played a crucial role in providing timely technical assistance (TA) on key activities and moving the Sector Wide Approach (SWAp) forward. This included assistance in the finalization of both NESP and ESIP, the development of the NESP Monitoring and Evaluation Plan and the production of key documents that facilitated the release of Fast Track Initiative funds. The general perception among donors was that EDSA was well placed to get things done; project staff had flexibility and could provide quick analyses in key areas. Recommendations The following actions are recommended to improve EDSA support to the MoEST’s institutionalization and implementation of PSIP:  EDSA needs to ensure that a national PSIP coordinator and national training team staff are identified before coming to an end.  EDSA needs to provide continued technical support to the MoEST to ensure the continuation of PSIP activities. This TA and capacity building support should “meet the needs” of the Ministry offices and match the existing skills and wishes of those offices and individuals. EDSA TA is seen as flexible and responsive; this should continue. This support should continue beyond the current life of EDSA.  For the rollout of PSIP, the MoEST should consider working through multiple offices best placed to implement and manage various aspects of the program. This could include the DBE, Department of Planning, Directorate of Inspection and Advisory Services or others suitably placed and capable. It is suggested that the 2 Recent e-mail communication suggested that a national coordinator had been identified. Education Decentralization Support Activity (EDSA) Mid-Term Evaluation 4 MoEST look beyond the packaging of PSIP as a project unto itself, but as related activities that should be carried out as regular duties in offices across the Ministry. District Capacity Building EDSA provided school planning and financial management training for District Based Capacity Development Teams (DBCTs) in all six districts. Those trained included DEMs, desk officers, district assembly members and Primary Education Advisors (PEAs). EDSA also provided resources for these teams to train community members and head teachers in every school community in the districts. Contrary to the expectations for the DCBTs, the teams that actually trained the school community groups included only PEAs. The DCBTs concept is crucial to the sustainability of the school and community participation and planning efforts, as DCBTs are slated to assist in rolling out the training to other districts beyond the phase 1 pilot districts as well as providing ongoing training. However, the DCBTs do not appear to be functional in the EDSA districts and did not function even to provide the initial training to school communities. Though the use of existing MoEST structures to facilitate training was inherent to the design of EDSA, another possibility originally considered in USAID’s solicitation was the use of NGOs to provide training to school communities. It is likely not essential for the MoEST to have an internal system to train communities, especially if this same system is responsible for providing direct support to teachers to improve classroom teaching. USAID has a rich history of supporting local NGOs in Malawi, especially in the area of community participation3 . In fact, the MoEST has directly supported NGOs in the education sector in the past, and EDSA is currently using NGOs to assist with community training in some zones and has recently started to expand in this area. It is increasingly being understood that solutions to the achievement of Education for All need to be built on the foundation of partnerships between governments, local communities and NGOs. Studies have found that NGOs in Malawi have a comparative advantage in working with communities in that they work predominantly at the grassroots level and are closer to the problems of schooling. They are, therefore, well positioned to devise appropriate solutions. Based on (1) the lack of success in developing DCBTs, (2) the necessity of the MoEST to focus direct efforts on key interventions more directly linked to learning, and (3) the historical model of using NGOs in USAID education interventions, it is suggested that MoEST reconsider the use of NGOs to facilitate community level training. The majority of the DEMs from the 6 districts hailed the EDSA supported DST as one of the most important contributions of the program. They described how they were able to produce and share data with district assemblies to help inform decisions on critical issues, such as the allocation of teachers or teachers’ houses. The DST has ignited interest in a new DEMIS platform that is being developed and will be rolled out to the new districts with EDSA assistance. Currently, training for new districts is to be carried out in a partnering arrangement between Phase 1 and Phase 2 districts. This partnering approach may, however, 3 Please see “Partners in Education: Key findings on the role of NGOs in basic education in Africa” USAID, 2003 and “The Changing roles of NGOs in Education in Malawi”, USAID and the Center for Educational Research and Training, 2003. Education Decentralization Support Activity (EDSA) Mid-Term Evaluation 5 place unreasonable demands on Phase 1 districts as they have other duties and obligations in home districts. Recommendations The following actions are recommended to strengthen the school community level training and to strengthen district capacity to utilize data:  The MoEST should reconsider the use of NGOs in providing training to school communities on financial management and school improvement. Relying only on education actors (PEAs) and the education sector to support community participation (including training and monitoring of grants) places undue stress on the education system, potentially reducing the ability of schools to deliver quality teaching in the classroom. The configuration of the DCBTs has proven unsustainable in prior projects (such as DSS efforts supported by DFID and JICA). EFA efforts will be more successful if they include a broader base of support outside of government.  The plan to use staff from one district to train other district staff needs to be realistically considered. This could be another use for NGO (private sector) training.  EDSA procurement plans are currently underway to develop new DEMIS software. The MoEST needs to develop a systematic plan to extend the DEMIS software to new districts. Community Participation and School Improvement Planning EDSA funded PEAs to provide financial management, school improvement planning, and community participation training to five individuals in all 1,084 school communities in the six districts (a total of 5,420 individuals). After the training, school communities were expected to develop school improvement plans (SIPs): all schools in the districts have developed SIPs. EDSA also provided grant funds to all school communities for the 2009- 2010 school year. The most common use of the grant funds in the 12 schools visited by the evaluation team was to hire assistant teachers (9 schools), followed by the purchase of exercise books and pens (3 schools). However, there was a wide gap between activities planned in the SIPs and what was actually funded. Also, the activities that were supported by the grants were very small in comparison to school level needs to reduce the pupil-teacher ratio and improve classroom learning. This somewhat elaborate planning process boiled down to modest improvements and inputs that should have been provided by the ministry directly. In one school a detailed SIP was found on the wall of the head teacher’s office, yet when compared to receipts for activities undertaken, nothing on the plan had been actually implemented. In the 12 EDSA school communities visited by the evaluation team, the perception of community members and head teachers was that community participation in the planning process had increased. This broader participation in planning was a departure from the more traditional focus on school maintenance and construction. Though formal SIPs were not found in the comparison schools, a comparably high level of community participation Education Decentralization Support Activity (EDSA) Mid-Term Evaluation 6 did exist. In EDSA districts, a wider variety of activities was funded beyond school supplies and maintenance that were funded with the Direct School Support (DSS) grant funds. One notable difference between EDSA and comparison schools, however, was the lack of support for OVCs in the latter. Recommendations The following actions are recommended for the MoEST to undertake and scale up these PSIP community participation activities:  Realistic expectations for the extent of community support in improving the management and quality of primary schools need to be established. Currently, school communities seem to have the fall back position of filling existing gaps (including recruiting and training assistant teachers and purchasing school supplies), which are supposed to be supported by the Government directly.  School Improvement Planning should include more realistic budgeting, and this should be reflected in the school community planning and financial management training. Lastly, financial management training guides need to be simplified. OVCs, Bursaries and Counseling As a means of increasing school and community support to OVCs, EDSA administered a bursary program for OVCs and provided school level training of “counselors” in a subset of zones in the six districts. To date, EDSA has provided bursaries to a total of 8,584 primary and secondary school boys and girls, exceeding the target of 7,000. Grants were provided to school communities who were responsible for identifying recipients and purchasing and distributing bursary items. Typically, recipients received a uniform, shoes, book bag, exercise books and pens. In all school communities visited, the process of selection seems to have been fair and transparent, and recipients were found to receive bursary items. Bursary recipients, community members and teachers alike strongly believe that the lives of the beneficiary OVCs had indeed improved, as being singled out for support from one’s school and community can be life changing. In all schools, community members felt that OVC needs were much greater than the level of support: many children remain vulnerable and at risk of dropping out of school. It is not unreasonable to conclude that attendance among bursary students has increased for several reasons: recipient OVCs with clean uniforms felt comfortable coming to school and were not chased away for lack of a uniform, they were generally happier about school, and they did not want to jeopardize future assistance by missing school. It was disheartening to hear that learners were being chased away from school because they did not have a uniform despite GOM policies prohibiting this. Bursaries for OVCs provide a psychosocial “boost” to recipients. Bursaries and counselor training in living positively are included in the Ministry’s own national education plan. Presumably they will be reflected in the national budget. Many donor partners also provide Education Decentralization Support Activity (EDSA) Mid-Term Evaluation 7 bursaries to students in various situations. Recommendations The following actions are recommended to strengthen OVC activities:  Bursaries should make learners fit in with their peers. Including items such as basic uniforms in the same style and material as other learners, basic footwear, basic school supplies such as pens and exercise books, and basic hygiene items such as soap) would also allow bursary funds to support more learners. Effectiveness of the Project Design The findings from this evaluation suggest that EDSA achieved most of the intended results and contributed to making decentralization a functional concept in the education sector in Malawi. Just because EDSA did what it set out to do, however, does not mean that these activities were the “right” or best things to do, or that they were implemented in the optimum manner for ensuring sustainable changes in education and decentralization in Malawi. Though EDSA has made major strides in promoting education sector reform and decentralization in the education sector in Malawi, the programmatic changes from USAID’s originally envisioned project design, including (1) a more singular focus on the MoEST, (2) the use of education systems to provide community training rather than NGOs, and (3) the absence of any link to PCAR may have had unintended programmatic consequences. Planning and financial management skills among community members are crucial to community development efforts across sectors. Working exclusively through education actors to increase community capacity in these areas places an undue burden for community participation and development on the education the sector, more specifically, the MoEST. As mentioned above, the use of internal district training teams is not working out in implementation and has not been sustained either in EDSA districts or through past efforts to develop such teams. USAID has a long history of working with NGOs in Malawi, and the MoEST has provided direct funding to NGO activities in the education sector in the past. A possible role for NGOs is to train school community groups. Recommendations The following actions are recommended for improving the effectiveness of EDSA design:  In the area of decentralization and community participation, efforts need to be made to work with and through non-education actors (especially at the district level), and to utilize NGOs4 in training school communities in areas that are outside of the MoEST’s mandate. MoEST efforts should focus on those inputs more closely linked to improving quality in the classroom. 4 EDSA has already started working with NGOs in a limited number of districts. Education Decentralization Support Activity (EDSA) Mid-Term Evaluation 8 II. DESCRIPTION OF THE PROJECT Established in 1891, the British protectorate of Nyasaland became the independent nation of Malawi in 1964. Malawi, located in southeast Africa, ranks among Africa’s most densely populated countries and among the least developed countries in the world. The economy is predominately agricultural, with about 80 percent of the population living in rural areas. It has an estimated population of over 15 million (World Bank Indicators, 2009), with 45 percent of the population below the age of 15 and a population growth rate of 2.7 percent. Malawi is also one of the world’s poorest countries and is ranked 164 out of 177 countries on the United Nations Human Development Index. In the last five years, however, the economy has grown at over 7 percent per year and GDP per capita has increased to $326.5 The Education Decentralization Support Activity (EDSA) provides assistance to the Ministry of Education Science and Technology (MoEST) to strengthen the decentralization process at the central, district and school levels. The program aims to assist the MoEST and other key stakeholder Ministries and development partners in making decentralization a functional concept within the education sector in Malawi. Funded through USAID/Malawi at $12.5 million over three years, EDSA started in February 2009 and is slated to end in February 2012. EDSA is funded through USAID’s EQUIP 2 mechanism and the implementing partners are the Academy for Educational Development (AED) and Research Triangle Institute (RTI). EDSA intends to achieve the following results:  Result Area 1: Strengthened policy and strategy articulation, interpretation and implementation  Result Area 2: Improved decentralization implementation, planning and data utilization for informed decision making  Result Area 3: Enhanced role and participation of communities in monitoring education service delivery These three areas focus on implementing a functional decentralization strategy within an education system facing many challenges. The primary school system is still trying to recover from the influx of students following the 1994 decision to eliminate school fees. Total primary enrollment increased by about 50 percent in just one year, going from 1.8 million in 1993 to 2.8 million in 1994. Enrollment has since increased to 3.8 million in 2010. The Gross Enrollment Rate (2010) was 115 percent, due to the enrollment of under and over-age children. Although a GER of 115 percent is a strong positive indicator of access to schooling, large classes6 with a pupil-teacher ratio in 2010 of 80:1 and a pupil-trained teacher ratio of 91:1, negatively impact the quality of education. Although the survival rate to 5 The World Factbook (Central Intelligence Agency,) estimates a GDP of $800 for 2010, placing Malawi 220 out of 228 countries. It also estimates a population of 15.8 million by July 2011. 6 The average primary school classroom size is 101 students Education Decentralization Support Activity (EDSA) Mid-Term Evaluation 9 Standard 8 has increased in recent years, it is still only 52 percent. The significant dropout rate and high rate of repetition result in an inefficient education system. For example, in the 2009-2010 school year, the dropout rate in Standard 1 was 13 percent and repetition was 25 percent. Finally, poor quality of education is also reflected in achievement tests. Proficiency in English reading halved from 1998 to 2004, reaching only 9 percent in 2004 (ESIP). An exogenous factor with important repercussions for education, HIV/AIDS is decimating the human resource base in Malawi, with UNAIDS (2009) reporting an adult HIV prevalence rate of 11 percent (among the highest in the world). It is estimated that 920,000 Malawians are living with HIV and that 51,000 people die of AIDS each year. An important strategy for the Government of Malawi, the MoEST, USAID, and other partners is promoting decentralization in the education sector. The global trend toward decentralizing education has been prompted by three overarching motives. First, decentralization is a fundamentally democratizing process that serves to increase community involvement and representativeness in educational decision making. Secondly, decentralization enables local authorities to prioritize planning and programming, resulting in improved service delivery and reduced strain on the central government. Lastly, empowering local authorities to prioritize their educational needs can lead to more efficient use of resources, reducing aggregate education spending. Schools and communities need to be active participants in shaping the education of their children, as they are the ones on the ground that best understand the local situation. They are in the best position to know about factors that may inhibit or support school attendance, the financial circumstances of schools and communities, especially the extent and nature of possible local support, and the basic needs and priorities of the schools. They can decide how to best support their schools – encouraging attendance, determining potential local support, not just with bricks and labor and local financial contribution – but also as a potential resource to support some aspects of the school program and basic monitoring activities. Ideally, the result of this participation is stronger schools that better meet the needs of their communities, an important aspect of Ministry goals and objectives for an education system capable of providing a foundation for development in Malawi. The implementation of a decentralized education system is central to the work of the MoEST in the next few years. Decentralization documents have been developed and widely distributed, including the National Education Sector Plan (August 2008), the Education Devolution Guidelines (October 2008), the National Strategy for Community Participation in the Management of Primary Schools (February 2007) and guidelines for district education planning. The policies included in these and other related documents form the context in which EDSA operates. EDSA is not introducing decentralization, but building upon a solid policy and practical base to help ensure that decentralization becomes a more functional system that contributes to strengthening education. The project helps to improve education sector management and governance by supporting and complementing key MoEST and Government of Malawi (GoM) policies, strategies and initiatives such as the National Strategy on Community Participation and Management of Primary Schools (NSCPMPS), the JICA-supported District Education Planning efforts and the World Bank and DfID funding for the Direct Support to Schools (DSS) program. Education Decentralization Support Activity (EDSA) Mid-Term Evaluation 10 EDSA also operates along with the initiatives of other donors and agreements with the GoM in supporting the overall education reform structure, the Sector-Wide Approach (SWAp). In January 2010, the Joint Financing Agreement was signed between the GoM and Development Partners, initiating the education SWAp. EDSA contributions to the SWAp include providing: 1) technical assistance to the MoEST, in particular the Directorates of Educational Planning and Basic Education on policy and system development and refinement; 2) assistance to districts in building capacities in management, governance, program implementation and evaluation; 3) grants to orphans and vulnerable children (OVCs), schools, zones, and districts; and 4) facilitating improved communication and coordination between the center and de-central levels by organizing fora and through the printing and dissemination of key MoEST documents. In addition, EDSA helps to address two underlying issues – HIV/AIDS and gender bias. It is the children and youth of the country who have been particularly hard hit by the pandemic, and the EDSA OVC bursary program is especially important in responding to the HIV/AIDS situation. The focus of the assistance is on establishing, operationalizing and strengthening systems at central and district levels and, through them, to the schools and communities. The ultimate goal is to produce systems that are effective in meeting school, zone and district needs in ways that are supported by national policies, procedures, and capability. At the central level, EDSA seeks to strengthen MoEST policy and strategy articulation, interpretation and implementation through assisting with refinement of national policies and guidelines. This includes finalization and, in some instances, translation of national level education related policies such as the NSCPMPS, guidelines for procurement for the Primary School Improvement Program (PSIP), and integration of HIV/AIDS and School Health and Nutrition strategies in PSIP. EDSA assists the MoEST in sensitizing districts and communities around these major policy documents. EDSA also provides technical assistance to the Ministry in support of the NESP and in the implementation of the MoEST’s PSIP, which is carrying out activities in the areas of direct support to schools and community participation in school management. EDSA activities in six districts have been used as “pilots” for the establishment of systems for both direct support and training of communities. Other activities include a monitoring and evaluation framework for a decentralization policy and guidelines for school bank accounts. At the district level, EDSA is improving decentralized implementation, planning and data utilization for informed decision making by enhancing capacities of districts to develop plans and generate data for planning processes. Activities included training of District Education Management Information System (DEMIS) officers in data collection, the development and use of the Decision Support Tool (DST), and the development of community training structures. EDSA focuses on strengthening the capacities of schools and communities through training in planning and financial management, especially the development of the School Improvement Plan (SIP), and the allocation of school grants and implementation of grant funded activities. In addition, EDSA has provided bursaries for Orphans and Vulnerable Children (OVCs) and has trained teachers in counseling and HIV care and support. Education Decentralization Support Activity (EDSA) Mid-Term Evaluation 11 The project originally targeted six districts (Nkhata-Bay, Dowa, Dedza, Mangochi, Blantyre Urban, and Mulanje), but more recently has been expanded to another six districts (Mzimba North, Blantyre rural, Salima, Thyolo, Zomba rural and Ntcheu districts). Lessons learned from EDSA’s work in the six districts are fed back to the MoEST for refinement, scaling up and national replication through PSIP. The MoEST plans to extend PSIP to all 34 districts by 2014. III. EVALUATION PURPOSE USAID has requested a Mid-Term Evaluation of the Education Decentralization Support Activity (EDSA). This report was meant to inform USAID/Malawi on EDSA’s performance and guide USAID/Malawi on which aspects of the program require strengthening or a reduction of investment. Specifically, the evaluation was expected to:  Assess the efficiency and effectiveness of the project design;  Provide lessons learned from the implementation of the project (including points for improvement of the current project and to inform future project design and project phase out);  Assess the sustainability and institutionalization of the EDSA activities in GoM activities and processes;  Provide overall recommendations to USAID/Malawi on whether there is value for the investments that the U.S. Government (USG) is providing through EDSA;  Provide information on the extent to which EDSA is achieving the overarching goals of supporting the MoEST and the Ministry of Local Government and Rural Development (MoLGRD) in the areas of operationalizing education decentralization, monitoring and evaluation, and community participation in school management. Broadly, the evaluation is meant to be formative, seeking to inform the extent to which intended results are being met and documenting lessons learned that can be used to improve project design and implementation. Ultimately, the evaluation will serve as a tool to inform decisions about whether activities merit extension or possible adjustments that may be needed. This evaluation was carried out with only 7 months remaining in the project (EDSA started in February 2009 and will end in February 2012). Thus, the evaluation focuses on priority activities that EDSA might undertake to help ensure sustainability before coming to an end and to identify lessons that would guide implementation of activities that may be undertaken by the MoEST. The primary audiences for the evaluation report are USAID, the GoM, development partners, implementing partners, and key stakeholders, especially at the decentralized levels. Education Decentralization Support Activity (EDSA) Mid-Term Evaluation 12 IV. METHODOLOGY In order to triangulate findings and strengthen the validity of recommendations, the evaluation team utilized a combination of methodologies best suited to answer the evaluation questions as stated in the RFTOP. This mixed methods approach included the review of extant data (school improvement plans, EDSA district baseline reports, etc.), direct observation of processes and systems (such as bank accounts and training structures), key informant interviews, and a case study in one district of the links between EDSA activities and school- and student-level outcomes. Two meetings were also held with key stakeholders at various stages of the evaluation process in order to get feedback on preliminary findings and recommendations. The methods are discussed in more detail below. REVIEW OF EXTANT DATA AND DOCUMENTS   An often overlooked and underutilized source of evaluation information is existing data and related project documents. In this case, especially those directly related to district and school visits. Before going to districts and schools, the evaluation team secured copies of EDSA district “baseline” reports. The team also obtained copies of school improvement plans (SIPs) and OVC bursary documents for the selected evaluation schools. This strengthened the school-level interviews and observations, as the team was able to ask about activities in more depth. Also, background documents provided information on the important MoEST policies that school communities were “sensitized” in and allowed the evaluators to look for key aspects of these policies in discussions. Many questions in the interview instruments were developed from information on activities and results reported in EDSA quarterly reports. DIRECT OBSERVATION OF PROCESSES AND SYSTEMS   Besides relying on beneficiary perception of the impacts of EDSA (beneficiaries with a vested interest in the continuation of assistance), the evaluation team looked for concrete “evidence” of impacts. For example, if a school/community told the evaluation team various items were purchased with the EDSA SIP grants, the evaluation team asked to see the items. To gage PEA involvement at the school, when possible, the team compared PEA information on school visits with official visitor logs. Charts on the walls depicting school planning information were compared to actual purchases and activities undertaken. In most schools, the team spent at least some time inside a classroom, though this was not a major focus of the evaluation. In one school that used grant funds to hire “assistant” teachers, this teacher’s class was observed. District computer databases, school bank books, SIP planning documents, and teacher attendance registers, were all viewed in the schools in various contexts. Processes that could not be observed were asked about: “Could you tell me who goes to the bank to withdraw the money? Do you take out all the money at one time?” Responses were verified by looking at the bank book. A critical advantage to direct observation is the fact that processes and outcomes can be studied in their natural settings. This contextualization provides a richer understanding of the situation, which may reveal conditions, problems, or patterns many key informants may be unaware of or unable to describe adequately. KEY INFORMANT INTERVIEWS   Key informant interviews are especially useful when the primary purpose of an evaluation is to generate recommendations and there is a need to understand motivation, behavior, and Education Decentralization Support Activity (EDSA) Mid-Term Evaluation 13 the perspectives of customers and partners. This approach is crucial for USAID/Malawi, as the experiences and perspectives of a wide array of stakeholders are paramount in this sector-wide planning environment. As selection of appropriate informants is critical for a quality evaluation, the evaluation team consulted with USAID and other stakeholders to ensure that the most appropriate individuals were included in the evaluation. Though many key informant interviews in this evaluation were exploratory and unstructured, in order to collect comparable and consistent information, structured instruments were utilized to guide interviews for the following stakeholder groups:  Ministry (national and district)  Coordinating Primary Education Advisor and Primary Education Advisor  Other donors and NGOs  Head Teacher  Teacher (TLIPO, counselor, teacher of bursary OVC)  SMC/PTA/Mothers’ Group/OVC committee/other village and community members  Learner (including Chichewa and English reading assessment for case study) Key stakeholders interviewed in each district included district education managers, education desk officers, EMIS officers, HIV/AIDS coordinators, and other relevant district officials. Coordinating Primary Education Advisors (CPEAs) for the district and Primary Education Advisors (PEAs) for the selected zones were also interviewed. At schools, head teachers, T’LIPO teachers or school counselors and teachers of bursary students were interviewed. Focus groups were conducted with the SMC, PTA, Mothers’ Group, OVC committee members and other village and community members. Six OVC bursary learners (three male and three female) were also interviewed at each school in the three EDSA districts for a total of 72 learner interviews. CASE STUDY OF THE LINKS BETWEEN EDSA ACTIVITIES AND SCHOOL￾AND STUDENT-LEVEL OUTCOMES A case study of the links between EDSA activities and student-level outcomes (school attendance), for OVCs was carried out in one EDSA district – Mulanje. Though student￾level indicators might be “higher level” outcomes than the three intended result areas (as articulated in the RFTOP), and outside of EDSA’s manageable interests, all partners contributing to the SWAp need to have these highest-level outcomes in mind. Also, as per the new USAID Education Strategy, all programmatic activities need to increase equitable access to schooling and improve student achievement and reading ability. In all student interviews in the three project districts, the evaluation team asked learners about their attendance before and after the bursaries, as well as their attendance in the two weeks prior to the evaluation. Head teachers, community members, and the teachers of bursary students were also asked about changes in bursary recipient attendance. In the case study district, an attempt was also made to compare stated attendance with teacher attendance registers. An evaluation team member also administered a short reading assessment in both Chichewa7 and English8 to all 24 learners in the case study district. 7 The instrument used was a subset (reading of syllables and a short passage) of the assessment tool developed with RTI assistance for the EGRA pilot activities undertaken in Malawi. Education Decentralization Support Activity (EDSA) Mid-Term Evaluation 14 SAMPLE SIZE AND COVERAGE   Interviews and direct observation of outcomes were conducted in three EDSA districts (Dedza, Mulanje, and Nkhata Bay9 ) and one comparison district (Ntchisi) and in two zones in each district10. Two schools were visited in each of the zones for a total of 16 schools. The zones were randomly selected from among those EDSA zones that included the OVC bursary component. Schools within the EDSA zones were also randomly selected11. GENDER CONSIDERATIONS   While undertaking the interview, the evaluators were always mindful of the interplay between gender and the major activity areas. For example, an attempt was made to include female teachers in the interviews whenever possible. Gender issues were also discussed with head teachers and community members alike. The extent to which gender-related activities were in the SIPs was also a focus. Data and analyses are gender disaggregated whenever relevant. V. FINDINGS, CONCLUSIONS AND RECOMMENDATIONS Overall, EDSA has achieved expected results in most major programmatic areas. In the six EDSA Phase 1 districts, PEAs have trained school community members in all 1,084 school communities (a total of 5,420 individuals), and all school communities have developed SIPs. All schools in the six districts were able to access funds directly through school bank accounts and procure items and implement activities based on these plans. With EDSA assistance, districts as well as school- and community-level stakeholders were sensitized in major MoEST policies (including the NESP, the Education Sector Implementation Plan (ESIP), and the NSCPMPS). The DST is in use in the six districts and hailed by district education managers as one of EDSA’s greatest accomplishments. Lastly, school communities in a the sub-set of zones that were provided grants were able to manage OVC bursary funds and provide much needed school and personal items to 8,584 primary and secondary school boys and girls. Underlying these accomplishments are the systems that EDSA helped develop to ensure that these activities would continue after the project comes to an end. These systems include providing training (including the development of materials) and providing direct financial support to schools through school bank accounts. All of the EDSA-supported activities mentioned above (with the exception of the bursaries) are intended to be carried out by the MoEST as part of the PSIP. The ministry has taken major steps towards institutionalizing 8 A short, timed reading passage from the beginning of the Standard 3 English reader was administered. It included three comprehension questions. 9 The RFTOP calls for one district in each region, Nkhata Bay is the only EDSA district in the Northern Region, Dedza was selected rather than Dowa in the Central Region since the PSSP-School Fees Project was recently undertaken in Dowa, and Mulanje was selected in the Southern Region to increase geographic spread of the sites. 10 In the comparison district, the schools were in four different zones. For a complete list of zones and schools, see Annex D. 11 In two cases, it was not possible to visit the originally selected school (in one case, the school was transferred to another zone that was not an EDSA bursary zone, and in the other case, the school was inaccessible during given time frame of the study). Education Decentralization Support Activity (EDSA) Mid-Term Evaluation 15 these EDSA supported systems. In fact, activities are already being “scaled up” to another six districts and the plan is to have them in every district nation-wide within the next four years. Currently, 2,229 schools (45 percent of public schools in Malawi) are in PSIP districts and are part of the program. Bearing in mind EDSA’s overall accomplishments, it is necessary to look at key activity areas in terms of, not only expected results, but also the larger context in which the activities are operating, in order to assess the value and sustainability of these activities and the ability of the MoEST to provide continued support. To this end, EDSA achievements, prospects for sustainability, and future direction of programming were considered in the following areas:  Central Level Policy Articulation and Implementation  District Level Capacity Building  Community Participation and School Improvement Planning  OVCs, Bursaries, and Counseling  Project Design Effectiveness CENTRAL LEVEL POLICY AND STRATEGY ARTICULATION, INTERPRETATION AND IMPLEMENTATION One result area in EDSA’s strategy was “Strengthened Policy and Strategy Articulation, Interpretation and Implementation.” In this area EDSA was successful in assisting the MoEST in finalizing major policy documents (NESP, ESIP), implementing these policies at the school level, and using its activities in the six districts as a piloting and launching pad for the Ministry’s own efforts in decentralizing education and increasing community participation through PSIP. Articulation and Implementation of National Policy Besides playing a key role in the finalization of ESIP and NESP, EDSA was instrumental in summarizing national-level policies (including the NESP, ESIP, and the NSCPMPS) and translating the documents into two local languages (Chichewa and Tumbuka) to facilitate dissemination at the school and community level. EDSA also increased awareness at the district level and school level of the support needed to successfully implement decentralization policies. As one senior donor partner representative put it, “EDSA took issues to the district level; they summarized the documents to speak to the districts.” One senior level MoEST office director said that over the past 20 years one of the “weakest links” in education has been guaranteeing that policy “hits the ground” and asserted that EDSA achieved this by ensuring people at the grassroots understand national policy. The district officials interviewed were all familiar with the national level policy documents. At the 12 schools in the EDSA districts visited for this evaluation, all head teachers were aware of the documents and most had copies. They would discuss the different sections of their SIPs as those activities contributing to one of the three major NESP thematic areas (access and equity, quality and relevance, and management and governance), demonstrating their “fluency” with the terminology. In the focus groups, most community groups had members who could also discuss these strategy elements. Only in two schools had the community groups not seen the documents in the local language. Education Decentralization Support Activity (EDSA) Mid-Term Evaluation 16 Though EDSA “sensitized” stakeholders at all levels of the system on these national-level policy documents, some aspects of the documents themselves may need revisiting. The NSCPMPS places extremely high expectations on communities, placing them as the “fall￾back” partner for all things that need to happen at the school. Though EDSA has tried to help clarify the roles in the devolution guidelines, a lack of clarity with regard to roles is still pervasive. The confusion is seen even within EDSA and PSIP -- the use of PEAs to train community members is contrary to both the NSCPMPS and the devolution guideline. Information sheets to clarify the devolution guidelines may be too long and exclude precise and actionable recommendations12. In all three EDSA districts visited in this evaluation, funds were being used to pay for assistant teachers who typically taught their own class and did nothing to alleviate class overcrowding, yet according to national-level policy, it is the MoEST that is responsible for recruiting and paying for teachers directly. Additionally, funds were being used to buy exercise books, which are also to be provided by the Ministry directly, though provision is sporadic and insufficient in quantity. In one school, the ministry had provided just over 4,000 exercise books for the school year when the actual need is over 10,000. In this case, the school grants add extra support at the school level, but, for the most part covering inputs (such as assistant teachers and exercise books) that are supposed to be provided by the Central Ministry directly. EDSA as pilot for PSIP EDSA activities in the six districts also served as a pilot for national- level efforts in the areas of decentralization of education and community mobilization. The pilot efforts are slated to be “scaled-up” nationally through the Ministry’s new Primary School Improvement Program. As mentioned above, EDSA has assisted the Directorate of Basic Education in the implementation of PSIP, which is contributing to two key ESIP activities in the areas of management and governance: institutionalization of DSS and increased participation of local communities and SMCs in primary school management. EDSA contributed to a major accomplishment in the institutionalization of DSS (and a major accomplishment in education decentralization in Malawi) through the creation of school bank accounts for the disbursement of Ministry funds directly to schools. The successful piloting of the flow of funds from EDSA directly to school bank accounts, and the transparent management of these funds, formed the basis for the request to the Accountant General for the approval of the use of these accounts for direct MoEST support nation-wide. The first Government funds were recently sent from the Ministry of Finance and should pass through the districts and reach the schools “any day now;” at least by the end of the fiscal year on June 3013. Seventeen years after the declaration of free primary education, funds will flow directly from the Central Ministry to schools for the first time – a decidedly major step toward institutionalizing DSS and fiscal decentralization in the education sector. In contrast to the EDSA/PSIP system, which deposits money into school bank accounts, under DSS (the 12 In the information sheet on Continuous Professional Development, for example, a recommendation is made that “clarification with respect to roles and responsibilities and financing are required” without actually providing this clarification. 13 The evaluation team just learned that this deadline was met and that all schools were able to access Ministry funds. Education Decentralization Support Activity (EDSA) Mid-Term Evaluation 17 previously developed model that is still in use in non-EDSA districts), funds are sent to districts that then pay the vendors for items procured by the schools. The MoEST is transitioning to take on leadership of PSIP activities and several key internal mechanisms have been put in place to ensure institutionalization. These include allocated government funding for PSIP and the allocation of office space in the MoEST. Other key elements not yet finalized include the appointment of a national PSIP coordinator and national training team. During interviews with major stakeholders within and outside of the Ministry closest to PSIP, a common, unified message was clear – there needs to be a national MoEST PSIP coordinator and a national training team and EDSA needs to ensure these components are in place before the project ends. Stakeholders believe that EDSA played a key role in leveraging the government to get PSIP launched and that if EDSA were phased out, the efforts would fail or seriously diminish. Though MoEST intentions to roll out PSIP nationwide are evident (including the designation of funds and the identification of the National Coordinator), there is an expressed need (by the MoEST and donors) for current and continued external technical assistance beyond the life of EDSA. This proposed national PSIP coordinator and national training or resource team should be supported by external technical advisors in key areas. Incorporating the divisional offices and staff is included in the PSIP documents, though this concept is in the early stages of development. Given that EDSA has worked primarily with the Directorate of Basic Education (DBE) and its director, DBE is one obvious “home” for the PSIP coordinator and related activities. DBE also came through clearly in many interviews as the leading office. That being said, it is suggested that the MoEST looks at the types of activities being undertaken by PSIP and takes a broader view of Ministry participation beyond DBE. Which office is best placed and has existing capacity to implement and manage the PSIP activities? Just because PSIP was launched from a project does not mean that these activities need to continue to be “bundled” in an almost project-like fashion in one office. Which office is best placed to oversee and implement the generalization of the district- and school-level training? For the EMIS training at the district level and the rollout of decentralized EMIS, it could be the EMIS or Monitoring and Evaluation (M&E) units in the Planning Department. If PSIP promotes a stronger link between school community planning efforts and quality in the classroom (linking efforts to Primary Curriculum and Assessment Reform), which is strongly suggested, then efforts would include the Directorate of Inspection and Advisory Services. TA provided to increase human capacity should focus efforts on key technical individuals in addition to more political office directors. As one ministry interviewee stated, “When exiting, it does not fall on high-up politicals [to keep activities going], but falls back to senior technical heads.” TA also needs to meet the specific needs of the recipients. Some ministry interviewees were in favor of “one on one” technical assistance with one ministry official working with an external technical advisor. Others appreciated periodic and specific technical assistance, as they were their “own TAs” and felt that they did not learn as much with more directed technical assistance. EDSA support to the SWAp EDSA has been an important partner in Malawi’s SWAp, playing a key role in finalizing major policy documents (such as ESIP and NESP) and ensuring MoEST compliance for the release of FTI funds. Unlike other major donors supporting basic education in Malawi, Education Decentralization Support Activity (EDSA) Mid-Term Evaluation 18 USAID does not provide direct budgetary support to the “pooled” education sector fund but programs funds through discrete projects. Ever mindful of being in sync with the government’s own planning processes, EDSA has worked closely with MoEST offices (namely the Director of Basic Education) to ensure that activities are aligned as closely as possible with the NESP, ESIP, and the SWAp that the Ministry and donors worked so hard to launch. To this end, EDSA has been instrumental in providing technical assistance to support document preparation, printing, and dissemination, policy articulation and sensitization, and piloting of system innovations that helped move the SWAp forward. The project design allowed for this flexibility and responsiveness and was a major strength of the project. USAID’s support for modifications to EDSA to allow this flexibility should also be commended. As mentioned above, EDSA’s piloting of major ESIP activities such as direct support to schools and community participation, were a major contribution to the SWAp as these activities are being rolled out by the MoEST as part of PSIP. EDSA also provided crucial documents (such as financial management training manuals) that helped to trigger the release of FTI funds. Donor partners were mixed in their reviews of EDSA’s contributions and its role in promoting the SWAp, but the criticism had more to do with USAID’s lack of participation in the pooled funding than on EDSA’s efforts. Most felt that, even though there was no direct funding, EDSA provided timely technical assistance for analysis and document production. One senior level donor partner said, “EDSA is not visible, they work behind the scenes. At the end of the day, EDSA helped the deliverable capacity of those agencies…We rarely heard about EDSA in those meetings, but those of us who knew realized its worth.” The general perception was that EDSA was well placed to do several things, was flexible, and could provide quick analyses in key areas. One donor partner said, “By design or luck…they are quite responsive. And they are helping the Planning Department look more responsive.” EDSA has contributed to some of the major accomplishments in the sector, as other areas of activity under the SWAp appear to have stalled. EDSA has produced policy briefs, or “fact sheets” that are analyses intended to guide policies in critical NESP areas. Fact sheets were developed on issues such as “Lessons learned on school based capacity development” and “Considerations and Policy Implications for Enhanced Direct Support to Schools”. It is difficult to measure the extent to which these type of documents are “used” or contribute to the dialogue and process, especially if we take this to mean that they are widely distributed and read. These documents can be useful for selected key individuals at the policy level – even if these individuals are only briefed on findings or if these technical findings make their way into policy and strategy documents. It is important to document lessons and consolidate technical thinking and work to ensure (as a technical advisor) that these findings are reflected in policies and activities, and this is what EDSA did. Fact sheets have been used for training and to inform dialogue. Monitoring and Evaluation In the area of M&E, EDSA played a key role in the development of the Monitoring and Evaluation Framework for the NESP (though this has not yet been finalized). EDSA also provided short-term technical assistance in several areas, including the development of a unit work plan and a research plan. These documents were found to be useful to the M&E office. EDSA also provided financial support to the EMIS unit to carry out their routine training Education Decentralization Support Activity (EDSA) Mid-Term Evaluation 19 and data collection efforts for the 2011 EMIS. That being said accomplishments at the central level may have fallen short of both EDSA’s and the M&E office’s expectations. One member of the EDSA staff said that though “the focus has always been there, it has been more challenging and, perhaps, less successful than other efforts.” Partly this was because EDSA may not have been “set up to provide the type of support that M&E needed,” but it also could have been due to internal MoEST dynamics between the M&E office and EMIS and donor partner wrangling over the provision of a long-term technical advisor for M&E. The M&E office’s role in district-level training was also minimal. As PSIP moves forward, (1) an increased role for the M&E office in the Planning Department with regard to monitoring the use of grants, and (2) stronger links between the M&E office and districts needs to be developed. Clarification of Roles and Responsibilities and Establishment of Linkages EDSA assisted the MoEST in clarifying roles and responsibilities at the various levels principally through their simplification of the NESP and ESIP documents. EDSA produced simplified versions of these documents in two local languages and are printing 30,000 copies for nationwide distribution. The NESP, for example, includes two key pages on roles and responsibilities, and this topic was an important part of their training programs in the districts. EDSA helped to forge linkages between various ministries and institutions relating to the establishment of the system for Government grant distribution to schools which involved the DBE, the Department of Planning and the Ministry of Finance. EDSA’s work with the education Director of Finance helped facilitate the flow of funds from the Ministry of Finance to the Local and National Government, which then ensured that the money would flow to the district assemblies, and from there to the school bank accounts. Another area in which EDSA helped to foster linkages is with the OVC program. In conducting the study "Quality of Life for OVC and HIV Infected Learners in the Learning Environment," EDSA collaborated with the Ministry of Gender, Women, Children and Community Development (MGWCCD) and the Department of Nutrition HIV and AIDS. They also consulted with the Malawi Network of People Living with HIV and AIDS, Teachers Living Positively with HIV and AIDS (T’LIPO), and the Malawi Health Research Committee. EDSA also collaborated with the MGWCCD, especially at the District level, to articulate challenges faced by OVC and HIV positive learners. There were limited, but unsustainable, efforts by EDSA to establish linkages with non￾education partners in the district-level training. EDSA had some initial success in involving people from social welfare and the HIV/AIDS focal person and sought to involve the Director of Finance from the district assemblies in financial management training. For the school planning activities, EDSA tried to involve persons from public works and the social sector. This was not sustained, due to turnover and a lack of responsiveness on the part of these non-education actors. Recommendations The following actions are recommended for EDSA’s support to the MoEST’s institutionalization and implementation of PSIP: Education Decentralization Support Activity (EDSA) Mid-Term Evaluation 20  EDSA needs to ensure that a national PSIP coordinator and national training team staff are identified before coming to an end.  EDSA needs to provide continued technical support to the MoEST to ensure the continuation of PSIP activities and the implementation of other important and related policies. This technical assistance and capacity-building support should “meet the needs” of the Ministry offices and match the existing skills and wishes of those offices and individuals. Technical assistance is not “one size fits all” and many aspects cannot be preplanned. In addition to supporting office directors, senior level technical heads and technical staff should also be provided appropriate levels of support. EDSA technical assistance was seen as flexible and responsive and this should continue. This assistance will be needed required beyond the current life of EDSA.  For the rollout of PSIP, the MoEST should consider working through multiple offices best placed to implement and manage various aspects of PSIP. This could include the Directorate of Basic Education, Department of Planning, the EMIS unit and the M&E office, Directorate of Inspection and Advisory Services, or others suitably placed and capable. Though the definitive recommendation about where in the Ministry PSIP or components of PSIP should be housed is beyond the scope of this evaluation, it is suggested that the MoEST look beyond the packaging of PSIP as a project unto itself. These related activities should be carried out as regular duties in offices across the Ministry. If divisional offices are to be included, their role and the effects on implementation would have to be considered. Central level oversight of the roll out of the DEMIS should be a part of PSIP.  Monitoring and Evaluation efforts need to be coordinated between EMIS and the M&E unit, and should be incorporated into PSIP.  Roles and responsibilities laid out in Ministry documents need to be followed by relevant actors at all levels of the system. Clarification of roles and responsibilities from the Central Ministry down still lack clarity, especially when it comes to the roles and responsibilities of communities in the delivery and monitoring of educational inputs (this is discussed in more detail in the report section on community participation). DISTRICT CAPACITY BUILDING EDSA’s second result area, “Improved decentralization implementation, planning and data utilization for informed decision making,” focused on the training of various district-level actors in M&E, financial management, school planning, and on issues related to HIV/AIDS and gender. It also included the development of decentralized EMIS and planning tools (the Decision Support Tool and School Assessment Charts.) A key premise of the school and community level activities is that all training to the schools and communities flows from, and is supported by, the district (and zonal) level. EDSA does no school or community level training directly. Education Decentralization Support Activity (EDSA) Mid-Term Evaluation 21 District Level Training At the district level, EDSA staff trained a team made up of District Education Managers (DEMs), Education Desk Officers, CPEAs, PEAs, and typically one non-education individual, such as the district director of finance, other district assembly members (district social welfare officer or M&E Officer) or the director of planning or public works. Bringing together district-level officials from different offices was seen as a positive step. According to one DEM, it made them feel like they were working together. He said “the director of public works goes around to schools – before they didn’t go even when invited.” The intention was for a subset of this group to form a District Capacity Building Team (DCBT). EDSA provided training for DCBTs in all six districts14. In total, 164 individuals (100 males and 64 females) who make up these DCBTs were trained in school planning and financial management of school grants, increasing the role of communities in school planning and financial management and increasing the role of women in SMCs and PTAs. EDSA then provided funding for the DCBTs to provide training for school community members (five school community members from each of 1,084 schools) in these areas15. Though, as mentioned above, some of the initial training included non-education actors, the teams that actually trained school community members at the zonal level consisted of education actors only, namely PEAs and CPEAs who held training sessions in zonal teacher development centers for each school community in the zone. One district official said, “EDSA tried, but other groups [like social welfare] are not coming through. They stopped attending meetings.” Staff turnover of trained personnel is also an issue. This means that the task of training communities relies solely on education staff and the MoEST. This is a big task for the MoEST to take on alone. PEAs are crucial to many aspects of school reform in Malawi; they are responsible first and foremost for providing training and school-level and classroom support directly impacting the quality of learning in the classroom. According to both the devolution guidelines and the NCSPMPS, training of communities falls outside of the PEA’s scope of duties. Another planned function for PEAs in EDSA and PSIP is to monitor grants at the school level. Almost every PEA that we spoke to mentioned the difficulty of getting out to the schools due to other training duties that took precedence16. Though PEAs visited many schools regularly, in one school visited by the evaluation team, the PEA had not been seen at the school during the five months that the head teacher had been there (they are supposed to visit each school once a term or about four times a year). In another school, according to the visitors book, the PEA had not been to the school in over six months. Training of community members and increasing community participation is a theme that runs across sectors and could be a shared responsibility of ministry offices beyond education. The cascade model of training employed in EDSA, which was piloted in the six phase 1 districts, is already being replicated in the phase 2 districts. This practice is slated for eventual 14 EDSA has also started training in phase 2 districts. This expansion will not be discussed here but follows the same model. 15 Training in HIV/AIDS and OVC support is discussed in the OVC and bursary section of this report. 16 It should be noted that several other USAID-supported activities, the Tikwere IRI program, Read Malawi, and the Malawi Teacher Professional Development Support Program, also use PEAs as their main conduit to the schools. Education Decentralization Support Activity (EDSA) Mid-Term Evaluation 22 extension to all districts nationwide as part of the implementation of the PSIP. The current plan is for all districts to be incorporated into PSIP, and using this model of school community planning and financial management training by DCBTs, by 2014. DCBTs are crucial to the sustainability of school and community participation and planning efforts. They are slated to assist in rolling out the training to other districts beyond the Phase 1 pilot districts and also for the continual training that communities will need (given turnover in SMC and PTA chairs), and their need for ongoing support in general. Discussing the sustainability of DCBTs may not be completely relevant, however, because it does not appear that they currently exist even in some of the EDSA districts. This model of training does not appear to have been sustained in districts with past DSS support and when asked about the DCBTs in Dedza (an EDSA Phase 1 district), a district official told us that they were “not really functional” but that “when the resources are there, perhaps they can come together.” In the comparison district of Ntchisi, an attempt was made to develop this same type of training team under DSS in 2005/200617. When the assistance ended, the training stopped. One district official reported that there was no longer a training team. When asked how sustainability could have been improved, he said, “We should have taken up the challenge as an office, but our funds were low.” He went on to say, “You can’t expect people to come to training with nothing” (such as an allowance). A major donor partner in the area of district development reported that, even in districts supported by previous district training efforts where training teams exist, they are no longer able to train due to lack of resources. The issues of allowances came up time and again as a necessary “cost” of providing training (they were even included in the school community developed SIPs). Allowances are an issue government-wide and policies and practices need to be reviewed. Though the use of training through existing MoEST structures was inherent to the design of EDSA, another possibility originally considered in USAID’s solicitation was the use of NGOs to provide training to school communities. Does the MoEST need to have an internal system to train communities? Is the same system that is responsible for providing direct support to teachers in their ability to improve classroom teaching? USAID has a rich history of supporting local NGOs in Malawi, especially in the area of community participation. In fact, the MoEST has directly supported NGOs in the education sector in the past and EDSA has utilized NGOs to train communities in some districts. In Nkhata Bay, for example, EDSA used three NGOs (TEMWA, Foundations for Children’s Rights and YONECO) to train communities. While the training approaches varied, they were all able to deliver the training to the school communities. The DEM’s office commended their efforts. Issues that still need to be addressed include the question of allowances (which exists whether using NGOs or not) and the institutionalization of mechanisms to ensure accountability on the part of NGOs. 17 One notable difference between DSS and EDSA training is that under EDSA, the training of capacity building teams is considerably longer (nine days for EDSA compared to one day for DSS.) A larger issue is the underlying premise that it is possible to take a group of professionals from various offices and with varied existing responsibilities, train them, and then expect that these individuals would constitute a semi-permanent training team at the district level – with or without funds to carry out training activities. Education Decentralization Support Activity (EDSA) Mid-Term Evaluation 23 It is increasingly being understood that solutions to the achievement of Education For All need to be built on the foundation of partnerships between governments, local communities and NGOs. Studies on NGOs in Malawi found that they have a comparative advantage in working with communities in that that they work predominantly at the grassroots level and are closer to the problems of schooling and therefore in a position to devise appropriate solutions. Based on the lack of success in developing DCBTs, and the necessity of the MoEST to focus efforts on key interventions more directly linked to learning and the historical model of using NGOs in USAID education interventions, it is suggested that MoEST reconsider the use of NGOs to facilitate community level training. Training Materials EDSA developed training materials for all trainings. These materials were crucial for developing standard guidelines for training and school planning. The guide for financial management was a trigger for the release of Fast Track Initiative (FTI) funds – an important contribution to the sector. As the training is scaled up through PSIP, these materials will be important for ensuring uniform delivery of the training. It is important that, as the training is delivered, the effectiveness of the training is monitored regularly and materials adapted to reflect weaknesses or gaps. In discussions with district officials and PEAs, the evaluation team heard that some of the planning materials may go into too much unnecessary detail given the length of the training. This caused the training to feel rushed. Additionally, some of the school improvement planning forms that communities had to fill out were duplicative and could be streamlined. One district accountant said that some of the financial management language could be simplified and that some concepts were difficult to translate. School improvement materials should also include a discussion of planning within a realistic budget. Data for Decision Making As mentioned earlier, one of the key objectives of EDSA was to strengthen decentralized institutions in planning and the use of information at the various levels. A major capacity￾building effort undertaken by EDSA in this area focused on increased “information for decision making.” The EDSA team undertook initial assessments in the districts and found that ED*Assist (the national level EMIS) was the software being used for decentralized data management. The district EMIS officers could enter data but there was no way to manipulate or actually use the data at the district level using ED*Assist making it not adaptable to function as a DEMIS. At the same time, the baseline assessment found that there was ad hoc use of data among DEMIS officers in EXCEL. This ad hoc use of district level data was expanded into the Decision Support Tool (DST). The DST was developed from files found on district computers that DEMIS officers themselves had created and used, and contained district level information reflecting district level priorities. The DST allows districts to manipulate the data and generate information and reports relevant to their district needs (along zone, constituency and Traditional Authority dimensions). Over time, however, the demands exceed EXCEL-based tools’ capacity. EDSA and the EMIS unit at the central level discussed the development of new software that would allow for the expansion of this district level tool. As plans moved forward in this area, however, procurement restrictions placed on AED prohibited moving ahead with this activity. Recently this restriction was lifted and plans for procuring this software are moving forward through EDSA. Education Decentralization Support Activity (EDSA) Mid-Term Evaluation 24 In focus group discussions, the majority of the DEMs from the six districts hailed the DST as one of the most important contributions of EDSA. They described how they were able to produce data, take to the district assemblies, and help inform decisions on critical issues such as the allocation of teachers (using the school level pupil teacher ratio) or teachers’ houses. The DST seems to have been successful in getting education managers to use data to drive decision making. However, there was no concrete example of an actual decision that had been made (and implemented) due to this new tool. In fact, in an EDSA quarterly report there is a discussion of the pervasiveness of politics in making school-level decisions, even in the face of contrary data. It should also be mentioned that in the comparison district indicators such as pupil-teacher ratio were also said to be used to make allocation decisions. The DST also enabled the production of School Assessment Charts (SACs), tools that are intended to help school-level planning at the district level. SACs were found in some schools (though in most cases these were the SACs produced by EDSA as a sample; in one case, the team saw a school-level SAC developed at the district level.) Some school heads reported having received three days of training and claimed to have used the SACs as a planning tool with community members. However, the only examples given, such as the need for more teachers and more teacher houses, did not require any special data to determine these priorities. Furthermore, there was no evidence that these SACs were being used by the schools, or that the PEAs were using the SAC sheets in their work with the schools. Besides the development of the DST, EDSA was also responsible for training district offices in data management and skills training. DEMIS officers in all six districts were trained. In all three districts visited for this evaluation, it was possible to see the DST on the computer and DEMIS officers were able to walk the evaluation team through it. As such, the technical skills to manipulate the DST information appear to exist. This training has already been generalized to Phase 2 districts (which mentioned above are transitioning to PSIP districts) and will eventually be spread to all 34 districts. The current plan is for Phase 1 districts to provide training to the six new Phase 2 districts in a “mentor” relationship – each Phase 1 district is paired with a Phase 2 district. Though district officials interviewed expressed their ability and desire to help neighboring districts, this plan could be complicated by the fact that the district staff have their own district roles and responsibilities. It does not appear that District Education Plans are linked to School Improvement Plans, and neither is linked to zonal improvement plans. Part of this has to do with the plan development cycle – DEPs are three years, SIPs are one year. It is also complicated by the unpredictable nature of budgets and the timing of fund availability. In order to link these plans, both need to be developed based on realistic, timely and predictable budgets. In one district we were told that the DEP is developed once funding levels are known. In the EDSA districts, SIPs are developed without reference to budget or funding levels. In order to link plans at various levels (SIPs, DEPs, or even zonal improvement plans) responsibilities need to be clarified and they need to be realistic and based on realistic budgeting. District Links to Schools and Central Ministry We heard from several school communities that links between the district and the schools had been strengthened through EDSA. In one case, a school needed new tin sheets to replace a roof that had been blown off. They took their plan (which indicated the need for a new roof) to the DEM and the district provided the funds for a new roof: “The EDSA Education Decentralization Support Activity (EDSA) Mid-Term Evaluation 25 training told us to work with the government. Before the training, we had no way of communicating with them.” Though links between schools and districts may have been strengthened through EDSA, several DEMs expressed concern about the “missing link” between the districts and central level. One DEM said that they see EDSA working hand in hand with the Ministry during meetings in Lilongwe where the Director of BE or Controller for Accounting attend, but they perceive that EDSA got the blessing of the Ministry to carry out the training and related work and that EDSA staff come to the districts by themselves. The DEM went on to say, “That link is missing. If EDSA pulls out, the structure exists from the district down,” but before EDSA exits, they have to develop the structure from the district to the central level. Another DEM said that the Ministry needs to be left with the idea that they need to communicate with the districts: “We only have coordination with the Ministry through EDSA; this link needs to be strengthened.” Links that are crucial to the implementation and sustainability of PSIP include Central Ministry oversight and involvement in district training efforts (even if these training efforts include NGOs), and increased communication. The Devolution Guidelines As mentioned earlier, though EDSA has tried to help clarify the roles in the devolution guidelines, a lack of role clarity is still pervasive. Information sheets to clarify the devolution guidelines may be too long and exclude precise and actionable recommendations. EDSA did not do as much with the Devolution Guidelines as perhaps was originally envisioned, in part, because other donors were focusing in this area: JICA had previously worked on the guidelines and GIZ may support this work following an assessment of the status of the guidelines to be conducted in the near future. DEMs were mixed on their reported knowledge of the guidelines. When asked, several DEMs said that they were familiar with the guidelines and that roles were well understood. In reality, however, roles in major areas (including the provision of teachers and basic school supplies and school construction) were not clear as evidenced by the provision of many of these inputs by school communities despite this being Central Level Ministry responsibility as laid out in the guidelines. Recommendations District capacity building to support school and community training is integral component of EDSA and crucial to the success of PSIP. The following recommendations are made for improving district capacity and providing training as PSIP moves forward.  The MoEST should reconsider the use of NGOs to provide training to school communities on financial management and school improvement. Relying only on education actors (PEAs) and the education sector to support community participation (including training and monitoring of grants) places undue stress on the education system, potentially reducing the ability of schools to deliver quality learning in the classroom. The configuration of the DCBTs has proven unsustainable in prior projects and is likely not sustainable, or even functional, in its current form.  Planning and financial management training materials should be assessed and modified. The training may go into too much detail in some areas and there is some duplication. The steps and forms could be streamlined and simplified, especially the Education Decentralization Support Activity (EDSA) Mid-Term Evaluation 26 financial management materials. SIP training materials should include a discussion of planning based on realistic budgets.  As the new DEMIS software is developed (with EDSA assistance), the MoEST needs to develop a systematic plan to extend it to the new districts, including training and use of the software as a part of PSIP. Currently the plan is for Phase 1 districts to mentor and support Phase 2 districts. DEMIS officers will require support to make this training plan more realistic and sustainable, perhaps by using Central Ministry EMIS or M&E staff or private sector EMIS specialists.  Before DEPs and SIPs can be meaningfully linked, both need to be based on realistic budgets as well as clear roles and responsibilities at all levels of the system. This is important so that it is clear who is responsible for which inputs.  Links between the districts and the Central Ministry need to be strengthened through PSIP. This may be done through the divisions (as part of the PSIP plan) and with the active participation of the MoEST National Coordinator for PSIP. Links that are crucial to the implementation and sustainability of PSIP include Central Ministry oversight and involvement in district training efforts (even if these training efforts include NGOs) and increased communication.18 COMMUNITY PARTICIPATION AND SCHOOL IMPROVEMENT PLANNING EDSA’s third result area is “Enhanced role and participation of communities in monitoring education service delivery.” Community participation is crucial to the success of education decentralization in Malawi. In order to increase and direct this participation, EDSA supported the training of head teachers and school communities in financial management and school improvement planning. School communities were also “sensitized” around the need for, and nature of, community participation as it contributes to national education sector policies and goals as well as the NSCPMP. The participatory development of the SIP, accessing funds directly from school/community bank accounts, and successfully funding and implementing activities from these SIPs is the culmination of EDSA efforts. These elements of EDSA are also part of the PSIP, which is slated to be implemented in all districts and all school communities over the next four years. Training for School Communities EDSA provided funding for and trained the PEAs to provide training19 for five individuals in all 1,084 school communities in the six districts. Five people were trained from each school community, including the head teacher, the deputy head teacher, the head of the SMC, the head of the PTA, and the head of the mothers’ group. In some cases, a village chief and/or the SMC treasurer were also included. The training sessions were held in the zonal Teacher Development Centers for all school communities in that zone and included a 18 For some training sessions, such as HIV related sessions, MoEST presence in training at the district level has been regular. 19 Training in HIV/AIDS and OVC support is discussed in the OVC and bursary section of this report. The district level training is discussed in the section on support to districts. Education Decentralization Support Activity (EDSA) Mid-Term Evaluation 27 total of six days of training in the areas of school improvement planning, financial management, and sensitization of community members on NSCPMPS. Training materials were developed for all training sessions and incorporated information on HIV/AIDS and increasing the role of women in community participation. After the training, school communities were expected to develop SIPs and set up bank accounts. EDSA-funded school improvement grants were then sent to the school bank accounts for the 2009-2010 school year. This was a way to ensure that the “system” for getting funds directly to schools was in place. This represents a major step with regard to giving schools and communities direct control over grant funds and for achieving fiscal decentralization in education in Malawi. All schools in the six districts were able to set up bank accounts and successfully access the EDSA grant funds. The evaluation team just learned that all schools in the six districts were also able to access the Ministry funds using the system established with EDSA assistance. In an attempt to increase transparency, the stipulation that either the head teacher or deputy head AND either the SMC chair or the PTA chair had to be present to withdraw funds was placed on the accounts. Attention needs to be paid to the issue of bank fees, however, as both the transaction and monthly fees add up. In the three EDSA districts visited by the evaluation team, interviews were conducted with teachers and/or deputy head teachers who had been trained, as well as with community members who had been trained (community members and teachers who were not trained were also interviewed). Head teachers and community group members that were interviewed (community groups members were interviewed in focus groups) were able to explain the types of training they had received, the school improvement planning process, and the acquisition of funds from the bank (in some cases, the evaluators also reviewed the bank books). Most were also able to describe the procurement process and distribution of items. It should be noted that besides the school improvement grant, EDSA provided school communities in a subset of zones a grant for OVC bursaries. School community groups were also expected to manage these funds20. In almost all of the schools visited, the evaluation team was able to see a copy of the 2009-2010 SIP. In most cases, 2010-2011 SIPs were also seen indicating the continuation of the formal planning process in at least one cycle after the training. In several cases, SIPs were posted on the wall in the head teacher’s office. School Improvement Planning In an attempt to link school-level planning to national level goals, school communities were trained to allocate budgets and plan their activities around the three major NESP goal areas: quality and relevance (50 percent), access and equity (40 percent), and governance and management (10 percent). Examples of planned activities under quality and relevance from the 12 SIPs in the evaluation schools included hiring of assistant teachers, the purchase of school supplies (mostly pens and exercise books), activities to reduce absenteeism and lateness, and HIV/AIDS activities on issues such as reducing stigma. Under access and equity, planned activities focused exclusively on construction or rehabilitation activities. And under governance and management, planned activities included the training of SMC and PTA members and the purchase of office supplies. 20The use of the OVC grant funds is discussed in more detail in the section on OVCs and bursaries. Education Decentralization Support Activity (EDSA) Mid-Term Evaluation 28 In the 12 sample schools, the most common use of the school improvement grant funds was to hire assistant teachers (nine schools) followed by buying exercise books and pens (three schools). Two of the schools that hired an assistant teacher also funded SMC/PTA training activities and one held a school-based in-service teacher training (INSET) on radio education. However, there was a wide gap between activities planned in the SIP and what was actually funded. In those SIPs that included budget figures, the total costs of carrying out envisioned SIP activities far exceeded available grant funds. In one school, the total cost of undertaking all planned activities was ten times more than the grant amount. So while many plans were grand, in the end, the funded activities were quite modest. Construction and rehabilitation activities could not be undertaken within the budget. Though HIV-related activities may have been in the plans (they were include in three SIPs), none were actually undertaken with budgeted funds. In the comparison district schools that were provided with DSS grant funds comparable to the EDSA grant, funds were also used to purchase pens and exercise books along with minor school maintenance (such as purchasing locks for classroom doors.) The size of DSS grants vary depending on school enrollment and is restricted to two components – school supplies and maintenance/rehabilitation. Though all of the comparison district school communities also had a process to develop SIPs (sometimes referred to as action plans), the actual SIP documents were only seen in one of the four schools. The SMCs and PTAs participated in the planning process, and in one of the four schools the involvement of the community was especially impressive and included wide participation from the village. This is not surprising given that NGOs have supported community development and sensitization efforts in the education sector for decades (in many cases funded by USAID). There are a few notable differences between EDSA grants and grants under DSS. EDSA grants were managed directly by schools; funds were deposited into and accessed from their bank accounts. DSS funds were managed through the district councils that purchased items and paid vendors directly. EDSA grants provided more flexibility than DSS funds (which restrict use to school supplies and maintenance and rehabilitation). The SIPs developed through EDSA generally included a wider vision and linked activities to NESP goals. Community Participation In the 12 EDSA school communities visited by the evaluation team, the perception of community members and head teachers was that community participation in the planning process had increased. In the EDSA districts, community participation had grown to include an increased role in the planning process and regular communication between at least the SMC and PTA chairs and the head teacher. This was a departure from the more common and traditional focus on school maintenance and construction. Besides planning, the major support provided to schools by the communities was in the form of bricks, sand, and labor for construction. Many communities also provided direct financial support through funds raised by village heads or through direct contribution of parents. This was also true in the comparison districts. EDSA’s contribution to improved quality of schooling comes from the SIP activities that are undertaken with EDSA grant funds. While it is true that (1) impacts at the classroom or learner level often take years to observe, (2) the focus of EDSA was not on increased Education Decentralization Support Activity (EDSA) Mid-Term Evaluation 29 learning in the classroom, and (3) there are other partners (Ministry and donor) focusing on improved learning outcomes, these EDSA-funded activities appear to have the potential to make only a modest contribution to overall efforts to improve learning in the classroom. As mentioned earlier, EDSA grants afforded school communities the ability to hire assistant teachers and to purchase basic school supplies. According to government policy documents, these are inputs that the Ministry itself should be providing directly. Assistant teachers were often teaching their own classes as there were not enough teachers to cover every grade (they didn’t alleviate overcrowding in classes) and schools received exercise books from the government sporadically and in insufficient quantities. Teacher shortages in Malawi are rampant, especially in the rural areas. Supporting school communities in this way to hire their own teachers (even assistants) and purchase basic materials may be a “stop gap” measure to ensure that inputs get to the schools, but it should not be seen as a replacement for fundamental Central Ministry support as spelled out in the NESP and ESIP. According to the NCSPMPS and the devolution guidelines, SMCs are responsible for “monitoring the quality of learning in the school.” It is suggested that the roles of communities be reconsidered to be more in line with what they are best placed to do, not in their proximity to schools. Communities are sending their children to school in incredible numbers and at great personal cost (even before fees are figured in.) They are providing labor and materials for school construction. Can they be expected to monitor the quality of learning in the classroom, a complex concept for actors at any level of the system? Community members are the only “volunteers” in this decentralized system; all others (from the center to the districts and schools) are paid professionals. The devolution guidelines state that communities need to develop budgets to fund activities and “mobilize as many resources as possible from communities to complement resources from government and development partners.” This high level of participation is meant to ensure “ownership and sustainability.” Most communities do participate in school development, but unclear and unrealistically high expectations will neither increase ownership nor sustainability. Grants to schools in the PSIP districts are slated to triple in 2011/2012 school year compared to 2010/2011 (and go up again until falling to below the current level by 2012- 2013.) At the peak level, schools planned to fund activities similar to those that they were already funding in addition to the purchase of 25 desks in one case and the construction of a teacher’s house in another (both items that the Ministry is supposed to provide directly). EDSA may have changed the nature of community participation in the six districts from a focus on brick-making21 to one of more direct participation in school planning and grants management. This type of involvement in planning was found in non-EDSA schools and even in some EDSA schools previously supported by LINK to Community Development One head teacher of an EDSA, formerly LINK, school said there was no difference to planning as a result of EDSA. While communities have been sensitized and trained for 21 Although the NSCPMPS states that community participation such as construction cannot be sustained and that it requires a higher level of involvement in order to sustain support and increase ownership, brick making is one community contribution that has actually endured and been sustained over the years and the significance of this contribution should not be minimized. Education Decentralization Support Activity (EDSA) Mid-Term Evaluation 30 decades in Malawi, the training conducted by EDSA was intensive and culminated in more detailed plans with wider visions. For the first time, communities are able to relate their activities to national policies. This is viewed by some as being critically important for the overall implementation of national goals. One notable difference between EDSA and comparison schools was the lack of support for OVCs in comparison schools. While comparison schools acknowledged that there were learners who were especially needy, they were unable to provide any support (at the same school, teachers were paying out of their own pockets to hire an assistant teacher). However, it is not possible to tell if the support to OVCs in the EDSA schools will continue beyond the life of the bursary program. The communities may be more involved with the schools and the SIP process may be more sophisticated, but in the end, items funded with grants were similar in EDSA and the DSS schools. The exception was the hiring of assistant teachers – the funded activity most closely associated with increased learning – though the actual use of these assistants may not translate into increased learning as would be expected. In one school of 700 learners with eight teachers, one assistant teacher was assigned to teach Standard 3 and another Standard 6, freeing up two teachers to teach 30 standard 8 students along with the regularly assigned teacher (making three Standard 8 teachers.) The class size in Standard 1 (over 300 students) and Standard 2 (over 200) remained unmanageably high with one teacher each. So while the school level pupil-teacher ratio improved on paper, classes for the youngest learners stayed at the same exceedingly high level. Unfortunately, this type of allocation was not an isolated incident. As mentioned earlier, SIPs typically included plans for a much more elaborate level of activity than could be supported by the grant. There was evidence that this could be disillusioning and hamper the enthusiasm for planning in the future. There did not appear to be much discussion in the SIP planning manual on realistic budgeting. The other side of this is that, in one case, a school was able to “sell” some of the ideas in their plans to external donors, so while they were outside of their budget, having them spelled out in the plans made it possible for external assistance. Recommendations EDSA’s work with the MoEST to develop a system of training school and community members, establishing bank accounts, developing SIPs, and procuring funds and implementing activities will continue in Malawi as PSIP. The following actions are recommended for PSIP in undertaking and scaling up these activities:  Realistic expectations for community support to primary schools need to be established. Currently, communities seem to have the fall back position of providing for existing gaps, most of which are supposed to be supported by the Government directly. SMCs cannot be responsible for monitoring the quality of learning in the classroom. Communities can and already do make large contributions to support education including sending their children to school at great personal cost (beyond fees), supporting construction efforts, and providing other financial support.  Community level training for school improvement planning should include realistic budgeting and be streamlined as much as possible. More elaborate planning may not Education Decentralization Support Activity (EDSA) Mid-Term Evaluation 31 lead to much different or better activities being implemented, and cumbersome planning may not be sustainable.  The burden of bank fees on school community bank accounts needs to be monitored. One EDSA report on the issue suggested a public-private partnership with banks whereby they were able to waive some of the fees associated with these accounts. OVCS, BURSARIES AND COUNSELING As a means to increase engagement of schools and communities to support OVCs and teachers in HIV prevention and care, EDSA administered a bursary program for OVCs and provided school level training of “counselors” in a subset of zones22. To date, EDSA has provided bursaries to a total of 8,584 primary and secondary school boys and girls: 7,445 to primary school students (3,666 males and 3,779 females) and 526 to secondary school females (slightly surpassing the target of 7,000.) EDSA also developed training materials in several OVC/HIV-related topics and trained teacher counselors in 300 schools. Bursaries for OVCs The bursary funds were given to the school communities in the form of a grant and they determined how the funds were to be spent (from a list of acceptable items) and selected the bursary recipients. During our visits to schools in EDSA districts, we interviewed six OVC bursary recipients at each school – three boys and three girls ranging from standard 3 to standard 8. In total, 72 children were interviewed. They were asked about the items they received, if they had been attending school before they received the bursary assistance, if they had ever been counseled by a counselor or teacher at the school, what other kinds of support they may need and changes resulting from the bursaries (school attendance and home and school life.) The 24 learners in the case study district of Mulanje were questioned in more depth about reasons for changes in school attendance. Typically, learners received two uniforms, a pair of shoes, a book bag or back pack, exercise books and pens. In some cases they were given umbrellas, school instruments, lotion, laundry soap, slips, underwear, rulers, socks and cotton wool (the latter only in the case of adolescent girls.) In several schools, the bursaries were also used to pay school fees levied upon every student. For most schools, the evaluation team brought with them a list of bursary recipients and a list of items recipients were to have received; the team was able to confirm that all of the interviewed students did, in fact, receive these items. Head teachers were able to produce receipts for the expenditures when asked. Head teachers and community members confirmed that a participatory process was used to select the bursary recipients based on criteria laid out for them in the training. In all school communities the process of selection seems to have been fair and transparent. In a couple of cases, more recipients were initially identified than could be supported and the process for making the final determinations was not clear to all community members 22 OVC committee members were trained in school management and school planning in so far as they were heads of SMCs or PTAs. There was no discrete training for OVC committee members. Education Decentralization Support Activity (EDSA) Mid-Term Evaluation 32 (though even in these cases, it was agreed that all children who received the bursaries were truly in need.) A major goal of providing the bursary items was to make these vulnerable students feel like their peers so they did not need to be ashamed of being without basic items. In most schools, the bursary recipients did not stand out from their peers; in fact, a number of learners interviewed were not wearing their uniforms (due to a lack of soap for washing) and most were not wearing the issued shoes When asked, students reported that this was due to lack of polish, socks or appropriate fit23. In one school, all of the OVCs were gathered in brand new uniforms, wearing their shoes and carrying umbrellas, backpacks and even rulers. Of course this was an uncomfortable situation and should be avoided, and it raised the issue of the extent to which these learners should be provided for beyond their peers. If bursary items are to make OVCs feel equal to their peers, items should focus on the basic necessities and not go beyond. This would include a basic uniform, the same style as their peers, basic footwear, basic school supplies (pens and exercise books) and hygiene items (soap, Vaseline, under garments, etc.) Furthermore, decreasing the amount provided may make it possible to support more needy learners. All 72 learners in the three districts were attending school before they received the bursaries. This is an indication that the bursaries did not bring new learners (out-of-school children) into the system, but they supported those already attending school to decrease the chances they may drop out. In one school, even after finding out that they were on the list to be bursary recipients, four learners (two male and two female) still dropped out of school due to early marriage or the necessity to work. It was not possible to determine exactly the extent to which the bursaries prevented recipients from dropping out, though this suggests that the bursary is not enough, in all cases, to prevent this from happening. Community members, teachers and bursary students alike were extremely appreciative of the bursary assistance and believed that as a result, school attendance of learners had increased and learners’ lives had improved in general. It should be noted that in the comparison district, school communities recognized the elevated needs and existence of OVCs, but there were no activities to provide support besides efforts by donors or NGOs directly. The anecdotal evidence suggested that student attendance had improved: all 72 learners said their attendance had improved as did interviewed teachers who had bursary students in their classes. Independent evidence of increased attendance was not easy to come by, however, especially in the absence of baseline data. Even so, the evaluation team found that eight of the 24 bursary students interviewed in Mulanje said they missed school at least once in the past two weeks. For bursary recipients for which it was possible to check their attendance in the class registers (in the case study district), their absenteeism levels appeared to be equal to non-bursary students in their class (in other words, we found they were not missing school more than non-recipients, though one might expect that OVCs would miss school more often than non-OVCs.) Based on the anecdotal evidence from community members, teachers and students themselves combined with the review of attendance registries, it is not unreasonable to conclude that attendance among bursary students has increased. There are direct and indirect 23 It is not possible to know if this was due to their desire to “blend in” with other students. Education Decentralization Support Activity (EDSA) Mid-Term Evaluation 33 ways that the bursary items may have affected their attendance. Though all 72 learners in the three districts reported that they had attended school regularly since receiving the bursary, as mentioned above, in the Mulanje sample, only 10 provided explanations for this increase and all 10 said that it was uniform-related: either they did not have clothes before (N=3), they did not have soap to wash the clothes (N=1) or they were previously sent home due to lack of uniform (N=6). Teachers also mentioned that students were told that if they did not attend school they would not receive bursary items in the future. When asked if or how the bursaries have changed the lives of the recipients, we often heard that they were now happy to come to school, which also could have led to increased attendance. Their teachers confirmed this, saying that recipient learners were now more confident and their self-esteem had improved. Getting unbiased information from project beneficiaries who have a vested interest in the success of a project is not easy. That being said, based on feedback from teachers and bursary recipients themselves, it is not unreasonable to conclude that the lives of bursary recipients had indeed improved as a result of the bursaries – being singled out for recognition and support from your school and community can be life changing. Students told us that they now smile when they did not before, they are no longer “anxious,” and they are just like other children. One girl reported that receiving the bursary items made her forget about the death of her parents. The bursaries do seem to have provided a “psycho social” boost to learners and the school community in general. It was disheartening to hear from so many students that they were chased away from school in the past because they did not have a uniform. USAID sensitization activities in the late 1990s were instrumental in supporting legislation forbidding children to be denied access to schooling for lack of a uniform (this was also a main focus on the free primary education policy established in 1994.) Great effort was made to sensitize communities in this area. Now, USAID is providing uniforms to some through bursaries while so many others still come to school in rags and may be chased away because of it. There needs to be a discussion reconciling school and community support for uniforms with national policy and the desire to remove financial barriers to schooling. As mentioned above, of the 24 learners, eight reported missing school in the past two weeks (three females and five males), and uniforms still present the largest reason for absenteeism. Four of the eight absences were due to lack of uniform or soap (to wash the uniform), two were due to learner illness, one due to caregiver illness and one due to lack of a 100 Kwacha school fee. In the end, findings indicate that attendance has likely increased due to a combination of students having clothing/uniforms to wear and thereby not being chased away, increased student enthusiasm for school and, perhaps, fear of losing the bursary in the future. When asked about other things the bursary students needed, the needs mentioned most frequently among the 72 learners were blankets (N=35), clothes (N=33), soap (N=24), food (N=19) and secondary school fees (N=10). Other items mentioned include shoes, bicycle, exercise books and book bags. Items mentioned did not vary by gender. Community members also expressed additional needs for the children that extended beyond the school – food and blankets were the biggest needs. Community members and teachers also expressed their desire for a bit more flexibility in order to buy these other items not Education Decentralization Support Activity (EDSA) Mid-Term Evaluation 34 specified on the “EDSA” list24. They also mentioned the additional assistance that OVCs needed at home (such as food and blankets) in addition to the assistance they were receiving in school. Another issue commonly mentioned was the need for more bursaries since the need was much greater than the number of children currently receiving the benefits. Counseling EDSA and MoEST counterparts trained district-level trainers (typically PEAs) in HIV care and support topics. The HIV-related training model at the district and zonal level included a higher level of collaboration between the MoEST and EDSA. PEAs then provided three training sessions (overall five days of training) for selected school counselors (one per school) in the areas of counseling for OVCs (2.5 days of training), psychosocial support/stigma and discrimination (2 days) and living positively (1.5 days). T’LIPO members were targeted for training; if the school had no T’LIPO presence, a counselor was selected who was perceived to be “compassionate” and sensitive to the issues. In the 12 evaluation schools, two of the 12 counselors were T’LIPO members. The training materials used were adapted from the Ministry of Gender and Child Development. Linking this to central level Ministry support, the MoEST has incorporated the “positive living” training into the ESIP. One expectation from the training was the establishment of a system of one-on-one OVC counseling at the school. When asked if there were anyone at the school they had talked to about challenges they were facing, none of the 72 OVC bursary recipients reported talking to anyone at the school. In Mulanje, when probed about who they could talk to if they really needed to, the most frequent responses were a teacher at the school (not the counselor) (N=7 boys, N=10 girls); grandma, aunt or older relative (N=4 boys, N=2 girls); a friend or sibling (N=3 boys and N=4 girls). Only two learners mentioned the EDSA-trained school counselors. An additional two learners said they had no one to talk to. Eleven of the 12 trained counselors in the evaluation schools were male. When discussing the types of individual counseling provided by the counselors, they described counselor￾initiated interactions with students (often girls) around inappropriate student behavior and group counseling sessions or discussions. This does not mean that the training was without merit: several trained counselors mentioned changes to their own personal behavior due to the training (one mentioned that he got tested for HIV) and the provision of information to other teachers and community members about what they had learned. One counselor said that other teachers had gotten tested for HIV after her recommendation. Counselors and head teachers also mentioned group “counseling” or discussions with the students. In some schools we learned of other efforts taking place to provide counseling to OVCs and other students through Mothers’ Groups. This evaluation cannot speak to the effectiveness of that training, but it is worth considering who may be best placed to provide this type of counseling to support learners in need. Though we did not see evidence of OVC referrals to other types of services (such as health care facilities), EDSA activities in this area were conducted mainly in Blantyre Urban. In Blantyre Urban, EDSA worked more intently on identifying HIV positive learners and provide referrals to other related services. 24 It should be noted that this list was developed through negotiation between PEPFAR (who initially wanted to fund only school “fees”) and EDSA. Education Decentralization Support Activity (EDSA) Mid-Term Evaluation 35 Recommendations Bursaries for OVCs provide a psychosocial “boost” to recipients. Bursaries and counselor training in living positively are included in the Ministry’s own national education plan and, presumably, will be reflected in the national budget. Many donor partners also provide bursaries to students in various situations. The following recommendations are made to support these activities as they move forward:  Bursaries should make learners fit in with their peers, not stand out. Including items such as basic uniforms in the same style and material as other learners, basic footwear, basic school supplies such as pens and exercise books and basic hygiene items such as soap would also allow bursary funds to support more learners.  The MoEST, schools and communities need to reconsider the role of uniforms as both a national policy and community sensitization issue. Requiring uniforms is another form of fee placed on the learners and their families and is contrary to national policy. While the bursary recipients can now attend school in their new uniforms, hundreds of children are still “chased away” because school communities appear to support a uniform requirement.  Consider who is best placed to provide one-on-one counseling to OVCs. Just because teachers are at the schools does not make them best placed or suited to provide this type of counseling (it is also being done by the Mothers’ Group in some cases.) That HIV related training did not lead to one-on-one counseling likely has more to do with overly high or misplaced expectations than a failure on the part of the teacher counselors. There are other merits to aspects of this type of training (for teachers and communities) that may warrant its extension. EFFECTIVENESS OF THE PROJECT DESIGN The findings from this evaluation suggest that EDSA achieved most of the intended results and contributed to making decentralization a functional concept in the education sector in Malawi. Just because EDSA did what it set out to do (and what was suggested by and agreed upon with USAID), however, does not mean that these activities were the “right” or best things to do or implemented in the optimum manner for sustainable changes in education and decentralization given the realities in Malawi. One area the evaluation team was asked to explore was “the effectiveness of the project design,” recognizing that, even if a project hits its targets, it is crucial to know if those were in fact the right targets to be aiming for. The design of EDSA includes notable modifications from the original solicitation description of possible activities and modes of implementation. Some changes were made to the design before the award and some after in response to the realities on the ground. These changes included a minimized role for non-education ministries and actors (the Ministry of Land and Rural Development, district assemblies and NGOs, for example) and the absence of the linking of EDSA to Primary Curriculum Assessment Reform (PCAR). Education Decentralization Support Activity (EDSA) Mid-Term Evaluation 36 The move away from non-education sector actors came about as the project evolved for various reasons. The focus on MoLRD25 at the central level was minimized because EDSA wanted to work closely within the MoEST SWAp. As such, the decision was made to “embed” EDSA or components of EDSA into the MoEST directly. This decreased EDSA’s ability to engage with other non-education (or even non-Ministry) actors, as EDSA staff felt it was inappropriate to “run around” and engage other ministries while working with and within MoEST structures. Instead, the belief was that the MoEST should engage other ministries, and it was the MoEST’s job to take the lead in this type of wider GOM collaboration. While attempts were made to include non-education actors at the district level (and many non-education district staff were trained), individual participation waned on various working groups as the project moved forward and, in the end, major activities were being carried out nearly exclusively by education staff. That being said, EDSA (though with and through the MoEST, not on its own) did work directly with the National Local Government Finance Committee to get approval of the school bank accounts – a groundbreaking accomplishment in the fiscal decentralization of education in Malawi. The emphasis on using NGOs to support school-level training was revised due to “capacity deficiencies” and the desire to “reinforce and work through as many government structures as possible”26. It was also the wish of the MoEST to work directly through its systems for carrying out such activities as training school community members. They did not want USAID supporting the creation of “parallel” structures (for service delivery.) The focus on PCAR and learning was never part of AED’s project design. The EDSA project element of complementing PCAR was translated into the development of user￾friendly training manuals. In AED’s proposal, USAID’s original request for linking and complementing PCAR and other cross-cutting issues was translated as follows: “Activity 3.3: Link to and complement the Primary Curriculum Assessment Reform (PCAR) and cross-cutting issues of gender equity and HIV/AIDS prevention and support affecting communities and schools most directly. Functional literacy. Work under this activity will focus on 1) developing user￾friendly SMC training materials and 2) HIV/AIDS communication materials.” EDSA Focus on Education Sector EDSA’s focus away from non-education actors and the use of the existing system of technical assistance to schools through PEAs means that support to communities for increased participation in schools is placed squarely in the laps of the education sector. If PEAs are training school communities in issues of school management and finance and community participation in general, they have less time to focus on issues that more directly influence teaching and learning in the classroom. According to the devolution guidelines, this type of community-level training is not the responsibility of the PEA. It is true that the PEAs and this system of direct support to school are there, but that does not mean use of this system to train communities is the best approach or even appropriate. Does the MoEST need a sustainable system to train community members? 25 Though as mentioned before, EDSA has worked with the National Local Government Finance Committee to establish the school community bank accounts. 26 Recently, EDSA has been working to bring a scaled down version of NGO support to schools to the project. This has been hampered in the past by AED’s restrictions on signing new contracts. Education Decentralization Support Activity (EDSA) Mid-Term Evaluation 37 This is not to say that community members do not need to be trained but that the education system may not need to be direct service providers in this area. USAID and EDSA want to ensure that the MoEST is in the lead, and it was the Ministry that was against the use of NGOs for the delivery of training to the communities. But there is a role for USAID as a provider of technical assistance to bring these issues to the fore. The MoEST does NOT need to be the sole provider of support to community participation in the education sector and likely does not need to build a sustainable system to do so. Literacy and reading are not included in the NESP but are a major focus of USAID’s education program in Malawi because they are deemed to be crucial to the success of education reform in Malawi. The essence of NESP and ESIP should be upheld and respected, but details of implementation at the activity level are not spelled out in any implementation documents, and these are the details that should be discussed and negotiated. There is still the issue of NGO capacity in Malawi. It is USAID/Malawi’s belief that the capacity of NGOs in the education sector may not be strong in areas of grants management. The DEM in Nkhata Bay commended the work the NGOs were doing to support community training in education participation. This evaluation did not include a review of NGO capacity. USAID has a long history of supporting and working with NGOs in the education sector in Malawi. The Creative Center for Community Mobilization (CRECCOM, which was started with USAID assistance) has been working with school communities nationwide for decades in community sensitization to support schooling, among other issues. There are NGOs in other sectors (health, for example) that could be tapped to provide such training at the community level. Support for community participation and capacity building at this level includes training in financial management, accountability and planning – areas not unique to education. EDSA Links to PCAR The elimination of the links to PCAR and the potential impact on learning on EDSA activities is even more direct. There is virtually no substantive discussion of improved learning among project activities in the proposal or subsequent work plans. The central directorate responsible for school quality and that has overall responsibility for the implementation of PCAR, the Directorate of Inspection and Advisory Services (DIAS), was not involved in any EDSA activities27. This has serious implications, not only due to the possible loss of focus on learning and quality in the classroom, but for the sustainability of using PEAs for training as the DIAS is their home office. The contribution of EDSA efforts to improved learning in the classroom was, ultimately, assistant teachers in schools, pens and exercise books. In the absence of other quality-related support at the school level, these inputs were very small compared to the need for increasing the quality of the learning environments in any of the schools visited in this evaluation. An example that was presented earlier was the hiring of two assistant teachers, which then freed two senior teachers, who then were assigned to teach the 30 Standard 8 learners (making a total of three teachers for Standard 8), but leaving one teacher to teach 300 Standard 1 learners and one teacher to teach 200 Standard 2 learners. In the other schools that hired assistant teachers, these teachers often were assigned to teach a class that had no teacher 27 In one meeting, EDSA responded to this type of assertion saying that it is the MoEST who should designate which offices in the ministry EDSA works with, it is not their place to make these determinations. Education Decentralization Support Activity (EDSA) Mid-Term Evaluation 38 rather than alleviative overcrowding in other classes. Linking communities to PCAR could be as easy as training them about crucial elements of learning, which include chalk, textbooks, present and engaged teachers and reasonable class sizes and teacher allocation. In Mulanje, the case study district, basic reading assessments in both Chichewa and English28 were administered to 24 pupils ranging from Standard 3 to Standard 8 (there was one learner from Standard 3.) These assessments were not meant to be definitive, precise measures of reading ability across districts or even schools but to provide general information on reading abilities among some learners in the schools in the six EDSA districts. Of the 24 learners assessed, 17 (10 boys and seven girls) could read with comprehension in Chichewa. In English, six of the learners assessed (three boys and three girls) could read with comprehension, another four learners (three girls and one boy) could read but not comprehend what they were reading. One Standard 8 girl who was sitting her leaving exam the following week admitted that she could “read but not understand” English. These low levels of literacy are commensurate to the levels found in other efforts to monitor the quality of learning such as SACMEQ. Increased reading ability is beyond the mandate of EDSA, both in scope and timing (EDSA is only a three-year project.) That being said, if a critical assumption of a project is that someone else (another donor or the government) is responsible for inputs more directly related to learning and ultimately for learning impacts, and neither these inputs nor the impacts materialize, there needs to be a strategic shift in the design of the project. In the evaluation schools visited, no coordination was found between EDSA and any other USAID/Malawi education activities whose mandates are more directly linked to improved learning in the classroom. For example, in the four schools visited in Mulanje, only one school was following the USAID-supported IRI Tikwere radio program, though another two initiated the program when asked about it. One class was so crowded and loud it was not possible to hear the radio and the lack of textbooks meant that the teacher had to write notes on the board, making him fall behind in the radio lesson. As has been discussed in other sections of this report, EDSA has made major strides in promoting education sector reform and decentralization in the education sector in Malawi. At the school and classroom level, other non-EDSA interventions to more directly support the quality of learning in the classroom are not present, and “whole school development” was not a functional reality in the EDSA schools that were visited for this evaluation. The following recommendations for EDSA activities are suggested:  In the area of decentralization and community participation, efforts need to be made to work with and through non-education actors (especially at the district level) and to utilize NGOs29 in training school communities in areas that are outside of the MoEST’s mandate. MoEST efforts should focus on those inputs more closely linked to improving quality in the classroom. 28 The Chichewa assessment included a subset of EGRA instruments that were developed for assessment of Standard 1. The English assessment included a reading passage from the beginning of the Standard 3 PCAR English textbook. 29 EDSA has already started working with NGOs in a limited number of districts. Education Decentralization Support Activity (EDSA) Mid-Term Evaluation 39 APPENDIX A. SCOPE OF WORK SECTION C – DESCRIPTION/STATEMENT OF WORK C.1 OBJECTIVE AND GENERAL DESCRIPTION C.1(a) Introduction The Education Decentralization Support Activity (EDSA) aims at assisting Ministry of Education Science and Technology (MoEST), Ministry of Local Government and Rural Development (MoLGRD) and other key stakeholder Ministries and development partners in making decentralization a functional concept within the Education sector. Funded through USAID/Malawi at $11.5 million over three years, EDSA began on February 20, 2009, with an end date of February 28, 2012. EDSA targets improving education sector management and governance by supporting and complementing key MoEST and GOM policies or strategies and initiatives such as the National Strategy on Community Participation and Management of Primary Schools (NSCPMPS), the JICA-supported District Education Planning efforts and the ongoing World Bank and upcoming DFID￾support to the GOM’s Direct Support to Schools (DSS) initiative, both through discreet and pooled funding, including the recent FTI Catalytic Funding for which the World Bank will be the Supervising Entity. EDSA also builds on, complements and bridges with major decentralization efforts supported by GTZ and DED. The EDSA activity has been designed and implemented with the intention of playing a valuable role in providing enhanced services and impact at the school level and providing improved mutual accountability and reporting at decentralized levels of the education system. The activity is designed to provide support at three levels: a) Central-level policy support, b) Districts support and c) Communities and schools, towards three major intended result areas, linked to specific program areas. Below are the results EDSA intends to achieve: Intended Result Area 1: Strengthened policy and strategy articulation, interpretation and implementation Intended Result Area 2: Improved decentralization implementation, planning and data utilization for informed decision making Intended Result Area 3: Enhanced role and participation of communities in monitoring education service delivery Providing direct technical assistance to MOEST in the area of decentralization policy implementation and planning, EDSA collaborate work with MoEST to establish core expertise at the central and district offices to develop, assess, and improve an effective decentralized system at the school level that is supported by clear policies, procedures and guidelines. EDSA works intensively in six districts and divisions to implement and help establish (where necessary) new systems in the result areas above. The focus of the assistance is on establishing, operationalizing and strengthening systems at central and district levels so that the systems are effective in meeting school; zone and district needs and that are clearly supported by national policies, procedures, and Education Decentralization Support Activity (EDSA) Mid-Term Evaluation 40 capability. This process of scaling up builds on the proven system, and draws on the expertise developed in the center and target districts (Blantyre Urban, Mulanje, Dedza, Dowa, Mangochi and Nkhata – Bay). The EDSA support and approach ensures sustainability through institutionalization of the procedures and systems at the district level, creating leaders and champions of the approach among district management, both along the education line personnel, as well as local government/administration. This is contingent on conditions for success existing in the policy framework, and that sufficient time and resources are devoted to the scaling up process. C.1(b) Objectives USAID/Malawi requires the services of a contractor to conduct a Mid-Term Evaluation of the Education Decentralization Support Activity (EDSA). The final contractor’s report will inform USAID/Malawi on how EDSA has performed and guide USAID/Malawi on areas that require strengthening or reducing scope of investments. Specifically the mid￾term evaluation will seek to:  Assess the efficiency and effectiveness of the project design  Provide lessons that have been learnt from the implementation of the project  Assess the sustainability and institutionalization of the EDSA activities in Government of Malawi activities and processes.  Provide overall recommendation to USAID/Malawi on whether there is value for the investments the United States Government is providing through EDSA. USAID/ Malawi expects a final mid-term evaluation report which will at a minimum provide information in the following key areas: (i) Extent to which EDSA is achieving the overarching goals of supporting the Ministry of Education Science and Technology (MoEST), and Ministry of Local Government and Rural Development (MoLGRD) systems in strengthening the following areas: making education decentralization a functional concept; governance, monitoring & evaluation and community participation in school management being realized; (ii) Lessons learned, good practices and points for improvement to make possible adjustments for current implementation of EDSA and to inform future project design. (iii) Potential project phase – out issues or elements that could be replicated in future projects. General comments may include the impact of other related MoEST or donor supported efforts in connection with or related to EDSA and the extent to which these are influencing results earmarked under the activity and its expected targets, objectives and output. C.2 STATEMENT OF WORK USAID/Malawi requires a contractor to conduct a Mid-Term Evaluation of the Education Decentralization Support Activity (EDSA) implemented by Academy for Education Development (AED) with support from the Ministry of Education Science and Technology (MoEST) and the Ministry of Local Government and Rural Development (MoLGRD). Education Decentralization Support Activity (EDSA) Mid-Term Evaluation 41 Ultimately, the evaluation will serve as a tool to inform decisions to determine whether the activity merits extension and with what scope. Broadly, the mid-term evaluation is meant to be a formative evaluation, seeking to inform the extent to which intended results are being met. Specifically, the effectiveness of implementation lessons and practices to improve project design and implementation. C.3 OVERALL RESULTS AND INDICATORS This section sets forth results (outcomes of contactor’s performance) requirements, and performance standards (minimum standards that the contractor must meet) that must be met to USAID’s satisfaction. The Final Mid-Term Evaluation Report shall be evidence￾based and respond to the Key questions and evaluation areas outlined below. The key questions listed below are not exhaustive. Offerors are strongly encouraged to propose additional or alternate questions, but the study should at a minimum answer the following: C.3(a) Result I - Program & Activity Design, Policy and Management Systems 1. Program Activity Design i. Determine the extent to which EDSA is achieving the overarching goals of supporting MoEST systems in strengthening the following areas: i. Decentralization policy, ii. Monitoring and evaluation and ii. Community participation in school management. Has the project design and implementation been responsive to Government of Malawi’s (GoM) strategic plans in the Education sector? 2. Lessons Learning i. Identify lessons learned, good practices and points for improvement that will inform possible adjustments for current implementation of EDSA ii. What are other related MoEST or donor-supported efforts in connection with or related to EDSA? How have these efforts affected this USAID-supported project, and to what extent are the results attributable to EDSA interventions? iii. What other areas or categories should this activity include as it pertains to leveraging and/or complementing pooled or sector budget support? iv. Review EDSA communication and information strategy. Are EDSA policy briefs circulated widely and facilitate informed policy decision making? 3. Sustainability i. Identify potential project phase-out challenges/opportunities. ii. Identify activities or elements of activities that could be replicated throughout the country, as well as the extent to which they could be replicated and the conditions necessary. iii. Determine if and to what extent the project will be sustainable if replicated. iv. What evidence is there that the host-country government has the capacity and will to take over the project when USAID funding ends? Are there bridging activities that the project might undertake to increase the likelihood of host country funding? Education Decentralization Support Activity (EDSA) Mid-Term Evaluation 42 4. Key Recommendations i. What would be crucial issues to be resolved if the activity is continued? ii. What issues need consideration in increasing the programmatic breadth? iii. What are key issues if activity is scaled up to the national level; is it plausible to scale it up with full or reduced scope? iv. Address any capacity-building and policy reform issues that require particular attention in continued implementation of the activity. v. Address any other options and recommendations that may be determined by the Offeror prior to or during the evaluation that are of critical essence to the life of the activity. vi. Are there options for USAID/Malawi in increasing the effectiveness and efficiency of this activity? Detail these options. C.3(b) Result II – Increased decentralization implementation, policy & strategy articulation, role participation and monitoring service delivery. In answering these questions, the Offeror shall particularly focus on the following sub questions per the areas highlighted below. 1. Improved decentralization implementation, planning and data utilization for informed decision making i. Assess the effectiveness of the dissemination and use of the Devolution Guidelines by district structures ii. Examine the extent to which EDSA has contributed to roles and responsibilities clarification at all levels of effort and functional linkages that EDSA has influenced amongst sector ministries, local assemblies and the Ministry of Finance and other institutions. iii. Analyze the effect of capacity-building of relevant institutions at district, zone and school levels, including, but not limited to training, iv. Assess how EDSA has functionally linked School Improvement Plans, District Education Plans and District Development Plans. This assessment should also look at relations that have been strengthened through EDSA’s work in respect to prior efforts by Development Partners and GoM to implement this; v. Assess EDSA efforts in decentralizing Education Management Information Systems (EMIS); vi. Assess EDSA’s efforts to promote incorporation of relevant HIV activities into the District Education Plans and its outcome; vii. Evaluate program success in improving educational access and achievement among selected OVCs; viii. Examine the extent to which HIV and AIDS is mainstreamed in the curriculum. 2. Strengthened policy and strategy articulation, interpretation and implementation i. Assess EDSA contributions to the development of MoEST Policy Framework; ii. Assess the extent of involvement of EDSA in the development of the implementation operation plans for the National Strategy on Community participation and Management of Primary Schools; iii. Examine level of effort of EDSA in updating the Education Sector HIV/AIDS and School Health and Nutrition Strategy and plans; iv. Assess contributions of EDSA on aspects of SWAp preparation and implementation Education Decentralization Support Activity (EDSA) Mid-Term Evaluation 43 efforts, particularly in the area of Monitoring and Evaluation (M&E); v. Examine efforts to strengthen/establish service linkage for OVCs including access to nutritional supplements and healthcare. 3. Enhanced role and participation of communities in monitoring education service delivery i. Explore how EDSA has strengthened school management on governance, information management and planning capacity; ii. Assess the linkages that EDSA has conducted to complement the Primary Curriculum and Assessment Reform (PCAR) and cross-cutting issues of gender equity and HIV/AIDS prevention affecting communities and schools; iii. Examine how EDSA has leveraged, built on and strengthened Civil Society Organizations’ skills and networks for improved coordination in support provisions in the education sector; iv. Assess efforts in equity access to build the capacity of TLIPOs to administer and monitor grants; v. Examine the performance of school OVC committees and their role in planning, delivery and monitoring of services C.4 REPORTS AND DELIVERABLES In addition to the requirements set forth for submission of reports in Sections I and J, and in accordance with AIDAR clause 752.242-70, Periodic Progress Reports, the Contractor shall submit reports, deliverables or outputs subject to the deadlines specified in Section F.4 of this RFTOP, as further described below to the COTR (referenced in Section G.2). The contractor will also be responsible for submitting the following deliverables: The following deliverables and reports are required under the Task Order. All deliverables and reports will be in English unless otherwise noted. If for local community distribution, both English and Chichewa are required. The Contractor and the Contracting Officer Technical Representative (COTR) have the authority to make small changes to the deliverables and reports specified below. Any such alteration must not change the basic substance of the deliverable, require funds beyond the amount obligated or exceed the total ceiling price or any budgetary limitation. Each deliverable shall conform to the performance standards as described in the Scope of Work, Section C. C.4(a) Evaluation plan The Contractor shall develop an evaluation plan for USAID approval. The plan should describe the methodology to be used, including sample size and coverage (relevant to activity geographic and demographic coverage), techniques (including comparisons of project and non-project areas and stakeholders, where possible), site selection, information sources, interview protocols, etc. The methodology should clearly describe how the team will work with the mission staff, partner NGOs, and government and school/community stakeholders during the evaluation, i.e. providing input into the evaluation plan, frequency of progress briefings, providing feedback to the draft report, and participating in the final report out. The offeror shall also propose a table of contents for the report Education Decentralization Support Activity (EDSA) Mid-Term Evaluation 44 C.4(b) Work plan The Contractor shall prepare and submit a detailed work plan based on the proposed methodology that describes activities and provides a schedule for implementation for USAID review and approval at the start of the contract. The Contractor /consultant team must also hold working sessions with NGOs, key partners and other stakeholders. The initial working session should be held upon arrival in country to solicit input for the work plan. Within ten days of arrival in country, the Contractor shall present the finalized work plan to mission staff and clarify any questions relating to the task. C.4 (c) Instruments/Tools The Contractor shall submit a copy of sample instruments or tools that will be used to implement their methodology or approach and draft final tools for use in the study. These shall also include, but not be limited to, proposed survey modalities, use of focus group discussions, etc. C.4(d) Periodic briefings and reports The Contractor shall provide progress briefings and reports to USAID Mission on weekly basis. The Contractor shall use e-mail, phones and hard copies in meeting this requirement. Minutes of the key reporting meetings will be recorded by the Contractor. USAID will concur on key issues after these meetings. The Draft Report will be presented to key stakeholders (at least 30 participants) identified during the evaluation by the contractor. USAID/Malawi will have to concur to the list and this should take a form of a one – day workshop organized and financed by the contractor. C.4(f) Findings and Recommendations Presentation The Contractor shall provide a brief of findings and recommendations to MoEST, USAID/Malawi, other GoM departments and Ministries and other key stakeholders identified collaboratively with USAID/Malawi. This shall take the form of a half-day event that gathers relevant stakeholders for feedback and comments to be incorporated in the final draft of the document. The contractor in liaison with USAID shall organize this half day event at the MoEST Conference Room. Costs related to this half – day event will at least include refreshments and report reproduction costs. The contractor shall be responsible for meeting the costs. C.4(g) Final report The final report format will be agreed with the USAID/Malawi Education Team, but should at minimum include:  USAID branded cover page  Executive summary  Major findings  Conclusions and recommendations The Contractor shall submit 50 well bound printed copies of the final report and an electronic copy in Microsoft Word format, PDF with any supporting documentation in Word, XL Publisher or other relevant software. Education Decentralization Support Activity (EDSA) Mid-Term Evaluation 45 C.5 OVERARCHING ELEMENTS AND IMPLEMENTATION MODALITIES C.5(a) Support MoEST Role EDSA as an activity is designed to work at multiple levels and with a variety of stakeholders. While this activity already supports crucial MoEST, EDSA and USAID priority areas, the activity is designed to ensure a harmonization and/or coordination with other priority partners’ initiatives to reduce program implementation constraints and transaction costs. C.5(b) Building Local Capacity The Offeror shall, to the maximum extent possible, use Malawian staff, technical experts, and institutions in carrying out the evaluation of EDSA under the resulting Task Order. C.5(c) Geographical Coverage EDSA works intensively in six districts and divisions to implement and help establish (where necessary) new systems in the result areas outlined in section C.3.(a) and C.3.(b) above. The focus of the assistance is on establishing, operationalizing and strengthening of systems at central and district levels so that the systems are effective in meeting school; zone and district needs that are clearly supported by national policies, procedures, and capability. This process of scaling up builds on the proven system, and draws on the expertise developed in the center and target districts (Blantyre Urban, Mulanje, Dedza, Dowa, Mangochi and Nkhata-Bay). C.5(d) Gender Considerations Equity should be addressed with a focus on gender and orphans and other vulnerable children. The mid-term evaluation shall provide address details on the effect and results of the project interventions on men, women, girls and boys. C.5(e) Audience The primary audiences for the evaluation report shall be USAID, Government of Malawi, Development Partners, Implementing Partners and key stakeholders especially at decentralized levels. END OF SECTION C Education Decentralization Support Activity (EDSA) Mid-Term Evaluation 46 APPENDIX B. PERSONS CONTACTED UNITED STATES U.S. Agency for International Development Joe Kitts Senior Education Advisor Catherine Powell Miles Senior Education Advisor Marisol Perez Education Officer Academy for International Development Carrie Williams Home Office Project Director MALAWI U.S. Agency for International Development Eric Loken Acting Mission Director Chikondi Maleta Program Management Specialist Ramsey Sosola Program Management Specialist Oghale Oddo Program/Project Development and Analysis Kristine Herrmann Education and DGTeam Leader Beth Deutsch PEPFAR/OVC Advisor Thomas Briggs Senior Policy Advisor EDSA Staff Joan Owomoyela COP Alastair Rodd Senior Education Analyst, RTI Oscar Mponda Senior Development Advisor David Balwanz Program Manager, AED Donors and Partners CIDA McPherson Jere Education Specialist DFID Don Taylor Education Advisor Foundation for Children’s Right, Mzuzu, Nkhata Bay Kondwani Botha Jenifer Mkandawire Rose Munthali GIZ Jorgen Friis Principal Sector Advisor, Basic Education Program Juergen Buchholz Team Leader, Basic Education Program Education Decentralization Support Activity (EDSA) Mid-Term Evaluation 47 JICA Kimura Hatsue Project Formulation Advisor (Education) Lingstone Chiona Programme Officer LINKS Community Development Choice Mchenga Development Officer TEMWA, Nkhata Bay Junior Mlindane K. Jumbo Projects Coordinator UNICEF Pawan Kucita Chief, Education WFP Martin Mphangwe MoEST – Center John Bisika PS for Education, Science and Technology Macnight Kalanda Director of Basic Education Raphael Agabu Director of Inspection and Advisory Services Patrick Lapukeni Director of Planning Grace Milner Principal Planning Officer, Department of Planning Martin Masanche Senior Planning Officer C. B. Nyirenda Senior Planning Officer Mr. Khozi Principal Planning Officer G.A. Kachepa Director of Finance Nick Hall SWAp Technical Advisor on Finance Charles Mazinga Deputy Director, School Health, Nutrition, HIV/AIDS & AIDS Chris Naunje Acting SWAp Coordinator Other Ministries/Offices–Center B.S.M. Mangulama Director, Office of Public Procurement Arnold M.J. Chirwa Assistant Director, Office of Public Procurement Peter Makanga Chief Professional Development Officer, Office of Public Procurement Stuart Ligomeka PS, Ministry of Local Government and Rural Development MoEST – Districts Blantyre Urban Ruth Samati-Limbali DEM Dowa Wilton Thengoliweta Desk Officer Education Decentralization Support Activity (EDSA) Mid-Term Evaluation 48 Dedza Billy Chikhwana Banda DEM Grace Sikeya (Mrs Tosia) DEMIS Officer Mary Gondwe Principal Accountant Catherine Bunya Assistant Focal Person, Department of HIV/AIDS, General, Disabilities, and School Health and Nutrition Caxton Pat Chiphaka CPEA Alvencio Chaima Counselor, Student. 8 teacher, Mchokera School, Chilanga Zone Grenard Chaperesa Head Teacher, Mchokera School, Chilanga Zone Maxwell H. Makumbi PEA, Kapiri Zone Peter A.C. Siginala Head Teacher, Kaundu School, Kapiri Zone Wycliff Khomara Std 7 and Counselor, Kaundu School, Kapiri Zone Chrissie Charles Lupenga (Mrs. Kapinama) Head Teacher, Takumana Primary School, Kapiri Zone Hoystoni Mulodzeni DHT, Takumana Primary School, Kapiri Zone Hendrix Walter Katengeza Teacher and Counselor, Takumana Primary School, Kapiri Zone Gladson Mkongolo PEA, Chilanga Zone Joseph F.L. Mchinga Head Teacher, Mbirima Primary School, Chilanga Zone Robert L. Kafa Counselor and P1 teacher, Mbirima Primary School, Chilanga Zone Mulanje Brown Nkweu DEMIS officer GremMollenSausa Desk Officer Kalipa CPEA Vincent Namutwa Accountant Henry Maruwo Coordinator for HIV/AIDS Rosemary Mnyaka PEA, Masubi Zone Lewis Nkowa Head Teacher, Ntiza LEA primary school, Masubi Zone Charles W. Joshua Deputy Head Teacher, std 8 teacher, Counselor, Ntiza LEA primary school, Masubi Zone Francis Mwimaniwa Std 3 teacher Ntiza LEA primary school, Masubi Zone Tracio Perete Head Teacher, std 6 teacher, Mtepuwa primary school, Masubi Zone Josiah Nkuma Deputy Head Teacher, Mtepuwa primary school, Masubi Zone Patrick Resesha Std 2 teacher, TLIPO counselor, Mtepuwa primary school, Masubi Zone Edith Meja Std 7 teacher, Mtepuwa primary school, Masubi Zone Mary Zimba PEA , Mathambi Zone Chrissie Ndalama Head Teacher, Nalusa Primary School, Mathambi Zone Feston Chingama Std 1 teacher, counselor, Nalusa Primary School, Mathambi Zone Sydney William Bondo Std 8 teacher, Nalusa Primary School, Mathambi Zone Education Decentralization Support Activity (EDSA) Mid-Term Evaluation 49 Mm Kathumba Head Teacher, Mitochi School, Mathambi Zone Clement Magaleta Std 1 teacher, Counselor, Mitochi School, Mathambi Zone Mike Syton Std 6 teacher, Mitochi School, Mathambi Zone Nkhata Bay M.W.J.Moyo DEM Emmanuel Kumwenda Desk Officer, Primary David Banda Desk Officer, EDSA Wilfred Gandali Kaunda DEMIS Office Beatrice Chirwa Coordinator of School Health and Nutrition, H/A, and Gender Maesteyn M. Msuku PEA, Ruarwe Zone Brian T. Chaumula Acting Head Teacher, Ruarwe Primary School, Ruarwe Zone Dick K. Mkandawire Counselor and P8 teacher, Ruarwe Primary School, Ruarwe Zone Macdonald C.M. Kondowe DHT, New Salawe Primary School, Ruarwe Zone E. K. Njikho TLIPO, New Salawe Primary School, Ruarwe Zone Godfrey Kondowe Counselor and P 4, New Salawe Primary School, Ruarwe Zone Edrian E. Kalenuo PEA, Mazamaba Zone Winston A. Kaira Head Teacher Kadeta Primary School, Mazamba Zone Happy K. Mwale Acting Head Teacher, Mazamaba Primary School, Mazamba Zone Simon Mjiva Shaba Head Teacher, Mazamaba Primary School, Mazamba Zone MycorineKamanga DHT and TLIPO, Mazamaba Primary School, Mazamba Zone Ntchisi A.K.B. Ntandika DEM R.E. Alubana Desk Officer for Primary Education Cosmas S. Lupiya DEMIS Officer Gerald Mwale HRM Officer McPherson A.K. Phiri PEA, Chikho Zone P. A. Kadseghe Acting Head Teacher, Chikho II School, Chikho Zone Chrissie Nkhoma Acting Head Teacher, Kanyulunyulu Primary School, Malomo Zone Chrissie Kayera Teacher and TLIPO Member, Kanyulunyulu Primary School, Malomo Zone Hammex ImaliThano PEA, Mpherere Zone Yohane V. Patisoni Head Teacher, Kawaza Primary School, Mpherere Zone Alick Mazonzi Head Teacher, Landira Primary School, Mbuyedziko Zone Education Decentralization Support Activity (EDSA) Mid-Term Evaluation 50 APPENDIX C. REFERENCES EDSA Publications Malawi: AED, July, 2010. Associate Award Revised Program Description Malawi: AED, Included in Fiscal Year Report, October. 2009. District Profiles: Mulanje Blantyre City Assembly Dedza Dowa Nkhata Bay Malawi: AED, Undated. Fact Sheets Considerations and Policy Implications for Enhanced Direct Support to Schools Disclosure of HIV/AIDS Status Gender and Women’s Participation in Primary School Management; SMC & PTA Lessons Learned on EDSS Grants Lessons Learned on School-Based Capacity School Grants Disbursement Process Malawi: AED, 2010 Miscellaneous Analysis of Bank Charges to School Bank Accounts (DRAFT). Johathan Sandfore Banda. What promotes and what negates education decentralization. Malawi: AED, May, 2009. Performance Monitoring and Research Plan Quarterly and Annual Reports End of Fiscal Year Report. October 2009. Quarter 1 – FY 2010. October-December 2009. Quarter 2 – FY 2010. January-March 2010. Quarter 3 – FY 2010. April-June 2010. Quarter 4 – FY 2010. July-September 2010. Quarter 1 – FY 2011. October-December 2010. Quarter 2 – FY 2011. January- March 2011. Malawi: AED, Dates various. SIPs, FOGs and OVC Documents (School Improvement Plans, Fixed Obligation Grants, Orphans and Vulnerable Children) Dedza Kaundu Primary School Mbirima Primary School Education Decentralization Support Activity (EDSA) Mid-Term Evaluation 51 Mchokera Primary School Takumana Primary School Mulanje Mitochi Primary School Nalusa Primary School Ntiza Primary School Nkhata Bay Kadeti Primary School Mazamba Primary School Ruarwe Primary School Malawi: AED. Work Plan: Year 1 (March 2009-June 2010). April 2009. Work Plan: Year 2 (June 2010-June 2011). September 2010. Malawi: Ministry of Education, Science and Technology, Dates Various. Community Brief on Education Support. National Strategy for Community Participation in Primary School Management; Education Support for Orphans and Other Vulnerable Children, Including HIV/AIDS-Positive Children. March 2011. Devolution Guidelines, December 2004. Education Decentralization Support Activity Grants to District-Based NGOs/CBOs. October 2010. Education Sector Implementation Plan 2009-2010. September 2009. Education Sector Research, Monitoring and Evaluation Framework for Implementation of NESP. November 2010. Education Statistics. 2008. Education Statistics. 2010. HIV & AIDS Guidelines, 2010. Joint Sector Review: Aide Memoire. November 2010. National Education Sector Plan 2008-2017. 2008. National Strategy for Community Participation in Primary School Management. August 2004. Primary School Improvement Programmed; Financial Management Guidelines. Working Manual. February 2011. Quality of Life for OVC and HIV/AIDS Infected Learners in the Learning Environment: Formative Action Study Report. April 2010. Report on the National Conference on Education. August 2005 School Improvement Planning Guidelines and Training Notes (Draft). March 2010. Strategic Plan: HIV & AIDS Response in the Education Sector 2009-2010. (No date) Teacher Development Centers (TDCs). Their Management, Use and Maintenance. November 2009. The Maintenance and Rehabilitation of Primary Schools. December 2009. Malawi: USAID, 2010. Malawi Education Sector Capacity Development Strategy and Plan RFTOP. Education Decentralization Support Activity (EDSA) Mid-Term Evaluation 52 Malawi: World Bank, 2010. Project Appraisal Document on a Proposed {Loan/Credit} in the Amount of (US$50 million) to the Government of Malawi for the Improvement of Quality Education in Malawi Education Decentralization Support Activity (EDSA) Mid-Term Evaluation 53 APPENDIX D. DISTRICTS VISITED BY ZONES AND SCHOOLS District Zone Schools Visited Nkhata-Bay Ruarwe New Salawe Ruarwe Mazamba Kadeti Mazamba Dedza Kapiri Kaundu Takumana Chilanga Mchokera Mbirima Mulanje Masumbi Ntiza Mtepuwa MathambiNalusa Mitochi Ntchisi Chikho Chikho Malomo Kanyulunyulu Mbuyedziko Landira Mpherere Kawaza Quick Count Central = 13 Districts = 27 Teachers = 45 For more information, please visit http://www.socialimpact.com Social Impact, Inc. 2300 Clarendon Boulevard Suite 300 Tel: (703) 465-1884 Fax: (703) 465-1888 www.socilaimpact.com