STRATEGIC OBJECTIVE 2 CLOSE-OUT REPORT USAID/GHANA – February 2005 I. BASIC IDENTIFYING INFORMATION Strategic Objective: Increasing the Effectiveness of the Primary Education System SO Numbers: 641-0128 / 641-0129 SO Approval Date: September 26, 1996 Performance Period: FY 1996 to FY 20051 Geographic Area: Ghana (countrywide program) Initial Obligation: $8,397,000 (FY 1996) Last Obligation: $7,223,731 (FY 2004) Total Cost: $57,867,000 ($51,867,000 in project assistance and $6,000,000 in non￾project assistance as of September 2004) Estimated Host Country Contributions: $17,700,000 Implementing Academy for Educational Development (AED) Partners: (1) Educational Development Center (EDC); (2) Catholic Relief Services (CRS); (3) The Mitchell Group (TMG); (4) Harvard Institute for International Development (HIID); (5) International Foundation for Education and Self-Help (IFESH); (6) Education Assessment and Research Center (EARC); (7) World Education. II. HISTORY OF STRATEGIC OBJECTIVE 2 (SO2 or QUIPS) USAID’s Strategic Objective 2, Increasing the Effectiveness of the Primary Education System, was a program designed to help Ghana’s education system progressively address district and school level constraints to quality education. SO2 was also known as the Quality Improvement in Primary Schools (QUIPS) program. QUIPS became a relatively complex program that combined project assistance and non-project (i.e., budgetary) assistance to support Ghana’s constitutionally mandated program of free, compulsory, universal basic education (fCUBE). QUIPS worked in collaboration with the Ministry of Education, Youth and Sports (MOEYS)2 and the Ghana Education Service (GES—the implementing agency of the Ministry responsible for pre-tertiary education) to achieve the objective of increasing effectiveness in the primary education system. QUIPS was designed to develop, demonstrate and replicate the conditions and processes necessary for improving school standards and ultimately pupil learning throughout the education system by establishing Partnership School/Communities (PSCs). QUIPS began on September 26, 1996. Program activities ended on September 30, 2004. QUIPS built on USAID's prior experience in Ghana in the Primary Education Program (PREP). PREP was a $35 million program embarked upon in 1991. It was designed to strengthen the policy and institutional frameworks required to improve the primary education system by addressing key economic, financial, institutional, and social constraints. Knowledge gained from PREP provided the foundation and rationale for the development of the QUIPS program based on targeted interventions at the school and policy levels to improve and implement initiatives that support quality education. The primary focus of the QUIPS program was on developing school quality standards through a Model Schools Program through project assistance that could deliver cost effective quality education replicable 1 Program activities ended on September 30, 2004 but QUIPS will not formally end until June 30, 2005 in order to complete evaluation related activities. 2 The Ministry of Education, Youth and Sports (MOEYS) changed its name to the Ministry of Education (MOE) in January 2005. on a national scale while at the same time working on national level policy and program reforms through non-project assistance to create a more supportive environment for quality primary education. The three original key program components included: (1) increasing community participation; (2) improving the quality of teaching and learning; and (3) improving education management. As the program evolved, two additional components were added to address girls’ education and HIV/AIDS prevention. The QUIPS program started out relatively simply as two projects, but soon mushroomed into a complex amalgam of several projects, some of which were overlapping. The first two projects included: (1) Improving Learning through Partnerships (ILP); and (2) Community School Alliances (CSA). The QUIPS/ILP project proposed: (1) improving the learning environment and effective teaching and supervision at the school and classroom level; and (2) improving educational policies and programming at the national level. The QUIPS/CSA project proposed: (1) improving community participation in primary schools at the local level; and (2) training education managers to use data for decision making at the district level. The ILP project was originally intended to be implemented nationwide by the Academy for Education Development (AED) through a five-year contract. The CSA project was originally intended to be implemented nationwide by the Educational Development Center (EDC) through a five-year cooperative agreement. However, when it became clear that it would take too long to reach the 330 anticipated schools, Catholic Relief Services (CRS) was provided a grant to operate in the North. Because they already had a school feeding program in the northern regions funded through PL-480, they were able to jump start operations. In addition, the QUIPS program needed assistance in developing the capacity to monitor and evaluate primary education programs. The Mitchell Group (TMG), with assistance from their sub-contractor the Harvard Institute for International Development (HIID), was awarded a two-year contact to manage and coordinate the monitoring, evaluation and dissemination of results of the program as well as build capacity primarily at the district level to collect and process school information. About halfway through the QUIPS program, two additional projects were added to address issues in girls’ education and HIV/AIDS. The Strategies for Advancing Girls’ Education (SAGE) project implemented a three-pronged girls’ education strategy, including system, school and community support activities. This project was implemented by AED. The Strengthening HIV/AIDS Partnerships in Education (SHAPE) project worked with local NGOs to strengthen and support HIV/AIDS awareness programs in junior and senior secondary schools (JSS and SSS). This project was implemented by World Education. Finally, a decision was made to give a grant to International Foundation for Education and Self-Help (IFESH) to improve teaching/learning opportunities for improved literacy. IFESH implemented the Primary School Teacher Training Project (PSTTP), which provided American volunteers the opportunity to come to Ghana to transfer their skills in English language, methodology and curriculum development at Teacher Training Colleges in Ghana. QUIPS has been succeeded by a new education strategic objective that was approved in May 2003 as part of the Ghana Mission's Country Strategic Plan for 2004-2010—SO8: Improved quality of and access to basic education. SO8 builds on the successes and lessons learned from QUIPS. III. CHANGES IN THE STRATEGIC FRAMEWORK The overarching objective of the SO2/QUIPS program was to increase the effectiveness of the primary education system. The strategic objective had the following original seven Intermediate Results: • IR 2.1: Improved environment for learning • IR 2.2: More effective teaching • IR 2.3: Greater community involvement 2 • IR 2.4: Primary education curriculum revised and instructional materials developed and tested along with assessment standards and procedures • IR 2.5: Education personnel management system improved • IR 2.6: Capacity and authority of districts and schools increased • IR 2.7: School quality information and analysis improved Based on the experience of the implementing partners in the first few years of the project, there was a streamlining of the intermediate results of the strategic objective. One of the major factors which influenced this change was the March 2001 QUIPS Mid-Term Assessment, which recommended modifying the implementation of the management component to make it more focused on specific district education management improvements that would make the results more sustainable. As a result, QUIPS consolidated its original seven intermediate results into the following four intermediate results: • IR 2.1: Improved teaching and supervision • IR 2.2: Improved education management • IR 2.3: Increased community involvement • IR 2.4: Improved learning environment IV. TOTAL SO2 COSTS The figures below show that $53,000,000 was originally obligated under SO2, of which $35,000,000 (66%) was in the form of project assistance, with the remaining $18,000,000 (34%) consisting of non￾project assistance. Non-project assistance was established to support Ghana’s efforts to reform the primary education system and provide encouragement for policy change. However, the MOEYS/GES did not have the capacity to absorb all the non-project assistance funds. That is, non-project assistance was originally intended to provide financing to the education sector through released tranches based on evidence that negotiated benchmarks were being met. Conditions for the release of tranches were to be determined annually by MOEYS/GES and USAID through a review process of the mutually agreed on benchmarks. However, a high level of turnover of division heads within MOEYS/GES over the life of the program meant that this annual review process became non-functional and that senior division heads within MOEYS/GES were never fully cognizant of the conditions and requirements for the release of the tranches. As a result, through several amendments between 1999 and 2003, $12,000,000 was transferred from Non-Project Assistance to Project Assistance. Over the life of the QUIPS program, an additional $4,867,000 was added to project assistance. Thus, project assistance rose from 64% to 90% of all funds.3 The total cost share contribution of the Government of Ghana was estimated at more than $17,700,000 and included in-kind contributions. (Figures in the tables below do not include cost share data and are current as of 09/30/04.) Total SO2 Costs Original (FY 1996) Actual (FY 2004) Project Assistance $35,000,000 66.0% $51,867,000 89.6% Non-Project Assistance $18,000,000 34.0% $6,000,000 10.4% TOTAL SO2 FUNDS $53,000,000 $57,867,000 V. STRATEGIC OBJECTIVE PARTNERS AND ACTIVITIES Three primary recipients made up the original SO2 partner consortia. They included: the Academy for Educational Development (AED); Educational Development Center (EDC); and Catholic Relief Services (CRS). There were also several contractors who were also SO2 partners and included: The Mitchell 3 See Annex 1 for a more detailed breakdown of the distribution of funds over the life of the project. 3 Group (TMG) with sub-contractor Harvard; the International Foundation for Education and Self-Help (IFESH); and Education Assessment and Research Center (EARC). When two additional components were added to the SO2 following the mid-term assessment, AED became involved in addressing girls’ education and World Education got involved in the HIV/AIDS prevention component. Academy for Educational Development (AED): AED implemented the Improving Learning through Partnerships (ILP) project in 86 districts in the south. AED focused on five activities: (1) implementing relevant curriculum and using instructional materials at the school level; (2) providing in-service teacher professional development; (3) improving district management, instructional management, and school administration; (4) helping to improve the school’s physical infrastructure; and (5) administering district grants to enable districts to extend QUIPS ‘best practices’ to non-partnership schools and communities. When gender was added after the mid-term assessment, AED also implemented the Strategies for Advancing Girls’ Education (SAGE) project whose primary activity was to mobilize public and private sector constituencies to improve the educational participation of girls. Educational Development Center (EDC): EDC implemented the Community School Alliances (CSA) project in the same 86 southern districts as AED. EDC focused on three activities: (1) increasing community awareness, responsibility and advocacy for education; (2) strengthening community school support organizations; and (3) enhancing community participation in designing, implementing, and monitoring school improvement efforts. Following QUIPS Mid-Term Assessment, EDC added a fourth activity: building the capacity of the District Education Office (DEO) to undertake monitoring and evaluation of education programs. Catholic Relief Services (CRS): In 24 districts in the north of Ghana, CRS implemented both the community-based and school-based interventions as well as the district management training component of the QUIPS Program. Simultaneously, CRS also implemented a school feeding program in the same 24 districts using PL-480 funding whose aim was the promotion of school enrollment and attendance, especially among girls. The Mitchell Group (TMG): TMG, with assistance from its sub-contractor, Harvard Institute for International Development (HIID), managed and coordinated the monitoring, evaluation and dissemination of results of the QUIPS program as well as built capacity primarily at the district level to collect and process school information. The Mitchell Group’s Performance Monitoring and Evaluation Project (PME) had two objectives: 1) to design and implement a monitoring and evaluation system for QUIPS; and 2) to assist the MOEYS in developing capacity for monitoring and evaluating the primary education program. International Foundation for Education and Self-Help (IFESH): IFESH implemented a program that was intended to bring experienced teachers from the United States to teach English as a second language and promote the use of teaching and learning materials at eight Teacher Training Colleges and in the Department of Primary Education at the University of Cape Coast. They had two main activities: (1) Teacher Training College based interventions focusing on the provision of skills in English language, methodology and curriculum development; and (2) Visiting basic education schools for teaching practice observations and providing guidance to teacher trainees. World Education: When HIV/AIDS prevention was added as a separate component after the mid-term assessment, World Education implemented the Strengthening HIV/AIDS Partnerships in Education (SHAPE) project whose primary activity was strengthening the capacity of Ghanaian civil society organizations (CSOs) and the MOEYS/GES School Health Education Program (SHEP) to more effectively plan, design, and implement innovative activities that mitigate the impact of the AIDS epidemic on Ghanaian schools. 4 VI. IMPACT OF STRATEGIC OBJECTIVE ACTIVITIES Overall, QUIPS worked collaboratively with the MOEYS/GES, districts, communities, and schools nationwide. QUIPS supported direct interventions in select communities and schools within all districts by providing in-service training to teachers, head teachers, circuit supervisors, and district officers. It increased community awareness through the support it gave to community school organizations. It contributed to improving the management capacity at the district level by providing training in planning, budgeting and financial management of resources. It enhanced the school environment by providing grants for the implementation of school improvement infrastructure activities. Each year approximately sixty schools and communities were selected by MOEYS/GES to participate in the QUIPS program. Each grouping or cohort received two years of interventions, which occurred at the community, school and district levels. These interventions promoted an enabling environment, improved classroom instruction, improved school and district management, and promoted community participation. By 2004, QUIPS had provided direct interventions in approximately three model schools and associated communities in all of Ghana’s 110 districts.4 QUIPS improved student learning and performance, and operated effectively in the schools and classrooms where the QUIPS program had been introduced. There was also an increase in community participation and a sense of ownership among community school organizations as well as an increase in teacher knowledge and motivation. This “Model School” approach worked in the communities and schools where the QUIPS program had been implemented. School Management Committees (SMC) became more pro-active in dealing with District Education Offices (DEOs) because of QUIPS training. However, although some District Directors of Education (DDEs) acknowledged school/community level improvements brought about by QUIPS, DDEs lacked initiative to apply recognized QUIPS-induced gains to non-QUIPS schools. They lacked an appreciation or understanding at the district level of the benefits of building community involvement into school improvement endeavors. The March 2001 Mid-Term Assessment suggested that the QUIPS program was less effective at the district and national levels than it was at the school and community levels. Lack of organizational change of District Education Offices under the fCUBE initiative meant that DEO operational interactions with schools and community school organizations remained a challenge. At the national level, a lack of communication and coordination with the MOEYS/GES resulted in a negligible influence of non-project assistance on national policies and programs aimed at nurturing a more conducive environment supportive of quality education. The issue of whether changes in implementing partners’ activities remedied this condition will be addressed in the final SO2 evaluation.5 VII. SUSTAINABILITY OF SO2 ACTIVITIES QUIPS program activities just ended on September 30, 2004. Therefore, it is too early to judge the degree to which activities undertaken by USAID’s implementing partners under the QUIPS program were sustainable. Some of the activities may not yield results for years, which makes determining sustainability at this stage a premature conjecture. However, it is clear from both the QUIPS Mid-Term Assessment and implementing partners’ final reports that QUIPS was extremely effective at the community and school levels, reaching upwards to the district level. But without the explicit and proactive support of the senior leadership of the MOEYS/GES as well as the coordinated support of donors and civil society in the financing and reform of basic education, it is difficult to say whether any activities undertaken by programs like QUIPS are sustainable at this juncture in the reform of basic education. An issue that the final SO2 evaluation will address is whether QUIPS activities may be sustainable given that national and district systems and resources for service delivery are not yet fully functional. Successful implementation 4 There are now 138 districts. 5 See Annex 2 for a more detailed discussion of the impact of implementing partner projects. 5 of the fCUBE strategic objective of strengthening management efficiency across all sectors is a critical condition for sustaining any donor funded interventions at the district and national levels, including QUIPS. But it is beyond the scope of this SO2 Close-out Report to determine whether fCUBE’s management efficiency objective has been successfully implemented. This is an issue that requires more in-depth analysis that may be more appropriately performed during the final evaluation of the program. VIII. SUMMARY AND USEFULNESS OF PERFORMANCE INDICATORS Performance indicators for SO2 were altered after the QUIPS Mid-Term Assessment in March 2001 in order to focus explicit attention on measurable change in the four impact areas: teaching practices, community involvement, supervision and management, and the learning environment. The following performance indicators are the final ones developed for SO2.6 • SO2: Increasing the Effectiveness of the Primary Education System Indicator 1: Percentage of students in partnership schools passing criterion referenced test in English and mathematics (national/partnership comparison). Indicator 2: Percentage of increased pupil learning in mathematics, English literacy and spoken English in partnership schools. Indicator 3: Percentage of partnership communities demonstrating sustained community involvement in education process. Indicator 4: Percentage of pupil dropout in partnership schools compared to national rate. • IR 2.1: Improved teaching and supervision Indicator 2.1.1: Percentage of teachers using pupil-focused instructional practices in partnership schools. Indicator 2.1.2: Percentage of teachers using effective classroom management techniques in partnership schools. Indicator 2.1.3: Percentage of head teachers using effective supervisory techniques in partnership schools. • IR 2.2: Improved education management Indicator 2.2.1: Number of QUIPS schools that demonstrate the ability to effectively manage material resources. Indicator 2.2.2: Number of partnership districts that demonstrate the ability to undertake a systematic process for planning and prioritizing resource allocations for education projects. • IR 2.3: Increased community involvement Indicator 2.3.1: Percentage of communities active in school decision-making in partnership schools. Indicator 2.3.2: Percentage of communities using participatory methodologies in initiating school projects in partnership schools. • IR 2.4: Improved learning environment Indicator 2.4.1: Number of schools adopting techniques to promote equity in the partnership districts. Indicator 2.4.2: Number of communities and schools in partnership districts implementing a community-school improvement plan (C-SIP). Substantial programmatic results were achieved under QUIPS. These achievements were well captured by the results indicators associated with SO2. SO2. A major reason for the success of the indicators is 6 See Annex 3 for a more detailed discussion of the performance indicators. 6 that evolved as the SO2 program evolved. The implementing partners were able to continuously measure these IRs throughout the life of the QUIPS program. The indicators are useful for assessing performance management because they address the key issues related to improving the quality of education in schools. They progressively address district and school-level constraints in four areas: classroom instruction, school and district management, community participation, and enabling environment. IX. LESSONS LEARNED The general approach of the QUIPS Model School worked effectively. In the communities and schools where the QUIPS program had been involved, the holistic Model School approach resulted in a collective school/community vision for education. There was strong community participation and ownership as well as increased teacher knowledge and motivation, which led to measurable quality results and enhanced pupil learning. Parents felt more confident and acquired a sense of ownership and involvement. The use of Appreciative Inquiry as a framework for improving educational quality was a key component in school change. In the Appreciative Inquiry approach, the emphasis is on identifying strengths and assets rather than problems, building a vision of what is possible and making a plan for change. Teachers, head teachers, parents and pupils were active participants in the creation of their schools as places of growth and learning. In this approach all participants had a role to play in creating change. The school-based INSET (In-Service Education for Teachers) approach was a key mechanism for teacher improvement. Applying the Appreciative Inquiry approach to INSET activities had a powerful impact on teacher change and school development. Use of District Teacher Support Teams (DTSTs) was also important to sustainability. Follow-up visits to schools between in-service trainings need to be built into any future teacher improvement strategy. Teacher Training College (TTC) staff provides an important resource in developing and conducting in-service teacher professional development. Micro-grants are an effective means of improving the school learning environment. The purchase of school furniture, textbooks, exercise books and sports equipment had a very positive effect on teacher morale and improved the quality of teaching and learning which, in turn, resulted in increased enrollments and community support for schools. Use of non-project assistance is not an effective means of strengthening the policy/management environment of the Ministry of Education, Youth and Sports (MOEYS). Non-project assistance was supposed to provide financing to the government budget to support the reform of the primary education system and encourage policy changes. However, a lack of sustained policy dialogue between USAID and the MOEYS/GES due to the high level of turnover of division heads within MOEYS/GES meant that the government was unable to absorb and use the non-project assistance funds either efficiently or effectively. This resulted in the need to reprogram funds from non-project assistance to project assistance in order to target specific needs related to the QUIPS program in the field. Close collaboration with MOEYS/GES staff in materials development was critical to sustainability. Provision of training and support of MOEYS/GES staff in the use of new materials was an effective part of the overall strategy. In order to be effective, management training needs to be very practical and applied to tasks at hand. Using the administration of infrastructure development grants as a mechanism for training the District Education Office and District Assembly was important for building senior management teamwork. School infrastructure improvement projects were most effective when their scope was restricted to available resources. Cost-savings were achieved in those cases where a partnership approach through community cash and in-kind contributions was used and District Assembly contributions were provided upfront. 7 The district grant mechanism provided an important vehicle for developing decentralized capacity in implementing educational reforms. District grant program resources were most effective when focused on a limited set of schools and activities. Having regional education offices play an oversight/support role in district grant programs was also an important element of success in developing decentralized capacity. Interventions used to build participation, management, organization, budgeting, planning and communication capacity among different community groups was critical to sustaining community involvement and ownership in support of education. QUIPS worked intensively with 439 communities throughout 110 districts in Ghana, resulting in significant and sustained improvements across all focus areas of the project. QUIPS also worked with the GES to provide a modified approach to more than 2,200 additional communities and management training for School Management Committees (SMC) and Parent Teacher Association (PTA) executives from more than 9,000 communities across the country. Working with all stakeholders in the community, including teachers, to develop and implement School Performance Improvement Plans (SPIPs) provided a realistic roadmap for improving school quality in community-identified priority areas. This approach paved the way for capacity development training of SMC and PTA executives and the provision of micro-grants to community organizations (supplemented by internally generated resources) to support the successful completion of SPIPs targets. Collaborating with MOEYS/GES education personnel at national, regional, and district levels on development of SMC/PTA project materials and intervention strategies led to capacity building of MOEYS/GES to replicate and mainstream SMC/PTA initiatives at both the community and district level. Over the life of the QUIPS program, there was a consistent transfer of implementation responsibilities to MOEYS/GES partners. For example, the GES took the majority of the responsibility for the implementation of the SMC/PTA Training Expansion Program and shared responsibility for the training delivery of the Monitoring and Evaluation Capacity Building Program, which were two activities that benefited the entire country. To support this transfer the QUIPS program provided regular and targeted capacity building in the areas of community mobilization, SMC/PTA training, and monitoring and evaluation. In the last year of the program, for example, more than 14,000 teachers, 2,200 MOEYS/GES officers, and 43,000 community representatives received training. Building the capacity of local NGOs and working with the National Service Secretariat (NSS) strengthened Ghana’s social development professional resource base. The QUIPS program partnered with the NSS to train and post more than 350 university graduates as community development workers with several local Ghanaian NGOs to deliver community-based education improvement programs and with an education assessment NGO to design and implement a Monitoring and Evaluation Capacity Building Program. Working with MOEYS/GES to develop innovative, accessible tools and methodologies for the design, implementation and assessment of participatory community development and district management initiatives developed effective and transportable materials. The materials, which included manuals, instruments, and information, education and communication (IEC) tools, have been adopted as official documents by the MOE/GES. By serving as a ‘testing ground’ for the development of innovative strategies for achieving the goals of fCUBE, QUIPS influenced operational policies of the MOEYS/GES for improving community participation in education. By partnering with the MOEYS/GES in the design and delivery of project activities and materials, the QUIPS program generated ownership within the government that led to the mainstreaming of the SMC/PTA training program and materials, the adoption of a shared format for all school-community level planning, and the creation of a post of Community Participation Coordinator (CPC) at the District Education Office (DEO). Through these and other activities, QUIPS advanced the dialog on the importance of community participation in education at the national level, garnered the support of many top government officials, and changed the way in which GES interacts with the community. 8 Incentives are key to adapting new practices. A key assumption of the ILP project was that teachers and GES staff would be intrinsically motivated to apply new ideas, innovations, and approaches in carrying out their professional responsibilities. QUIPS’ experience under ILP suggests that there were a considerable number of teachers and other GES professional staff at the district, regional, and national level who were unwilling to put in the extra effort to adopt new practices without the provision of tangible inducements, such as additional benefits in the form of higher levels of per diem to participate in training, other in-kind benefits (e.g., study tours), awards for outstanding performance, opportunities for further formal education, and performance-based promotions. Professional commitment and competence leads to system-wide support of teachers’ professional development. The ILP strategy relied on strong circuit supervisor and head teacher support as mechanisms for ensuring that new teacher practices and school/classroom management practices were institutionalized. QUIPS’ experience under ILP suggests that a considerable number of circuit supervisors were not able to fulfill adequately this support role. In some cases, the cause was attributable to logistical constraints (i.e., transportation or payment of travel costs). In a substantial number of cases, however, it appeared that circuit supervisors were not sufficiently committed or lacked the necessary professional skills to provide professional support to teachers and schools. Teacher supervision results in more effective use of instructional time. Experience in the ILP project suggests that GES, district education offices, and head teachers should focus more on ensuring that instructional time in the classroom is used efficiently and effectively. This is based on impromptu and regular school visits which revealed a pattern of teacher absenteeism, lack of punctuality, and inefficient use of classroom instructional time. This requires support supervision, as well as a readiness by authorities to take necessary disciplinary action when teachers are not performing up to standards. High score on sustainability index is a strong predictor of continued growth. The goal of the CSA project was that each partnership community show a moderate to high sustainability level by the end of the project. The sustainability level represents an estimate of a community’s probability of sustaining a process or result. QUIPS developed an index to predict the extent to which changes that had taken place within the community would not only be sustained, but that the community would continue to move in a more positive direction. QUIPS found that 72% of communities from all cohorts achieved a moderate to high sustainability level by the end of project interventions, with more than half scoring in the high sustainability range while only 13% were determined to have no sustainability potential. National level institutionalization leads to sustainable CSA Intervention Schemes and Tools. At the national level, the MOEYS/GES institutionalized many of the materials and processes that had been developed over the course of the CSA project. MOEYS/GES adopted the SMC/PTA Handbook as official SMC and PTA materials and subsequently conducted training for all school-communities in Ghana using GES trained district-level personnel. MOEYS/GES carried out community mobilization activities and developed School Performance Improvement Plans (SPIPs) in at least 20 school-communities in each district using the CSA developed PLA process. MOEYS/GES established Community Participation Coordinator (CPC) positions within each district to facilitate the implementation of community participation activities. The National Service Secretariat (NSS) incorporated the CSA Facilitator Community Mobilization Manual into its training of National Service Volunteers. MOEYS/GES requested NSS to assign at least two NSS personnel to each district to support community mobilization activities and supported these assignments with district budget allocations. Several District Education Offices (DEOs) developed strategies for replicating CSA activities in select school-communities. These DEOs were able to identify funding from their district director budgets, district assembly common funds, and donor institutions. Effective promotion of girls’ education requires a multi-sector approach. In order to be sustainable, effective promotion of girls’ education cannot be limited to a single organization or sector of society. Several factors impinge on the promotion of girls’ education. To be able to address the issues effectively and achieve the objectives of the project, a variety of relevant organizations and sectors must be engaged. 9 HIV/AIDS awareness training in education requires strong leadership support. Sustainability requires: (1) developing special criteria to identify trainers (e.g., a broad cascade approach); (2) providing a series of intensive training activities for those persons who serve as trainers; and (3) providing special training to district directors and head teachers of SSS, JSS and primary schools to ensure that they provide the leadership support necessary for effective training is in place. X. RELATED STUDIES AND ASSESSMENTS “QUIPS Joint Program Description (Revised Final Copy).” Lisa Franchett, EHRDO, USAID/Ghana. April 2002. “QUIPS Mid-Term Assessment.” Authored by: Margaret Bonner, Cameron Bonner, Joy du Plessis, Ash Hartwell, and Celia Marshall. March 2001. “Strategic Objective No. 2: Increasing the Effectiveness of Primary Education.” Peter Kresge, SO2 Team Leader. USAID/Ghana. September 26, 1996. XI. CLOSE-OUT REPORTING INSTRUMENTS “Community School Alliances Project Final Report.” Submitted by EDC, Inc. December 1, 2004. Prepared by Kay Leherr, Kingsley Arkorful, Jerry Boardman and Ron Israel. “Strengthening HIV/AIDS Partnerships in Education Project. Final Program Report.” Submitted by World Education. December 2004. “Close Out Report Quality Improvement of Primary Education (QUIPS).” Submitted by CRS/Ghana. December 2004. “Quality Improvement in Primary Education Schools. Improving Learning through Partnerships Project (QUIPS/ILP) Final Report Cooperative Agreement.” Submitted by AED. November 2004. “Strategies for Advancing Girls’ Education (SAGE) Ghana Project. Phase II. Final Report.” Submitted by AED. October 2004 “Quality Improvement in Primary Education Schools. Improving Learning through Partnerships Project (QUIPS/ILP) Final Report.” Submitted by AED. September 2003. “QUIPS/IFESH Annual Report. September 30, 2001 – August 31, 2002. Primary School Teacher Training Project Grant Extension for One Year. (New Focus).” Submitted by IFESH. XII. HUMAN RESOURCE CONTACTS Lisa Franchett, Supervisor Program Officer, USAID/DAKAR/PRM Email: lfranchett@usaid.gov. Tel. +221-869-6100. Ext. 3502 Elsie Menorkpor, Education Development Specialist, USAID/ACCRA/EHRD Email: emenorkpor@usaid.gov. Tel. +233-21-228440. Ext. 213 William Osafo, Education Development Specialist, USAID/ACCRA/EHRD Email: wosafo@usaid.gov. Tel. +233-21-228440. Ext. 203 10 Annex 1: Distribution of Funds between Project Assistance and Non-Project Assistance over Time Date Project Assistance Cumulative Total Non Project Assistance Cumulative Total Summary Obligation per Amendment SOAG Obligation (Cumulative) Non Project Assistance Project Assistance Planned Life of Project Funding Original 5,897,000 2,500,000 8,397,000 8,397,000 18,000,000 35,000,000 53,000,000 06/26/1997 9,847,000 2,500,000 3,950,000 12,347,000 18,000,000 35,000,000 53,000,000 09/30/1997 9,847,000 6,500,000 4,000,000 16,347,000 18,000,000 35,000,000 53,000,000 09/30/1997 9,872,000 6,500,000 25,000 16,372,000 18,000,000 35,000,000 53,000,000 05/14/1998 14,672,000 6,500,000 4,800,000 21,172,000 18,000,000 35,000,000 53,000,000 09/30/1998 14,672,000 10,500,000 4,000,000 25,172,000 18,000,000 35,000,000 53,000,000 08/13/1999 22,267,000 10,500,000 7,595,000 32,767,000 18,000,000 35,000,000 53,000,000 06/06/2000 27,587,000 10,500,000 5,320,000 38,087,000 14,000,000 39,000,000 53,000,000 09/25/2000 27,587,000 10,500,000 0 38,087,000 14,000,000 39,000,000 53,000,000 03/23/2001 27,587,000 10,500,000 0 38,087,000 14,000,000 39,000,000 53,000,000 06/21/2001 33,444,000 10,500,000 5,857,000 43,944,000 14,000,000 50,750,000 64,750,000 07/17/2002 39,644,000 10,500,000 6,200,000 50,144,000 14,000,000 50,750,000 64,750,000 08/23/2002 44,144,000 6,000,000 0 50,144,000 9,500,000 55,250,000 64,750,000 09/27/2002 44,644,000 6,000,000 500,000 50,644,000 6,000,000 51,867,000 57,867,000 08/22/2003 51,867,000 6,000,000 7,223,000 57,867,000 6,000,000 51,867,000 57,867,000 11 Annex 2: Detailed Discussion of Impact of ILP and CSA Projects Impact of ILP—Improving Learning through Partnerships Project • Development of Curriculum and Instructional Materials: ILP worked collaboratively with GES to develop instructional support materials that complemented the new curriculum and syllabi and helped strengthen human resource capacity in these areas throughout the GES system. These materials were initially pilot–tested in QUIPS partnership schools and later adopted by non-partnership schools. • Promotion of In-Service Teacher Professional Development Program: A total of 275 schools in 86 districts benefited from the in-service teacher professional development program. The overall effect on pupil learning and changes in teacher behavior and head teacher practices was very positive. • Improvement of Education Management: ILP supported the MOEYS’s commitment to decentralization by designing management training programs that built human capacity to plan and manage education resources more effectively at the district and school levels while also helping to refine and support implementation of policy initiatives at the national level. More than 700 GES officers in 86 districts benefited from a series of management training programs and their skills were directly applied in preparing district grant work plans. Some 550 trainers were trained to introduce supervisory reference manuals to head teachers and circuit supervisors throughout the country. A similar number were also trained to introduce GES’s new performance appraisal system from district to school level personnel (i.e., DEO staff, head teachers and teachers). • Introduction of District Grant Mechanism: The district management training strategy was redesigned to focus directly on practical management skills that could be applied in developing plans and then implementing, monitoring, and evaluating activities to be funded under the QUIPS district grant program. • School Infrastructure Improvement: ILP helped improve the school learning environment through rehabilitation and construction of school facilities in 275 partnership schools. Specific project activities that were undertaken included: the addition of 632 new classroom blocks, rehabilitation of 247 existing blocks, building of 81 toilet facilities and teachers’ quarters, and the provision of classroom furniture. Impact of CSA—Community School Alliances Project • Community Level Impacts: CSA monitored project impact in the 347 communities where CSA operated. CSA worked with communities in six different cohorts, with each cohort receiving an average of two years of support from the CSA project. Impact analysis of all cohorts indicated that significant improvement was achieved in nine focus areas. More than three-quarters of communities achieved high performance in the areas of participation (83%), empowerment (78%), gender (78%), and partnerships (75%). Almost two-thirds of the communities demonstrated high performance in the areas of management (73%) and resources (66%). At least half of the CSA communities achieved high performance in the areas of quality (62%), trust (61%), and transparency (50%). • District Level Impacts: Following the March 2001 Mid-Term Assessment of the QUIPS program, CSA implemented a Monitoring and Evaluation Capacity Building program in all 110 districts. Each district participated in an 18-month training cycle that had three components: (1) residential M&E training; (2) residential computer training; and (3) on-site support training. The program focused on enhancing the use of indicators, data collection, data management, data analysis, dissemination, and use of information. Each district also received a computer system to support the implementation of M&E activities. Impact analysis revealed that the districts had substantial improvements in performance in all focus areas. 12 Impact of SAGE—Strategies for Advancing Girls’ Education The Strategies for Advancing Girls’ Education (SAGE) project aimed to mobilize public and private sector constituencies to improve the educational participation of girls. It worked with the education system and the community to promote values and skills, such as girls’ self-esteem, leadership, gender equity, safe schools and prevention of sexually transmitted diseases. It also brought in the business sector and media to gain support for helping girls gain access to education, stay in school, and achieve. The SAGE project worked with the Peace Corps, Girls' Clubs, and the Girls Education Unit of the Ministry of Education, Youth and Sports in mobilizing local communities and to train both government officials and community organizations on issues of girls' education in 70 communities. SAGE also helped create a Media Task Force that produced a TV documentary on SAGE Ghana. Impact of SHAPE—Strengthening HIV/AIDS Partnerships in Education World Education’s efforts under SHAPE focused on: (1) building the capacity of the MOEYS/GES School Health Education Program (SHEP) and Ghanaian civil society organizations (CSOs) to effectively support HIV/AIDS prevention programs; (2) increasing the knowledge, attitudes, and behaviors of students, parents and teachers related to HIV/AIDS; (3) increasing the number of school-based activities related to HIV/AIDS education, prevention and support; (4) increasing the capacity of the education sector to respond to the epidemic; and (5) increasing the capacity of teacher training colleges to address HIV/AIDS. 13 Annex 3: Detailed Discussion of Performance Indicators • Indicator SO2.1: Percentage of students in participating QUIPS schools passing Criterion Referenced Test (CRT) in English and Mathematics. The Criterion Referenced Test (CRT) was supposed to be a key performance indicator of the quality of teaching and learning as a measure of pupil learning outcomes. The CRT was designed to monitor national levels of attainment in Math and English among children completing six years of primary school. Results from this test were only available for the 1999 and 2000 academic years. MOEYS did not process or release data in subsequent years and finally stopped administering the test altogether because the results were not considered adequate measures of student progress. The CRT is now being replaced by a new minimum national standards test called BECAS, which is supported by USAID and is expected to be administered in June 2005. SO2.1--CRT 1997 1998 1999 2000 2001 2002 2003 2004 Mathematics Planned n/a n/a 7.0% 9.0% 8.0% 8.0% 8.0% 8.0% Actual--National 2.7% n/a 4.0% 4.4% n/a n/a n/a n/a Actual--QUIPS n/a n/a 7.7% 6.3% n/a n/a n/a n/a English Planned n/a n/a 11.7% 13.7% 12.7% 12.7% 12.7% 12.7% Actual--National 6.2% n/a 8.7% 9.6% n/a n/a n/a n/a Actual--QUIPS n/a n/a 14.7% 13.2% n/a n/a n/a n/a • Indicator SO2.2: Increased pupil learning in Mathematics, English literacy, and spoken English in partnership schools. The QUIPS achievement tests are aligned with the goals of the QUIPS program and are designed to track pupil learning growth over time. By supplementing the CRT with the QUIPS achievement tests, it was possible to evaluate both impact on the system overall and impact on pupil learning effectiveness. Test results indicate that the QUIPS program exceeded its targets in math although spoken English and English reading were below target. SO2.2--QUIPS Achievement Test 1998 1999 2000 2001 2002 2003 2004 Mathematics Planned 28% 33% 60% 60% 60% 60% Actual 23% 30% 34% 66% 67% 63% 64% English Reading Planned 49% 54% 55% 55% 55% 55% Actual 44% 51% 55% 54% 54% 52% 53% English Speaking Planned 65% 60% 60% 60% Actual 58% 54% 49% 53% • Indicator SO2.3: Percentage of communities demonstrating sustained community involvement in education process. Sustained community involvement is defined as communities that provide evidence of continued involvement with schools after the QUIPS interventions have been completed. This is measured by compiling community ratings in 5 focus areas known to be related to sustainable community involvement: (1) empowerment; (2) participation; (3) partnerships; (4) resources; and (5) transparency. The QUIPS program performed extremely well in promoting greater community participation in schools which has 14 been shown to contribute positively to teaching and learning outcomes. The level of community involvement in schools supported under the program consistently exceeded the target each year. SO2.3--Sustained Community Involvement 2001 2002 2003 2004 Planned n/a 75% 75% 75% Actual 100% 95% 94% 97% • Indicator SO2.4: Rate of pupil dropout in partnership schools compared to national rate. National dropout rates are based on the annual primary school census conducted by the Ministry of Education, Youth and Sports (MOEYS). QUIPS school-based data were collected by AED and CRS. Rate of pupil dropout is defined as the residual dropout rate, or the proportion of pupils registered in a given school year who fail to enroll the following year. Rate of pupil dropout was considered the most appropriate indicator for monitoring QUIPS impact on pupil participation because Ghana’s policy required that primary school pupils are promoted. Even though all schools may not adhere strictly to the “no repetition” policy, pupil promotion rates tend to be relatively high (and rates of pupil repetition tend to be low). Therefore, pupil dropout was considered to be a more meaningful performance indicator for addressing pupil participation. Furthermore, primary school pupil dropout, particularly for girls and particularly for rural communities in Northern Ghana is known to be a problem in Ghana. Planned results for the QUIPS schools on pupil dropout, though expected to be lower than the National levels, was not anticipated to be dramatic because pupil dropout is impacted by a wide range of factors, many of which may not be related to the school and community interventions (e.g., family economics and health factors). The impact of the sustained community involvement in schools may have been a major factor contributing to the low dropout rates of pupils in schools supported by the QUIPS program. Although there was a slight upturn in the dropout rate achieved in 2003, the result of 4.2% was still significantly lower than the national rate of 9%. SO2.4--Dropout Rates 1998 1999 2000 2001 2002 2003 2004 Planned--QUIPS Total 8.0% 7.0% 7.0% 5.0% 5.0% 5.0% Males 8.0% 7.0% 7.0% 4.5% 4.5% 4.5% Females 8.0% 7.0% 7.0% 5.5% 5.5% 5.5% Actual--National Total 9.1% n/a 7.2% 5.4% 9.0% 9.0% 9.0% Males 8.8% n/a 6.7% 4.8% 8.0% 8.0% 8.0% Females 9.4% n/a 7.6% 6.0% 10.0% 10.0% 10.0% Actual--QUIPS Total n/a 4.9% 3.6% 4.8% 3.0% 4.2% Males n/a 4.7% 2.7% 4.5% 1.9% 3.8% Females n/a 5.1% 4.5% 4.8% 3.9% 4.6% IR.2.1: Improved Teaching and Supervision • Indicator 2.1.1: Percentage of teachers using pupil-focused instructional practices in Partnership Schools. Classroom observation data were collected by AED and CRS Teacher Trainers in all QUIPS schools at baseline and at the end of each school term during the 2-year QUIPS interventions. Remarkable progress was made in terms of introducing teachers to new teaching methodologies and approaches and supporting them to improve their lesson preparation and delivery skills. The percentage of teachers using learner centered approaches exceeded the planned targets. Increases in teachers using pupil-focused instructional practices were associated with the increase in math. However, it remains unclear why these 15 trends in improved instructional practices did not carry over to the results of spoken English and English reading achievement. Indicator 2.1.1: Pupil-Focused Instruction 1998 1999 2000 2001 2002 2003 2004 Planned n/a 20% 35% 55% 62% 62% 62% Actual 15% 37% 47% 71% 57% 69% 73% • Indicator 2.1.2: Percentage of teachers using effective classroom management techniques in Partnership Schools. The classroom observation instrument scoring criteria were associated with five items that measure teacher’s ability to productively manage the classroom: (1) effectively use physical space; (2) minimize use of classroom time for routine tasks; (3) maintain control and discipline in a fair and consistent manner; (4) use productive grouping strategies; and (5) facilitate pupil-teacher interaction in the classroom. Although the aggregate result indicates that the overall target for the last reporting period was exceeded by six percentage points, results from 3 out of the 6 cohorts did not met the target set. Thus there remained a significant number of QUIPS schools where the practices associated with the indicator had not been sufficiently mainstreamed. Indicator 2.1.2: Effective Classroom Management 1998 1999 2000 2001 2002 2003 2004 Planned n/a 20% 35% 55% 50% 50% 50% Actual 7.6% 46% 50% 53% 44% 56% 56% • Indicator 2.1.3: Percentage of head teachers using effective supervisory techniques in Partnership Schools. Effective head teacher supervisory practices were measured by ratings on two Supervisory and Classroom Management Instrument (SCMI) items: (1) number of visits paid to teachers for observation; and (2) number of staff meetings held. As in all years of the program the indicator target of 61% was not met by the program in the reporting year (FY 2004). A number of factors including the non-detachment of head teachers from classroom teaching responsibilities in a number of QUIPS schools could have affected the performance of the program on the indicator. Indicator 2.1.3: Effective Supervisory Techniques 1998 1999 2000 2001 2002 2003 2004 Planned n/a 65% 70% 75% 61% 61% 61% Actual 60% n/a 69% 64% 34% 33% 39% IR 2.2: Improved Education Management • Indicator 2.2.1: Number of QUIPS schools that demonstrate the ability to effectively manage materials resources. Schools that demonstrate the ability to manage materials resources are defined as schools that provide evidence of keeping up-to-date and accurate records which track the flow of construction materials, texts and other teaching/learning materials. Indicator 2.2.1: Manage Material Resources 1999 2000 2001 2002 2003 2004 Planned n/a 80 135 183 239 239 Actual 1 81 114 151 224 235 16 • Indicator 2.2.2: Number of District Education Offices that demonstrate the ability to improve the planning, implementation, monitoring and evaluation of education programs. This indicator is based on District Education Office (DEO) applications and work plans for accessing District Grant funding and progress reports on the implementation of District Grant activities submitted to USAID by AED and CRS. For a district to be counted as meeting the result, it would have had to prepare an acceptable work plan, and a progress report linked to the District Grant Mechanism. This indicator presupposed the training of District Education Office staff in a system of utilizing available and relevant information to prioritize, plan and allocate resources to the implementation of activities linked to improving teaching and learning. Indicator 2.2.2: Planning, Implementation, Monitoring and Evaluation 1999 2000 2001 2002 2003 2004 Planned n/a n/a n/a 14 23 46 Actual n/a n/a - 12 48 38 IR 2.3: Increased Community Participation • Indicator 2.3.1: Percentage of communities active in school decision-making in Partnership Schools. Communities judged to be active in school decision making are defined as communities that reach high performance in the Empowerment focus area, as measured by the Best Practices Assessment Instrument. Indicator 2.3.1: Communities Active in School Decision￾Making 1998 1999 2000 2001 2002 2003 2004 Planned 65% 70% 70% 65% 65% 65% Actual 11% 87% 78% 80% 78% 70% 85% • Indicator 2.3.2: Percentage of communities using participatory methodologies in initiating school projects in Partnership Schools. Communities using participatory methodologies in initiating school projects are defined as communities that reach high performance in the Participation focus area as measured by the Best Practices Assessment Instrument. Indicator 2.3.2: Communities Using Participatory Methodologies 1998 1999 2000 2001 2002 2003 2004 Planned 50% 70% 70% 70% 65% 70% Actual 19% 74% 86% 78% 80% 71% 71% IR 2.4: Improved Learning Environment • Indicator 2.4.1: Number of schools adopting techniques to promote equity in the Partnership Districts. Schools where the majority of teachers (=>75%) encourage all children to participate, particularly girls, are considered to have successfully adopted techniques to promote equity. Teacher performances were based on ratings on the two items: (1) encouragement given to girls to participate; and (2) all learners are encouraged to participate. Indicator 2.4.1: School Adopting Equity Techniques 1998 1999 2000 2001 2002 2003 2004 Planned n/a 35 50 120 183 239 239 Actual 23 32 55 157 246 331 331 17 • Indicator 2.4.2: Number of communities and schools in Partnership Districts implementing Community School Improvement Plans. Community-School Improvement Plans are a planning tool to assist the school in developing and achieving targets in the areas of: teaching and learning; school management; infrastructure/access; and community participation. Schools/ communities that were successful in the following two areas were considered as successful implementers of the C-SIP: (1) meeting annual targets; and (2) modifying targets annually to meet the changing requirements of school improvements. Indicator 2.4.2: Implementing CSIPs 1998 1999 2000 2001 2002 2003 2004 Planned 36 75 125 183 239 239 Actual - 40 80 124 140 158 176 18