GHANA CASE STUDY SUMMARY REPORT EVALUATION OF SUSTAINED OUTCOMES JUNE 26, 2017 This publication was produced for review by the United States Agency for International Development. It was prepared by Management Systems International, A Tetra Tech Company; and Development and Training Services, a Palladium company, for the E3 Analytics and Evaluation Project. GHANA CASE STUDY SUMMARY REPORT EVALUATION OF SUSTAINED OUTCOMES Contracted under AID-OAA-M-13-00017 E3 Analytics and Evaluation Project DISCLAIMER The author’s views expressed in this publication do not necessarily reflect the views of the United States Agency for International Development or the United States Government. Ghana Case Study Summary Report: Sustained Outcomes Evaluation ii CONTENTS Acronyms...........................................................................................................................iv Map of Ghana......................................................................................................................v Acknowledgements...........................................................................................................vi Executive Summary.........................................................................................................vii Introduction .........................................................................................................................................vii The Case: QUIPS................................................................................................................................vii Evaluation Methodology...................................................................................................................viii Case Study Findings............................................................................................................................ ix Conclusion........................................................................................................................................... xii I. Introduction........................................................................................................... 1 A. Sustained Outcomes Evaluation Objectives................................................................... 1 B. Overview of Quality Improvement in Primary Schools and Community Schools Alliance..................................................................................................................... 2 II. Case Study Methodology...................................................................................... 4 A. Evaluation Design.................................................................................................................. 4 B. Phase I Evaluation Research ............................................................................................... 6 C. Phase II Evaluation Research.............................................................................................. 9 D. Data Analysis........................................................................................................................11 E. Study Limitations and Facilitators...................................................................................13 F. Case Study Team................................................................................................................15 III. Ghanaian Context ............................................................................................... 15 A. Broad Ghanaian Context..................................................................................................15 B. Structure of the Ghanaian Education System ..............................................................16 IV. QUIPS/CSA Project Activities........................................................................... 19 V. Case Study Findings ............................................................................................ 21 A. Evaluation Question 1 .......................................................................................................21 B. Evaluation Question 2 .......................................................................................................24 C. Evaluation Question 3 .......................................................................................................25 D. Evaluation Question 4 .......................................................................................................28 VI. Conclusion............................................................................................................ 29 Annexes............................................................................................................................ 30 Annex A: Timelines......................................................................................................... 31 Annex B: Focused Activity Map..................................................................................... 33 Annex C: Formal Education System Map..................................................................... 34 Ghana Case Study Summary Report: Sustained Outcomes Evaluation iii Annex D: List of Interviewees........................................................................................ 35 Annex E: List of Documents Consulted........................................................................ 37 Annex F: Ghana Case Study Team Profiles.................................................................. 39 Ghana Case Study Summary Report: Sustained Outcomes Evaluation iv ACRONYMS CPC Community Participation Coordinator CSA Community Schools Alliance E3 Bureau for Economic Growth, Education, and Environment (USAID) EQ Evaluation Question fCUBE Free and Compulsory Universal Basic Education GES Ghana Education Service GoG Government of Ghana IR Intermediate Result LER Office of Learning, Evaluation, and Research (USAID/PPL) MoE Ministry of Education (Ghana) MSI Management Systems International PPL Bureau for Policy, Planning, and Learning (USAID) PS Primary School PTA Parent Teacher Association SMC School Management Committee QUIPS Quality Improvement in Primary Schools UPE Universal Primary Education USAID United States Agency for International Development Ghana Case Study Summary Report: Sustained Outcomes Evaluation v MAP OF GHANA Ghana Case Study Summary Report: Sustained Outcomes Evaluation vi ACKNOWLEDGEMENTS This report would not have been possible without the time and effort freely given by all those involved in helping to organize site visits, speak with or write to the case study team, review data interpretation and reports, and dig through mounds of old files. We thank you. From the individuals who met us in offices in Accra and Kumasi, to those that met with us under a tree in a school yard, the team acknowledges the generous and patient individuals who allowed us to interview them for this study. We aimed to capture their stories about their personal interaction with the activities associated with a USAID intervention and the impact that intervention had on their lives and their communities. We tried, in our way, to weave these stories together much like the famed Asante Kinte cloths from that part of Ghana. Ghana Case Study Summary Report: Sustained Outcomes Evaluation vii EXECUTIVE SUMMARY Introduction This report looks at one outcome of the United States Agency for International Development’s (USAID’s) Ghana Quality Improvement in Primary Schools (QUIPS) program. The QUIPS program constitutes one of four case studies being conducted for an ex-post systems evaluation that is exploring the factors that contribute to sustained USAID outcomes. The Office of Learning, Evaluation, and Research in USAID’s Bureau for Policy, Planning, and Learning (USAID/PPL/LER) commissioned this evaluation to address four evaluation questions (EQs): • EQ 1: Were USAID-intended outcomes sustained? • EQ 2: What other outcomes resulted from the project (positive/negative) and were these outcomes sustained? • EQ 3: What has contributed to or hindered sustaining the outcomes? • EQ 4: How are the outcomes perceived and valued by those with significant stakes in the project? To address these questions, the evaluation looked at one outcome delivered by the Community Schools Alliance (CSA) project implemented under QUIPS: Enhancing parent engagement in local education. The Case: QUIPS In 1996, the Ghanaian Ministry of Education (MoE) launched the Free Compulsory Universal Basic Education Program (fCUBE). The goal of fCUBE was to implement universal primary education (UPE) and provide an opportunity for every school-age child in Ghana to receive a quality basic education by 2005. fCUBE had three primary components: improve the quality of teaching and learning; improve efficiency in management; and increase access and participation (e.g., raising attendance levels by all students, with a focus on girls, and increasing parental involvement in schools). USAID’s Basic Education Strategy in Ghana (1997–2004) aimed to support the MoE to expand quality primary education, including through broad decentralization initiatives. The primary USAID mechanism to support fCUBE was QUIPS, an umbrella program delivered from 1996 to 2004 through several projects, including the Community Schools Alliance (CSA) project in the south of the country. CSA, delivered by Education Development Center, used public awareness activities, participant rural appraisals, and capacity-building training for school managers and leaders of parent-teacher associations (PTAs) and school management committees (SMCs) to build support for education, parent engagement, and shared responsibility for school management. An external evaluation of QUIPS in 2005 found that CSA contributed to shifts in community behavior related to: (1) supporting school quality; (2) supporting girls’ education; (3) strengthening school management; and (4) participatory planning and design. It also found that QUIPS contributed to five significant and relevant results: • Increased awareness of, interest in, and commitment (on the part of communities and parents) to the process of education. • Strengthened relations between teachers and the community. • Strengthened community leadership for improving schools through the SMC/PTA and traditional leadership structures. Ghana Case Study Summary Report: Sustained Outcomes Evaluation viii • Improved community support of schools via enrollment drives, community contributions, and maintenance of the school, including food support, books, and volunteer teachers. • Heightened sense of parental responsibility for their children’s well-being and education. Evaluation Methodology As requested by USAID, the evaluation used a systems approach1 as the guiding methodology. USAID defines systems thinking as a “set of analytic approaches—and associated tools—that seek to understand how systems behave, interact with their environment, and influence each other. Common to these approaches is a conviction that particular actions and outcomes are best understood in terms of interactions between elements in the system.” 2 Accordingly, the methodology encouraged a case study that reflects holism: that this was not an evaluation of QUIPS nor of the CSA activity. Prior to the case study research, the case study team lead and education specialist conducted preparatory activities that informed the case study research design. These activities included exploratory interviews, document review, and developing early system maps, timelines, and a program description. These activities also informed the case study team planning meeting, during which the team received training on the approach, selected the outcome of interest, refined the evaluation tools (including timelines and systems maps), identified an initial list of key stakeholders, and selected sites at which to conduct the study. Outcome Selection The case study team selected the QUIPS outcome of “enhancing parent engagement in local education” for several reasons. First, evidence from the final evaluation of QUIPS showed that this outcome had been at least partially achieved – a precondition for sustainment. Second, under this outcome, QUIPS established formal and informal relationships between education institutions and tracing the intervention’s sustainability through the presence of these units and relationships seemed plausible. Third, the outcome was closely linked to improved education quality and improved student learning, important goals of USAID education programming. Finally, the community-level intervention provided a manageable context to explore, given the constraints of time, resources, and personnel. The team chose to limit the study to one outcome due to time and resources constraints, and the belief that exploring a single outcome would be a sufficient unit of analysis to begin to understand the influences on sustainability after USAID funding has ceased. Site Selection and Data Collection Using information identified during preparatory research, the case study team identified research sites that were shown to be high-performing in 2004, where people familiar with QUIPS remained and were accessible for interviews. The case study team selected the Ashante region of Ghana, and within that region, one district (Kwabre) that had since been divided into two: Kwabre East and Efigya. Within these districts, the team selected seven communities (and the schools within these communities) that satisfied the sampling criteria. 1 This approach requires that donors and evaluators resist conceptualizing solutions in overly simplistic, acontextual, and ahistorical ways (Miller, 2016). Systems thinking demands avoidance of a single perspective (Flood, 1999; Midgley, 2000) and is the antidote to dogmatism in social problem-solving (Reynolds & Holwell, 2010). Applying systems thinking to problem-solving has the potential to lead to the identification of solutions that bring in broader, innovative thinking, and do not place the donor’s intervention at the center of change, or at the heart of an evaluation (Meadows, 2008). 2 USAID’s Local Systems: A Framework for Supporting Sustained Development (April 2014). Ghana Case Study Summary Report: Sustained Outcomes Evaluation ix The team conducted 57 interviews: 4 with key experts at the national level, 8 with regional- and district￾level officials, 3 with former QUIPS staff and consultants living abroad, and the remainder at the local level. In the middle of the data collection, the team began to reach data saturation on certain elements of the narrative; by the end of data collection, the team had reached data saturation for the narrative provided in this report. Study Limitations and Facilitators This case study faced several limitations, including constraints on time and resources and the presence of cognitive biases on the parts of respondents and researchers. Very few CSA documents were available to the case study team, which strained the team’s understanding of CSA and required it to spend considerable time conducting qualitative research to piece together what happened in CSA. Finally, although the team included three Ghanaians, none were from the Kumasi region and their cultural backgrounds did not match those of the respondents. This “outsider” perspective may have limited both the team’s ability to reframe interview questions to gather the best information and interpret the informants’ responses. The study also benefited from several facilitating factors. First, the team’s composition played a critical role in data collection and analysis, combining broad sector knowledge and networks at the national, regional, and local levels. Second, initial preparatory activities, including the development of a practical research guide and thorough training on the methodology, ensured that the local team was well prepared for, and supported, during field work and data analysis. Finally, selecting a successful program to investigate encouraged respondents to speak candidly and with enthusiasm to the case study team, which influenced the richness and depth of the data. Case Study Findings Through qualitative data, the case study team developed several key findings relating to the extent to which the outcome was sustained in the communities studied. Was the USAID-intended outcome sustained? (EQ 1) Finding: Positive attitudes about the importance of education and promoting school initiatives were sustained. Data strongly suggest that CSA introduced a new concept to communities – that parents should have some measure of authority over how their children were educated. Since 2004, each community has examples of parents working to improve the school grounds (e.g., painting, cleaning, digging latrines). Finding: Community members and school officials involved with CSA continue to believe in community responsibility or participation in school management. Parents, teachers, and principals who were trained by or involved with CSA described its legacy as its message: that all members of the community should share responsibility for school management and children’s education. These typically had to do with issues such as corporal punishment, teacher absenteeism, or interpersonal problems between parents and school staff. Finding: Community members and school officials who were not involved with CSA are less likely to view community participation in school management favorably. The data showed differences between those who were trained by CSA and those who came after the project closed. Head teachers and classroom teachers who were not trained under CSA provided perspectives on community engagement in school management that were often negative. Head teachers and teachers were not in favor of parent engagement in the actual teaching and learning process. Ghana Case Study Summary Report: Sustained Outcomes Evaluation x Finding: Some community relationships were maintained at varying levels of functioning. Some evidence showed that the inter-relationships established between various units within the local school community still existed in some communities 12 years after the close of CSA. However, there is considerable variance among communities in how these relationships functioned, and their usefulness. Explanations for the variance included personal factors, lack of systems and structures, and the level of knowledge or skills. In other communities, the team identified negative relationships between classroom teachers and parents. Finding: Community Participation Coordinator role exists, though its functions have largely discontinued. SMCs, PTAs, structures of tribal or traditional authority, and religious organizations were well established in the communities prior to CSA. The key district-level position introduced through CSA was the Community Participation Coordinator (CPC). The CPC was delegated to supervise community participation activities across a range of local communities within a district. Twelve years after the conclusion of CSA, the CPC role exists; however, its actual function has been largely discontinued. What other outcomes resulted from the project (positive/negative) and were these outcomes sustained? (EQ 2) Finding: Formal engagements between districts and communities developed and continue to exist. Twice a year, since the implementation of CSA, a district-level stakeholder meeting takes place for planning and problem solving. No concrete results were identified from these continued engagements. Finding: Individuals involved with the project were empowered by and benefitted from career growth. Substantial evidence demonstrated how individuals involved in the original CSA project received life-changing opportunities for career development. For example, education managers at the national, regional, and district levels spoke of individuals who had gone through CSA as a trainer of trainers and emerged with an improved ability to present, facilitate, and lead community engagement. What has contributed to or hindered sustaining the outcomes? (EQ 3) Sustaining community support for education is reliant on multiple systems, mainly community, political, financial, and education. The effective intersection of these systems influenced the initial success, and the eventual lack, of sustainability of the CSA’s achievements. Factors that Contributed to Sustained Outcomes Finding: Cultural attitudes toward volunteerism contributed to the sustainment of the PTA and community participation in education. One factor that facilitated the sustained PTA and CSA system was that it coincided with local attitudes about volunteerism. However, parental assumptions that schools are the government’s responsibility contradicted this factor. Finding: Structures that existed during CSA still exist and are functional. The structures that existed during CSA (though not established by CSA) exist today and are functional. Factors that Hindered the Sustainment of Outcomes Finding: Economic obstacles/disincentives to parental participation. The national and regional economies had a powerful influence in reshaping traditional communities. Many rural families had begun to engage in retail commerce either as employees or owners of roadside retailers. This often meant that adults were out of the home early—before children were up—and on their way to work. The result was that many children were left on their own to feed themselves and go to school. When parents Ghana Case Study Summary Report: Sustained Outcomes Evaluation xi returned to their communities in the evenings, they had little time or energy left for attending PTA meetings or engaging in other school activities. Parents with a higher (or perceived higher) economic status than most of the community provided a different economic reason for not attending PTA meetings. These respondents noted that they stopped attending PTA meetings because the meetings provided the PTA only with the opportunity to solicit money. Related to this finding is that various parents reported that PTA meetings resulted only in their being asked to donate in-kind, such as labor to improve the school grounds and facilities. Finding: Traditional systems eroded support for active and democratic school management committees. The evaluation took place in the Ashante region, the heart of the traditional tribal system. The hierarchy of paramount chief, local chiefs, and sub-chiefs wielded significant authority in the education, community, and political system. A permanent seat on the SMC fueled an incongruous dynamic that pit nascent ideas about democracy and power-sharing against traditional ideas about the absolute authority of tribal chiefs. Traditional systems combined with democratic ones created confusion in the community. Finding: The MoE’s policy of rationalization weakened relationships between teachers and the community. The MoE’s policy of rationalization resulted in teachers periodically being relocated from one school to another. The logic of rationalization is to prevent individual teachers and head teachers from becoming too close to the communities, as that may breed corruption. Thus, moving teachers to other communities thwarted the strengthened parent/community/teacher relationships. Finding: Funding constraints or de-prioritization weakened the role of the CPC, who served as a facilitator for community engagement. Under CSA, the CPC played a key role as a supervisor, mentor, and motivator of community participation activities. After the CSA project closed, the CPC became an unfunded position with little budget to travel to support schools. Finding: The number and popularity of private schools in Kumasi increased during CSA and likely reduced the number and proportion of engaged parents of public school students. The number of private primary schools in a district capital north of Kumasi increased from four in 1993 to 64 in 2004. Data suggested that private education drained off the articulate parents, as well as the better-performing learners. Finding: High rates of migration into Kumasi of families with exposure to CSA awareness￾raising activities weakened community participation in school activities. Informants reported a significant increase in transient populations, with people from the rural community relocating to Kumasi and people from remote northern villages moving in to the rural outskirts of Kumasi. This meant that most people who experienced the consciousness awareness programs of CSA were no longer in the community and new community members did not receive the same type of sensitization. How are the outcomes perceived and valued by those with significant stakes in the project? (EQ 4) Informants who had participated in the original training and implementation phase of CSA had very favorable opinions of the outcome achieved by the project. However, informants who were not part of the CSA intervention had only minimal regard, if that, for the outcome achieved by CSA. Other themes provide insight into how outcomes are perceived and valued include: • Traditional leaders did not likely value the role of CSA that encouraged parental engagement and often worked through other-than-traditional venues. Ghana Case Study Summary Report: Sustained Outcomes Evaluation xii • School officials who were not involved in CSA activities viewed SMC as stepping outside its roles in trying to engage in education management. • Parental viewpoints suggested that for some parents, SMCs were merely vehicles to ask for money, while other parents who valued their role in their children’s education found the new models (e.g., PTA) to bring confusion with regard to whom to engage with to resolve education issues. These parents did not highly value CSA or its remaining influences. Conclusion Three contextual factors influenced the sustainability of CSA outcomes. Namely, the institution of UPE and the widespread idea around parents and communities understanding their role in encouraging children to attend school. The growth of democracy and civic engagement combined with economic growth and its impact on the stability of rural and small community relationships and institutions were also contributing factors to understanding the sustainability of the CSA outcome. Three structural factors related to the sustainability of CSA outcomes. Two that should have supported CSA’s sustainability include a decentralized system supports district and local school management, and a financial need at the school level to engage community support that builds community deliberation and consensus. A factor that discernibly contradicted CSA included a government policy that routinely relocated teachers and administrators and counteracted the school community relationships that CSA aimed to strengthen. One of the main objectives of CSA was to encourage parents to support their children’s education by finding meaningful ways for them to support the children’s schools. In some school communities, while parents’ engagement in school affairs may have diminished over time, it appears that support for education remains high. However, data suggest that while most parents accept their critical role in ensuring that children are enrolled in and attend school (thus awareness was raised and maintained in 2016), parental engagement with schools, such as school management and financial responsibilities, is not strong, collaboration among school personnel and parents is often weak, and the lines of authority for managing schools confound parents and communities. Several factors combined to militate against the sustainability of the CSA outcome, including government policies and the fluidity of communities. The evidence suggests that CSA influences seemed to have little to no value when they introduced new systems or ideas that are contrary to, appear to be contrary to, or challenge authority, whether it is in educational, traditional, or cultural structures or systems. Ghana Case Study Summary Report: Sustained Outcomes Evaluation 1 I. INTRODUCTION This document provides key findings for one of four case studies conducted as part of an ex-post evaluation examining what factors contribute to sustained outcomes from international development interventions. The United States Agency for International Development (USAID) intervention selected for study that is discussed in this report is the USAID/Quality Improvement in Primary Schools (QUIPS) with a specific focus on part of that activity, the Community Schools Alliance. A. Sustained Outcomes Evaluation Objectives The Office of Learning, Evaluation, and Research in the United States Agency for International Development’s Bureau for Policy, Planning, and Learning (USAID/PPL/LER) commissioned this evaluation, and USAID’s Bureau for Economic Growth, Education, and Environment (E3) Analytics and Evaluation Project designed and implemented it.3 The primary purpose of the evaluation was to identify factors contributing to sustained outcomes from international development interventions. These factors encompass programmatic characteristics (e.g., the design, management and implementation, monitoring and evaluation, and learning of projects) and contextual features, including local systems. A secondary purpose was to record and learn lessons from the process of designing and implementing an ex-post evaluation taking a systems approach. These lessons are included as an annex to the final evaluation synthesis report.4 Since the secondary purpose of the evaluation is to learn lessons about designing and delivering an ex￾post evaluation using a systems approach, the case study methodology sections that follow include more detail than would otherwise typically be found in a case study or evaluation report. 1. Evaluation Questions As per USAID’s approved Statement of Work and subsequent written evaluation design, 5 this study addressed the following evaluation questions (EQs): • EQ 1: Were USAID-intended outcomes6 sustained7? • EQ 2: What other outcomes resulted from the project8 (positive/negative) and were these outcomes sustained? • EQ 3: What has contributed to or hindered sustaining the outcomes? 3 The E3 Analytics and Evaluation Project is implemented by a team lead, Management Systems International (MSI), and team partners Development and Training Services, a Palladium company, and NORC at the University of Chicago. 4 https://dec.usaid.gov/dec/content/Detail_Presto.aspx?vID=47&ctID=ODVhZjk4NWQtM2YyMi00YjRmLTkxNjktZTcxMjM2NDBmY2Uy&rID=NTA2NDk2 5 The Evaluation Design Proposal: Evaluation of Sustained Outcomes is available at http://pdf.usaid.gov/pdf_docs/PA00M8CN.pdf and includes the evaluation’s Statement of Work as Annex A. 6 ‘Outcomes’ are defined here as the conditions of people, systems, or institutions that indicate progress or lack of progress toward achievement of project/program goals. Outcomes are any result higher than an output to which a given output contributes, but for which it is not solely responsible. Outcomes may be intermediate or end outcomes, short- or long-term, intended or unintended, positive or negative, direct or indirect (USAID Automated Directives System 200-203). 7 ‘Sustained’ refers to something that has been maintained or continued over time. In this evaluation, the reference is to the intended outcome of the USAID activity and its condition in the present time, i.e., some years after the activity’s funding ended. 8 ‘Project’ refers to a set of executed interventions over an established timeline and budget that are intended to achieve a discrete development result through resolving an associated problem. It is linked to the Country Development Cooperation Strategy (CDCS) Results Framework. More succinctly, a project is a collaborative undertaking with a beginning and end that is designed to achieve a specific purpose. Based on consultations between the Project team and USAID/PPL/LER, it was agreed that case study research for this evaluation will focus on what the Agency currently defines as an “activity,” or a sub-component of a project that contributes to a project purpose Ghana Case Study Summary Report: Sustained Outcomes Evaluation 2 • EQ 4: How are the outcomes perceived and valued by those with significant stakes in the project? The evaluation team used empirical evidence to compare the status/achievement of outcome(s) at present to their status/achievement at the end of USAID funding, and explored how the USAID activity contributed to whether an outcome was sustained. 2. Evaluation Audience The primary audience for the evaluation is USAID/PPL, which may use the evaluation to inform program cycle guidance and tools related to the design, implementation and monitoring and evaluation of projects and activities. This includes changes in USAID’s approach to sustainability analysis during project design and guidance, and identifying factors that may foster sustainability in project design and implementation. Other audiences include USAID Missions and individuals involved in the design, implementation, monitoring and evaluation of international development projects. While the evaluation may be of special interest to USAID staff who work on basic education programs, the evaluation findings on factors that influence sustainability may be of interest to USAID staff working in areas other than education. B. Overview of Quality Improvement in Primary Schools and Community Schools Alliance In 1996, the Ghanaian Ministry of Education (MoE) launched the Free Compulsory Universal Basic Education Program (fCUBE). The goal of fCUBE was to provide opportunity for every school-age child in Ghana to receive a quality basic education by the year 2005. fCUBE had three primary components: improve the quality of teaching and learning, improve efficiency in management, and increase access and participation (e.g., raising attendance levels by all students, with a focus on girls, and increasing parental involvement in schools). USAID’s Basic Education Strategy in Ghana (1997-2004) aimed to support the MoE to expand quality primary education, including through broad decentralization initiatives. The primary USAID mechanism to support fCUBE was the umbrella program QUIPS, which was delivered from 1996 to 2004 through several projects. The total value of QUIPS was $75,567,000, which included $51,867,000 in USAID project assistance,9 $6,000,000 in USAID non-project assistance10 as of September 2004, and $17,700,000 in Estimated Host Country Contributions. 11 9 “Project” refers to a set of executed interventions over an established timeline and budget that are intended to achieve a discrete development result by resolving an associated problem. It is linked to the Country Development Cooperation Strategy (CDCS) Results Framework. More succinctly, a project is a collaborative undertaking with a beginning and end that is designed to achieve a specific purpose. Based on consultations, the E3 Analytics and Evaluation Project team and USAID/PPL/LER agreed that case study research for this evaluation would focus on what the Agency currently defines as an “activity,” or a sub￾component of a project that contributes to a project purpose. 10 “Non-project assistance is also known as program assistance. The distinguishing feature of program assistance is the way USAID resources are provided. Under this mode, USAID provides a generalized resource transfer, in the form of foreign exchange or commodities, to the recipient government. This contrasts with other types of assistance in which USAID finances specific inputs, such as technical assistance, training, equipment, vehicles, or capital construction. (This distinction parallels distinctions in law and previous USAID usage between project and non-project assistance.)” See Automated Directives System Glossary of Terms (USAID, 2014), available at: https://www.usaid.gov/sites/default/files/documents/1868/glossary.pdf. 11 See USAID/Ghana, Strategic Objective 2 Close-Out Report (February 2005). Ghana Case Study Summary Report: Sustained Outcomes Evaluation 3 QUIPS was delivered through several projects, 12 the most significant of which were: • Improving Learning through Partnerships, delivered by Academy for Education Development, worked directly with district and school personnel to help improve teaching and learning practices in primary schools in southern Ghana. • Community School Alliances Project, delivered by Education Development Center, worked to increase community awareness and responsibility and advocacy for education, to strengthen School Management Committees (SMCs) and Parent Teacher Associations (PTAs), and to enhance community participation in the design, implementation, and monitoring of school improvement efforts. It also was delivered in southern Ghana. • Training and technical assistance, delivered by Catholic Relief Services, provided in northern Ghana and similar to what was being delivered under QUIPS in the south. The subject of this case study is Community Schools Alliance (CSA) project delivered in southern Ghana. Community Schools Alliance Project CSA intended to enhance community support for education through three objectives: • Ensure parents enroll their children in basic education and that they take responsibility that children attend school regularly. Public education services targeted many families that had previously not participated in basic education. CSA used public awareness campaigns to encourage parents to ensure that their children were: (1) enrolled in school, (2) adequately fed, (3) had at least the minimum school supplies, and (4) attended school regularly. • Motivate parents to support school improvement projects through cash or in-kind contributions. Expanding enrolments meant that most government funds went directly to salaries of teachers, school administrators and managers, while infrastructure and school maintenance spending from regional and local education budgets was strained. CSA built capacity in local PTAs to serve as fund-raising organizations and to provide a pool of parent and community volunteer labor. • Build capacity to SMCs to provide oversight to school decision-making. Launched in 1996, SMCs were advisory committees composed of local education and political representatives. SMCs served as a communication conduit between national, regional and district level political authorities and the local school community. SMCs also were expected to address management issues that important implications for school operations - e.g., budgets, performance monitoring, teacher truancy and absenteeism. CSA provided intensive training to members of school management committees and stimulated the management capacity of SMCs through the introduction of School Performance Improvement Plans (SPIPs). The program also incentivized the SMCs through a system of project-supported school improvement grants. USAID structured the CSA to build the capacity of each unit (e.g. family, SMC, PTA) and strengthen the partnership among individuals and units within the community, thereby creating a strong alliance for the support and supervision of local schools. Section IV provides additional details on CSA. 12 There were seven QUIPS projects in total: Community School Alliances Project, Improving Learning through Partnerships, Performance Monitoring and Evaluation Project, Strategies for Advancing Girls’ Education, Primary School Teacher Training Project, Strengthening HIV/AIDS Partnerships in Education, and the Catholic Relief Services Training and Technical Assistance project. Ghana Case Study Summary Report: Sustained Outcomes Evaluation 4 II. CASE STUDY METHODOLOGY This section describes: • The overall design of the study, including the use of a systems approach to address the evaluation questions; • The research approach that enabled the team to identify appropriate outcomes for analysis and sites and informants for primary research; • The methodologies and approaches used to analyze primary and secondary evidence; and • Limitations and factors that hindered or facilitated the collection and analysis of data to address the evaluation questions. Key factors that the evaluation team used to identify Ghana QUIPS as an appropriate case for inclusion in this study included a focus on basic education, evidence of outcome achievement related to a basic education objective, a relatively benign (non-conflict affected) research environment, and the sufficiency of documentation describing project activities. To limit the length of this report, these criteria are not explained here, but Annexes C and D of the Evaluation Design Proposal present them in detail. 13 The rationale for selecting Community Schools Alliance component is described in more detail on page 8. A. Evaluation Design As USAID requested, the case study team used a systems approach14 as the guiding methodology. USAID defines systems thinking as a “set of analytic approaches—and associated tools—that seek to understand how systems behave, interact with their environment, and influence each other. Common to these approaches is a conviction that particular actions and outcomes are best understood in terms of interactions between elements in the system.”15 Accordingly, the methodology encouraged a case study that reflects holism. While this performance evaluation is examining factors that contributed to sustaining project outcomes, this is not an evaluation of Ghana QUIPS itself, nor does its presentation reflect a typical program evaluation. To implement the systems approach, the team was guided by three specific concepts in collecting and analyzing data. 1. Commitment to multiple perspectives – the team explicitly probed to understand how different groups and individuals perceived issues/situations, differentiating between an individual or group’s common role (e.g., teachers) versus their individual values and motivations. 2. Understanding interrelationships – the team was mindful specifically of the importance of understanding relationships among actors within the system, including how these relationships change over time and have non-linear and emergent aspects. 3. An awareness of boundaries – the team recognized the need to establish explicit boundaries for in-depth research encompassing some or all of considerations of context, actors, relationships, and perspectives, with due consideration of available resources and research capacity. 13 See: http://pdf.usaid.gov/pdf_docs/PA00M8CN.pdf. 14 This approach requires that donors and evaluators resist conceptualizing solutions in overly simplistic, acontextual, and ahistorical ways (Miller, 2016). Systems thinking demands avoidance of a single perspective (Flood, 1999; Midgley, 2000) and is the antidote to dogmatism in social problem-solving (Reynolds & Holwell, 2010). Applying systems thinking to problem-solving has the potential to lead to the identification of solutions that bring in broader, innovative thinking, and do not place the donor’s intervention at the center of change, or at the heart of an evaluation (Meadows, 2008). 15 USAID’s Local Systems: A Framework for Supporting Sustained Development (April 2014) Ghana Case Study Summary Report: Sustained Outcomes Evaluation 5 In addition to the written evaluation design, the evaluation team lead created a guide (the “Evaluation Guide”) to translate the more conceptual description of systems methods from the design document into what is intended to be a practical guide to help case study team members apply systems approaches during field work for each of the country case study. The Evaluation Guide is available as Annex D to the Final Evaluation Report that will synthesize findings from all four case studies. 16 Guided by the systems approach, the team used mixed methods to answer the evaluation questions. The team collected primary qualitative data and relied upon secondary qualitative and quantitative data taken from administrative sources, the project implementer, and a 2005 final evaluation of QUIPS. The approaches used to answer each evaluation question are described below, and further described in the Evaluation Guide. Evaluation Question 1 Were USAID-intended outcomes sustained? The team sought both qualitative and quantitative data to understand whether USAID’s intended outcome was sustained. The team identified relevant indicators of outcome achievement and attempted to identify secondary data during the field work. However, the only indicator data identified were collected following the cessation of CSA in 2004 (for a study published in 2005). The team collected qualitative data to trace forward in time the condition of the outcome to understand how the outcome may have changed over time, and what else it may have influenced. This analysis considered a temporal aspect, as it was possible that an outcome was sustained for a period, but not up to the present. During the preparatory research, the team made a preliminary determination where possible of the extent to which outcomes have been sustained; during the in-depth field research, it confirmed and, again where possible, elaborated on the determination through open and semi-structured interviews, program documents, and literature reviews. Evaluation Question 2 What other outcomes resulted from the project (positive/negative) and were these outcomes sustained? The team purposively sampled key actors in the system, and drawing upon open-ended and semi￾structured interviews, probed for unanticipated outcomes with a focus on processes and outcomes linked to USAID’s efforts. The team also reviewed program documents and literature reviews, and gathered data using systems methods such as maps and timelines. Evaluation Question 3 What has contributed to or hindered sustaining the outcomes? The team gathered data from sampled actors in the system and using semi-structured interviews probed for in-depth information. They also drew information from document reviews and literature reviews. The team used systems maps and time lines to gather data that described activity achievements and results over time – noting how the activity contributed to changes in the system (specifically changes in structures, processes, quality, and behaviors). 16 https://dec.usaid.gov/dec/content/Detail_Presto.aspx?vID=47&ctID=ODVhZjk4NWQtM2YyMi00YjRmLTkxNjktZTcxMjM2ND BmY2Uy&rID=NTA2NDk2 Ghana Case Study Summary Report: Sustained Outcomes Evaluation 6 Evaluation Question 4 How are the outcomes perceived and valued by those with significant stakes in the project? The team relied on qualitative methods, such as open-ended and semi-structured interviews, to gather data with which to develop an in-depth understanding of how intended and unanticipated outcomes were and are perceived and valued by a wide range of actors and institutions. B. Phase I Evaluation Research 1. Document Collection and Review, Literature Review, and Exploratory Interviews Activity Document Review The case study team collected available CSA activity documents that could be identified from public and non-public sources. The team used key documents17 to: (1) identify the outcomes18 achieved by CSA and (2) describe the actors, dynamics, and events that influenced and/or continued to influence education delivery with respect to these outcomes. While limited, these data provided descriptive information, including: • The original context and evolution over nearly two decades; • The activity and its implementation; • Outcomes achieved by the activity; • Systemic factors and key actors affecting achievement of outcomes; and • Partial list of the key national and education policy changes relevant to the evaluation. Review of Contextual Literature The team reviewed literature to develop a better understanding of the formal education system and formal and informal actors that influence the delivery of education in Ghana. The literature review, in conjunction with the activity document review, was used to inform the exploratory interviews, draw initial systems maps, and begin a timeline of key events. See Annex E for a list of documents consulted. Exploratory Interviews Prior to the Team Planning Meeting, which was held in-country, an in-country team member interviewed four people who had significant roles in the implementation of QUIPS. The data from these interviews provided valuable insights about the activity’s design, aims, and implementation. Drawing on the interview and document review data, the case study lead and the research manager (team members based outside of Ghana) developed a presentation on QUIPS that briefly summarized the activity, intended outcomes and identified results. Further, they prepared a descriptive contextual timeline that described political, cultural, economic and educational changes in the identified timeframe. These initial steps were critical to the evaluation and yielded important evidence with which to further 17 Key documents included the following: USAID/Ghana, Strategic Objective 2 Close-Out Report (USAID 2005); A look at learning in Ghana: the final evaluation of USAID/Ghana's quality improvement in primary schools (QUIPS) program (USAID, 2005); and The Community School Alliances Project: Advancing the State of the Art of Community Participation in Basic Education (Education Development Center, undated). 18 The evaluation focused on one outcome of interest documented in activity documents which was identified as achieved during the life of the activity. Ghana Case Study Summary Report: Sustained Outcomes Evaluation 7 refine the evaluation process. The Team used the resulting documents and presentations at the Team Planning meeting. 2. Planning and Scoping Trip During April 2016, the case study team held a team planning meeting in Accra, Ghana. Prior to this, the case study team identified two education experts who were familiar with the QUIPS activity and the education system in the period before, during, and since the completion of the project. The education experts participated in the planning session and prepared for their engagement by reviewing the QUIPS project summary, early mapping, and timelines. The planning session had four objectives: • Introduce systems evaluation and the evaluation approach to the case study team. • Present a range of possible outcomes and collectively identify the one outcome to be explored; identify the boundaries of that system and the geographical areas; and generate a list of key actors. • Collect information to adapt the approach, and its related evaluation tools, and further inform context timelines. • Identify logistical and other issues related to the evaluation and to plan a way forward. During the team planning session week, the team interviewed six Ghanaian local education experts who were familiar with QUIPS. These interviews provided a grounded perspective on the USAID intervention (QUIPS and CSA) and the situation it had been designed to influence. The case study team’s education experts identified additional documents and worked with the team to supplement the draft descriptions by describing supplementary education, political, cultural and economic events that took place prior to, during, and after QUIPS and CSA. With the support of the education experts, the case study team selected an outcome on which to focus. This is further described in Section 3: Outcome Selection. The case study team then decided on the research boundaries of the ‘system’ in which the education sector change took place19 and which would be subject to primary research. Concurrent with the setting of boundaries, the team identified potential key informants and developed a focused system map that would subsequently be used as a tool in interviews with key informants and as part of the data analysis. Key Point: The planning session provided a process that engaged with the necessary information to select the outcome of interest for the case study. This was accomplished through data and information being prepared prior to the planning and scoping trip, and an organized process during the week in country for an intensive, guided review of data. Data included draft maps that identified key actors and relationships, and an extensive timeline that noted economic, educational, social, and global initiatives, activities, actions, or policies for the time under study. 3. Outcome Selection The case study team selected for its unit of analysis the CSA intervention and its outcome: Enhancing parent engagement in local education. Choosing one outcome resulted from several factors. First, in-depth, empirical knowledge required substantial resources, which only permitted exploring one outcome. Second, this is not an evaluation of QUIPS’ performance; instead, it is an evaluation that aims to understand what influences sustainability in a setting, long after USAID funding has ceased. Selecting one outcome can provide a solid place with 19 Two criteria were used to establish the boundaries for the system of enquiry: (1) were the elements/factors likely to be connected to the potential outcomes and are likely to have influenced sustainment; and (2) were the elements/factors accessible, and could potentially be investigated within the time and resources of this evaluation. Ghana Case Study Summary Report: Sustained Outcomes Evaluation 8 which to begin to understand this, and address the USAID’s evaluation questions. Third, the evaluation approach was ambitious but had been tested only recently in Namibia. It was prudent to focus efforts in Ghana on only one outcome to determine if this resulted in useful information. The case study team considered each of the following Intermediate Results (IRs) from QUIPS as potential outcomes of interest: • IR1: Improving the quality of teaching and learning; • IR2: Building capacity for decentralized school management; • IR3: Increasing community involvement in schools; and • IR4: Improving the physical learning environment Working with the education experts in the planning session, the case study team used the following four criteria20 to select the outcome of interest: 1. Intensity of intervention: Did CSA activities relating to this outcome continue for the duration of the project? 2. Sufficiency of data: Is it likely that there will be sufficient data available to understand the nature of the outcome and the extent to which it was sustained? 3. Centrality to improving learning performance: Recognizing that improved student learning is the ultimate objective of basic education, was the outcome likely to have contributed to this objective? 4. Existence of “ripple effects”: Is the outcome likely to have contributed to continuing changes to the delivery of basic education following the completion of the project?21 The outcome of ‘Enhancing parent engagement in local education,’ which aligned most closely with IR2, met all research criteria. The intervention achieved the goal of building capacity of community units such as PTAs and SMCs and established formal and informal relationships between these units that, based on early evidence, is in existence in at least some form in 2016. Tracing the intervention’s sustainability through the presence of these units and relationships was plausible and early data suggested the outcome had likely influenced other factors, thus meeting the first, second and fourth criteria. Further, the outcome is closely linked to improved education quality and improved student learning, meeting the third criteria. Finally, the community-level intervention provided a manageable context to explore given the constraints of time, resources, and personnel. There was one quantitative indicator tracked during CSA that measured aspect of parent engagement in local education: Indicator 2.3: Percentage of communities demonstrating sustained community involvement in the education process. In 2004, an evaluation documented that performance on this indicator surpassed the target (set at 75 percent), with 97 percent of the communities demonstrating sustained involvement, with communities sampled in the year after the QUIPS interventions were withdrawn.22 The case study team investigated whether it would be possible to collect data corresponding to this indicator as part of the research, but determined it was not feasible. The term ‘sustained’ was not defined in the available documentation and there was insufficient methodological detail available to recreate this metric for present application. 20 The outcome selection criteria are developed more fully in the Evaluation Guide annexed to the final evaluation report. 21 One consideration implicit in the concept of ripple effects is that there should be evidence that the outcome was achieved – a necessary condition for sustainability. Ghana Case Study Summary Report: Sustained Outcomes Evaluation 9 C. Phase II Evaluation Research Primary field research for this case study took place during the first two weeks of September, 2016. Prior to the site visits to Ghana for primary data collection, the team reviewed all data, which included primary data (i.e. interview data), and secondary data such as project, government, and other relevant documents that covered the QUIPS period. Based on the information, the case study team lead then developed semi-structured interview guides, and refined timeline and maps that would be used to collect data. Using these guides, the team conducted semi-structured interviews at the national, regional, district and community level. 1. Sampling Approach The scope of QUIPS necessitated narrowing down the case study to focus on one QUIPS implementation area. CSA was chosen for several reasons, most importantly that the outcome under study was community engagement and the CSA component focused on that QUIPS outcome. A few other factors supported the decision to select CSA, such as CSA providing more relevant written documentation than other components of QUIPS and there being clear implementation of one intervention (as opposed to several in the same area by the same implementer), which would likely better enable respondents to link what happened in CSA to what remains today. This remainder of this sub-section describes the criteria applied to select sites and key informants. The case study team applied the sampling criteria relying on information collected during scoping research and engaged with a Ghanaian education expert who validated the sampling decisions. Site Selection During the planning session, the team and education experts applied two selection criteria to identify the geographic region in which research would be conducted: • Sites should be data rich (e.g. sites have people that were there when the implementation took place; sites have previous evaluation data). • Sites should be accessible (e.g. the team can physically get there within the evaluation time and budget; the team is likely to be granted permission to visit that site) Using information provided by the education experts, the initial interview data, and the data analyzed during the literature and desk review, the case study team applied the sampling criteria. The team selected the Ashante region, and within that region, one district (Kwabre) which had since been divided into two districts: Kwabre East and Efigya. Within these districts, the team selected seven communities (and the schools within these communities) that satisfied the sampling criteria – i.e., they were shown to be high performing in 2004, people remained that were familiar with or experienced QUIPS, and were accessible for interviews. The decision to visit only two districts and seven schools was also based on considerations of logistics and available resources. The focus on seven schools enabled the team to delve into deep conversations within communities. The seven schools selected for primary research are listed in Table 1. TABLE 1: SCHOOLS VISITED IN SELECTED REGION School Location District Region Dumanaf Primary School (PS) Rural Kwabre East Kumasi/Ashante Ntribuoho PS Rural Kwabre East Kumasi/Ashante Krobo PS Rural Kwabre East Kumasi/Ashante Ghana Case Study Summary Report: Sustained Outcomes Evaluation 10 Ejuratia PS Rural Kwabre East Kumasi/Ashante St. Mary’s Combined Rural Kwabre East Kumasi/Ashante Boamang PS Rural Afigya Kumasi/Ashante Adumakasekese Rural Afigya Kumasi/Ashante Key Informant Selection The case study team worked with the education experts during the planning session to identify potential key informants. The team applied four selection criteria: • High likelihood of being able to contact this person, and their willingness to engage. • High relevance in terms of their ability to talk about the topic (e.g. are they in contact with the initial implementation area; are they knowledgeable about the technical and/or local area) • Ability of the range of respondents to yield different perspectives on the outcome of interest. • Three actors in each of the categories (see Annex J of the Evaluation Guide), to the extent possible with higher representation in the implementer and beneficiary categories. Although many potential informants (roles or individuals) were identified during the planning session, the process allowed for the probability that additional persons would be identified during research from whom valuable information could be obtained. To identify these additional respondents, the team used snowball sampling.23 Instances of snowball sampling use included the following: • An informant provides the name of an additional respondent with specific valuable information about either the condition of an intended outcome or an unplanned outcome. • An informant provides information about an individual or individuals who can speak knowledgeably from a unique (or relatively unique) perspective about why the outcome was or was not sustained. • After questioning the informant about who may disagree with them or provide a unique perspective, the informant provides an additional informant. Potential Bias of Purposeful Sampling Purposeful sampling strategies were used at all levels, from case selection, to site selection, to the selection of individuals. The primary reason for adopting this sampling approach was practical. To gather data in a school system for a project that ended 12 years’ prior, the team needed to identify individuals who remembered, and at best were involved in, the project and would be familiar with the context today. Randomly selecting schools and hoping that to find people who remembered QUIPS would likely not have provided data necessary to answer the evaluation questions. The team selected schools and respondents that would be more likely to provide information of sufficient depth and detail to rigorously identify key factors influencing outcome sustainment – and which could credibly inform future decision-making with regards to program cycle planning. While purposeful sampling enabled solid empirical data collection and contributed to an effective evaluation, this sampling approach is inherently biased. Findings drawn from this study may be analytically generalizable, but they are not representative of CSA or USAID education interventions more generally. 23 Snowball sampling is a non-probability sampling technique in which researchers ask existing, identified respondents to identify other individuals who would have relevant information or views and would be appropriate to participate in the study. Ghana Case Study Summary Report: Sustained Outcomes Evaluation 11 Further discussion of and justification for the team’s adopted sampling approach can be found the Evaluation Guide. 24 Triangulation Triangulation is a method that avoids dependences on the validity of any one source. Denzin25 and Patton26 distinguish four types of triangulation, all of which the case study team used throughout its research: multiple research methods, multiple sources within one method, multiple analysts, and multiple theories and perspectives. 2. Primary Data Collection The team conducted 57 interviews: four key experts at the national level and eight regional and district￾level interviews, three interviews with former QUIPS staff members and consultants living abroad, with the remainder conducted at the local level. The research team provided each respondent with an oral informed consent which is noted on each of the interview notes. Towards the middle of the data collection, the team began to reach data saturation on certain elements of the narrative, and by the end of data collection, the team had reached data saturation for the narrative provided in this report. D. Data Analysis The data analysis for the case study took place in four iterative phases. Pre-Fieldwork The case study lead, research manager, and education specialist reviewed and analyzed existing QUIPS/CSA documents, external literature, and initial interview data to understand the activity, outcomes associated with CSA, the nature and modalities of the interventions implemented by CSA, and the key actors. These data were used to inform the design of the data gathering tools and selection of the sites. During Fieldwork At the end of each day of fieldwork, the case study team members reviewed their interview notes against the evaluation questions and their understanding of the various systems to incorporate insights gained from the research conducted that day. The didactic review created structured time for the team members to reflect on the data to inform and identify data gaps, to confirm and disconfirm evidence, and use this information to guide subsequent interviews and focus the narrative – adapting the interview guide as appropriate. The iterative process also enabled the case study team to identify initial findings that were confirmed, disconfirmed or required further data and to revise and improve the interview guides. 24 https://dec.usaid.gov/dec/content/Detail_Presto.aspx?vID=47&ctID=ODVhZjk4NWQtM2YyMi00YjRmLTkxNjktZTcxMjM2ND BmY2Uy&rID=NTA2NDk2 25 Denzin, N. The Research Act: A Theoretical Introduction to Sociological Methods. New York: McGraw-Hill (1978). 26 Patton, M.Q. Qualitative Research & Evaluation Methods, Integrating Theory and Practice, 4th Ed.. St. Paul: Sage Publications Inc. (2015) Ghana Case Study Summary Report: Sustained Outcomes Evaluation 12 Data Analysis Workshop Following the conclusion of field research, the case study team participated in a three-day data analysis workshop. Prior to and in anticipation of the workshop, the team had a one-day data preparation activity. The purpose of this activity was: (a) to organize the data that had been collected, (b) to again clarify understanding of the research questions, and (c) to set ground rules for data analysis. Case Study Lead Dr. Jim Wile led the analysis workshop sessions. Throughout the analytical process, which included data and researcher triangulation, Dr. Wile guided the discussion with questions to promote team reflection on the data and its analysis against the evaluation questions. All data analysis was undertaken manually (i.e., no software was used) and notes were taken by Dr. Wile and research manager Julian Glucroft, as well as by the researchers. Ensuring Analytical Consistency During the initial stages of the workshop, the case study team reviewed the purpose of the evaluation and discussed both the evaluation questions and the proposed analytical process to ensure consistent understanding. The team also formulated a set of ground rules to ensure that each researcher was following a similar procedure for analyzing the data and discussed the potential impact their own implicit biases on analysis – agreeing on the need to think broadly about the research questions but at the same time to look for nuance in informants’ responses. This commitment to discussing each interview comment as a team was deemed the most reasonable approach to dealing with team members’ biases and interpretations. Prior to conducting analysis pertaining to any of the evaluation questions, the case study lead read aloud the evaluation question and each team member would review the notes from one interview for interviewee responses that addressed that question. Team members then read one sample response aloud and discussed whether the response was coded appropriately. This process raised some questions regarding the literal meaning of evaluation questions and the underlying intent of those questions. The team continued this process until each of the first interview notes had been fully coded (for the first evaluation question) and the team had reached consensus that each response appropriately pertained to the target evaluation question. Interview Coding The case study lead provided each team member with notes for 8 to 12 interviews, ensuring that no researcher reviewed interviews that they had conducted. Each researcher was provided with four differently colored highlighters, each corresponding to one of the evaluation questions. Team members then worked independently to color code all responses in their sets of interview notes that corresponded to the first evaluation question. When all had completed the coding of interview notes for Evaluation Question 1, the team then repeated the process of reading aloud the second evaluation question, agreeing the literal meaning and implied intent of this question, and coding one interview for discussion before coding the remainder of their set. This process was repeated for each evaluation question. After all interview notes were coded, each team read aloud the coded responses pertaining to Evaluation Question 1, noting the role and identity of respondents with unique identifiers. The research manager inputted these responses verbatim into a document that was projected on the screen in front of the team, allowing any team member to discuss and agree whether responses had been properly coded. The use of unique identifiers allowed the team to observe whether certain groups or roles (e.g., head teachers) had a shared perspective or whether diverse community members had a shared Ghana Case Study Summary Report: Sustained Outcomes Evaluation 13 perspective on specific issues. This process was repeated for each evaluation question and required approximately two days to complete. Identification of Themes and Concepts During the third day of the workshop, the team reviewed the entire data set and identified important themes and concepts emerging from the data. For example, concepts such as responsibility, power and ownership appeared consistently in interviews. The analytical workshop concluded with a discussion of the systemic factors that seemed to influence the sustainability of the outcomes of the CSA intervention. This discussion was a formal synthesis of the daily discussions the team had been having during its regular de-briefing sessions during the data collection activity. With the aid of the context map, time line, and focused map, data demonstrated how systems began to emerge as powerful influencers. These included the: (1) education system, (2) economic system, and (3) cultural system. A more detailed description these are found in the Findings’ section. The workshop also featured a brief presentation and discussion with a USAID/PPL/LER representative about the evaluation purpose, which was followed by a presentation by the research team about the case study. Post-Workshop Research and Analysis Using the findings that were confirmed following the three-day analysis workshop, the case study lead wrote the initial narrative, which was then reviewed by the case study team and evaluation team lead. The review identified a few weak areas and some slight gaps in the data, which then led to additional discussions and research. To address these gaps, the case study team conducted five additional structured interviews. The case study lead used these data to triangulate findings and the new insights to strengthen the narrative. He then provided the improved narrative to the education experts and one team member for review which resulted in slight changes. The case study lead then shared the revised draft with USAID/Ghana. E. Study Limitations and Facilitators 1. Study Limitations The case study team identified six possible limitations to the study. The nature of this study does not allow for broad generalization of findings: This case study was implemented using systems-thinking approaches that recognize the complexity of systemic change and the importance of understanding change in context. The study relied on purposive approaches to select the case, in addition to the outcome of interest, research sites, and respondents. The findings from this case study are therefore not generalizable (in any statistical notion of the word) to either the intervention as whole or to USAID basic education projects. The findings in this report may, however, be analytically transferrable where a donor or implementer has assessed and found similarities in the factors producing outcomes and the effect of context. Incomplete Documentation from QUIPS/CSA: There were very few CSA activity documents available, such as proposals, work plans, monitoring plans, or reports. The documents available provided weak descriptions regarding the activity and its results. This strained the team’s understanding of CSA, which then required a considerable amount of effort, mostly through qualitative interviews, to piece Ghana Case Study Summary Report: Sustained Outcomes Evaluation 14 together what happened in CSA. The data and report were not negatively influenced; the lack of written data only meant that the team had to spend an inordinate amount of time triangulating basic descriptions. Resource Constraints: Due to resource constraints, the case study team focused the research on one region (Ashante), and within that region, seven school and school communities. It is possible that were the study to have been conducted in a larger number of regions or with a larger number of schools and school communities, a different perspective and additional factors that contributed to or hindered the sustainment of the outcome of interest in those communities may have told a different story. However, the region chosen received a concentrated effort by the CSA intervention and therefore had the highest probability of yielding data to answer the evaluation questions. Resource constraints do not suggest any plausible limitations on understanding what happened in the seven communities studied. Further, the team lead does not view the resource constraints as a limitation as data saturation was met that enabled the team to answer the evaluation questions based on one case. The intent was not to study CSA; the intent was to understand if something remained, and if so, what factors contributed that finding. Time Constraints: The complexity of the study and research required considerable time. Each interview required at least one hour to conduct. Combined with the need for daily debriefs and analysis and travel to research sites, this limited research to approximately two interviews per researcher per day. In addition, two researchers also were unable to participate fully in field research for religious and other personal reasons. During the research, it became apparent that even two full weeks was barely sufficient to probe the surface of community life, its evolving relationships, and individual as well as community priorities. The limitation meant that some potential areas that could have been further explored and triangulated as potential findings were not fully researched. The findings presented in this evaluation are those that were triangulated. Essentially, more time may have allowed a longer narrative with more details and does not influence the findings presented in the study. Insider/Outsider Dynamics: Through the research, the team aimed to understand fundamental issues of power and authority, and the values and priorities that dominate daily community life. This type of research benefits from an insider perspective, both to know how to re-frame an interview question, and how to interpret the informant’s response. Although our research team included three Ghanaians, none were from the Kumasi region and did not bring cultural backgrounds that matched the respondents. The outsider perspective may have resulted in limited insights. Recognizing that the data interpretation may have been limited due to cultural challenges, the team consistently engaged interviewees in data interpretation. The researcher summarized what was said, and then checked the interpretation of that response with the respondent. Further, as the research progressed, the researcher used these summaries and interpretations to check their meaning with additional local respondents. In this way, the researchers engaged people form the Kumasi region in data interpretation which mitigated potential misunderstandings. Respondent Cognitive Biases: Key informants constituted the primary source of information in answering all evaluation questions. As is well known, interview data are prone to cognitive biases on the part of the respondent and/or the interviewer, such as social desirability or acquiescence bias. The case study team mitigated potential cognitive biases in the research to ensure the validity and reliability of its findings using systematic triangulation of interview sources, appropriate selection of a range of interview participants and expert validation of data. Ghana Case Study Summary Report: Sustained Outcomes Evaluation 15 2. Facilitating Factors Several key factors facilitated the evaluation’s implementation. The team’s composition played a critical role in data collection and analysis, combining broad sector knowledge and networks at the national, regional and local levels. A structured guide, thorough training on the methodology, and continued support from the case study team lead ensured that the local team was well prepared for, and supported, during field work and data analysis. Further, selecting a successful program to investigate encouraged respondents to speak candidly and with enthusiasm to the case study team, increased the likelihood of identifying fluid outcomes, which in turn influenced the richness and depth of the data. A highly experienced case study team lead, who brought solid education research knowledge, facilitation skills, and deep knowledge of multiple education contexts, and was intimately involved in the evaluation’s design, greatly aided the study. F. Case Study Team Stephen McLaughlin, a US-based researcher, initially led the Ghana case study. Julian Glucroft, also U.S.- based, served as the research manager. The team’s Ghanaian-based researchers included: Vitalis Agana (Education Specialist), Yussif Seini Abdul-Rauf (Research Specialist), and Benjamin Yeboah (Research Specialist). During the interim period between the team planning meeting and the start of data collection the team underwent some staffing changes. Gideon Porbley replaced Benjamin Yeboah, and Dr. James Wile replaced Stephen McLaughlin as case study lead. Annex F provides profiles of each team member. III. GHANAIAN CONTEXT This section provides a description of the context in which QUIPS was delivered and in which it continues to influence the sustainment of the outcome of interest. It begins by discussing the Ghana context followed by a description of the education policy and the formal and informal structures that govern the delivery of basic education. A. Broad Ghanaian Context 1. History Ghana is an independent, democratic republic located along the Gulf of Guinea and the Atlantic Ocean. Ghana was one of the first African nations to gain independence from European colonial rule during the mid-twentieth century and has had a position of African and global leadership since 1957. As an emerging economy in Africa and a member of the Economic Community of West African States, Ghana has experienced both the advantages and the disadvantages of globalization, urbanization, and climate change. Each of these forces continues to re-shape the political, economic, educational, and cultural character of Ghana and each of these forces has had implications for communities in Ghana. The re￾emergence of a democratic government in 1992 after years of military rule revitalized Ghana’s economic and political life, from the nation’s capital to the grassroots communities galvanizing individual enterprises and civic participation. 2. Economic Development After an economic downturn during the military regime, the shift to a democratic government and rule of law, Ghana experienced a significant period of economic growth during the late 1990s and into the 21st century. Small business retail entrepreneurship emerged as the economic driver in the country. The Ghana Case Study Summary Report: Sustained Outcomes Evaluation 16 implication for rural communities is the shift from the agricultural to the retail sector people leading to rural populations relocating to metropolitan centers or spending time commuting to and from their places of business. The rural to urban shift has been accompanied by a more general transience as newcomers (typically from the northern sections) relocate in rural southern communities. Thus, many of the traditional relationships that helped provide stability to rural communities have deteriorated since the late 1990s. 3. Education shifts to Universal Primary Education In 1994, the Ghanaian government embraced a policy of universal primary education (UPE). UPE provided access to numerous children who had previously been unable to attend public school due to school fees. To meet the demands of newly-enrolled learners the country extended the development of new primary schools, especially in previously underserved communities, and schools which had previously been operated by various faith-based organizations were brought under government control. The shift to UPE placed a strain on Ghana’s human and financial resources, which had other negative implications. For example, to ensure that every classroom was staffed, districts were often forced to hire under-educated teachers, as the recruitment and retention of qualified teachers proved challenging. Most donor aid implemented teacher-support programs that overlooked the teacher colleges which traditionally have the responsibility to train teachers. After agreeing on the importance of adopting a goal of universal primary education to expand access to education, the Government of Ghana (GoG) became aware that in many communities, there were families who had never participated in the education system. Thus, the GoG instituted the fCUBE policy which called for free and compulsory education. The fCUBE policy included specific funding strategies, school construction and renovation plans, and a staffing plan. One challenge that remained was how to sensitize these parents to support their children’s education. As the formal education structure shifted, parents needed to also shift their role to one that supported their children’s education. Key elements to bring about that shift included having parents understand that children needed to attend school regularly, come to school equipped with stationary and supplies, and arrive well-fed and ready to learn. 4. Key Factors There are three contextual factors that influenced the sustainability of CSA outcomes. Namely, the institution of UPE and the widespread idea around parents and communities understanding their role in encouraging children to attend school. Then, the growth of democracy and civic engagement combined with economic growth and its impact on the stability of rural and small community relationships and institutions were also contributing factors to understanding the sustainability of the CSA outcome. B. Structure of the Ghanaian Education System 1. Formal Structures Public education in Ghana, both during the intervention and now, is divided into three main levels - pre￾school, primary, secondary – prior to tertiary education. • Pre-primary education is considered part of the basic education system. It is generally open to children aged 4-6 years. • Primary education lasts six years and is compulsory, with children starting to attend at the age of 6. Primary education is divided into two-three year cycles, lower and upper primary education. • Secondary education, since the 2007 education policy reform, lasts six to seven years with children typically starting at the age of 12. It consists of three years of junior high school Ghana Case Study Summary Report: Sustained Outcomes Evaluation 17 education, completing basic compulsory education and leading to the Basic Education Certificate Examination; and four years of senior high school education. Some technical or vocational senior secondary schools last three years. The figure below shows the relationship between the central administration and its implementing units, which was the structure during QUIPS and now: FIGURE 1: STRUCTURE OF GHANA’S FORMAL EDUCATION SYSTEM 2. Organization The MoE is responsible for policy, planning and monitoring performance of the formal education system, in addition to having legal and financial oversight. It is represented at the regional level by Regional Education Officers in ten regional offices, each supported by technical and administrative staff. Each region is divided into districts, of which there are 138 throughout the country. At the district level, the MoE is represented by District Education Officers and their supporting administrative and supervisory/technical staff. Within each district, individual schools are grouped geographically into circuits for the purposes of management and supervision. Depending on the geography, each circuit will include approximately 15-40 schools. An assigned Circuit Supervisor will visit the schools on a rotating basis to monitor and support education quality at the school level. In addition to providing supervision and mentoring, the Circuit Supervisor is also the liaison between the school and the District Education Office. At the school level, along with the head teacher or school principal, the Circuit Supervisor is a member of each School Management Committee in his or her circuit. The SMC provides a platform for communicating school needs to the district office (through the circuit supervisor) as well as providing a venue for the Circuit Supervisor to disseminate education policy and programs to the community leadership. Ministry of Education (Policy, Planning & M&E) Ghana Education Service (Implements policies on pre￾tertiary education) Regional Education Offices (10) District Education Offices (138) Pre-Primary, Primary, Jr. Secondary and Secondary Schools (organized by circuits) Ghana Case Study Summary Report: Sustained Outcomes Evaluation 18 The main implementing organization of the education system is the Ghana Education Service (GES). GES insures program implementation on a national scale through a cascade model of training of trainers. The targets of most GES programs are the professional development of in-service teachers. These programs (including QUIPS) aim to upgrade teachers’ knowledge and use of best practice teaching methods, including the use of innovative instructional materials. The centralized GES office disseminates teacher improvement programs through a system of training of trainers that ultimately leads to teacher workshops facilitated by trained Circuit Supervisors. The GES also works to strengthen the capacity of regional and district officers to manage programs, resources and for monitoring school and learner performance quality. In summary, the National Level MoE, through its representatives at the district level, are responsible for policy development, implementation, budgets, and hiring and deploying school staff. The GES is the unit within the MoE responsible for organizing in-service training and developing instructional materials. For example, if the MoE should determine the need to implement a new primary school curriculum, the GES’s curriculum specialists and staff would be responsible for developing or obtaining training materials and organizing workshop logistics necessary to implement the revised curriculum. 3. Policies and Practices The provision of public education in Ghana is based on a policy of decentralized operation. Each unit (regional, district and school) is responsible for the efficient implementation of programs and policies and for reporting data up to the national ministry through its Education Management Information System. Regional and District offices have autonomy over funds and staffing and for purchasing equipment, transport and fuel. District Education Officers have responsibility for placing teachers in schools within their district. At the school level, Head Teachers have some decision-making power in the management of the school infrastructure and maintenance of school property. For instance, head teachers and the school staff can decide how they will use the government grants; The annual grants are not earmarked for a specific purpose. An influential practice is the rationalization of the teaching and administrative workforce. Here, teachers, Head Teachers and district staff are routinely relocated after a period of six years in a post. The explanation for this practice is that relocating district and community level MoE staff limits the likelihood of corruption. However, rationalization also makes it difficult to build strong interpersonal relationships within the school and between the school personnel and the community. Even though Ghana spends over 20 percent of its annual budget to support education, at the school level Head Teachers have extremely limited resources to maintain the quality of the school environment. Thus, effective school administrators attempt to harness the economic resources within their school community to provide cash or in-kind assistance. Traditional tribal leaders, wealthy business people, and religious organization typically provide cash support while community members provide labor for maintaining the school grounds, construction of latrines, and the addition of classrooms. The SMC has no authority to raise money to supplement the government budget. The PTA typically provides in-kind labor to maintain the school grounds and the school building. These roles appeared to be clear. However, there was confusion among parents who believed that fCUBE guaranteed free education for all aspects. Thus, parents generally did not believe it was their responsibility to provide out of pocket for school supplies. The SMC played a role to counteract that belief, and encourage parents to provide the essential needs for their own children such as book bags, stationary, and pencils. 4. Funding structures Since independence, the financing of public education institutions is the responsibility of the central Government. By law–– the Education Act of 1961 and PNDC Law 207––the District Assemblies Ghana Case Study Summary Report: Sustained Outcomes Evaluation 19 provide buildings and equipment for basic education (primary and junior secondary schools) in their communities. Most of the Assemblies are, however, unable to adequately perform this responsibility, due to financial constraints. The central government is the main source of education financing. During the past ten years, a great percentage of public expenditure on education (around 85 percent) has been allocated to personnel salaries. Non-salary costs, such as the boarding fees of all students in special education schools, the allowances of all students in teacher training colleges, the total boarding fees for students in the universities, and the scholarships and bursaries for students in tertiary institutions, account for another 10 percent of this budget. Teaching and learning materials are also provided by the Ministry. In 1998, only 4 percent of the total government budget was allocated to investment expenditure. This percentage is wholly inadequate to maintain basic education facilities. Thus, at least one-third of basic schools do not have permanent structures. An alternative is cost sharing, which is practiced at all levels of education except primary and junior secondary public schools. Families and communities contribute, in cash or through communal labor, to the expansion of educational infrastructural facilities.27 Key Point: In this section, the evaluation team identified three structural factors related to the sustainability of CSA outcomes. Two that should have supported CSA’s sustainability include a decentralized system supports district and local school management and a financial need at the school level to engage community support that builds community deliberation and consensus. A factor that discernibly contradicted CSA included a government policy that routinely relocated teachers and administrators and counteracted the school community relationships that CSA aimed to strengthen. IV. QUIPS/CSA PROJECT ACTIVITIES Ghana’s 1992 constitution provides the legal framework to establish community-based SMCs, an avenue for legitimate local engagement and control over school management. In support of the MoE’s mandate to implement SMCs, USAID’s QUIPS initiative designed CSA to provide technical support. CSA had four aims: 1) Raise the awareness of parents and community members of the importance of public education. The combined phenomena of universal public education and decentralization meant that (a) public education was now accessible to many families that had previously been unable to afford schooling and (b) parents had new roles to play in the active engagement in school affairs. To generate awareness among parents and community members of the accessibility and attendant responsibilities associated with public education, CS undertook awareness raising activities including: • Producing community skits and dramas that explained the importance of public education to encourage parents to send their school-aged children to school regularly. • Producing and airing radio programs and public service announcements exhorting parents to take advantage of public education. • Developing and disseminating posters and brochures about the role of parental engagement. • Facilitating local discussions about the role the School Management Committees. 27 See http://www.ibe.unesco.org/fileadmin/user_upload/archive/Countries/WDE/2006/SUB￾SAHARAN_AFRICA/Ghana/Ghana.htm Ghana Case Study Summary Report: Sustained Outcomes Evaluation 20 2) Build the capacity of SMCs and PTAs CSA provided capacity-building training courses for SMC and PTA volunteers on such topics as school finances and budgets, membership management, facilitating meetings, and activities to attract community engagement. CSA also provided capacity-building trainings for school principals and head teachers to improve their general level of professionalism and introduce the collaborative relationship with SMC chairs and PTA secretaries. The original intention was that the teacher professional development component of QUIPS would include attention to supporting school-community partnerships between classroom teachers, parents, and other community members, however this was never realized. 3) Introduce School Improvement Procedures CSA worked with community members and leaders to define educational priorities and develop a School Performance Improvement Plan (SPIP), which identified actions that the community agreed to undertake to improve primary school quality. The types of activities identified in SPIPs included building new facilities, monitoring teacher performance, and making sure children attended school and did their homework. The SPIP also established the procedures that the SMC and Head Teacher would adopt to promote collaboration. CSA also provided resources and technical assistance enable Ghanaian communities to implement their SPIPs, including in the form of small grants awarded to participating school communities that had completed their SPIP. When CSA ended, it was intended that schools would have access to the government’s UPE capitation grants - which were ultimately minimal by comparison and not regularly available. 4) Coordinate School Management Activities The CSA recognized that strengthening the capacity of individual units within the community such as the SMC and the PTA were necessary but not sufficient for strong community-school alliances. The program encouraged SMCs to work with traditional leaders, religious organizations, local politicians, trade unions, and other influential, political and cultural leaders and groups. CSA encouraged collaboration by creating the role of and training a Community Participation Coordinator (CPC). The CPC, typically a role taken on by the district supervisor, was responsible for supervising SMCs and the level of community engagement in the school communities within his/her district. Further, the CSA provided CPCs with transport so that they could visit school communities. What was achieved at the end of CSA? CSA used public awareness activities, participant rural appraisals, capacity-building training for school managers and leaders of PTAs and SMCs to help build support for education, parent engagement, and shared responsibility for school management. During the life of the program it had successfully implemented training, provided and distributed grants to schools that had developed and submitted school improvement plans (jointly-developed through school head teacher and SMCs) and implemented a monitoring and support mechanism via the Community Support Coordinator. The external evaluation of QUIPS (2005, Mitchell) drawing on baseline and end-line data, suggested that the greatest shifts in community behavior, were observed in: (1) supporting school quality (2) supporting girls’ education; (3) strengthening school management; and (4) participatory planning and design. The five significant and relevant results attributable to the QUIPS program included: • Increased awareness and interest in, and commitment on the part of communities and parents to, the process of education. • Strengthened relations between teachers and the community. Ghana Case Study Summary Report: Sustained Outcomes Evaluation 21 • Strengthened community leadership for improving schools, through the SMC/PTA and traditional leadership structures. • Improved community support of schools via enrollment drives, community contributions, and maintenance of the school, including food support, books, and volunteer teachers. • Heightened sense of parental responsibility for their children’s well-being and education. Data collected in 2016 confirmed that these findings occurred during the program’s operational phase. The evaluation aimed to understand how these perceptions and activities were perceived today and whether any of these activities remained as robust as they were during the original intervention, or if they in some way influenced. V. CASE STUDY FINDINGS Each evaluation question is addressed separately. The data identified and analyzed in this section resulted from the case study team’s explicit awareness of boundaries. The question of what is in and what is out is critical and not often easy to determine when exploring beyond the initial USAID-intended outcome. The themes identified in this section emerged through iterative analysis of multiple perspectives within one geographical location and within a specific timeframe. A. Evaluation Question 1 Was the USAID-intended outcome sustained? The USAID indicator, which only captures one aspect of the outcome enhancing parent engagement in local education, was measured immediately after CSA ended, and suggested that the outcome was achieved. In 2005 an evaluation, using the indicator percentage of communities demonstrating sustained community involvement as the measurement of success, identified that 97 percent of the targeted communities demonstrated sustained community involvement in the education process.28 The same data were not available for any other year beyond the evaluation (with the data collected in 2004 and reported in 2005). The case study team confirmed the unavailability of data for the USAID indicator. These data are not presently collected by: schools, school districts, regional education offices, national education directorates, other non-profit organizations or other donors. Therefore, the team drew on empirical qualitative data to assess the extent to which the outcome of interest has been sustained. These qualitative, triangulated data provided evidence that showed some remaining CSA influence in the communities studied. Finding A.1 Positive attitudes of the importance of education and promoting school initiatives was sustained. Data strongly suggest that CSA introduced a new concept to communities - that parents should have some measure of authority over how their children were educated. Almost all the informants expressed the opinion that the program objective of enhancing community support for education was a valuable contributor to the positive perceptions parents had about educating their children, and supporting initiatives at the school. Since 2004, there were examples in each community of parents working to improve the school grounds (painting, cleaning, digging latrines). However, few parents or 28 A Look at Learning in Ghana: The Final Evaluation of USAID/Ghana’s Quality Improvement in Primary Schools program (The Mitchell Group). 2005. Ghana Case Study Summary Report: Sustained Outcomes Evaluation 22 community members saw that they had responsibility for school matters such as staffing, budgeting, long-term planning, or infrastructure. Finding A.2 Community members and school officials involved with CSA continue to believe in community responsibility or participation in school management. Parents, teachers and principals who were trained or involved with CSA described the legacy of CSA as its message that responsibility for school management and children’s education should be shared among all members of the community. These typically had to do with issues such as corporal punishment, teacher absenteeism or interpersonal problems between parents and school staff. Finding A.3 Community members and school officials that were not involved with CSA are less likely to view community participation in school management favorably. The data showed differences between those who were trained by CSA, and those who came after the project closed. Head teachers and classroom teachers who were not trained under CSA provided a perspective on community engagement in school management which were often negative. Head teachers and teachers were not in favor of parent engagement in the actual teaching and learning process. For example, when parents monitored teacher absenteeism or SMC members observed classroom teaching, head teachers and teachers’ perception of parent engagement was not viewed as useful. Finding A.4 Some community relationships maintained at varying functioning levels. As CSA activities focused on awareness raising and capacity building at the personal individual level, these activities were designed to lead to improved integration among individuals and community groups. Capacity-building programs for SMC Chairs aimed at improved management and budgeting of SMCs, but also how to work across community linking with PTAs, the Head Teacher, and other non-school individuals and entities in the community. The goal was to move from personal engagement to forming organizational relationships which would, in turn, create a critical mass for quality improvement action. Some evidence showed that the interrelationships established between various units within the local school community still existed in some communities twelve years following the close of CSA. However, there is considerable variance among communities in how these relationships functioned, and their usefulness. Several communities reported that Head Teachers, classroom teachers and PTA secretaries had engaged with parents for school improvement. Three examples illustrate how communities used engagement and collaboration to address problems in the community related to education. Two examples included the lack of classrooms and poor teacher housing, which were addressed by community members working together with the teachers. The third example focuses on children’s nutrition where a significant number of children came to school hungry. Thus, children were buying unhealthy snacks from vendors just beyond the school yard. The head teacher, the PTA and the SMC mobilized the community to build and staff a school kitchen that could provide children with nutritious meals. The reason given for these three examples taking place was always explained as a personal factor, nothing systematic or structural, or related back to knowledge or skills learned through CSA. In other communities, negative relationships were reported, with a disconnect between classroom teachers and Ghana Case Study Summary Report: Sustained Outcomes Evaluation 23 parents. Parents, teachers, and head teachers spoke about instances of mistrust and antagonism between parents and school personnel, though the reason for the mistrust was not clear. Finding A.5 Community Participation Coordinator role exists though its functions have largely discontinued. SMCs and PTAs, and structures of tribal or traditional authority and religious organizations were well￾established in the communities prior to CSA. The key district-level position introduced through CSA was the CPC. The CPC was delegated to supervise community participation activities across a range of local communities within a district. Twelve years after the conclusion of CSA the CPC role exists, however the actual function of the CPC had been largely discontinued. The role has been re-assigned to one of the District Education Officers (for example, subsumed into the responsibilities of the Child Welfare Officer) or reallocated to the Circuit Supervisors. The reasons for the function being discontinued was explained by the district and regional level respondents who noted that following the close of CSA the CPC officer may have retained his or her title, but lack of budget for vehicle or fuel did not permit the CPC officer to engage with communities. Finding A.6 Consciousness-raising activities are no longer sustained. There was no evidence in any community of consciousness-raising activities. Radio programs and public service announcements had ceased and no new programs or public service programming had been developed. In some communities, there was the occasional use of a loudspeaker to announce PTA meetings and other school related events. Further, there was no evidence that formal training activities for head teachers and community members had been continued. Respondents did not know who would have had responsibility for replicating the training after CSA concluded. Some respondents, such as SMC chairpersons, noted that they had received training under CSA and remained in their position since being selected as SMC chair during the original program implementation, though they had received no further training on consciousness-raising activities. Other respondents, typically head teachers and PTA secretaries, stated that they had been appointed to their positions without specific training on community participation in school management decision-making. The activity of providing school improvement grants discontinued with the close of CSA. However, the GoG has implemented a program of capacitation grants. The grant amounts are awarded to each school based on the reported pupil enrolment. These funds (although not regularly disbursed) are intended to be distributed directly to local schools and can be used at the school’s discretion. The data did not clearly link the school improvement plan to the grants. Finding A.7 Publications, manuals, handbooks and self-assessments no longer used. CSA produced, used and distributed several training manuals, handbooks, community self-assessments, and supportive materials. These resources were used to train school management leaders, head teachers, and PTA secretaries. Respondents who had participated in, or who had knowledge of, CSA confirmed that some of the resources produced during CSA still existed twelve years after the close of the program. However, these respondents did not report any use of the printed resources. Interviewees who had no knowledge of CSA also had no knowledge of any of CSA resources. Some of these Ghana Case Study Summary Report: Sustained Outcomes Evaluation 24 respondents reported having to learn how to lead an SMC “on the job” unaware that resources existed that might have helped prepare for effective leadership. Looking more broadly, there was no evidence that suggested that materials are used or had been adopted or adapted by government, universities, or any external organization. Program documents and interviews with those familiar with CSA did not identify any evidence that suggested continued use (e.g. sustainability) of any materials developed had been built into the CSA design. B. Evaluation Question 2 What other outcomes resulted from the project (positive/negative) and were these outcomes sustained? The case study team identified two unanticipated outcomes that can be linked to CSA during its implementation. Finding B.1 Formal engagements between districts and communities. Twice a year, since the implementation of CSA, a district-level stakeholder meeting takes place for planning and problem solving. The meeting includes representatives from community-level actors and district level education officials. The participants are representatives of each school management committee (e.g. the SMC Chairs) within the district as well as Circuit Supervisors and Head Teachers. There were no concrete results identified from these continued engagements. Finding B.2 Individual empowerment and career growth. Substantial evidence demonstrated how individuals involved in the original CSA project had provided them with life-changing opportunities for career development. Education managers at the national, regional and district level spoke of individuals who had gone through CSA as a trainer of trainers and emerged with an improved ability to present, facilitate, and leader community engagement. Head teachers reported that PTA meetings were typically important venues for women to give voice in the community. For example, women attended more meetings than men and were often the more outspoken during meetings. The limitations of the study did not permit observing PTA meetings or conducting focus groups with women who participated in these meetings to further understand how, or if, these meetings influenced them or their children’s education. Key Point: The data identified and analyzed in this section resulted from the case study team’s explicit awareness of boundaries, which enabled in-depth focused research and data triangulation. While two themes identified the relevance of the collaboration, data were not clear regarding how this finding influenced education, if at all. The second theme was not explored to the extent needed to confirm how the PTA may have positively influence women in the community. Community engagement, and PTA participation in particular, were important venues for empowering women. The link between mothers’ positive attitudes about school and their children’s attendance and performance is a universal phenomenon. Indirectly, then, this activity helped create women advocates for education. Ghana Case Study Summary Report: Sustained Outcomes Evaluation 25 C. Evaluation Question 3 What has contributed to or hindered sustaining the outcomes? Section C describes the various factors that the case study team identified that contributed to or hindered the sustainment of the outcomes mentioned above. Sustaining community support for education is reliant on multiple systems, mainly community, political, financial, and education. The effective intersection of these systems influenced the initial success, and the eventual lack, of sustainability of the CSA’s achievements. Two factors contributed to sustaining the outcome, while six factors hindered CSA outcome sustainment. Contributed to Sustaining the Outcomes Finding C.1 Cultural attitudes toward volunteerism contributed to the sustainment of the PTA and community participation in education. The one factor that should have facilitated the sustained PTA and CSA system was that it coincided with local attitudes about volunteerism. Most respondents noted that volunteerism is a valued component of Ghanaian rural life. For example, many respondents spoke about family support and neighbor to neighbor collaboration. Therefore, the notion of volunteer support to the benefit the school (as in donated labor or cash) fit within the Ghanaian traditions of community engagement. What may have contradicted the cultural system are the factors interweaved in above sections, such as many parents interpreting the GoG’s fCUBE to mean that schools are the sole financial responsibility of the government, and therefore financial accountability for functioning schools lies with the government. Finding C.2 Structures that existed during CSA exist and are still functional. The structures that existed during CSA (though not established by CSA) exist today and are functional. School personnel including regional and district education authorities, head teachers, and unit committee representatives reported that the PTAs and SMCs were effective in disseminating and collecting information at the community level. These data suggest remnants of the CSA remaining (the idea of parental involvement) is sustained by functional structures that exist in the educational system, and a personal factor, as explained in Finding A.2. Hindered the Sustainment of Outcomes Finding C.3 Economic obstacles/disincentives to parental participation The national and regional economies had a powerful influence in reshaping traditional communities. Land inheritance had left most rural framers with subsistence-sized plots of land. While these could be used to grow food crops such as cassava and maize it did not support a cash income. Thus, many rural families had begun to engage in retail commerce either as employees or owners of roadside retailers. This often meant that adults were out of the home early—before children were up—and on their way to work. The result was that many children were left on their own to feed themselves and go to school. When parents returned to their communities in the evenings there was little time or energy for Ghana Case Study Summary Report: Sustained Outcomes Evaluation 26 attending PTA meetings or engaging in other school activities. Parents who spent so much time out of their communities were often only loosely connected to the school in their community. Parents with a higher (or perceived higher) economic status than most of the community provided a different economic reason for not attending PTA meetings. These respondents noted they stopped attending PTA meetings because the PTA meetings only provided the PTA with the opportunity to solicit money. Related to this finding is that various parents reported that PTA meetings only resulted in their being asked to donate in-kind, such as labor to improve the school grounds and facilities. Finding C.4 Traditional systems eroded support for active and democratic School Management Committees. The evaluation took place in the Ashante region, the heart of the traditional tribal system. The hierarchy of paramount chief, local chiefs and sub-chiefs wielded significant authority in the education, community and political system. With a permanent seat on the SMC there was an incongruous dynamic that pit nascent ideas about democracy and power-sharing against traditional ideas about the absolute authority of tribal chiefs. The tension between traditional authority and democracy played out in how people in the community often raised issues with or about education. The data show that school issues are often reported through traditional processes, such as talking to the local tribal chief, rather than through the modern democratic SMC system. One example is when a community objected to the placement of a new head teacher in their school. The parents brought their concern to the local tribal chief for resolution. A second example from a different community demonstrates the use of tribal system over the democratic SMC system. Here several education issues were at a standstill because the local traditional authority had died and there was no agreement on a new chief. Without a tribal leader, no solution or recommendation by an SMC would be authorized. Finally, traditional systems combined with democratic ones created confusion in the community. Local land and property in the Ashante region of Ghana typically remain under the local traditional authority. Most respondents talked about a confusion over who had responsibility for the local school—the MoE or the local traditional authority. The lack of clarity on the matter of who was the responsible authority in school management provides insight as to why, in some cases, the SMCs are considered irrelevant to the decision-making process. The traditional system of values is also reflected in the way community members understood the concept of authority and decision-making. The system is hierarchical and obedience is not optional. The deference toward people in traditional roles of authority ran counter to the democratic values embodied in a local SMCs. In other words, the idea that local parents had the authority to challenge people with responsibility for school quality or to make demands on representatives of government authority was counter to traditional ideas. The lack of continued support for democratic values likely eroded the SMCs’ vitality. Further, a respect for authority and the hierarchical nature of rural communities is reflected in PTA and SMC leadership positions. In most SMCS reviewed, the person serving as SMC chair had been in that position for the past sixteen years and those serving as PTA secretaries or SMC chairs were the wealthier community residents, suggesting that while a structure remains (the SMC or PTA) the democratic values are not fully embraced. Ghana Case Study Summary Report: Sustained Outcomes Evaluation 27 Finding C.5 The MoE’s policy of rationalization weakened relationships between teachers and the community. A new policy that mandated free education and the government’s move toward decentralizing education necessitated strengthening community school alliances. Nonetheless, the MoE’s policy of rationalization meant that teachers are periodically relocated from one school to another. While this is typically within the same district, rationalization can be used to send teachers, including head teachers, to remote locations. The logic of rationalization is to prevent individual teachers and head teachers from becoming too close to the communities in which they work which addresses a concern that familiarity may breed corruption. Regardless of the legitimacy of this concern, what resulted is that head teachers and teachers who had been part of their present community at the time CSA was implemented were moved to another community, thwarting the strengthened parent/community/teacher relationships. Finding C.6 Funding constraints or de-prioritization weakened the role of Community Participation Coordinator, which served as a facilitator for community engagement. The MoE’s financial constraints negatively impacted sustaining the CSA outcome. Under CSA the CPC played a key role as a supervisor, mentor and motivator of community participation activities. The CPC position was carved out of the human and financial resources at the District Education office. After the CSA project closed, the CPC became an unfunded position. The CPC’s duties were added on to other (existing) positions at the district level (typically the District Welfare officer). Even more critical was loss of funds to support travel to local communities to support school community alliance activities. With no single person responsible for managing these important relationships and activities, the CPA activities were reduced to filling an oversight role. Data from each community in the study depicted a significant drop in community participation activities following the discontinued support from the district’s Community Participation Coordinator. Finding C.7 The number and popularity of private schools in Kumasi increased during CSA and likely reduced the number and proportion of engaged parents of public school students. The increase in the private sector schools, and children moving to them, influenced the CSA outcome. The number of private primary schools in a district capital north of Kumasi increased from four in 1993 to 64 in 2004. While data were not conclusive why parents shifted their children to private education, data suggested that private education did drain off the most concerned and articulate parents, as well as the better performing learners. Finding C.8 High rates of migration into Kumasi of families with exposure to CSA awareness-raising activities weakened community participation in school activities. Perhaps more dramatic was the population shift that resulted from economic growth. Informants reported a significant increase in transient populations, with people from the rural community relocating to Kumasi and people from remote northern villages moving in to the rural outskirts of Kumasi. Together, these demographic shifts meant that most people who experienced the consciousness awareness programs of CSA were no longer in the community and new community members did not Ghana Case Study Summary Report: Sustained Outcomes Evaluation 28 receive the same type of sensitization. Like the rationalization of teachers and head teachers, when the population of the community shifted the school-community alliance relationships weakened.29 Key Point: In understanding perspectives and interrelationships, the evaluation has boundaries that define what is included and what is not. Therefore, the evaluation needs explicit reason for (e.g. criteria) including or excluding something. While a systems evaluation aims to be holistic, ‘holism’ is not about trying to deal with everything, but being methodical, informed, pragmatic and ethical about what to leave out.30 It also involves being transparent about boundaries and keeping what is outside of established boundaries in our “peripheral vision”. The factors that supported and hindered the sustainability can be found in the financial, political, education, private sector, traditional and cultural, and economic systems. D. Evaluation Question 4 How are the outcomes perceived and valued by those with significant stakes in the project? Nearly all the informants expressed the opinion that the program objective of enhancing community support for education was a valuable contributor to the positive perceptions parents had about educating their children—and most informants cited increased enrollments and reduced truancy because of communities engaging in education activities. Informants who had participated in the original training and implementation phase of CSA had very favorable opinions of the outcome achieved by the program. They frequently shared stories about how CSA had energized their community and how it had raised support for education, in general. However, informants who were not part of the CSA intervention had only minimal regard, if that, for the outcome achieved by CSA. Teachers and head teachers, for example, viewed their relationship with the PTA and the SMC as “we tell them what we need and they get it for us.” This hardly reflects the intention of organizing local community members to engage in school management decision making. Other themes provide insight into how outcomes are perceived and valued include: • Traditional leaders did not appear to appreciate the concept of new community structures challenging their role, or parents using those structures to engage in education. These findings suggest that traditional leaders did not likely value the role of CSA which encouraged parental engagement and often through other than traditional venues. • School Officials who were not involved in CSA activities viewed SMC as stepping outside of their roles in trying to engage in education management. • Parental viewpoints suggested that for some parents, SMCs were merely vehicles to ask for money, while other parents who valued their role in their children’s education, found the new models (e.g. PTA) to bring confusion with regards to whom to engage with to resolve education issues. These parents did not highly value CSA or its remaining influences. Key points: These data, which build on and pull from all preceding sections, emanated from the evaluation’s commitment to multiple perspectives, providing insight to how the same situation can be “seen” in different ways. We describe the behavior of persons, organizations and groups based on their perceptions of the situation, rather than some logic defined by someone else. Analyzing data enabled differentiation and examination of individual and group roles (e.g. teachers) versus their individual values and motivations. We also highlight how understanding can be enhanced by consideration of dynamic aspects (i.e. when interrelationships affect the behavior of a situation over time) and sensitivity of inter- 29 Data showed that the trend toward retail merchandizing increased considerably in the last 10 years (post-CSA). 30 This description is heavily based on the work of Williams and Hummelbrunner, and Beverly Parsons. The team is also informed by USAID’s Local Systems: A Framework for Supporting Sustained Development (April 2014). Ghana Case Study Summary Report: Sustained Outcomes Evaluation 29 relationships to context. Finally, we drew on nested systems, recognizing that each actor is also a member of multiple systems (e.g. education sector is connected to financial systems, traditional systems, and political systems). The evidence suggests that CSA influences seemed to have little to no value when they introduce new systems or ideas that are contrary to, appear to be contrary to, or challenge authority, whether it is in educational, traditional or cultural structures or systems. VI. CONCLUSION One of the key objectives of QUIPS during its implementation was to support the decentralization of the education system. The CSA component aimed to support the government’s newly organized SMCs, and was grounded in the government’s need to elevate community and parental support for the education of their children. At the end of the CSA intervention there had been an increased level of parent and community engagement with children’s education, from parents enrolling children in school to engaging with school infrastructure projects. By 2016, data suggest that while most parents accept their critical role in ensuring that children are enrolled and attend school (thus awareness was raised and maintained in 2016), how parents engage with schools, such as school management and financial responsibilities are not strong, collaboration among school personnel and parents is often weak, and the lines of authority for managing schools confound parents and communities. Capacity-building for SMC and PTA on such topics as school finances and budgets, membership management, facilitating meetings, and activities to attract community engagement, are non-existent, as are capacity building for school principals and head teachers to improve their general level of professionalism and introduce the collaborative relationship with SMC chairs and PTA secretaries. No school performance improvement plans were identified, and cash awards grants for participating schools that had completed the school performance improvement plan are no longer funded. In 2002, the CSA recognized that strengthening the capacity of individual units within the community (e.g. SMC and the PTA) were necessary but not sufficient for strong community-school alliances. CSA encouraged SMCs to work with traditional leaders, religious organizations, local politicians, trade unions, and other influential, political and cultural leaders and groups. CSA encouraged that collaboration by creating the role of and training a CPC, which is no longer highly functional. Several additional factors combined to militate against the sustainability of the CSA outcome. Government policies required the rationalization of teachers and head teachers undermined community integration and counteracted the school community relationships that CSA aimed to strengthen. Communities in Ghana are also in constant change with new economic opportunities where parents travel for better economic opportunities, leaving less time, and sometime no time, to supervise their children’s nutrition, health, and schooling. While economic and political factors are complex in how they negatively influenced the CSA outcome, more simple factors also played a role. An example is the CPC role that still exists, however the actual function of the CPC had been largely discontinued and lack of a budget for fuel for CPC’s to visit communities is non-existent. While some structural innovations remained at the community and district level, the degree of vitality of these structures and collaboration has significantly diminished since the close of the QUIPS/CSA intervention. While some of this may be attributed to the lack of support (e.g. training, training materials) for PTA and SMCs, conflicting systems played a detrimental role – a democratic one instituted by the government and supported by CSA, and a traditional system of authority that appears to have trumped the democratic system. In sum, one of the main objectives of CSA was to encourage parents to support their children’s education by finding meaningful ways for them to support their children’s school. In some school communities, while parents’ engagement in school affairs may have diminished over time, it appears that support for education remains high. Ghana Case Study Summary Report: Sustained Outcomes Evaluation 30 ANNEXES Annex A Timelines Annex B Focused Activity Map Annex C Formal Education System Map Annex D List of Interviewees Annex E List of Documents Consulted Annex F Team Member Profiles Ghana Case Study Summary Report: Sustained Outcomes Evaluation 31 ANNEX A: TIMELINES Ghana Case Study Summary Report: Sustained Outcomes Evaluation 32 Ghana Case Study Summary Report: Sustained Outcomes Evaluation 33 ANNEX B: FOCUSED ACTIVITY MAP Acronyms CSA Community School Alliances Project CPC Community Participation Coordinator DEO District Education Office GNAT Ghana National Association of Teachers PTA Parent Teacher Association SMC School Management Committee ANNEX C: FORMAL EDUCATION SYSTEM MAP Acronyms DEO District Education Office MMDA Metropolitan, municipal, and district assemblies PTA Parent Teacher Association REO Regional Education Office SMC School Management Committee Ghana Case Study Summary Report: Sustained Outcomes Evaluation 35 ANNEX D: LIST OF INTERVIEWEES Name QUIPS Position/Role Current Position/Role Current Location Stella Ofori-Atta N/A District Education Director AfigyaKwabre District Education Office Owusu Agyemang K. N/A Deputy Director of Inspection AfigyaKwabre District Education Office Daniel Owusu Frimpong N/A Budget Officer AfigyaKwabre District Education Office Opoku Philomena N/A Girl Child Officer AfigyaKwabre District Education Office Soloman Asmah N/A Circuit Supervisor AfigyaKwabre District Education Office Samuel Asanti Mensah C/S Retired Kwabre East District Education Office Michael K. Awua CPC Coordinator Retired Kwabre East District Education Office Godwin Seshimey Statistics and Planning Officer Statistics and Planning Officer Kwabre East District Education Office Kusi Appiah Kingsford N/A Circuit Supervisor (including Krobo) Kwabre East District Education Office Kennedy Osei N/A GNAT District Secretary Kwabre East District Cornelia Damah N/A SHEP Coordinator Kwabre East District Education Office Afrifa Anthony N/A Head Teacher Boamang R/C Pry. Adu-Acheampong Philip PTA Secretary Boamang Fordjour Charles S. M. C Chairman Boamang Francis Boadi N/A Teacher and PTA member Boamang Lucy Addai Teacher Teacher Boamang Nana Kwobene Obeng N/A SMC Chairman Boamang Esther Ackon Not present during QUIPS Headmistress NTRIBUOHO Nana Kwame Amofa Beneficiary Traditional Authority SMC member NTRIBUOHO Georgina Achiaa PTA Member Unit committee Member NTRIBUOHO Edwin Kwaku Duan N/A PTA Secretary (Teacher) NTRIBUOHO Baiden Arhin Fredrick N/A Head Teacher Ejuratia Primary Brefo Emmanuel Assemblyman, PTA Chair Retired Ejuratia Primary Serwaah Janet Teacher Teacher Ejuratia Primary Alex Boakye PTA Chairman S.C. Chairman Adwumakase-kese Rev. Solomon Ansu N/A Teacher, School Manager Adwumakase-kese Honorable Yaa Pokuah N/A Assembly Woman Adwumakase-kese Nana Yeboah Mensah Head Teacher Krongihene/Acting Chief Adwumakase-kese Ghana Case Study Summary Report: Sustained Outcomes Evaluation 36 Name QUIPS Position/Role Current Position/Role Current Location Teachers focus group N/A Teacher, PTA secretary Adwumakase-kese Charlootte Fiwaa Aikins N/A Head Mistress Adwumakase-kese Ntim Yaw Boakye SMC Chairman SMC Chairman Aboabogya Methodist Comfort Addai PTA Member Nil Aboabogya Methodist Freda Asante N/A Head Teacher Aboabogya Methodist Michael Osei N/A District Assembly Aboabogya Methodist Albert Boakye N/A Reverend & JHS teacher Aboabogya Methodist Hayford Fosu Unit Com. Rep. Unit Com. Rep. Dumanafo R/C Primary Asuming Okyere John SMC Member SMC Member Dumanafo R/C Primary Community member focused group [Multiple] Community members and PTA chair Dumanafo R/C Primary Dorothy Mahama N/A Headmistress Dumanafo R/C Primary Komfort Sefa Boache N/A Assembly Woman Dumanafo R/C Primary Richard Boakye PTA Chair SMC Chairman Krobo R/C Primary Ernest Kwesi Brefo N/A PTA Secretary and Teacher Krobo R/C Primary Jenifer Appiah Bempah District NSS officer Regional Director National Service Secretariat – Regional Office Reverend Kwame Adjei N/A Director of Kumasi Metropolitan National Service Secretariat – Regional Office Joe Korangteng GES officer (trainee and trainer) Head of Inspectorate of Basic Schools Regional Education Office [Anonymous] N/A Girl Child coordinator Regional Education Office Mary Owusu Achiaw District education officer (trainee and trainer) Regional Director of Education Regional Education Office Augustine Tawiah N/A Kay Leherr Education Development Center, M&E officer and then CSA Chief of Party Education Expert Steve Manu Academy for Educational Development, Director of QUIPS/ Improving Learning through Partnerships Education Expert Bright Wireko￾Brobby District Grant Support Coordinator, Community School Alliances Project Community Development Specialist Stephen Adu N/A Acting Deputy Director for Management Services in GES Ghana Education Service Lesley Casely￾Hayford Team Member, QUIPS evaluation. Education Specialist Ghana Case Study Summary Report: Sustained Outcomes Evaluation 37 ANNEX E: LIST OF DOCUMENTS CONSULTED Final Implementer Reports Education Development Center. The Community School Alliances Project: Advancing the State of the Art of Community Participation in Basic Education, 2004/2005. International Foundation for Education and Self-Help. IFESH Primary School Teacher Training Project Final Report, December 2003. Academy for Educational Development. Quality Improvement in Primary Schools. Improving Learning through Partnerships Project. Final Report, September 2003. Academy for Educational Development. SAGE Final Activities Report, July 2002. Final Evaluation Report The Mitchell Group. A look at learning in Ghana: the final evaluation of USAID/Ghana's quality improvement in primary schools (QUIPS) program, 2005 Quarterly and Periodical Reports Education Development Center. QUIPS/CSA Quarterly Report. 1 April 2000 to 30 June 2000. Education Development Center. QUIPS/CSA Quarterly Report. 1 July 2000 to 30 September 2000. Education Development Center. QUIPS/CSA Quarterly Report. 1 Oct 2000 to 31 December 2000. Education Development Center. QUIPS/CSA Quarterly Report. 1 January 2001 to 30 March 2001. Education Development Center. QUIPS/CSA Quarterly Report. 1 July 2001 to 30 September 2001. Education Development Center. QUIPS/CSA Quarterly Report. 1 January 2002 to 30 March 2002. Education Development Center. QUIPS/CSA Quarterly Report. 1 April 2002 to 30 June 2002. Education Development Center. QUIPS/CSA Quarterly Report. 1 July 2002 to 30 September 2002. Education Development Center. QUIPS/CSA Quarterly Report. 1 Oct 2002 to 31 December 2002. Education Development Center. QUIPS/CSA Quarterly Report. 1 January 2003 to 30 March 2003. Education Development Center. QUIPS/CSA Quarterly Report. 1 April 2003 to 30 June 2003. Education Development Center. QUIPS/CSA Quarterly Report. 1 July 2003 to 30 September 2003. Education Development Center. QUIPS/CSA Quarterly Report. 1 Oct 2003 to 31 December 2003. Education Development Center. QUIPS/CSA Quarterly Report. 1 January 2004 to 30 March 2004. Education Development Center. QUIPS/CSA Quarterly Report. 1 April 2004 to 30 June 2004. Consultant Reports The Mitchell Group. Rebecca R. Corn Trip Report, March 13 -March 31, 2000. Ghana Case Study Summary Report: Sustained Outcomes Evaluation 38 The Mitchell Group. Wes Snyder Trip Report, March - April 2000. The Mitchell Group. Dr. Tom Welsh Trip Report, May - June, 2000. The Mitchell Group. Dr. Tom Welsh, November 1999. The Mitchell Group. Rebecca Rumbaugh Trip Report, November 2001. The Mitchell Group. David W. Chapman Trip Report August 2000. The Mitchell Group. John Meyer Trip Report, August 1999. The Mitchell Group. Rebecca R. Corn Trip Report, May to June 2000. The Mitchell Group. Wes Snyder Trip Report, November 1999 The Mitchell Group. Robert S. Barcikowski Trip Report, November – December 1999. The Mitchell Group. Richard S. SandmanTrip Report for a Test Development Assignment in Accra, Ghana, January 2000. The Mitchell Group. Rebecca R. Corn Trip Report, October – November 1999. Design Documents USAID/Ghana. Strategic Objective 2, Increasing the Effectiveness of Primary Education, September 1996. Education Development Center, Community School Alliances Project Draft FY 2004 Continuation Application, June 2003. Other Evaluation Reports ORC Macro. Support for Education Decentralisation in Ghana: USAID’s District Grant Mechanism, May 2003. DevTech Systems. Strengthening HIV/AIDS Partnerships in Education I and II Evaluation, April 2007. Other Assessments Education Development Center. ‘Best Practices’ in Community Participation in Education: Impact and Sustainability, March 2001. USAID/Washington. Evaluation of the Community Mobilization Component of Strategies for Advancing Girls’ Education (SAGE), October 2004. The Mitchell Group. Helping Children Learn in Ghana: Lessons Learned from QUIPS, June 2005. Academy for Educational Development. Strategies that Succeed Stories from the SAGE Project. DevTech Systems. Safe Schools Program Ghana Assessment Report, May 2004. Implementation Tools and Materials Education Development Center Inc. CSA Community Mobilization Activities Facilitator Manual, December 1999. Ghana Case Study Summary Report: Sustained Outcomes Evaluation 39 ANNEX F: GHANA CASE STUDY TEAM PROFILES Dr. James Wile, Evaluation Education Specialist and Case Study Lead (Primary Research Phase) Dr. James Wile is a senior level teacher development and literacy expert with over 17 years of technical and executive experience in the field of education development in the United States, South Asia, Africa, and Southern Europe. His experience includes research and evaluation, program design and implementation, university and school-based teacher education and systems development, and policy/advocacy initiatives. He has worked on projects that helped individuals and institutions leverage reading and writing activities for human expression and self-determination. Mr. Wile is committed to the goal of contributing to the professionalization of the education workforce with the ultimate aim of improving learning outcomes for children, families and communities. Dr. Wile holds a Ph.D. in Language and Literacy from The Ohio State University and a Masters in Reading and Curriculum from the University of Michigan. Stephen McLaughlin, Case Study Lead (Preliminary Research Phase) Mr. Stephen McLaughlin has over 25 years of research and evaluation experience, both in international and U.S. settings, in all aspects of education, as well as programming for youth development, HIV/AIDS education and for orphans and vulnerable children. His experience includes designing, conducting and managing evaluations, performance monitoring systems, and social science research for government and private sector organizations in 36 countries including the U.S. Results of his studies have been used in program improvement, policy revisions, designs of loan and technical assistance programs, management decisions, funding proposals, and future research. In his capacity of Education Program Development Adviser, Mr. McLaughlin has contributed to the programming of numerous donor-funded projects around the world. Mr. McLaughlin has Ed.D. in International Education and Masters in Education from the University of Massachusetts, Amherst. Julian Glucroft, Case Study Research Manager Mr. Julian Glucroft is a Technical Manager at Management Systems International, with three years working in the field of international development and social science program research. He has sound knowledge of theories and methods of international development project design, management, monitoring, and evaluation. His academic focus has been on food security and rural development in Sub￾Saharan Africa. Currently, he provides research and analytical services for MSI’s business development and technical staff. Mr. Glucroft has a Bachelor’s degree in International and Global Studies from Brandeis University and has a Master’s degree in International Development from American University. He is fluent in French and English, proficient in Spanish, and has basic knowledge of Wolof. Mr. Vitalis Agana, Case Study Education Specialist Mr. Agana is an education specialist and researcher, with technical expertise in qualitative and quantitative research methods and data analysis. Since 2006, Mr. Agana has worked for the Ghana Education Service, including most recently as the Assistant Director for Educational Planning, Research, Monitoring and Evaluation with the Tolon District Directorate of Education. His recent experiences include leading a team of 10 Research Officers and 20 research assistants to conduct base line study on the educational needs of girls and Children with Disability in the Talensi, and Nabdam Districts in the Upper East Region and leading a study on the use of sanitation and hygiene facilities in school communities. Mr. Agana holds an M.Phil. degree in Development Management from the University of Development Studies (Ghana) and BA degrees in Accounting and Sociology from the University of Ghana. Ghana Case Study Summary Report: Sustained Outcomes Evaluation 40 Yussif Abdul-Rauf, Case Study Research Specialist Mr. Yussif Abdul-Rauf has significant experience in leadership and administration as a field research team leader and a research coordinator. In the roles of both Senior Investigator and Research Associate, he has completed studies, evaluations and impact assessments in Ghana in the areas of energy, agriculture, sanitation and water management, micro-finance, entrepreneurship and poverty reduction, gender and diversity, and human rights. Mr. Abdul-Rauf has a Master’s Degree in Development Management a Bachelor’s Degree in Integrated Development Studies from the University of Development Studies (Ghana). Benjamin Asare Yeboah, Case Study Research Specialist Mr. Benjamin Asare Yeboah is an experienced monitoring and evaluation specialist with over five years of experience in designing and implementing monitoring frameworks and evaluations and conducting research. Mr. Yeboah has conducted this work in the areas of Food Security, Resilience, Child Vulnerability and Protection, Environments and Education. He has particular experience with the education sector in Ghana, having previously served as an Education Program Officer. Mr. Yeboah has a BS in Development Planning and Policy from the Kwame Nkrumah University of Science and Technology, Ghana. Gideon Selorm Hosu-Porbley, Case Study Research Specialist Gideon Hosu-Porbley has extensive research experience and is currently the part-time Research, M&E Officer for the United Kingdom Department for International Development’s STAR-Ghana Project. He is also an Adjunct Lecturer in Social Research Methods at the Ashesi University College, Ghana. Gideon has conducted academic and professional assessments in the fields of education, gender, HIV & AIDS, environment, water and sanitation, poverty reduction, participatory methodologies, governance and policy. Mr. Hosu-Porbley read Geography and Resource Development with Political Science for his B.A (Hons) degree and proceeded for an M.Phil degree in Geography and Resource Development, all at the University of Ghana. Gideon is currently a PhD candidate in Development Studies, at the University of Cape Coast (Ghana).