FINAL VERSION EVALUATION OF THE REPLICA PROJECT CARRIED OUT UNDER THE BASIC EDUCATION AND POLICY SUPPORT III (BEPS III) PROJECT WITH SUPPORT FROM USAID/UGANDA Dr. Mark Lynd, President School-to-School International April 2007 CONTENTS ACRONYMS........................................................................................................1 ACKNOWLEDGEMENTS .....................................................................................2 BACKGROUND ...................................................................................................1 OVERVIEW OF THE REPLICA DESIGN...............................................................2 6 components.................................................................................................2 Consultative process ......................................................................................3 Selection of districts and schools ..................................................................4 Cascade training.............................................................................................5 Support supervision & monitoring.................................................................6 Project management and implementation.....................................................6 Monitoring and evaluation.............................................................................7 Implementation constraints ..........................................................................7 EVALUATION DESIGN AND IMPLEMENTATION .................................................8 Objective and design of the evaluation..........................................................8 Evaluation questions......................................................................................8 Sample............................................................................................................9 Data collection and analysis ........................................................................11 Limitations of the evaluation.......................................................................11 FINDINGS.........................................................................................................12 Major achievements .....................................................................................13 Design ..........................................................................................................15 Training........................................................................................................17 Materials ......................................................................................................18 Support supervision & monitoring...............................................................20 The 6 components........................................................................................22 RECOMMENDATIONS OF RESPONDENTS .......................................................24 DISCUSSION ....................................................................................................24 Impact ..........................................................................................................24 Sustainability issues ....................................................................................25 RECOMMENDATIONS ......................................................................................27 ANNEX A: DATA COLLECTION TOOLS.............................................................29 ANNEX B: LIST OF PERSONS CONSULTED......................................................71 ANNEX C: MATERIALS REVIEWED..................................................................74 Evaluation of REPLICA/UGANDA Conducted by School-to-School International for Creative Associates International, Inc. Final version April 2007 p. 1 of 85 ACRONYMS BEPS Basic Education Policy Support Program CAO Chief Administrative Officer CC Coordinating Center CCT Coordinating Center Tutor CIP Community Integration Program DEO District Education Officer DIS District Inspector of Schools DPO Deputy Principal Officer FAWE Forum for African Women Educationalists G & C Guidance & Counseling IDP Internally Displaced Person M&E Monitoring and Evaluation MoES Ministry of Education and Sports (MoES) NREF Northern Region Education Forum PALS Performing Arts and Learning in Schools PIASCY Presidential Initiative on AIDS Strategy for Communication to Youth PTC Primary Teacher Colleges SBT School-Based Training SEO Senior Education Officer SNEGC Department of Special Needs Education, Guidance & Counseling TDMS Teacher Development and Management System TOT Training of Trainers Evaluation of REPLICA/UGANDA Conducted by School-to-School International for Creative Associates International, Inc. Final version April 2007 p. 2 of 85 ACKNOWLEDGEMENTS I would like to thank the following people for their assistance in the implementation of this evaluation: • Renuka Pillay, Chief of Party to the BEPS and UNITY projects and Director of the REPLICA Project, for her constant availability, candor and infectious optimism for designs such as REPLICA’s – ones with the potential to “change the paradigm” of education in Uganda and beyond; • Patrick Bananuka, TDMS Outreach Coordinator, for the critical help he provided in the conceptualization of the design, the development of data gathering tools late into the night, assistance with logistics in the field, and his willingness to share his seemingly endless stock of knowledge about REPLICA, the Ugandan education system and the country’s history in general; • Martin Opolot, M&E Officer for the UNITY Project, who assisted with the sample design as well as providing valuable background information on REPLICA’s development and management; • Martin Omago-Loican, Commissioner, Department of Special Needs Education, Guidance & Counseling, MoES and Mr George Opiro, Principle Education Officer, Guidance & Counseling Programs, for their very thorough and spirited orientation to the REPLICA project and feedback on data collected in the field; • Jeanne Moulton and Joanne Murphy of Creative Associates International, Inc. for providing timely information and advice throughout this consultancy, and for providing helpful feedback on the draft of this report. • All of the participants in this evaluation who gave generously of their time, often beyond the normal hours of work, to express their satisfaction and enthusiasm about this project while also providing frank and open comments on ways to improve it in its expansion stage. If any details are misrepresented or absent from this report, it is not the fault of those who worked so hard to assist with its development, but rather a reflection of my own limitations in adequately portraying a project notable not only for its complexity but for its extraordinary achievements in such a short period of time. Mark Lynd April 2007 Evaluation of REPLICA/UGANDA Conducted by School-to-School International for Creative Associates International, Inc. Final version April 2007 p. 1 of 85 EXECUTIVE SUMMARY The REPLICA project, a 1-year pilot project funded by USAID in Uganda, was designed to address the particular educational needs of children, teachers and parents in the conflict-affected zones of northern Uganda. REPLICA (Revitalization of Education Participation and Learning in Conflict Areas) was implemented under the BEPS III Project from July 2005 to July 2006, reaching 30 primary schools and 4 Primary Teacher Colleges in northern Uganda. After this pilot period, an evaluation was requested by USAID/Uganda in order to prepare for expansion to 1,700 schools in 2007. In February 2007, the consultant worked in Uganda for two weeks to gather data through interviews, focus group discussions and documentary analysis. In all, 127 people took part in the evaluation, including pupils, teachers, Head Teachers, and parents; district education officials; Primary Teacher College administrators, tutors, and students; the Regional Inspector; ministry officials; project staff; and a university professor. In general, the study found overwhelming support and enthusiasm for the REPLICA project at all levels, from primary schools to PTCs, from district education to central ministry offices. One of the biggest findings was the success of the Guidance and Counseling component, with evaluation participants citing improved relations among school staff and between staff and pupils; increased pride and empowerment on the part of model school pupils; and reportedly improved enrollment and retention in REPLICA model schools, especially amongst girls. Other positive aspects of the project included the effectiveness of the TDMS structure in project deployment, the quality of training received, particularly by trainers, the quality of print materials produced and provided by the project, and overwhelming satisfaction with the provision of girls’ kits and musical instruments. Several aspects of the REPLICA were cited as requiring rethinking or improvement. In particular, the School-Based Training model had minimal impact on teachers, and the Community Integration Component of the project seemed to have little impact in involving parents in schooling. Infrequent training for teachers and insufficient quantities of materials were also cited as some of the project’s biggest difficulties. This consultant also found three additional areas requiring attention if expansion is to be achieved successfully: the need to increase consistency, economy and ease-of-use of print materials; the need to development instruments and processes that could reliably measure the project’s progress and impact; and questions concerning the sustainability of models providing consumable materials. Based on these findings, 10 recommendations are made to ensure that changes are taken into consideration in REPLICA’s critical scale-up phase. Evaluation of REPLICA/UGANDA Conducted by School-to-School International for Creative Associates International, Inc. Final version April 2007 p. 1 of 85 BACKGROUND Over the last two decades, the story of northern Uganda has been one of suffering, violence and constant social upheaval as a result of war with the Lord’s Resistance Army. Entire villages have been destroyed or displaced by the conflict, with over half the population living in camps for Internally Displaced Persons (IDPs). Throughout this period, the Government of Uganda (GOU) has struggled to provide basic services to its citizens by establishing makeshift schools called “learning centers” in the IDP camps, and by supporting the mainstream schools (those not displaced by the conflict) in the villages that have remained to stay intact. Over time, a series of daunting problems has arisen, ranging from poor sanitization and hygiene to collapsed school management systems, overcrowded “classrooms,” limited instructional materials and traumatized teachers and children in learning centers and mainstream schools alike. So profound were the effects of the war that even during more peaceful periods, people continued to settle their differences through violent means. Throughout this period, a number of partners, donors, NGOs and UN agencies provided assistance to the Ministry of Education and Sports (MoES) in its efforts to maintain the quality of basic education services to people in the north. One important program was the USAID-funded Basic Education Policy Support (BEPS) program. Initiated in 2000, BEPS supported the MoES in educational policy dialogue and reform as well as the improvement of equity, efficiency and quality of basic education, both nationwide and in the north. More recently, BEPS focused much of its support on the north through the development of programs for head teacher and school management, curriculum revision, Guidance and Counseling, and support to the Northern Region Education Forum (NREF) advocacy group (BEPS III final report, pp. 5-6). In the last few years, the LRA has indicated its willingness negotiate a peace settlement. In response to this opportunity, a number of studies have been carried out to reassess the educational conditions and needs of people in the north. One such study, called Uganda: Opportunities, Challenges and a Way Forward (Bragin 2004), revealed that a number of good educational practices were already being implemented on a small scale in the north by local and international NGOs, particularly in the IDP camps. However, these practices were not being shared, as they were operated outside the Ministry and had no systemic means of being communicated and supported on a larger scale. The study also revealed the weaknesses in the provision of psychosocial support services, especially for teachers, pupils and students in conflict-affected areas.1 Based on this and other studies, a number of documents have been developed to provide a policy and legal framework for the creation of education programs for war-afflicted people in the north. One of these documents, the National Conceptual Framework emphasized the need for the use of psychosocial approaches in capacity development, mentoring, and support supervision in educational development, with a view to developing a child-friendly model of learning centers that could be replicated in Ugandan primary schools (MoES 2005b, p. 6-7). 1 Available from Creative Associates International, Inc. Evaluation of REPLICA/UGANDA Conducted by School-to-School International for Creative Associates International, Inc. Final version April 2007 p. 2 of 85 Based on the principles laid out in these documents, the MoES, along with numerous stakeholders, partners and NGOs, and especially with support from BEPS, developed a program called Revitalization of Education Participation and Learning in Conflict Areas (REPLICA). The REPLICA program was piloted in northern Uganda from July 2005 to July 2006 as a program under the BEPS project. In November 2006, the UNITY project was initiated with USAID funding, under which an expanded version of REPLICA will be implemented until November 2009. In order to ensure that lessons learned from the REPLICA pilot were applied to the scaled-up version under UNITY, USAID requested that an evaluation of the REPLICA pilot be done. This report is the result of that evaluation. OVERVIEW OF THE REPLICA DESIGN 6 components The main objective of REPLICA was to restore quality education services in northern Uganda. In particular, REPLICA was designed as a response to the following problems: • Children, and in particular girls, had been traumatized by the war and therefore needed to feel safe and secure in order to return to school and be able to learn. • Parents, teachers and school administrators, themselves victims of the war, were also traumatized and demoralized, often exhibiting minimal motivation to participate fully in school activities (i.e., “the waiting syndrome”). • The quality of teaching and learning had deteriorated significantly due to the difficulties associated with providing services for schools and systems constantly afflicted by war and upset by moving populations. To meet these challenges, REPLICA proposed a package of supports that integrated social, pedagogical and institutional services in order to re-establish schools as safe, welcoming and functional environments. REPLICA consisted of six components: • Guidance & Counseling/Psychosocial Care & Support, designed to help teachers, Head Teachers and Coordinating Center Tutors (CCTs) diagnose trauma in children in school-based settings, and provide assistance to help them cope. This component included the establishment of a Guidance & Counseling room in model schools and the 4 Primary Teacher Colleges (PTCs). • Peace Education, designed to help children understand the causes and effects of conflict and to acquire skills that would help them prevent or resolve conflicts without violence. This component, along with the Guidance & Counseling, included the concept of “the talking compound” in which signs were posted around the school yard to reinforce messages concerning positive self concept (e.g., “be focused on your future”), safe sex messages (e.g., “say no to bad touches”), and conflict resolution (“if someone hits you, throw him cotton”). Evaluation of REPLICA/UGANDA Conducted by School-to-School International for Creative Associates International, Inc. Final version April 2007 p. 3 of 85 • Leadership & Management, 2 designed to empower educational administrators in primary schools and PTCs to provide visionary and accountable leadership, optimize resource utilization and initiate positive change, and create sustainable collaborative networks with the wider community. The component included a peer exchange program in which teachers and Head Teachers visited neighboring schools to observe teachers, give feedback, and receive the same in their schools. • Girl Child Education Component, designed to promote girls’ education and improve the retention and completion of girls in primary school by building capacity for peer initiatives and increasing community support for girls’ education. The component included the provision of “Girl-Friendly Kits” to 8,000 girls. 3 The component also provided incinerators to schools for the disposal of menstrual pads after use. • Performing Arts and Learning in Schools (PALS), designed to make learning more relevant and participatory while at the same time restoring interest in local, positive traditions such as music and dance, as well as reinvigorating community interest in and support for schooling. This component also included the provision of a set of local instruments to all model schools and PTCs, and support for the publication of a locally-produced magazine in 3 of the PTCs. • Community Integration Program (CIP), designed to reverse the trend of community apathy and resistance to schooling – one of the major obstacles to school success documented in previous studies. The CIP included three sub￾components: community sensitization (e.g., school open days, testimonials), community outreach (e.g., community service) and community involvement (e.g., community provision of materials, school development plans). Each of these components was accompanied by training and materials for teachers and/or head teachers, described more later in this report. Consultative process One of the key elements of the REPLICA design was to ensure that the MoES and project partners and stakeholders played a genuine role in program design, policy determination, and implementation so that ultimately, ownership of the program would be assumed by the Ministry and the local population. To achieve this goal, REPLICA organized a variety of types of activity to facilitate stakeholder participation. These were: • Working Group meetings through which ministry officers and project partners could discuss specific project design and implementation issues. 2 Sometimes called Leadership & Governance. 3 The kits contained a packet of sanitary napkins, nail cutters, toothpaste and toothbrush, Vaseline, soap and soap dish, a piece of wrapping cloth (lesu) and a notebook and pen. The kits also contained a variety of reading materials on girls’ maturation and adolescence: short stories and poems were written by Ugandan authors and poets to motivate the girls to remain in school and achieve success. Evaluation of REPLICA/UGANDA Conducted by School-to-School International for Creative Associates International, Inc. Final version April 2007 p. 4 of 85 • Consensus-building meetings held both at the headquarters for the MoES officials and partners, and the district level for district education officials, PTC staff, head teacher representatives and other partners. • Public engagement meetings, one at the national level and four at the sub-regional level, for education officials and stakeholders to discuss issues affecting schools in their regions and ways of addressing these issues. Meetings were attended by Chief Administrative Officers (CAOs), District Education Officers (DEOs), District Inspectors of Schools (DISs), Senior Education Officers (SEOs), Local Council 5, Principals of four PTCs, and Deputy Principal Officers (DPOs). A total of 60 people participated in consensus building meetings. • Academic seminar with stakeholders: The theme of the conference was Opening dialogue on Peace Building, Conflict Resolution, and Leadership and Governance in Education. Participants included representatives from the MoES, public universities, UN organizations, civil society organizations, program implementers, cultural leaders and district education stakeholders. A total of 47 people attended the conference. • Frequent consultative and review meetings were held between the subcontractors,4 field implementers, and the MoES in conjunction with the technical BEPS III project staff. The major aim of these meetings was to review how the trainings were delivered at all levels, re-adjust and modify the implementation model in response to field realities, and generally to identify areas of improvement and challenges that required collective action. • Elders’ consultative meetings were conducted in the conflict-affected subregins of Teso, Lango and Acholi. The purpose of the elders consultative meetings was to give them an opportunity to share their wisdom and experience and to identify how best to motivate them to actively contribute to the improvement of performance in primary schools in their regions. These meetings were attended by cultural and religious leaders, SEOs, DEOs, DISs, Local Council 5, CCTs, and the representatives of Head Teachers for a total of 290 participants. REPLICA also organized political meetings, teacher union discussions, pupil circles and club activities. Selection of districts and schools REPLICA was piloted in 10 districts in the most war-affected areas in the North and Northeast of Uganda. The districts in the North were Gulu, Lira, and Apac, and the districts in the Norteast were Pader, Kitgum, Kaberamaido, Soroti, Katakwi, Amolatar, and Amuria. In each of 10 districts, 3 schools were selected for a total of 4 Subcontractors included Pincer Associates, FAWE/Uganda, Team Line, Sharon Cox. See ??? below. Evaluation of REPLICA/UGANDA Conducted by School-to-School International for Creative Associates International, Inc. Final version April 2007 p. 5 of 85 30 schools. Selection of the three schools in each district was based on the following criteria: • One was an “IDP learning center”: a school established in an IDP camp, drawing on different age groups and organized less formally than a normal school • One was a “mainstream school”: one that had not been moved, and • One was a “worst case scenario”: a school with weak infrastructure, unqualified teachers, etc. Additionally, four Core PTCs participated in the pilot. Tutors trained students and PTCs coordinated REPLICA in-service training activities through the Ministry’s Teacher Development and Management System5 (TDMS). The 30 model schools were selected by the Core PTCs and local education district. Cascade training Training, materials, concepts and processes were deployed through a 3-tier training cascade as follows: • National-level training (TOT) at which contractors, project implementers and ministry personnel trained “master trainers,” including pre-service tutors (i.e., PTC lecturers), outreach tutors (CCTs), PTC principals, DPOs, DEOs, Chief Officers and Inspectors of Services at a hotel or training center. Two workshops took place in the Soroti PTC and in Lira, each lasting approximately 21 days. A total of 125 master trainers were trained. • District-based training which was carried out in two ways: one for in￾service training and the other for pre-service training. For in-service training, CCTs trained Head Teachers and Senior Man and Woman Teachers from each of the 30 model schools, usually in workshops of 1- 2 week duration, at their Coordinating Centers (CCs). A total of 521 teachers were trained in these workshops. For pre-service training, PTC tutors trained students in the PTCs, usually through the incorporation of REPLICA content into course curriculum, though sometimes in a workshop format of several days as well. • School-based training (SBT): Head Teachers, assisted by the Senior Man and Woman Teacher from their school, were to train their fellow teachers in after-school and weekend workshops. According to the BEPS Final Report, all teachers in the 30 schools were trained in SBTs in all 6 component areas of REPLICA, reaching 18,904 girls and 19,762 boys (BEPS final report). 5 The TDMS structure, established in 1993 by the MoES with significant support from USAID, was a system designed to provide initial and ongoing training of teachers through its network of “Core-PTCs.” In the TDMS system, each Core-PTC has a network of Coordinating Centers (CCs) in its “catchment area.” Each CC, in turn, has 15-25 schools in its “catchment area” and is staffed by a CCT who ensures ongoing training of teachers and Head Teachers in his/her cluster at the CC. Approximately half the PTCs in Uganda participate in the TDMS structure as Core-PTCs; 4 participated in the REPLICA pilot. Evaluation of REPLICA/UGANDA Conducted by School-to-School International for Creative Associates International, Inc. Final version April 2007 p. 6 of 85 In addition to training, the TDMS structure was also used to deliver materials to target populations, both during their training and subsequently through the CC and PTC catchment area networks, and to deliver monies for training and support supervision, described below. Support supervision & monitoring Once a particular target group was trained, REPLICA used a system called support supervision and monitoring in which teachers and Head Teachers received regular visits by various personnel to ensure understanding, to provide support and to monitor progress. Support supervision and monitoring was organized at three levels: the national level, at which Ministry personnel provided support to the PTCs; the PTC level, at which college personnel visited the CCs and schools to ensure implementation; and at the school level, at which CCTs, contractors and District Education Officer personnel visited teachers and Head Teachers in the model schools. Financing of support supervision activities was ensured through the TDMS structure in which monies were passed from REPLICA through the MoES to the PTCs, then to the CCs and District Education Offices to ensure the implementation of support supervision and other project activities. Project management and implementation While day-to-day management of REPLICA was assured by the BEPS project, responsibility for implementation was shared by a number of organizations. The Department of Special Needs Education, Guidance & Counseling (SNEGC) in the MoES established the policy framework for REPLICA and took responsibility for quality assurance and control, ensuring systemic support to the program. The BEPS project provided conceptual guidance, technical support, funding and the monitoring and evaluation frameworks. BEPS collaborated with local organizations, including the NREF, PTCs and DEOs in managing technical and programmatic components. These included development of materials, trainings, public dialogue (sensitization), and school-level activities. To ensure the implementation of the six components, the MoES and BEPS engaged the services of three contracting groups and one individual. Pincer Associates was responsible for implementation of the Guidance & Counseling/Psychosocial Care & Support, Peace Education, and Leadership & Management components. Team Line was hired to implement the PALS component, the Forum for African Women Educationalists, Uganda Chapter (FAWEU, hereafter called FAWE) was hired to implement the Girl Child Education Component, and Sharon Cox was hired as an independent consultant to implement the CIP. Each contractor provided the following services: • An inception report, identifying the needs to be addressed in its Scope of Work, and a plan of action. • A baseline study of conditions in the 30 models schools in its implementation area (see “inception reports” by Pincer Associates, FAWE, Teamline and Sharon Cox in References, below). • Materials for its component(s). Evaluation of REPLICA/UGANDA Conducted by School-to-School International for Creative Associates International, Inc. Final version April 2007 p. 7 of 85 • Training of Trainers in its component area at the national level workshop. • Support supervision in the schools, and • A final report for its component. Monitoring and evaluation Early in the program, the MoES established a program monitoring plan that included activities; input, process, output and outcome indicators; time frames; persons responsible and resources required (Opiro, Wirefred George: 2006b). Examples of indicators include a talking compound with relevant and appropriately displayed messages, a functional guidance and counseling room, an integrated work plan, a school map, gender sensitive sanitation facilities, functional peace clubs, routine teaching of peace in the schools, community integration activities and use of performing arts in the schools. The contractors used these indicators in their baseline and follow-up activities, the findings of which constituted their baseline, quarterly and final reports. Implementation constraints The above description might leave one in awe of the volume of work accomplished by REPLICA in so little time – just one calendar year, from July 2005 to July 2006, with effectively 3 months of actual implementation during the school year 2006. Such awe is warranted, particularly in light of the constraints faced by the project during its short life. These include the following: • The creation of new districts: After REPLICA started with 8 districts, the government created new districts which required REPLICA to expand to 10 and eventually to 13. With the creation of each new district came the requirement to plan project activities within the new administrative structures, and to train newly added district officers – all with additional budget implications. • The constant threat of war and insecurity, especially in the Pader district, affected project planning and implementation and, of course, raised constant concerns about staff and partner safety. • According to numerous reports, the low levels of knowledge and skills on the part of teachers in model schools meant that significant time was required to train teachers in relatively basic concepts. • The transfer of teachers and head teachers resulted in lost time as new personnel continually needed to be trained from the beginning. • The decongestion of IDP camps led to significant teacher and pupil movement from school to school, with new schools being created and, concomitantly, no system of schools statistics to help with tracking the changes. • Because of a full school calendar from July to December, some components of REPLICA had a late startup – for example, Team Line did not begin training activities until the December holidays. • Local and national elections eclipsed all other activities during the months of January and February 2006; as such, REPLICA was not able to begin working Evaluation of REPLICA/UGANDA Conducted by School-to-School International for Creative Associates International, Inc. Final version April 2007 p. 8 of 85 in schools until the end of March, and involvement in the schools ended in June. • One result of the recent history of war in the north was a lack of infrastructure, resulting in the need to use hotels for basic training events, which significantly impacted the budget. Lack of infrastructure also meant that demonstrations in neighboring schools were not always possible. Given these and other constraints, the achievements of REPLICA are indeed impressive. Nevertheless, these constraints suggest a number of design elements that will need to change when REPLICA expands from 30 to 1,700 schools in its next phase. These suggestions are presented in the recommendations section at the end of this report. EVALUATION DESIGN AND IMPLEMENTATION Objective and design of the evaluation The objective of this evaluation was to provide program managers with the information they need to modify REPLICA in order to better meet government needs while at the same time providing information on how best to scale up in its expansion stage under UNITY. Such an evaluation is crucial if the REPLICA pilot is to have served as a basis for learning about what works and should therefore be continued, as well as what did not work and should therefore be modified or eliminated. The design of the evaluation was qualitative, based primarily on reports from project implementers, MoES personnel, stakeholders and beneficiaries, as well as observation of inputs in model schools and colleges (e.g., G&C rooms, incinerators), and analysis of project monitoring and evaluation reports, and training and instructional materials. Evaluation questions The principle questions of this evaluation were the following: • What are the strengths and weaknesses of the REPLICA program in terms of implementation and design? and • How should REPLICA retool to prepare for the next phase? In order to answer these questions, a number of domains of inquiry were identified, including the quality of training, materials, the 6 components of REPLICA, support supervision & monitoring, project design, project management, and questions concerning project scale-up. Based on these domains, a question matrix was developed to identify specific questions, indicators and data sources. From the matrix, Mr. Bananuka, who had served as the TDMS Outreach Coordinator (assisting with the REPLICA project), and the evaluator developed 6 data gathering tools to be used in the field and in Kampala (see Annex A for samples of tools). Evaluation of REPLICA/UGANDA Conducted by School-to-School International for Creative Associates International, Inc. Final version April 2007 p. 9 of 85 Sample Purposive sampling was used to select the districts, schools and individuals to be interviewed for this evaluation; that is, project implementers, stakeholders and beneficiaries were stratified by geographic location and role, then selected on the basis of the criteria that follow. The districts were selected based on the following criteria: • Effects of the war: Since people in the North were the most significantly affected by the war, more districts in the North were selected. • Importance of PTCs: Given the central role of the PTCs in the implementation of REPLICA, it was decided that all 4 colleges must be visited; hence, their districts were given priority. • Accessibility: In order to maximize the amount of data that could be collected in a 2-week period, the most accessible districts were selected. Based on these criteria, five districts were selected for inclusion in this evaluation: Gulu (N), Kitgum (N), Apac (N), Soroti (NE) and Amuria (NE). In each district, the Core-PTC was visited, including the Prinicipal and DPO, interested tutors and students. In the three districts with District Education Offices, the DEO was also interviewed – in Kitgum, with his DIS, SEO and another inspector. In Gulu, the Principal Inspector of the Education Standards Agency (ESA) was also interviewed. Within each of the districts selected, model schools to be visited were identified on the basis of the following criteria: • Depth orientation: Because of the importance of the model schools in the REPLICA program, it was decided that the different actors there – Head Teachers, teachers, pupils, parents and CCTs – should be interviewed separately. The decision was therefore taken to spend 3-4 hours in each school which, as a consequence, limited the number of schools in the sample to a total of four. • REPLICA categories: At least one school from each of REPLICA’s three initial school selection categories was to be visited: mainstream, IDP Learning Center or Worst Case Scenario. • Balance between strong weak schools: REPLICA’s distinctions did not necessarily distinguish between strong and weak schools – e.g., a mainstream school could be strong or weak – so the criterion of strong and week schools was added. With the assistance of UNITY staff Patrick Bananuka and Martin Opolot, UNITY staff familiar with the REPLICA design, the following schools were selected: Evaluation of REPLICA/UGANDA Conducted by School-to-School International for Creative Associates International, Inc. Final version April 2007 p. 10 of 85 Table 1: Primary schools included in the evaluation Region District School Classification N Gulu Abera PS IDP Learning Center Strong N Kitgum Pajimo Army PS Worst case scenario Weak NE Amuria Angera Wera PS Mainstream Strong N Apac (Loro PTC) Aculbanya PS* Mainstream Strong * A visit had originally been planned to visit Bara Midyang PS, a “worst case scenario” (weak) school but because the Head Teacher had not prepared for our visit, Aculbanya Primary School was visited instead.6 Data were collected both from the field and from actors from a variety of organizations in Kampala, including representatives from the REPLICA project; the UNITY Project; the Ministry’s SNEGC; FAWE; Pincer Associates; Team Line; and Kyambogo University. In all, 127 people participated in interviews or focus groups for this evaluation as follows (see Annex B for a list of persons consulted): Table 2: Number of study participants Ministry MoES personnel 3 DEO staff 6 UNITY Project staff 3 Primary schools (4) Head teachers & Deputy Head Teachers 6 Teachers 14 * Pupils 50 ** Parents 26 *** CCTs 13 PTCs (4) PTC administrators 11 Subcontractor staff 10 Tutors 8 Other Inspector of Schools, ESA 1 Lecturer, Univ of Kyambogo 1 Principal Inspector of Schools, ESA 1 Total 153 * 12 male, 3 female ** 24 male, 26 female *** 24 male, 26 female 6 Though the classification of these schools suggest that we mainly visited strong schools, in fact Abera PS had recently been moved to a new location and was in the process of re-establishing itself; few teachers were around, almost no materials were in evidence. Thus, though it still had capable leadership, Abera might temporarily be considered a weak school. Evaluation of REPLICA/UGANDA Conducted by School-to-School International for Creative Associates International, Inc. Final version April 2007 p. 11 of 85 Data collection and analysis Data were collected from February 12-23, 2006 in Uganda by this evaluator with significant assistance from Patrick Bananuka, who participated in the initial interviews and focus groups in the North to ensure that comprehension of REPLICA implementation and the roles of interviewees in its implementation. The remaining interviews and focus groups were conducted solely by this evaluator. Interviews in the PTCs were conducted in a focus group format, with questions posed to the entire group. Representatives from each group – administrators, tutors and students – were expressly encouraged to answer each question. Interviews with district education officials were conducted individually if only one official was available; in Kitgum, the interview was conducted in a group. In contrast, interviews and focus groups at the primary schools were conducted in turns, each interviewee responding by role as follows: • Interview with the Head Teacher and Deputy Head Teacher together • Interview with teachers (n=3-6, balance of male and female) • Interview with 6 girls P5-7 • Interview with mixed group of pupils P5-7 (n=6 or more) • Focus group with parents (n=6 or more, balance of fathers and mothers) • Interview with the CCT. During week two of the consultation in Uganda, the evaluator conducted a preliminary analysis of the data and presented it to a group of approximately 40 stakeholders, beneficiaries and interviewees at the Namirembe Guest House. The group validated the preliminary results of the evaluation, and raised several issues concerning the problems associated with gathering school statistics in the North, issues of community involvement and best practices (to be documented by UNITY), the issue of sustainability with girls’ pads and PTC magazines, and the appropriate role of the district offices in future REPLICA activities. Upon return to the US, the evaluator completed the analysis using qualitative methods – i.e., by identifying patterns and themes in the responses of participants in the evaluation, triangulating the responses of one group with those of another, and finally identifying the most frequent and salient findings for this report. The evaluator added his own observations based on his assessment of participants’ responses, project reports, materials and the status of the project, and included them in the Findings and Discussions sections of this report. Limitations of the evaluation In the opinion of this evaluator, the conditions of this evaluation permitted an adequate sampling of project activities and outputs to provide an objective and critical appraisal of project strengths and problems. Nevertheless, as with all research, this evaluation was subject to a number of limitations. They were as follows: • Evaluation time frame: During his 2-week stay in Uganda, the evaluator was able to perform evaluation design, data collection and preliminary analysis Evaluation of REPLICA/UGANDA Conducted by School-to-School International for Creative Associates International, Inc. Final version April 2007 p. 12 of 85 activities, leading to a presentation of the initial findings. He used an additional week in the US to complete the analysis and write the draft report. While this time frame permitted adequate collection of data, more time would have permitted a broader sampling and perhaps a more thorough review. • Limited sample of the project: Another limitation of this study related to time was the length of the project: though the life of the project was one year, actual implementation time after baselines were conducted, elections were held and project staff were able to enter the schools, was only about 3 months. This evaluation, then, represents a look at a one-year project that was fully operational for only a quarter of that time - and in its pilot form at that! • Overlapping interventions: As noted above, a number of interventions were not unique to REPLICA, thus limiting the extent to which results can be attributed to the project. For example, the REPLICA baseline found that some intended beneficiaries had received training prior to REPLICA in psychosocial care, guidance and counseling, human rights, and financial management, among others. All schools reportedly had co-curricular activities at the time REPLICA began, including scouting, debating and peace building – three types of clubs cited in this evaluation attributed to REPLICA. In some cases, similar (“overlapping”) interventions were provided by other education partners – e.g., TASO provided interventions in guidance & counseling and AVSI in psychosocial care. In other cases, overlapping interventions were actually conducted by BEPS, such as the Presidential Initiative on AIDS Strategy for Communication to Youth (PIASCY), which also fostered the use of suggestion boxes, talking compounds and G&C. Thus, the problem of overlapping interventions raises questions about our knowledge of the impact of REPLICA vis-à-vis other projects and the extent to which results can be attributed to REPLICA alone. • Insufficient reliable monitoring data: As noted above, significant efforts were made in REPLICA to develop and use monitoring instruments in each of the 6 components of the project. However, as the discussion on M&E below explains, most data collected from these exercises lack the characteristics necessary to make them comparable or to derive conclusions based on measurable changes. Moreover, much internal monitoring data was not available to the evaluator, such as financial information and materials distribution figures. As a result, this evaluation focused its attention principally on the perceptions of project beneficiaries, anecdotal reports and observation of project outputs (e.g., materials), rather than outcome measures such as changes in knowledge, skills, attitudes, practices or beliefs, or efficiency measures such as relative cost of project components. FINDINGS This section examines partners’, stakeholders’ and beneficiaries’ perceptions about REPLICA’s achievements, then the strengths and difficulties associated with particular aspects of the project. Evaluation of REPLICA/UGANDA Conducted by School-to-School International for Creative Associates International, Inc. Final version April 2007 p. 13 of 85 Major achievements When asked “What were REPLICA’s biggest achievements?”, respondents overwhelmingly cited experiences that can be grouped into three categories: improved relations, pride and empowerment, and improved enrollment and retention. Improved relations Teachers, head teachers and CCTs reported that the REPLICA project had brought about a palpable change in relations in schools: among teachers, between teachers and Head Teachers, between teachers and parents, and between adults and children. Some teachers reported that where they used to see their work as individuals, now they were more inclined to plan together, work together to solve problems, consult one another and observe one another in class. In some schools, the hierarchical relationship between teacher and Head Teacher has been replaced by one of greater collegiality and respect, with teachers participating in school planning and management: “We all see ourselves as leaders.” In another school, the name of “the dispute committee” had changed to “the counseling committee.” Similarly, respondents reported that the traditional line between teachers and parents had been blurred as a result of REPLICA. Some parents reported that where they used to be reluctant to enter the school grounds, now they come to school to talk to teachers to discuss their children’s progress or problems. A number of teachers and parents reported greater community involvement and interaction in general. Numerous teachers reported that the biggest difference REPLICA made was in relations between themselves and pupils, the biggest outward sign being the abandonment of corporal punishment. Parents also noted similar changes; one father said he had abandoned the cane for guidance and counseling “because it works better.” Teachers and parents also spoke of increased respect pupils had for adults. Ministry officials and school personnel alike spoke of increased self discipline and respect amongst pupils and as a result, reduced violence. “Instead of My children are now coming closer to me, not running away. We used to beat them, we didn’t guide them. Now they come to the teacher with their problems without fearing their teachers. Teacher We used to collide in the work; now no more collisions. We used to see beating a child as a way to teach. Now we ask, “Why is a child doing that?” You consult, find out what is his problem, then solve help him solve it. Parents Now big kids tell younger ones what is wrong, rather than the teacher always being the one. And in class, older kids come up to assist. Sometimes they say “let’s wait for the teacher.” They’re happy. Teacher We value education more. We look at it with wider eyes than before. Parent Evaluation of REPLICA/UGANDA Conducted by School-to-School International for Creative Associates International, Inc. Final version April 2007 p. 14 of 85 fighting, they discuss and help each other,” one teacher stated. According to some interviewees, there has been greater discipline and less violence amongst teachers as well. Improved relations were also noteworthy at the PTCs where students attested to being helped by their tutors and, on occasion helping their tutors themselves. The feeling of teamwork and collegiality experienced amongst teachers in the primary schools was also shared by students and tutors in the PTCs. Pride and empowerment According to those interviewed in this evaluation, REPLICA has contributed to a sense of pride in primary schools and PTCs. One source of pride is the performing arts and debating competitions, organized through the PALS component, in which pupils and students proudly described the competitions they had won. Pupils also spoke proudly of clubs to which they belonged in their schools. Adults also exhibited pride in achievements related to REPLICA: one Head Teacher displayed a trophy on her desk awarded by World Vision for her work in organizing teacher exchanges with other schools – an activity organized as part of the support supervision component of REPLICA. Some Head Teachers and teachers spoke of children’s sense of increased empowerment due to REPLICA. One teacher cited an example where the Minister had come to the school for a presentation and a pupil handed her the microphone saying “you have 5 minutes to talk.” Some spoke of girls being empowered to speak out on defilement, with teachers’ and parents’ awareness raised about this issue as well. This evaluator indeed noted the forthrightness, sense of dignity and self-worth of pupils in some schools as they spoke freely about girls’ education, peace education, and life in their schools. Importantly, some felt free to ask questions of the evaluator – a significant indication of children’s self-esteem and understanding of their right to self-expression. However, this observation was not consistent across schools: parents, teachers, and pupils were more apt to share stories of pride and improved relations in “strong schools,” particularly those where teachers and parents were more verbal about their role in sharing leadership in the school. At the PTCs, some students reported increased confidence after contributing articles to their school magazines. Others reported that as a result of leadership and management training, girls assumed more responsibilities in the colleges. More students were also coming forward in student elections. This evaluator was struck by the pride with which students spoke in focus groups, in some cases signing the attendance sheets as “Member, Red Cross Club” or “Health Minister.” Last year, a father and mother were punishing their child seriously. The boy discussed it with the senior woman teacher. The parent was abusing the child. The parent came to the school, and the situation got better. Head Teacher Evaluation of REPLICA/UGANDA Conducted by School-to-School International for Creative Associates International, Inc. Final version April 2007 p. 15 of 85 Improved enrollment and retention One of the most frequently cited impacts of the REPLICA project was the increased enrollment of pupils in schools, especially girls. Head Teachers and teachers alike reported that since REPLICA had arrived, more children were attending school. In particular, they reported that the girls’ kits had increased girls’ enrollment and retention. In some cases, girls came to school after the kits were distributed, and were told that they should “be good” in order to receive one next year. Some teachers and administrators claimed that REPLICA had resulted in a reduction of early marriages, a claim supported by some parents who attested to their increased awareness of the importance of sending their girls to school. Some Head Teachers reported that as a result of REPLICA inputs, including the kits, arts programs, and clubs, some children were returning to school after having dropped out, and that some children were coming from other schools to REPLICA schools “because they are more child-friendly.” Teachers and Head Teachers also said that pupils were dropping out less, getting better marks and achieving higher pass rates than in previous years. These claims notwithstanding, it is important to note that while Head Teachers were able to produce enrollment statistics, no standardized manner of collecting or reporting the data was evident; nor was demographic information available that might also explain enrollment changes (e.g., decongestion of IDP camps, school closures, population movements, etc.). Whatever the numbers might say, conditions in REPLICA schools are palpable – a phenomenon that led PIASCY evaluators to ask the DPO at the Loro PTC why certain schools in their review were different, to which the DPO responded, “They’re REPLICA schools.” The following section presents a more detailed look at interviewees’ perceptions of REPLICA’s interventions, both at the primary and PTC level, in terms of project design, training, materials, support supervision and management issues, as well as observations of the evaluator. Design Consultations As noted above, REPLICA was designed with significant input from stakeholders and intended beneficiaries through a variety of consultative mechanisms. According to the Commissioner of the SNEGC, the lead department within the MoES for REPLICA, REPLICA was the only project that had “penetrated the issue of cultural leaders” – i.e., created a genuine consultative role for them – and in so doing, was beginning to command the respect of a wide range of Ugandans. This, he said, was a big achievement. One DEO noted that through the various consultative processes Before, girls were just assets to be sold; now we know that even girls can do something, their knowledge is the same as boys. This is why their enrollment has increased. “We brought more girls to the school because education was improved. Parents Evaluation of REPLICA/UGANDA Conducted by School-to-School International for Creative Associates International, Inc. Final version April 2007 p. 16 of 85 used in REPLICA, “all key stakeholders had been trained” and thus capacity had been built. Use of subcontractors The use of subcontractors in the implementation of REPLICA clearly facilitated the development of materials and systems and the implementation of a host of activities that, given the short time frame of the pilot, might have been otherwise impossible to achieve. Subcontractors also brought significant and diverse technical expertise to the project, thereby providing a richer pool from which to draw for the training of counterparts and stakeholders. If subcontractors were able to bring quality, variety and capacity to REPLICA, they also provided services that were viewed at times to be uneven. According to the perceptions of some staff at the PTCs, the subcontractors provided more services for the CCs and schools than they did for the PTCs. Interviews with project staff revealed that the quality of inputs varied from contractor to contractor. As the descriptions of training and materials below illustrate, the output of each contractor varied substantially in terms quantity, quality and impact. According to some reports, services were delivered less well by contractors in some instances, owing to their relative inexperience for specific tasks assigned to them. In these instances, project staff sometimes felt that contractors were not providing a “value added,” rather, they were demanding time of project staff just to perform the duties for which they had been hired. According to REPLICA training participants (e.g., teachers, Head Teachers, CCTs, DEOs, tutors), training activities were competently executed, though support supervision was often insufficiently structured or too infrequent. In the view of this evaluator, baseline and monitoring reporting were weak across subcontractors (see the discussion of M&E below), though reasons for these difficulties may be a result of the M&E design itself. Of the final reports evaluated for this report, those conducted by Sharon Cox and FAWE presented the most useable data. TDMS & DEO The choice to use the TDMS structure to deliver the REPLICA project was universally praised across respondents, even district education office personnel. TDMS is clearly a well-established system in Uganda, understood by all and capable of delivering a program such as REPLICA. The choice to implement REPLICA through TDMS nevertheless raises perennial concerns about the role of the District Education Offices in this program over the long term. It is important to note that not all District Office personnel raised concerns about the DEO’s role in a program such as REPLICA. When concerns were raised, however, two principal comments were made: (1) District personnel are sometimes excluded from REPLICA events – indeed, of MoES events as well. One officer said that “sometimes the Commissioner goes straight to the field, without stopping here or informing us of his/her actions.” (2) The DEO lacks the resources to carry out its support supervision role properly. Yet even according to the DEOs themselves, REPLICA invites them to its events and provides some resources to assist with support supervision. Would more Evaluation of REPLICA/UGANDA Conducted by School-to-School International for Creative Associates International, Inc. Final version April 2007 p. 17 of 85 involvement or support be a remedy? Numerous respondents made it clear that passing funds or materials through the district offices would undoubtedly lead to delays in project execution and should therefore be avoided. The Principal Inspector recommended that TDMS file its reports through the DEO to the Commissioner of Teacher Education. However, the Commissioner has no legal authority over the District Offices, so while such a provision might improve information sharing, it could not result in any structural difference in how the project is implemented. The question therefore remains as to how to meaningfully integrate the DEOs while developing capacity not only at the PTCs but also at the district level. Training As described above, REPLICA used a cascade model of training consisting of national, mid-level (PTC or CC) and school-based events. One general finding was that the cascade within the TDMS structure was supported by all respondents, including district officers. This section examines the quality of training at each level as perceived by project participants. National-level training Education officials, PTC administrators, CCTs and tutors alike spoke highly of the training they had received at the national level, noting good content, easy-to-use materials, participatory activities, good facilitation, and a friendly environment with participants sharing experiences. Some noted that they felt empowered by the training and sufficiently equipped to train at the next level. Respondents were mixed on the amount of time required for the training (21 days), some saying it was adequate, others indicating that it was too long and should be broken up. Some subcontractors noted that time was sometimes insufficient for them to fully present their portion of the workshop – at times being informed at the last minute, for example, to reduce their presentation from 5 days to 2. One contractor noted that participants were fatigued toward the end of the workshop, when his session was scheduled to commence. Some respondents noted that a “one-off” was not sufficient and that “remedial training” was necessary to reinforce newly-acquired skills. District-based training The in-service workshops held by CCTs for Head Teachers and Senior Man and Woman Teachers received the same positive reports as the regional-level trainings concerning content, materials, facilitation and conditions. Again, respondents (CCTs, Head Teachers and teachers) were split on their evaluation of the timing (8 days on average), some saying it was too short, others too long. The pre-service training also received generally positive reports from PTC students, who said that as a result of the training, they understand the messages and “feel they have a place.” Some tutors commented, however, that PTC-based training tended to be carried out “in doses” and needed to be put together in a more coherent manner. Some tutors also complained that their colleagues had not been trained, leaving a disproportionate burden on them to ensure all training in their institutions. Evaluation of REPLICA/UGANDA Conducted by School-to-School International for Creative Associates International, Inc. Final version April 2007 p. 18 of 85 At Gulu PTC, an administrator lauded a third kind of district-based training through which REPLICA had come to his college and delivered a 3-hour tailor-made training on program planning. School-based training (SBT) A serious weakness of REPLICA is that in many cases training did not reach the school level and thus, at least in those cases, had no impact on teaching or learning. Most persons interviewed for this evaluation indicated that the SBT format of training is ineffective. Interviews in primary schools indicated that attendance at SBT workshops usually averages below 50%. One PTC administrator reported that according to his institution’s monitoring data, only about 20% of Head Teachers who are trained in REPLICA messages even carry out training in their schools. In her final report, Sharon Cox noted the impact of this problem with the CIP component: Head Teachers and teaching staff that underwent training did not share the information and skills received with their colleagues. As a result the school management did not strategize on ways to use the school resources and skills to effectively implement both programs. This led to teachers feeling overwhelmed with their work loads as they were not receiving support from their work colleagues. When asked about the problems associated with SBTs, respondents most often said that teachers were resentful of their colleagues who had benefited from financial incentives for their training (e.g., allowances for lunch, transport), whereas teachers in SBTs did not. Another reason was that teachers often do not respect their peers, even Head Teachers, as much as someone from outside the school. One Head Teacher reported that “teachers listen to headquarters more than they listen to Head Teachers.” Other reasons given included the following: • Trainings are too short or infrequent: though the usual pattern was one weekend per month, some respondents reported less frequent training – in once case, once per term. • Not enough time is allotted to each training event. • Trainers have insufficient materials such as flip chats, manilas, and books. • The low level of teachers mitigates the quality of training; teachers often have only a basic mastery of content, thus requiring longer, more intensive training than the SBT model offers. Materials Interviewees were shown a list of print and other materials (e.g., girls’ kits, incinerators, etc.) provided by the REPLICA project, then asked to identify which ones had been the most useful. In terms of print materials, those produced for Guidance & Counseling/Psychosocial Support were cited most frequently, followed closely by the Peace Education materials. Respondents noted the user-friendliness of the materials, the simple language, and their pertinence to the context in northern Uganda, though some noted the theoretical language in some of the Evaluation of REPLICA/UGANDA Conducted by School-to-School International for Creative Associates International, Inc. Final version April 2007 p. 19 of 85 materials.7 The PALS materials ranked third, Girls’ Education and CIP tied for fourth and Leadership & Management last. This is a curious finding since Leadership & Management was viewed as the second most successful component. This discrepancy suggests that the impact of Leadership & Management might have been achieved more through its training and support supervision activities than through its materials. As noted in Limitations of the study above, a conclusive assessment of the success of REPLICA materials is not currently possible due to the late delivery of the materials. In one school (Abera), when the question concerning the materials’ effectiveness was posed, teachers could not respond because the materials had not yet arrived (the school was an IDP learning center and had just moved). In other instances, participants showed us packages of materials that had just arrived, and had not even been unpacked. The FAWE Final Report (2006a) supported this observation, stating that materials on the REPLICA components generally, and Girl Child Education component specifically, were inadequate in all the project schools: 73% of schools had some REPLICA materials – albeit in insufficient numbers – and 23% indicated they had no materials at all. When the materials were present, the most prevalent were Guidance and Counseling and Peace Building (FAWE 2006a). It should also be noted that familiarity with the materials was relative, as most participants had received them in draft form. As a result, few beneficiaries were familiar with the materials in their final form, so showing them the materials did not help remind them of the materials under discussion. In the primary schools, respondents were more effusive about the non-print materials, especially the girls’ kits, but also the incinerators and the musical instruments. By far, the biggest complaint about materials at all levels was their quantity: even when they had been delivered on time, beneficiaries felt that they were not in sufficient quantities. Formulas for numbers of materials per institution were not clear in this evaluation, but the Loro PTC reported that they had received 3 copies of materials in Peace Education, Leadership & Management and other components – clearly not enough to equip the library, all tutors and students. As to the quality of the print materials, a cursory analysis by this evaluator revealed, in general, a high level of quality in terms of layout (attractive, easy to use), comprehensive design (including trainer’s guides, toolkits, worksheets, reference books, lesson plans), and content (simple, clear explanations, relevant and accurate information). However, a review of the materials suggests two problems: • Lack of uniformity: Across the 6 components, there is a remarkable range of materials. The Guidance and Counseling materials are clearly the most fully (and well) developed. This is not surprising since, as noted above, G&C had begun as a BEPS intervention and implemented in several districts two years prior to REPLICA. At the other end of the spectrum is PALS, with no published materials, only training materials that were handed out in workshops. 7 The DEO in Kitgum noted that in the Therapeutic Play materials, language had scared people and was difficult to understand. One of the challenges in this volume, he noted, was to make the materials less theoretical. Evaluation of REPLICA/UGANDA Conducted by School-to-School International for Creative Associates International, Inc. Final version April 2007 p. 20 of 85 Leadership & Management and Girl Child Education each have one module or guide, and Peace Education has modules available for lower and upper primary. The sheer difference in architecture of the materials might partially account for the difference in perceptions concerning their quality. These differences have important implications concerning ease of use and cost in the scaling up phase of REPLICA (discussed in Recommendations below). • Inadequate evaluation of effectiveness and efficiency: Ministry officials and subcontractors who developed the materials indicated that in most cases, some kind of initial evaluation of the materials had been conducted before printing them, either through a review of the materials by end users, or an evaluation of their use in a training setting. While such measures provide some information concerning the effectiveness of materials, the failure to use a standardized system of evaluating them “in real time” – i.e., during the development process under real or simulated conditions – is the probable reason for the variation in quality. For example: o The language sometimes vacillates from 2nd person to general description without any explanations to guide the reader. o Parameters of training and learning activities are sometimes unclear –e.g., how to set them up, time requirements, etc. o Activity descriptions are often cryptic or incomplete. o No assessment of learning is included in the training guides. o The high production quality of materials (e.g., glossy covers, 4-color printing) raises questions about sustainability, both in the REPLICA scale-up phase and afterwards. o Newer materials lack the depth of other already-existing materials. For example, the Leadership & Management module developed for REPLICA lacks the relevance, detail and clear training structure of the 4 BEPS-developed leadership modules which, in this evaluator’s view, are clearly superior and should be retained. In order to ensure that materials are achieving their intended objectives in terms of quality and cost, some kind of standard formative evaluation process is necessary for their development, or if materials are to be selected for use, a transparent process with objective criteria must also be used in order to ensure that the best materials are selected. Support supervision & monitoring Support supervision This evaluation found that, in the main, support supervision occurred on a regular and frequent basis across components. Patterns varied, from twice per term by CCTs on average, to three times or more for subcontractor staff in a given component. Most teachers were visited three times or more. Evaluation of REPLICA/UGANDA Conducted by School-to-School International for Creative Associates International, Inc. Final version April 2007 p. 21 of 85 According to teachers and Head Teachers, support supervision worked well. They found it helpful in consolidating knowledge and skills acquired at CC and PTC training, and in some cases, the manner of supervision changed their manner of working from punitive to more supportive modes. One CCT claimed, “Before, teachers hid during supervision; now they welcome it.” One type of support supervision about which participants were particularly enthusiastic was peer exchange program (described above). Respondents cited several difficulties with support supervision. Some teachers requested that the number of visits needed to increase in order to ensure consolidation of new knowledge and skills. The FAWE report (2006c) described scheduling difficulties experienced by the different subcontractors working in the same schools: “When you think of a date, you find other contractors already having meetings going on.” CCTs and DEOs noted that they did not have enough money to move around. Monitoring As noted above, the MoES, REPLICA and its subcontractors made substantial efforts in the beginning of the project to develop monitoring instruments for use by all project implementers. Indicators and tools were developed for each component and sent to the model schools for comments. It was unclear at the time of this evaluation whether these tools had been finalized, but each subcontractor did use its own tools to monitor progress and provide information via the quarterly and final reports. These important efforts notwithstanding, the lack of quantifiable and reliable indicators in this project resulted in a substantial body of monitoring data that provided many useful insights into project successes and difficulties, but little empirical data with which to reliably measure project results or impact. Specifically, the following problems were found in the baseline, monitoring and evaluation tools used in the REPLICA project: • Many qualitative items are expressed in vague terms – e.g., “school community activities – list” or “classrooms and furniture” or “teaching/learning materials.” Each of these items was listed without clear responses to tick, scales to fill in or rubrics to use as guides, rendering responses vague and subjective. • Responses to some items are left open, making analysis and comparison across schools and populations difficult. • Some items ask for information that would be impossible to use – e.g., enrollment data. As noted above, without reliable collection procedures and accompanying demographic data, it is impossible to determine whether increases in enrollment are due to project interventions, decongestion of IDP camps, recent trouble and movement in the area, or something else. The other important problem encountered in the monitoring component of this project was the basic lack of records at the school level. This was highlighted as one of the biggest problems faced by REPLICA in virtually all of the quarterly and final reports submitted by the subcontractors. The FAWE (2006a) report, for example, noted that the majority of schools lacked records on school activities, especially REPLICA components. The most recorded were Guidance and Counseling Evaluation of REPLICA/UGANDA Conducted by School-to-School International for Creative Associates International, Inc. Final version April 2007 p. 22 of 85 (19 of 30) and Girl Child Education (12 of 30). This problem illustrates the need for a simple, common system of reporting that can be instituted in all project schools if accurate school level records are to be kept. The 6 components Success of each component All respondents were asked to identify the component that they felt was the most successful in REPLICA. Of course, “successful” can mean a variety of things, from the quality of implementation to actual impact to relevance to the northern Ugandan context. The question was deliberately left open so that a broad range of responses would be generated. The question was followed up by “How was it successful?” and “Can you provide an example?” It is important to note that with the exception of CIP, the other 5 themes of REPLICA clustered fairly strongly; in most situations, respondents gave numerous responses to the question, suggesting that the real success was the package of inputs, and not just one or two. Nevertheless, when the responses of interviewees are tallied, Guidance and Counseling/Psychosocial Support emerged by far as the most successful of the components. Respondents explained that it had provided them with the skills they needed to help pupils and to help one another, and that it had improved relations and reduced violence both in school and at home. Respondents identified the second most successful component as Leadership and Management. Teachers, parents and Head Teachers spoke of the change in management in their schools: things were more orderly, planning had become easier, more was done through team work, and delegation had become more the norm than before. In all schools visited, this evaluator noted a plethora of corroborating evidence in the form of “talking offices” (signs on Head Teachers office walls), including vision and mission statements, integrated plans, organizational charts and inspirational quotations. Three components tied for third most successful component: Peace Education, Girl Child Education and PALS. In the primary schools, Peace Education and Girl Child Education ranked higher, whereas in the PTCs, Peace Education and PALS ranked higher. CIP was least often ranked successful of the 6 components. This finding was corroborated by project reports and interviews with project staff. The above rankings raise several questions about the relative success of the different components of REPLICA: • Was Guidance and Counseling/Psychosocial Support rated the highest because it was simply more developed than the other components? In particular, the Guidance and Counseling materials, which are the most exhaustive and arguably the most well-developed of the REPLICA materials, had been developed prior to REPLICA and were therefore ready early on, thereby giving them more exposure. Evaluation of REPLICA/UGANDA Conducted by School-to-School International for Creative Associates International, Inc. Final version April 2007 p. 23 of 85 • Guidance and Counseling and Peace Education are related themes and thus mutually reinforcing, perhaps more than any other two themes. Might the high ranking of each be a result of being accompanied by another, thereby making it in effect “a double theme”? • In what ways did the different components work together? How might REPLICA evaluate the different ways in which the different components interact, and how they might be modified to interact even better? • Why was CIP the least successful? The following section attempts to provide some answers for the last question. Reasons for difficulties with CIP The Community Integration Component (CIP) was rated the least successful of the 6 components by interviewees in this evaluation. Yet when asked about REPLICA’s greatest achievements, increased community involvement was cited as one of them. Why the contradiction? The answer might lie in the definition of “integration.” As noted in the description of REPLICA above, CIP consisted of three types of activity: community sensitization (schools sending messages to communities), community outreach (schools working in communities), and community involvement (communities working in schools). In this evaluation, respondents associated with primary schools (teachers, Head Teachers, parents) spoke of community integration in terms of improved involvement in school activities and parent-teacher relations. On the other hand, respondents associated with PTCs (Principals, PTOs, tutors and students) spoke of community integration in terms of outreach – e.g., cleaning paths, building wells, assisting in burials, etc. Such findings would suggest a difference in community integration emphasis between schools (involvement) and PTCs (outreach). Some project reports seem to corroborate this finding. For example, the FAWE final evaluation found that at the school level, community outreach activities had indeed been practiced the least – in 57% of the cases reviewed, as compared with community sensitization (80%) and community involvement (90%). Yet it is unclear what 90% community involvement means. In interviews with parents, one complaint was that they had not been invited for training as frequently as they would like.8 So while they might feel more welcome consulting teachers and participating in the life of the school, a more structured role for them, including training opportunities at the school, still seems to be missing. One report (Cox 2006) also raised issues around insufficient sensitization, weak CCT outreach and inadequate understanding of the community’s role in the school by Head Teachers. All of these findings suggest that a closer examination of the types of community involvement actually practiced at both the PTC and primary school levels, would shed greater light on the nature of their successes and 8 Though teachers also noted that one inhibiting factor was the “sitting allowances” that some community members expected (one NGO had paid 20,000 shillings a day to community participants). Evaluation of REPLICA/UGANDA Conducted by School-to-School International for Creative Associates International, Inc. Final version April 2007 p. 24 of 85 potential causes of their difficulties. Such an examination would help improve the CIP component of REPLICA in its expansion stage. RECOMMENDATIONS OF RESPONDENTS When asked how REPLICA can be improved, especially in its expansion stage, respondents overwhelmingly voiced their support for the project, that it continue and that it expand to as many schools as possible. In addition to this request, respondents made the following recommendations, presented in order of frequency: • Provide materials for all beneficiaries – e.g., one copy for each tutor, student, Head Teacher and teacher. • Provide more training and support supervision. • Distribute more girls’ kits and include something for the boys. • Improve conditions of teaching and learning: teachers’ houses, water at schools, classrooms, classroom furniture. • Improve transport and office support for CCTs, DEOs and DISs. • Provide training of all pre-service tutors. • Translate materials into local languages, especially material to be used in lower primary. • Provide recognition, promotion and incentives for teacher and Head Teachers. DISCUSSION This section examines several issues raised by the study that merit further reflection in order to increase the chances of success in REPLICA’s expansion stage. Impact Though few quantitative measures of the impact of REPLICA in its pilot phase are available, there are nevertheless abundant indications of its success. The greatest indication is the palpable level of enthusiasm on the part of stakeholders and beneficiaries for the REPLICA project which, to take them at their word, has transformed the school environment. Neighboring schools wish to be included in REPLICA, the Northern Region Education Forum (NREF) has recommended that REPLICA be scaled up, and post-primary institutions have requested that it be extended to their level as well. REPLICA content is being incorporated into primary school curricula by teachers and into the Primary Teacher Education curriculum at Kyambogo University. Again, the claim that REPLICA has been responsible for the changes claimed by respondents in this evaluation must be viewed with caution. With other projects, including BEPS, PIASCY and others intervening in areas such as guidance and counseling, peace education, talking compounds, and girl child education, it is impossible to attributed changes in these areas solely to REPLICA. However, reports gathered in this evaluation provide convincing evidence that REPLICA has at the very least reinforced gains in these areas, and has certainly pioneered Evaluation of REPLICA/UGANDA Conducted by School-to-School International for Creative Associates International, Inc. Final version April 2007 p. 25 of 85 various elements in the North as well (e.g., incinerators, Guidance & Counseling rooms, etc.). In a word, the objective of restoring quality education services in northern Uganda by re-establishing schools as safe, welcoming and functional environments has clearly been met. Indeed, the response has been overwhelmingly positive, undoubtedly exceeding the original expectations of project designers. Sustainability issues In spite of the immediate gains associated with some REPLICA inputs, careful consideration should be given to the question of sustainability. For example, while the girls’ kits were truly a sensation in the schools, driving up girls’ enrollment and generating significant enthusiasm at school and at home, reports also suggest significant jealousy on the part of boys. According to one report, some boys wished they were girls so they would get a bag. Reports also suggest increased dependence on donor aid – e.g., girls asking for new bags because theirs were spoiled, and asking for replacement pads because they could not afford to buy them. In one school, Save the Children had provided additional pads. Other schools were taking initiatives such as the use of donations and capitation grants to buy replacement pads, and the initiation of sewing clubs to teach girls to make pads using local materials. But the benefits of charity will forever be the province of the fortunate few, and school-based initiatives are by definition exemplary and therefore not scalable. Schools included in REPLICA II will need a different formula for encouraging girl child education if it is to be sustainable. Similarly, the sense of empowerment experienced by those who participated in the creation of the three college magazines under the PALS component is one of the undeniable successes of the REPLICA pilot. However, in order to continue production of magazines of this quality (glossy, 4-color), substantial editorial effort and expertise, not to mention resources for printing and distribution, must be constantly found. In order to make the magazines sustainable, PTCs were asked to sell their magazines – in one case, for 3,000 shillings, in another, 5,000. At the time of this evaluation, only 10 of 400 issues of Solot, one of the magazines, had been sold, and all PTCs reported being concerned about sales potential the production of future issues. At the presentation of preliminary findings for this evaluation, the recommendation was made to study the objective of these magazines, including purpose, production values, intended audience and mechanisms for sustaining production, in order to determine whether the effort should be maintained in its current form. The recommendation should be heeded. Synergy The six components of REPLICA created a synergy that provided a rich learning environment. This evaluation found that the synergistic aspect of REPLICA is indeed on of the most poignant aspects of its success: not only has REPLICA brought a number of appropriate and effective interventions to An example of synergy Most interesting for our cluster was having the children do performances for the parents and the community so the HIV prevention and care messages of PIASCY reached out of the school just as the CIP program trained us. We can now see how the REPLICA parts can work together and help reinforce the other part as one complete thing. We hold REPLICA close to our hearts and all we now can say is that we have to continue to work together to bring about change and make a difference in our children’s lives. Head Teacher Evaluation of REPLICA/UGANDA Conducted by School-to-School International for Creative Associates International, Inc. Final version April 2007 p. 26 of 85 the North, but the nature of the interventions is such that in many cases, they are mutually reinforcing. For example, curriculum components such as Peace Education and PALS reinforce each other’s messages. Guidance and Counseling provides an avenue to peace building that conventional peace education programs do not provide. Guidance and Counseling rooms are used both for dealing with trauma and for helping girls feel relaxed (most are equipped with beds, spare pads and changing rooms). The CIP and leadership components worked together to build a more ordered, and at the same trusting and collaborative, environment in the school. As a result of REPLICA Leadership and Management training, women assumed more responsibilities in the PTCs. Additionally, REPLICA components interacted positively with programs from outside REPLICA, such as PIASCY for health messages in the community, and the BEPS teacher education activities with a focus on cooperative learning. Importantly, certain synergies were not realized and, had they been, REPLICA might have witnessed greater success. For example, one REPLICA project staff noted that because much community work had not been done, girls’ education suffered, since many girls’ education issues are community-based. Thus the synergistic aspects of REPLICA, both realized and potential, represent a frontier that should be explored throughout the expansion and for years to come. M&E culture If synergy is one of the great frontiers of REPLICA, it is also a reason to be concerned about the difficulties inherent in evaluating such a complex design. Though this evaluator would not recommend trying to separate the six components, it is nevertheless of crucial importance that each area be tracked in order to provide sufficient background and comparative data that can then be used to inform programming decisions. The FAWE Final Report provides one example. The report notes that at the end of the project, only 14 of 30 schools had changing room for girls, 22 had washing facilities, 25 had separate latrines and 63% had no school maps. While these data already provide a “snapshot” of REPLICA schools in the end-of-project stage, it is difficult to know what they actually mean because they provide no comparative data. For example, how many schools had these inputs at the beginning of the project? Which schools did not have these inputs? What might be some factors associated with the absence of these inputs? The Guidance and Counseling Final Report provides a similar example, in which it notes that “40% of schools environments were not talking.” No description is provided as to whether this means no signs, too few signs, or signs that do not meet REPLICA requirements. Nor is there any indication which schools were not talking, or possible reasons associated with the lapse. As with the FAWE data, it is therefore impossible to use this information to make cross-district or cross-school comparisons, to make a judgment about the nature of the problem, or to determine how the problem should be addressed. The lack of such information is understandable at the pilot stage of any project where everything is new, inputs are varied, and actors have different ideas about what constitutes success. However, when REPLICA is fully operational in 1,700 Evaluation of REPLICA/UGANDA Conducted by School-to-School International for Creative Associates International, Inc. Final version April 2007 p. 27 of 85 schools, economies of scale will dictate that a simple system of generating comparative information will be of critical importance – an “M&E culture.” The groundwork for such a culture has already been laid in the conceptualization of key indicators and preliminary development of monitoring tools. The development of the system remains a task for REPLICA II. RECOMMENDATIONS Based on the findings and discussion points in this report, the following recommendations are offered in an effort to ensure that lessons learned from REPLICA I are considered in the design of the expansion of REPLICA. 1. Continue implementation of REPLICA through the TDMS structure. This includes the use of the cascade model at the regional and district levels, the use of the PTCs and CCTs as vectors for training and dissemination of materials and messages, the use of PTCs and CCs for the transmission of funds, the integration of pre-service and in-service training and materials distribution, and collaboration with the district offices in the implementation of REPLICA. 2. Explore alternatives to School-Based Training. Given the nearly unanimous view that SBT is “the weak link” in the training chain, explore the delivery of training to teachers through other formats. Some simple, low￾cost options include: • Sub-cluster training – e.g., groups of 3-5 schools with 15-25 teaches who can meet at one of the schools for regular training. • More frequent CC-based training. • Modularized training supported by more intensive supervision from CCTs and Head Teachers. 3. Expand DEO involvement. In light of concerns raised by district officers in this evaluation, examine avenues through which more meaningful involvement of DEO personnel is achieved, especially in the areas of training, support supervision, and reporting. Possible roles might include co-training roles, implementing key aspects of the monitoring plan, and contributing to the development of training materials. 4. Streamline training materials. In order to reach teachers in 1,700 schools, it will be important to develop a simple, standardized, easy-to-use package of training materials to be used throughout the REPLICA training system and that can be passed on, in time, to the Ministry for its continued use. These materials need not be developed from scratch; rather, they should draw on the excellent materials already available through REPLICA and other sources (e.g., BEPS). For each component, there should be a package that includes a training manual/reference book, teacher’s guide, tools for support supervision, and if relevant, learner’s materials. Materials should be written to be used in both pre-service and in-service contexts. 5. Establish an “M&E culture.” This would entail developing instruments that include simple, standardized, measurable indicators with clear choices, Evaluation of REPLICA/UGANDA Conducted by School-to-School International for Creative Associates International, Inc. Final version April 2007 p. 28 of 85 rating scales and/or rubrics; uniform and adequate training of monitoring staff; a system of gathering data on a regular (e.g., quarterly) basis; a system of entering, analyzing and sharing data electronically; regular stakeholder meetings where analyzed data are presented for discussion; and decision￾making based on discussions informed by these data. 6. Examine the sustainability of provision of consumables, especially girls’ kits, girls’ pads, and college-published magazines. 7. Examine the different ways in which CIP is used in schools and PTCs, and explore options for its expansion and improvement. These might include: a. Special training sessions for parents (i.e., not just relying on the cascade to reach them effectively). b. Incorporation of CIP into the mandates of the other 5 components, especially girls’ education. c. Consideration of special incentives for parent participation in schools (e.g., recognition awards, parent days). 8. Create a standard system for the formative evaluation of materials that ensures materials are evaluated in “real time,” as they are intended to be used in real or simulated situations. Such a system should include: a. A formative evaluation team dedicated to this task. b. Clearly defined procedures and tools for the formative evaluation of each material being produced. c. The keeping of records that document the results of evaluation activities, changes recommended and changes made. 9. Establish a mechanism for reviewing subcontractor capacity vis-à-vis the tasks which they are being engaged to execute. 10.Establish criteria for the following design issues: a. Selection of future schools, including schools least served by other donors/partners. b. Provision of materials in local languages (e.g., for P1-3). c. Provision of materials – i.e., how many of which materials to be distributed to each audience. d. Mechanisms to limit movement of staff (teachers, Head Teachers) out of model schools during the life of REPLICA II. e. Mechanisms for planning for and adjusting the program to the decongestion of IDP camps. Evaluation of REPLICA/UGANDA Conducted by School-to-School International for Creative Associates International, Inc. Final version April 2007 p. 29 of 85 ANNEX A: DATA COLLECTION TOOLS Evaluation of REPLICA/UGANDA Conducted by School-to-School International for Creative Associates International, Inc. Final version April 2007 p. 30 of 85 REPLICA EVALUATION, TOOL 1 REGION/DISTRICT: PRINCIPAL INSPECTOR OF SCHOOLS Instructions to researcher: To be administered to Principal Inspector alone. Duration: 1 hour. Materials: List of materials, examples if possible. Date: ……………………. Region: ………………………………… Name of Principal Inspector: …………………………………… Place of interview: ………………………………………………………… Special conditions (optional): …………………………………………………………. ………………………………………………………………………………………………… Explain to Inspector: Evaluation, content: materials, training, support supervision, what was learned, and design issues. Feel free to express yourself, do not have to agree with each other, praise and criticism are good, will help in going to scale, anonymity, 1 hour. Have you participated in the REPLICA project? [ ] yes [ ] no If yes, how? What kinds of activities have you participated in? …………………………………………………………………………………………………. Training Reminder: Training for REPLICA was in a cascade: 1. Regional level: National level trainers brought people together at the regional level: Tutors, CCTs, Disctrict Education Officers, Deputy Principal Outreach. 2. Coordinating Center level: Tutors, CCTs, DEOs and DPOs train 3 teachers from each school. NB All schools in a catchment area came to these trainings, but REPLICA only provided additional support (supervision etc) to the 3 model schools per district. 3. School level: 3 teachers in each school train their peers in “school-based training”. Training: School-based To your knowledge, did the school-based training take place (REPLICA model schools)? [ ] yes [ ] sometimes [ ] no If yes, were trainings organized: [ ] during the week [ ] on the weekends/ holidays? Evaluation of REPLICA/UGANDA Conducted by School-to-School International for Creative Associates International, Inc. Final version April 2007 p. 31 of 85 [ ] 2 or more times per week [ ] once a week [ ] once every 2 weeks [ ] once a month What percentage of teachers attended on average? [ ] 90-100% [ ] 50-90% [ ] fewer than half What problems did teachers or trainers have with school-based training? …………………………………………………………………………………………………. …………………………………………………………………………………………………. …………………………………………………………………………………………………. In your view, have any of REPLICA’s training events had any impact in the classroom? [ ] yes [ ] no If yes, what were they? …………………………………………………………………………………………………. …………………………………………………………………………………………………. …………………………………………………………………………………………………. …………………………………………………………………………………………………. Training: CC level (5 days) Did you attend any of these trainings? [ ] yes [ ] no If yes, how long was the training? [ ] less than one day [ ] one day [ ] 3 days [ ] 5 days other: ……. If yes, would you judge these workshops as: [ ] completely successful [ ] partially successful [ ] many problems If partially successful or many problems, what problems did you see? …………………………………………………………………………………………………. …………………………………………………………………………………………………. …………………………………………………………………………………………………. Training: Regional level (21 days) Did you attend this training? [ ] yes [ ] no If yes, what percentage of the participants actually attended? Evaluation of REPLICA/UGANDA Conducted by School-to-School International for Creative Associates International, Inc. Final version April 2007 p. 32 of 85 [ ] 90-100% [ ] 50-89% [ ] less than 50% If yes, would you judge this workshop as: [ ] completely successful [ ] partially successful [ ] many problems If partially successful or many problems, what problems did you see? …………………………………………………………………………………………………. …………………………………………………………………………………………………. …………………………………………………………………………………………………. Materials Please look at this list of materials. Are you familiar with them? [ ] yes [ ] no (Show materials. If interviewees change their mind, change previous answer.) Which of them were, in your view, the most useful? ………………………………………………………………………………………………… ………………………………………………………………………………………………… In what way were they useful? ………………………………………………………………………………………………… ………………………………………………………………………………………………… ………………………………………………………………………………………………… ………………………………………………………………………………………………… How would you rate these materials in terms of their appropriateness for conditions here in the north? [ ] very appropriate [ ] appropriate in some cases [ ] not appropriate If appropriate in some cases or not appropriate, how could these materials be improved to make them more appropriate? ………………………………………………………………………………………………… ………………………………………………………………………………………………… What else could be done to improve these materials? ………………………………………………………………………………………………… Evaluation of REPLICA/UGANDA Conducted by School-to-School International for Creative Associates International, Inc. Final version April 2007 p. 33 of 85 ………………………………………………………………………………………………… Were materials distributed on time? [ ] yes [ ] sometimes [ ] no If yes, what were problems with distribution? ………………………………………………………………………………………………… ………………………………………………………………………………………………… Support monitoring Have you made any visits to REPLICA schools? [ ] yes [ ] no If so, with what frequency? [ ] Once a week or more [ ] 2-3 times per month [ ] more than 3 times per month When you visited the schools, whom did you speak with at the school: [ ] teachers [ ] head teacher [ ] deputy head teacher [ ] head man teacher [ ] head woman teacher [ ] pupils [ ] other What kinds of problems were mentioned in regards to REPLICA activities? (For each problem, note who mentioned it.) ……………………………………………………………………………………………… ……………………………………………………………………………………………… ……………………………………………………………………………………………… ……………………………………………………………………………………………… What was learned The REPLICA project focused on 6 components: • Psychosocial care/guidance and counseling • Community integration • Girl Child education • Performing arts and learning • Leadership and governance • Peace education In which of these areas do you believe teachers learned the most (more than one answer possible? Evaluation of REPLICA/UGANDA Conducted by School-to-School International for Creative Associates International, Inc. Final version April 2007 p. 34 of 85 ………………………………………………………………………………………… Can you give an example of something you have seen that shows what teachers learned through the REPLICA project? …………………………………………………………………………………………………. …………………………………………………………………………………………………. …………………………………………………………………………………………………. …………………………………………………………………………………………………. What new attitudes have teachers acquired since participating in the REPLICA project? …………………………………………………………………………………………………. …………………………………………………………………………………………………. …………………………………………………………………………………………………. …………………………………………………………………………………………………. Project design questions REPLICA used 5 kinds of events to prepare for the project and build support: (1) Info gathering sessions before design, (2) consensus-building meetings, (3) Academic seminars with stakeholders, (4) Public engagement meetings and (5) Feedback meetings. Are you familiar with these events? [ ] yes [ ] no (If no, explain the events he hasn’t heard of – maybe he has, but doesn’t know the name.) Have you participated in any of these events? [ ] yes [ ] no If yes, please indicate which events you participated in: [ ] Info gathering sessions before design [ ] consensus-building meetings [ ] Academic seminars with stakeholders [ ] Public engagement meetings [ ] Feedback meetings How would you rate the quality of the event(s) you attended in terms of: Usefulness GOOD FAIR POOR Info gathering sessions before design consensus-building meetings Academic seminars with stakeholders Public engagement meetings Feedback meetings Evaluation of REPLICA/UGANDA Conducted by School-to-School International for Creative Associates International, Inc. Final version April 2007 p. 35 of 85 If fair or poor, please explain: …………………………………………………………………………………………………. …………………………………………………………………………………………………. Involvement & participation of beneficiaries GOOD FAIR POOR Info gathering sessions before design consensus-building meetings Academic seminars with stakeholders Public engagement meetings Feedback meetings If fair or poor, please explain: …………………………………………………………………………………………………. …………………………………………………………………………………………………. A central design feature of the REPLICA project was to focus energies on the TDMS structure for training, for distributing materials and monies, etc. The idea was that once the colleges received training, monies and materials, college personnel would then administer the project to the CCs and the schools. In your view, has this design worked? What problems have you noticed? …………………………………………………………………………………………………. …………………………………………………………………………………………………. …………………………………………………………………………………………………. …………………………………………………………………………………………………. For the REPLICA project, the colleges were supposed to co-opt district personnel and ESA staff to assist with support supervision activities in REPLICA schools. Do you think this system worked well, or is there another way that support supervision should be organized? [ ] worked well [ ] should be organized differently If organized differently, how? …………………………………………………………………………………………………. …………………………………………………………………………………………………. …………………………………………………………………………………………………. Do you think the REPLICA project empowered teachers, head teachers, pupils or parents? [ ] yes [ ] no Evaluation of REPLICA/UGANDA Conducted by School-to-School International for Creative Associates International, Inc. Final version April 2007 p. 36 of 85 If yes, can you give an example of something you have seen that showed they were empowered? …………………………………………………………………………………………………. …………………………………………………………………………………………………. …………………………………………………………………………………………………. What has been REPLICA’s biggest impact in terms of girl child education? …………………………………………………………………………………………………. …………………………………………………………………………………………………. …………………………………………………………………………………………………. The REPLICA project will be scaling up in the coming months to cover 1,700 schools. What difficulties can you foresee (e.g., administration, support supervision, training, materials)? …………………………………………………………………………………………………. …………………………………………………………………………………………………. …………………………………………………………………………………………………. (If time permits): How can REPLICA prepare for this stage? …………………………………………………………………………………………………. …………………………………………………………………………………………………. …………………………………………………………………………………………………. In summary, what in your view were REPLICA’s biggest successes and biggest problems? Success: …………………………………………………………………………………………………. …………………………………………………………………………………………………. …………………………………………………………………………………………………. Problems: …………………………………………………………………………………………………. Evaluation of REPLICA/UGANDA Conducted by School-to-School International for Creative Associates International, Inc. Final version April 2007 p. 37 of 85 …………………………………………………………………………………………………. …………………………………………………………………………………………………. Do you have any other comments you would like to add? …………………………………………………………………………………………………. …………………………………………………………………………………………………. …………………………………………………………………………………………………. Thank you for your time! Notes of researcher: …………………………………………………………………………………………………. …………………………………………………………………………………………………. …………………………………………………………………………………………………. …………………………………………………………………………………………………. …………………………………………………………………………………………………. …………………………………………………………………………………………………. Evaluation of REPLICA/UGANDA Conducted by School-to-School International for Creative Associates International, Inc. Final version April 2007 p. 38 of 85 REPLICA EVALUATION, TOOL 2 DEO, DIS Instructions to researcher: To be administered to DEO and DIS. Others can attend if desired. Duration: 1 hour. Materials: List of materials, examples if possible. Date: ……………………. District: ………………………………… Name of DEO: …………………………………… Name of DIS: …………………………………… Other interviewees (if relevant: Name:………………………………………. Function:…………………………………. Name: ………………………………………. Function: ………………………………… Place of interview: ………………………………………………………… Special conditions (optional): …………………………………………………………. ………………………………………………………………………………………………… Explain to interviewees: Evaluation, content: materials, training, support supervision, what was learned, and design issues. Feel free to express yourself, do not have to agree with each other, praise and criticism are good, will help in going to scale, anonymity, 1 hour. 1. Have you participated in the REPLICA project? DEO DIS [ ] yes [ ] no [ ] yes [ ] no 2. If yes, how? What kinds of activities have you participated in? DEO DIS Evaluation of REPLICA/UGANDA Conducted by School-to-School International for Creative Associates International, Inc. Final version April 2007 p. 39 of 85 Training Reminder: Training for REPLICA was in a cascade: 4. Regional level: National level trainers brought people together at the regional level: Tutors, CCTs, Disctrict Education Officers, Deputy Principal Outreach. 5. Coordinating Center level: Tutors, CCTs, DEOs and DPOs train 3 teachers from each school. NB All schools in a catchment area came to these trainings, but REPLICA only provided additional support (supervision etc) to the 3 model schools per district. 6. School level: 3 teachers in each school train their peers in “school-based training”. Training: School-based 3. To your knowledge, did the school-based training take place (REPLICA model schools)? [ ] yes [ ] sometimes [ ] no 4. If yes, were trainings organized: [ ] during the week [ ] on the weekends/ holidays? [ ] 2 or more times per week [ ] once a week [ ] once every 2 weeks [ ] once a month 5. What percentage of teachers attended on average? [ ] 90-100% [ ] 50-90% [ ] fewer than half 6. What problems did teachers or trainers have with school-based training? …………………………………………………………………………………………………. …………………………………………………………………………………………………. …………………………………………………………………………………………………. 7. In your view, have any of REPLICA’s training events had any impact in the classroom? [ ] yes [ ] no 8. If yes, what were they? …………………………………………………………………………………………………. …………………………………………………………………………………………………. …………………………………………………………………………………………………. …………………………………………………………………………………………………. Training: CC level (5 days) Evaluation of REPLICA/UGANDA Conducted by School-to-School International for Creative Associates International, Inc. Final version April 2007 p. 40 of 85 9. Did you attend any of these trainings? [ ] yes [ ] no DEO DIS [ ] yes [ ] no [ ] yes [ ] no 10.If yes, how long was the training? DEO [ ] less than one day [ ] one day [ ] 3 days [ ] 5 days other: ……. DIS [ ] less than one day [ ] one day [ ] 3 days [ ] 5 days other: ……. 11.If yes, would you judge these workshops as: DEO [ ] completely successful [ ] partially successful [ ] many problems DIS [ ] completely successful [ ] partially successful [ ] many problems 12.If partially successful or many problems, what problems did you see? …………………………………………………………………………………………………. …………………………………………………………………………………………………. …………………………………………………………………………………………………. Training: Regional level (21 days) 13.Did you attend this training? DEO DIS [ ] yes [ ] no [ ] yes [ ] no 14.If yes, what percentage of the participants actually attended? DEO DIS [ ] 90-100% [ ] 50-89% [ ] less than 50% [ ] 90-100% [ ] 50-89% [ ] less than 50% 15.If yes, would you judge this workshop as: DEO DIS [ ] completely successful [ ] partially successful [ ] many problems [ ] completely successful [ ] partially successful [ ] many problems 16.If partially successful or many problems, what problems did you see? Evaluation of REPLICA/UGANDA Conducted by School-to-School International for Creative Associates International, Inc. Final version April 2007 p. 41 of 85 …………………………………………………………………………………………………. …………………………………………………………………………………………………. …………………………………………………………………………………………………. Materials 17.Please look at this list of materials. Are you familiar with them? DEO DIS [ ] yes [ ] no [ ] yes [ ] no (Show materials. If interviewees change their mind, change previous answer.) 18.Which of them were, in your view, the most useful? ………………………………………………………………………………………………… ………………………………………………………………………………………………… 19.In what way were they useful? ………………………………………………………………………………………………… ………………………………………………………………………………………………… ………………………………………………………………………………………………… ………………………………………………………………………………………………… 20.How would you rate these materials in terms of their appropriateness for conditions here in the north? DEO DIS [ ] very appropriate [ ] appropriate in some cases [ ] not appropriate [ ] very appropriate [ ] appropriate in some cases [ ] not appropriate 21.If appropriate in some cases or not appropriate, how could these materials be improved to make them more appropriate? ………………………………………………………………………………………………… ………………………………………………………………………………………………… 22.What else could be done to improve these materials? ………………………………………………………………………………………………… Evaluation of REPLICA/UGANDA Conducted by School-to-School International for Creative Associates International, Inc. Final version April 2007 p. 42 of 85 ………………………………………………………………………………………………… 23.Were materials distributed on time? DEO DIS [ ] yes [ ] sometimes [ ] no [ ] yes [ ] sometimes [ ] no 24.If yes, what were problems with distribution? ………………………………………………………………………………………………… ………………………………………………………………………………………………… Support monitoring 25.Have you made any visits to REPLICA schools? DEO DIS [ ] yes [ ] no [ ] yes [ ] no 26.If so, with what frequency? DEO DIS [ ] Once a week or more [ ] 2-3 times per month [ ] more than 3 times per month [ ] Once a week or more [ ] 2-3 times per month [ ] more than 3 times per month 27.When you visited the schools, whom did you speak with at the school: DEO DIS [ ] teachers [ ] head teacher [ ] deputy head teacher [ ] head man teacher [ ] head woman teacher [ ] pupils [ ] other [ ] teachers [ ] head teacher [ ] deputy head teacher [ ] head man teacher [ ] head woman teacher [ ] pupils [ ] other 28.What kinds of problems were mentioned in regards to REPLICA activities? (For each problem, note who mentioned it.) ……………………………………………………………………………………………… ……………………………………………………………………………………………… ……………………………………………………………………………………………… ……………………………………………………………………………………………… What was learned Evaluation of REPLICA/UGANDA Conducted by School-to-School International for Creative Associates International, Inc. Final version April 2007 p. 43 of 85 The REPLICA project focused on 6 components: • Psychosocial care/guidance and counseling • Community integration • Girl Child education • Performing arts and learning • Leadership and governance • Peace education 29.In which of these areas do you believe teachers learned the most (more than one answer possible? ………………………………………………………………………………………… 30.Can you give an example of something you have seen that shows what teachers learned through the REPLICA project? …………………………………………………………………………………………………. …………………………………………………………………………………………………. …………………………………………………………………………………………………. …………………………………………………………………………………………………. 31.What new attitudes have teachers acquired since participating in the REPLICA project? …………………………………………………………………………………………………. …………………………………………………………………………………………………. …………………………………………………………………………………………………. …………………………………………………………………………………………………. Project design questions 32.REPLICA used 5 kinds of events to prepare for the project and build support: (1) Info gathering sessions before design, (2) consensus-building meetings, (3) Academic seminars with stakeholders, (4) Public engagement meetings and (5) Feedback meetings. Are you familiar with these events? DEO DIS [ ] yes [ ] no [ ] yes [ ] no (If no, explain the events they haven’t heard of – maybe they have, but don’t know the name.) 33.Have you participated in any of these events? Evaluation of REPLICA/UGANDA Conducted by School-to-School International for Creative Associates International, Inc. Final version April 2007 p. 44 of 85 DEO DIS [ ] yes [ ] no [ ] yes [ ] no 34.If yes, please indicate which events you participated in: DEO DIS [ ] Info gathering sessions before design [ ] consensus-building meetings [ ] Academic seminars with stakeholders [ ] Public engagement meetings [ ] Feedback meetings [ ] Info gathering sessions before design [ ] consensus-building meetings [ ] Academic seminars with stakeholders [ ] Public engagement meetings [ ] Feedback meetings 35.How would you rate the quality of the event(s) you attended in terms of: 36. DEO Usefulness GOOD FAIR POOR Info gathering sessions before design consensus-building meetings Academic seminars with stakeholders Public engagement meetings Feedback meetings If fair or poor, please explain: …………………………………………………………………………………………………. …………………………………………………………………………………………………. 37. DIS Usefulness GOOD FAIR POOR Info gathering sessions before design consensus-building meetings Academic seminars with stakeholders Public engagement meetings Feedback meetings If fair or poor, please explain: …………………………………………………………………………………………………. …………………………………………………………………………………………………. 38. DEO Involvement & participation of beneficiaries Evaluation of REPLICA/UGANDA Conducted by School-to-School International for Creative Associates International, Inc. Final version April 2007 p. 45 of 85 GOOD FAIR POOR Info gathering sessions before design consensus-building meetings Academic seminars with stakeholders Public engagement meetings Feedback meetings If fair or poor, please explain: …………………………………………………………………………………………………. …………………………………………………………………………………………………. 39. DIS Involvement & participation of beneficiaries GOOD FAIR POOR Info gathering sessions before design consensus-building meetings Academic seminars with stakeholders Public engagement meetings Feedback meetings 40.If fair or poor, please explain: …………………………………………………………………………………………………. …………………………………………………………………………………………………. 41.A central design feature of the REPLICA project was to focus energies on the TDMS structure for training, for distributing materials and monies, etc. The idea was that once people in the colleges received training, monies and materials, they would then administer the project to the CCs and the schools. In your view, has this design feature worked? What problems have you noticed? …………………………………………………………………………………………………. …………………………………………………………………………………………………. …………………………………………………………………………………………………. …………………………………………………………………………………………………. 42.The REPLICA project used a cascade model to deliver training to the school level: Ministry officials trained College personnel, who trained CCTs, who trained selected teachers from REPLICA schools who were, in turn, expected to train the other teachers in their schools. Are you aware of any problems experienced by people in the cascade? DEO DIS [ ] yes [ ] no [ ] yes [ ] no Evaluation of REPLICA/UGANDA Conducted by School-to-School International for Creative Associates International, Inc. Final version April 2007 p. 46 of 85 43.If yes, what were the problems? …………………………………………………………………………………………………. …………………………………………………………………………………………………. …………………………………………………………………………………………………. 44.For the REPLICA project, the colleges were supposed to co-opt district personnel and ESA staff to assist with support supervision activities in REPLICA schools. Do you think this system worked well, or is there another way that support supervision should be organized? DEO DIS [ ] worked well [ ] should be organized differently [ ] worked well [ ] should be organized differently 45.If organized differently, how? …………………………………………………………………………………………………. …………………………………………………………………………………………………. …………………………………………………………………………………………………. 46.Do you think the REPLICA project empowered teachers, head teachers, pupils or parents? DEO DIS [ ] yes [ ] no [ ] yes [ ] no 47.If yes, can you give an example of something you have seen that proved to you that people were empowered? …………………………………………………………………………………………………. …………………………………………………………………………………………………. …………………………………………………………………………………………………. 48.What has been REPLICA’s biggest impact in terms of girl child education? …………………………………………………………………………………………………. …………………………………………………………………………………………………. …………………………………………………………………………………………………. 49.The REPLICA project will be scaling up in the coming months to cover 1,700 schools. What difficulties can you foresee? …………………………………………………………………………………………………. Evaluation of REPLICA/UGANDA Conducted by School-to-School International for Creative Associates International, Inc. Final version April 2007 p. 47 of 85 …………………………………………………………………………………………………. …………………………………………………………………………………………………. 50.How can REPLICA prepare for this stage? …………………………………………………………………………………………………. …………………………………………………………………………………………………. …………………………………………………………………………………………………. 51.In summary, what in your view were REPLICA’s biggest successes and biggest problems? 52.Success: …………………………………………………………………………………………………. …………………………………………………………………………………………………. …………………………………………………………………………………………………. 53.Problems: …………………………………………………………………………………………………. …………………………………………………………………………………………………. …………………………………………………………………………………………………. 54.Do you have any other comments you would like to add? …………………………………………………………………………………………………. …………………………………………………………………………………………………. …………………………………………………………………………………………………. Thank you for your time! Notes of researcher: …………………………………………………………………………………………………. …………………………………………………………………………………………………. Evaluation of REPLICA/UGANDA Conducted by School-to-School International for Creative Associates International, Inc. Final version April 2007 p. 48 of 85 REPLICA EVALUATION, TOOL 3 FOCUS GROUP AT THE CORE PTCs WITH PRINCIPAL, DPO, TUTORS, STUDENTS Questions: 1. Training in REPLICA: Strengths and problems to be solved. 2. Materials in REPLICA: Strengths and problems to be solved. 3. The 6 components of REPLICA: Which was the most successful and why? 4. Design of REPLICA: What worked well? What should be modified? 5. Biggest achievement of REPLICA. 6. Recommendations for the expansion. Evaluation of REPLICA/UGANDA Conducted by School-to-School International for Creative Associates International, Inc. Final version April 2007 p. 49 of 85 REPLICA EVALUATION, TOOL 4 School: CCT, head teacher, deputy head teacher, senior woman teacher, senior man teacher, other teachers, pupils. Instructions to researcher: To be administered to CCT, head teacher, deputy head teacher, senior woman teacher, senior man teacher, other teachers, pupils. Duration: 3+ hours. Materials: List of materials, examples if possible. Part 1: Head teacher, deputy head teacher (45 min.) Date: ……………………. District: ………………………………… Name of School: …………………………………… Name of head teacher: ……………………………………….. Name of deputy head teacher: ……………………………………. Special conditions (optional): …………………………………………………………. ………………………………………………………………………………………………… Explain to HT and DHT: Evaluation, content: materials, training, support supervision, what was learned, and design issues. Feel free to express yourself, do not have to agree with each other, some questions separate, praise and criticism are good, will help in going to scale, anonymity, 1 hour. Have you participated in the REPLICA project? HT DHT [ ] yes [ ] no [ ] yes [ ] no If yes, how? What kinds of activities have you participated in? …………………………………………………………………………………………………. Training Training: School-based 1. Did REPLICA school-based training take place in your school? [ ] yes [ ] sometimes [ ] no a. If yes, were trainings organized: [ ] during the week [ ] on the weekends/ holidays? [ ] 2 or more times per week [ ] once a week [ ] once every 2 weeks [ ] once a month 2. What percentage of teachers attended on average? Evaluation of REPLICA/UGANDA Conducted by School-to-School International for Creative Associates International, Inc. Final version April 2007 p. 50 of 85 [ ] 90-100% [ ] 50-90% [ ] fewer than half 3. What problems did teachers or trainers have with school-based training? …………………………………………………………………………………………………. …………………………………………………………………………………………………. …………………………………………………………………………………………………. 4. In your view, have any of REPLICA’s training events had any impact in the classroom? HT DHT [ ] yes [ ] no [ ] yes [ ] no 5. If yes, what were they? …………………………………………………………………………………………………. …………………………………………………………………………………………………. …………………………………………………………………………………………………. …………………………………………………………………………………………………. Training: CC level (5 days) 6. Did you attend any of these trainings? [ ] yes [ ] no 7. If yes, how long was the training? [ ] less than one day [ ] one day [ ] 3 days [ ] 5 days other: ……. 8. If yes, would you judge these workshops as: [ ] completely successful [ ] partially successful [ ] many problems 9. If partially successful or many problems, what problems did you see? …………………………………………………………………………………………………. …………………………………………………………………………………………………. …………………………………………………………………………………………………. Materials 10.Please look at this list of materials. Are you familiar with them? HT DHT [ ] yes [ ] no [ ] yes [ ] no Evaluation of REPLICA/UGANDA Conducted by School-to-School International for Creative Associates International, Inc. Final version April 2007 p. 51 of 85 (Show materials. If interviewees change their mind, change previous answer.) 11.Which of them were, in your view, the most useful? ………………………………………………………………………………………………… ………………………………………………………………………………………………… 12.In what way were they useful? ………………………………………………………………………………………………… ………………………………………………………………………………………………… ………………………………………………………………………………………………… ………………………………………………………………………………………………… 13.How would you rate these materials in terms of their appropriateness for conditions here in the north? [ ] very appropriate [ ] appropriate in some cases [ ] not appropriate 14.If appropriate in some cases or not appropriate, how could these materials be improved to make them more appropriate? ………………………………………………………………………………………………… ………………………………………………………………………………………………… 15.What else could be done to improve these materials? ………………………………………………………………………………………………… ………………………………………………………………………………………………… 16.Were materials distributed on time? [ ] yes [ ] sometimes [ ] no 17.If yes, what were problems with distribution? ………………………………………………………………………………………………… ………………………………………………………………………………………………… Support supervision 18.Have you participated in peer support supervision? HT DHT [ ] yes [ ] no [ ] yes [ ] no Evaluation of REPLICA/UGANDA Conducted by School-to-School International for Creative Associates International, Inc. Final version April 2007 p. 52 of 85 19.If so, with what frequency? HT DHT [ ] once [ ] twice [ ] more than twice [ ] once [ ] twice [ ] more than twice 20.What did you do? 21.What did you learn? 22.What have you been able to apply in your school base on what you learned through support supervision? What was learned 23.The REPLICA project focused on 6 components: • Psychosocial care/guidance and counseling • Community integration • Girl Child education • Performing arts and learning • Leadership and governance • Peace education 24.In which of these areas have you learned the most (more than one answer possible)? ………………………………………………………………………………………………… ………………………………………………………………………………………………… 25.Can you give an example of something you have learned from these activities that you have applied in your school? …………………………………………………………………………………………………. …………………………………………………………………………………………………. …………………………………………………………………………………………………. …………………………………………………………………………………………………. 26.What new attitudes have you acquired since participating in the REPLICA project? …………………………………………………………………………………………………. …………………………………………………………………………………………………. …………………………………………………………………………………………………. Evaluation of REPLICA/UGANDA Conducted by School-to-School International for Creative Associates International, Inc. Final version April 2007 p. 53 of 85 …………………………………………………………………………………………………. 27.Do you get any support from the CCT? 28.How regular? 29.What kinds of activities? 30.How could it be made more helpful? 31.Do you think the REPLICA project empowered teachers, head teachers, pupils or parents? [ ] yes [ ] no 32.If yes, can you give an example of something you have seen that showed they were empowered? …………………………………………………………………………………………………. …………………………………………………………………………………………………. …………………………………………………………………………………………………. 33.What has been REPLICA’s biggest impact in terms of girl child education? …………………………………………………………………………………………………. …………………………………………………………………………………………………. …………………………………………………………………………………………………. 34.In summary, what in your view were REPLICA’s biggest successes and biggest problems? 35.Success: …………………………………………………………………………………………………. …………………………………………………………………………………………………. …………………………………………………………………………………………………. 36.Problems: …………………………………………………………………………………………………. …………………………………………………………………………………………………. …………………………………………………………………………………………………. 37.Do you have any other comments you would like to add? Evaluation of REPLICA/UGANDA Conducted by School-to-School International for Creative Associates International, Inc. Final version April 2007 p. 54 of 85 …………………………………………………………………………………………………. …………………………………………………………………………………………………. …………………………………………………………………………………………………. Thank you for your time! Part 2: Senior man teacher, senior woman teacher, other teachers Instructions to researcher: Teachers to be interviewed collectively. Duration: 3+ hours. Materials: List of materials, examples if possible. Date: ……………………. District: ………………………………… Name of School: …………………………………… Name of senior man teacher: ……………………………………….. Name of senior woman teacher: ……………………………………. Number of teachers total: …… M:……… F:……… Special conditions (optional): …………………………………………………………. ………………………………………………………………………………………………… Explain to teachers: Evaluation, content: materials, training, support supervision, what was learned, and design issues. Feel free to express yourself, do not have to agree with each other, some questions separate, praise and criticism are good, will help in going to scale, anonymity, 1 hour. 1. Have you participated in the REPLICA project? SMT SWT Other teachers [ ] yes [ ] no [ ] yes [ ] no [ ] yes [ ] no 2. If yes, how? What kinds of activities have you participated in? …………………………………………………………………………………………………. Training Training: School-based 3. Did REPLICA school-based training take place in your school? [ ] yes [ ] sometimes [ ] no Evaluation of REPLICA/UGANDA Conducted by School-to-School International for Creative Associates International, Inc. Final version April 2007 p. 55 of 85 4. If yes, were trainings organized: [ ] during the week [ ] on the weekends/ holidays? [ ] 2 or more times per week [ ] once a week [ ] once every 2 weeks [ ] once a month 5. What percentage of teachers attended on average? [ ] 90-100% [ ] 50-90% [ ] fewer than half 6. What problems did teachers or trainers have with school-based training? …………………………………………………………………………………………………. …………………………………………………………………………………………………. …………………………………………………………………………………………………. 7. In your view, have any of REPLICA’s training events had any impact in the classroom? SMT SWT Other teachers [ ] yes [ ] no [ ] yes [ ] no [ ] yes [ ] no 8. If yes, what were they? …………………………………………………………………………………………………. …………………………………………………………………………………………………. …………………………………………………………………………………………………. …………………………………………………………………………………………………. Training: CC level (5 days) 9. Did you attend any of these trainings? [ ] yes [ ] no 10.If yes, how long was the training? [ ] less than one day [ ] one day [ ] 3 days [ ] 5 days other: ……. 11.If yes, would you judge these workshops as: [ ] completely successful [ ] partially successful [ ] many problems 12.If partially successful or many problems, what problems did you see? …………………………………………………………………………………………………. …………………………………………………………………………………………………. Evaluation of REPLICA/UGANDA Conducted by School-to-School International for Creative Associates International, Inc. Final version April 2007 p. 56 of 85 …………………………………………………………………………………………………. Materials 13.Please look at this list of materials. Are you familiar with them? SMT SWT Other teachers [ ] yes [ ] no [ ] yes [ ] no [ ] yes [ ] no (Show materials. If interviewees change their mind, change previous answer.) 14.Which of them were, in your view, the most useful? ………………………………………………………………………………………………… ………………………………………………………………………………………………… 15.In what way were they useful? ………………………………………………………………………………………………… ………………………………………………………………………………………………… ………………………………………………………………………………………………… ………………………………………………………………………………………………… 16.How would you rate these materials in terms of their appropriateness for conditions here in the north? [ ] very appropriate [ ] appropriate in some cases [ ] not appropriate 17.If appropriate in some cases or not appropriate, how could these materials be improved to make them more appropriate? ………………………………………………………………………………………………… ………………………………………………………………………………………………… 18.What else could be done to improve these materials? ………………………………………………………………………………………………… ………………………………………………………………………………………………… 19.Were materials distributed on time? [ ] yes [ ] sometimes [ ] no 20.If yes, what were problems with distribution? ………………………………………………………………………………………………… Evaluation of REPLICA/UGANDA Conducted by School-to-School International for Creative Associates International, Inc. Final version April 2007 p. 57 of 85 ………………………………………………………………………………………………… Support supervision 21.Have you been visited by head teachers from other schools ? SMT SWT Other teachers [ ] yes [ ] no [ ] yes [ ] no [ ] yes [ ] no 22.If so, with what frequency? SMT SWT Other teachers [ ] once [ ] twice [ ] more than twice [ ] once [ ] twice [ ] more than twice [ ] once [ ] twice [ ] more than twice 23.Have you been visited by the CCT? SMT SWT Other teachers [ ] yes [ ] no [ ] yes [ ] no [ ] yes [ ] no 24.If so, with what frequency? SMT SWT Other teachers [ ] once [ ] twice [ ] more than twice [ ] once [ ] twice [ ] more than twice [ ] once [ ] twice [ ] more than twice 25.What did you and the supervisor do during these visits? 26.What did you learn? 27.What have you been able to apply in your school base on what you learned through support supervision? What was learned The REPLICA project focused on 6 components: • Psychosocial care/guidance and counseling • Community integration • Girl Child education • Performing arts and learning • Leadership and governance • Peace education 28.In which of these areas have you received training? • Psychosocial care/guidance and counseling • Community integration Evaluation of REPLICA/UGANDA Conducted by School-to-School International for Creative Associates International, Inc. Final version April 2007 p. 58 of 85 • Girl Child education • Performing arts and learning • Leadership and governance • Peace education 29.In which of these areas have you received materials? • Psychosocial care/guidance and counseling • Community integration • Girl Child education • Performing arts and learning • Leadership and governance • Peace education 30.In which of these areas have you learned the most (more than one answer possible)? • Psychosocial care/guidance and counseling • Community integration • Girl Child education • Performing arts and learning • Leadership and governance • Peace education 31.Can you give an example of something you have learned from these activities that you have applied in your school? …………………………………………………………………………………………………. …………………………………………………………………………………………………. …………………………………………………………………………………………………. …………………………………………………………………………………………………. 32.What new attitudes have you acquired since participating in the REPLICA project? …………………………………………………………………………………………………. …………………………………………………………………………………………………. …………………………………………………………………………………………………. …………………………………………………………………………………………………. 33.Do you think the REPLICA project empowered teachers, head teachers, pupils or parents? [ ] yes [ ] no 34.If yes, can you give an example of something you have seen that showed they were empowered? Evaluation of REPLICA/UGANDA Conducted by School-to-School International for Creative Associates International, Inc. Final version April 2007 p. 59 of 85 …………………………………………………………………………………………………. …………………………………………………………………………………………………. …………………………………………………………………………………………………. 35.What has been REPLICA’s biggest impact in terms of girl child education? …………………………………………………………………………………………………. …………………………………………………………………………………………………. …………………………………………………………………………………………………. 36.In summary, what in your view were REPLICA’s biggest successes and biggest problems? 37.Success: …………………………………………………………………………………………………. …………………………………………………………………………………………………. …………………………………………………………………………………………………. 38.Problems: …………………………………………………………………………………………………. …………………………………………………………………………………………………. …………………………………………………………………………………………………. 39.Do you have any other comments you would like to add? …………………………………………………………………………………………………. …………………………………………………………………………………………………. …………………………………………………………………………………………………. Thank you for your time! Evaluation of REPLICA/UGANDA Conducted by School-to-School International for Creative Associates International, Inc. Final version April 2007 p. 60 of 85 Part 3: Pupils, P5-7 Instructions to researcher: To be administered to Pupils: All P5-P7. 6: 3m, 3f. Duration: 20 min. Number pupils:….. boys:…… girls:….. 40.Have you ever heard of a program called REPLICA? [ ] yes [ ] no 41.If yes, what have you heard? ………………………………………………………………………………………………… ………………………………………………………………………………………………… 42.Have any of your teachers been teaching you about peace? [ ] yes [ ] no 43.If yes, what do they teach you? ………………………………………………………………………………………………… ………………………………………………………………………………………………… 44.Do you do other things outside of class to practice peacemaking? ………………………………………………………………………………………………… ………………………………………………………………………………………………… 45.Do you have a peace club in the school? If yes what do you do? ………………………………………………………………………………………………… ………………………………………………………………………………………………… 46.Do you have an unmarked box in the school? [ ] yes [ ] no 47.If yes, how do you use it? Give me an example. ………………………………………………………………………………………………… ………………………………………………………………………………………………… Part 4: Girls P5-7 Instructions to researcher: To be administered to 6 girls, 2 from each class. Duration: 20 min. Number girls: …… 48.Have you ever heard of a program called REPLICA? Number yes: ….. Evaluation of REPLICA/UGANDA Conducted by School-to-School International for Creative Associates International, Inc. Final version April 2007 p. 61 of 85 49.If yes, what have you heard? ………………………………………………………………………………………………… ………………………………………………………………………………………………… 50.Do your teachers teach you about girls’ education? Number yes: …… 51.If yes, what do they teach you? ………………………………………………………………………………………………… ………………………………………………………………………………………………… 52.Did you receive a girl friendly kit? (Ask to respond individually). Number yes:… 53.If yes, what’s in the kit? ………………………………………………………………………………………………… ………………………………………………………………………………………………… 54.Why were you given this girl-friendly kit? ………………………………………………………………………………………………… ………………………………………………………………………………………………… 55.Did boys receive a kit? Number yes: ….. 56.Why not? ………………………………………………………………………………………………… ………………………………………………………………………………………………… 57.Do you have it here? Number yes:….. 58.Do you still have the items? Which ones? 59.For those items that are finished, have your parents continued to buy you others? 60.Has your school provided you with a special changing room for cleaning yourself? 61. Does your school have a special latrine for girls? Evaluation of REPLICA/UGANDA Conducted by School-to-School International for Creative Associates International, Inc. Final version April 2007 p. 62 of 85 Thank you! Researcher’s notes: Part 5: CCT Instructions to researcher: To be administered to CCT. Duration: 30 min. Materials: List of materials, examples if possible. Name of CCT: …………………………………………… … male … female Explain to CCT: Evaluation, content: materials, training, support supervision, what was learned, and design issues. Feel free to express yourself, some questions separate, do not have to agree with each other, praise and criticism are good, will help in going to scale, anonymity, 1 hour. Training Reminder: Training for REPLICA was in a cascade: 7. Regional level: National level trainers brought people together at the regional level: Tutors, CCTs, Disctrict Education Officers, Deputy Principal Outreach. 8. Coordinating Center level: Tutors, CCTs, DEOs and DPOs train 3 teachers from each school. NB All schools in a catchment area came to these trainings, but REPLICA only provided additional support (supervision etc) to the 3 model schools per district. 9. School level: 3 teachers in each school train their peers in “school-based training”. Training: School-based 1. To your knowledge, did the school-based training take place in all 3 of your REPLICA schools? [ ] yes [ ] sometimes [ ] no 2. If yes, were trainings organized: [ ] during the week [ ] on the weekends/ holidays? [ ] 2 or more times per week [ ] once a week [ ] once every 2 weeks [ ] once a month 3. What percentage of teachers attended on average? [ ] 90-100% [ ] 50-90% [ ] fewer than half 4. What problems did teachers or trainers have with school-based training? …………………………………………………………………………………………………. …………………………………………………………………………………………………. Evaluation of REPLICA/UGANDA Conducted by School-to-School International for Creative Associates International, Inc. Final version April 2007 p. 63 of 85 …………………………………………………………………………………………………. 5. In your view, have any of REPLICA’s training events had any impact in the classroom? [ ] yes [ ] no 6. If yes, what were they? …………………………………………………………………………………………………. …………………………………………………………………………………………………. …………………………………………………………………………………………………. …………………………………………………………………………………………………. Training: Regional level (21 days) 7. Did you attend this training? [ ] yes [ ] no 8. If yes, what percentage of the participants actually attended? [ ] 90-100% [ ] 50-89% [ ] less than 50% 9. If yes, would you judge this workshop as: [ ] completely successful [ ] partially successful [ ] many problems 10.If partially successful or many problems, what problems did you see? …………………………………………………………………………………………………. …………………………………………………………………………………………………. …………………………………………………………………………………………………. Materials 11.Please look at this list of materials. Are you familiar with them? [ ] yes [ ] no (Show materials. If interviewees change their mind, change previous answer.) 12.Which of them were, in your view, the most useful? ………………………………………………………………………………………………… ………………………………………………………………………………………………… 13.In what way were they useful? Evaluation of REPLICA/UGANDA Conducted by School-to-School International for Creative Associates International, Inc. Final version April 2007 p. 64 of 85 ………………………………………………………………………………………………… ………………………………………………………………………………………………… ………………………………………………………………………………………………… ………………………………………………………………………………………………… 14.How would you rate these materials in terms of their appropriateness for conditions here in the north? [ ] very appropriate [ ] appropriate in some cases [ ] not appropriate 15.If appropriate in some cases or not appropriate, how could these materials be improved to make them more appropriate? ………………………………………………………………………………………………… ………………………………………………………………………………………………… ………………………………………………………………………………………………… ………………………………………………………………………………………………… 16.What else could be done to improve these materials? ………………………………………………………………………………………………… ………………………………………………………………………………………………… ………………………………………………………………………………………………… ………………………………………………………………………………………………… 17.Were materials distributed on time? [ ] yes [ ] sometimes [ ] no 18.If no or sometimes, what were problems with distribution? ………………………………………………………………………………………………… ………………………………………………………………………………………………… Support monitoring 19.With what frequency do you visit each REPLICA school? [ ] Once a month [ ] twice a month [ ] more than twice a month Other:…………….. 20.When you visit your schools, do you visit (tick all that apply): Evaluation of REPLICA/UGANDA Conducted by School-to-School International for Creative Associates International, Inc. Final version April 2007 p. 65 of 85 [ ] teachers [ ] head teacher [ ] deputy head teacher [ ] head man teacher [ ] head woman teacher [ ] pupils [ ] other 21.How many peer support meetings have you conducted in total? …… 22.How many peer group meetings (PGMs) have you conducted specifically to look at REPLICA issues? ….. 23.Have you ever conducted CPDs specifically geared to improving the professional development of teachers as far as REPLICA was concerned? [ ] yes [ ] no If yes, what issues were handled in those meetings? ……………………………………………………………………………………………… ……………………………………………………………………………………………… ……………………………………………………………………………………………… ……………………………………………………………………………………………… What was learned The REPLICA project focused on 6 components: • Psychosocial care/guidance and counseling • Community integration • Girl Child education • Performing arts and learning • Leadership and governance • Peace education 24.In which of these areas do you believe teachers learned the most (more than one answer possible? …………………………………………………………………………………………………. ………………………………………………………………………………………… 25.Can you give an example of something you have seen that shows what teachers learned through the REPLICA project? …………………………………………………………………………………………………. …………………………………………………………………………………………………. …………………………………………………………………………………………………. Evaluation of REPLICA/UGANDA Conducted by School-to-School International for Creative Associates International, Inc. Final version April 2007 p. 66 of 85 …………………………………………………………………………………………………. 26.What new attitudes have teachers acquired since participating in the REPLICA project? …………………………………………………………………………………………………. …………………………………………………………………………………………………. …………………………………………………………………………………………………. …………………………………………………………………………………………………. Project design questions 27.REPLICA used 5 kinds of events to prepare for the project and build support: (1) Info gathering sessions before design, (2) consensus-building meetings, (3) Academic seminars with stakeholders, (4) Public engagement meetings and (5) Feedback meetings. Are you familiar with these events? [ ] yes [ ] no (If no, explain the events they haven’t heard of – maybe they have, but don’t know the name.) 28.Have you participated in any of these events? [ ] yes [ ] no If yes, please indicate which events you participated in: [ ] Info gathering sessions before design [ ] consensus-building meetings [ ] Academic seminars with stakeholders [ ] Public engagement meetings [ ] Feedback meetings 29.How would you rate the quality of the event(s) you attended in terms of: Usefulness GOOD FAIR POOR Info gathering sessions before design consensus-building meetings Academic seminars with stakeholders Public engagement meetings Feedback meetings If fair or poor, please explain: …………………………………………………………………………………………………. …………………………………………………………………………………………………. 30.Involvement & participation of beneficiaries GOOD FAIR POOR Info gathering sessions before design Evaluation of REPLICA/UGANDA Conducted by School-to-School International for Creative Associates International, Inc. Final version April 2007 p. 67 of 85 consensus-building meetings Academic seminars with stakeholders Public engagement meetings Feedback meetings 31.If fair or poor, please explain: …………………………………………………………………………………………………. …………………………………………………………………………………………………. 32.A central design feature of the REPLICA project was to focus energies on the TDMS structure for training, for distributing materials and monies, etc. The idea was that once people in the colleges received training, monies and materials, they would then administer the project to the CCs and the schools. In your view, has this design feature worked? What problems have you noticed? …………………………………………………………………………………………………. …………………………………………………………………………………………………. …………………………………………………………………………………………………. …………………………………………………………………………………………………. 33.For the REPLICA project, the colleges were supposed to co-opt district personnel and ESA staff to assist with support supervision activities in REPLICA schools. Do you think this system worked well, or is there another way that support supervision should be organized? [ ] worked well [ ] should be organized differently 34.If organized differently, how? …………………………………………………………………………………………………. …………………………………………………………………………………………………. …………………………………………………………………………………………………. 35.Do you think the REPLICA project empowered teachers, head teachers, pupils or parents? [ ] yes [ ] no 36.If yes, can you give an example of something you have seen that proved to you that people were empowered? …………………………………………………………………………………………………. …………………………………………………………………………………………………. Evaluation of REPLICA/UGANDA Conducted by School-to-School International for Creative Associates International, Inc. Final version April 2007 p. 68 of 85 …………………………………………………………………………………………………. 37.What has been REPLICA’s biggest impact in terms of girl child education? …………………………………………………………………………………………………. …………………………………………………………………………………………………. …………………………………………………………………………………………………. 38.The REPLICA project will be scaling up in the coming months to cover 1,700 schools. What difficulties can you foresee? …………………………………………………………………………………………………. …………………………………………………………………………………………………. …………………………………………………………………………………………………. 39.How can REPLICA prepare for this stage? …………………………………………………………………………………………………. …………………………………………………………………………………………………. …………………………………………………………………………………………………. 40.In summary, what in your view were REPLICA’s biggest successes and biggest problems? 41.Success: …………………………………………………………………………………………………. …………………………………………………………………………………………………. …………………………………………………………………………………………………. 42.Problems: …………………………………………………………………………………………………. …………………………………………………………………………………………………. …………………………………………………………………………………………………. 43.Do you have any other comments you would like to add? …………………………………………………………………………………………………. …………………………………………………………………………………………………. Evaluation of REPLICA/UGANDA Conducted by School-to-School International for Creative Associates International, Inc. Final version April 2007 p. 69 of 85 …………………………………………………………………………………………………. Thank you for your time! End of part 1 Notes of researcher: …………………………………………………………………………………………………. …………………………………………………………………………………………………. …………………………………………………………………………………………………. REPLICA EVALUATION, TOOL 5 SCHOOL: PARENTS Date: ……………………. Number of parents: male …… female …… Title: ……………………………………………. Place of interview: ………………………………………………………… Special conditions: ………………………………………………………………………. ………………………………………………………………………………………………… 1. Have you heard of REPLICA? What have you heard? 2. What was your role in REPLICA? In which activities did you participate? 3. Did REPLICA change your children’s behavior? How? 4. Did REPLICA change teaching or conditions in your child’s school? How? 5. How did REPLICA change your knowledge, your role or your behavior as a parent? 6. What can be done to improve REPLICA in future? Evaluation of REPLICA/UGANDA Conducted by School-to-School International for Creative Associates International, Inc. Final version April 2007 p. 70 of 85 REPLICA EVALUATION, TOOL 6 INTERVIEW WITH SUBCONTRACTORS Date: ……………………. Name of organization: ……………………………………………….. Names of interviewees & titles: …………………………… ………………………… …………………………… ………………………… …………………………… ………………………… …………………………… ………………………… Place of interview: ………………………………………………………… Special conditions: ………………………………………………………………………. ………………………………………………………………………………………………… Questions: 1. What was your role and activities in REPLICA? 2. What is the background of your organization? 3. What materials did you develop and how? 4. What training activities did you conduct and how? 5. How did you conduct support supervision? 6. Do you have any monitoring records? 7. What was REPLICA’s greatest achievement? 8. What recommendations can you make for REPLICA’s expansion? Evaluation of REPLICA/UGANDA Conducted by School-to-School International for Creative Associates International, Inc. Final version April 2007 p. 71 of 85 ANNEX B: LIST OF PERSONS CONSULTED Abraham Were DPO, PTC Loro Aguti Martha Tutor, Guidance and Counseling, PTC Loro Aide Michael Tutor, Core PTC, Gulu Aisu Noel CT/DPO/Tutor, Soroti PTC Akenna Soloman Robert Man teacher, Abera PS Alan Tukwasibwe Finance Officer, FAWE/Uganda Alegan Amos CCT, Angole-Wera PS Amo Okwe-Okaka DEO Kitgum Amongin-Tulsei Jane Immaculate Tutor, Soroti PTC Amuge Jane Teacher, Angole-Wera PS Anuso Susan Senior Woman Teachaer, Angole-Wera PS Charles Kamwesigye Community Coordinator, Pincer Associates Dorothy Akankwasa Program Officer, FAWE/Uganda Erwaat John Peter Teacher, Angole-Wera PS Etapu Ogweta Joseph Tutor, Soroti PTC Etoyu Michael DEO Soroti Florence Kanyike Deputy Chief of Party, UNITY George Opiro Principle Education Officer, Guidance & Counseling Programs, SNECG Immaculate Mukasa Coordinator M&E, Pincer Associates Isaac Ogal CCT, Aculbanya PS, Apac Jerry Bagaya City Inspector of Schools, Maths – ESA Jimmy Oppio Advisor to Team Line John Akenna Man teacher, Abera PS John Bwayo Lecturer, Univ of Kyambogo John Ogwang Head Teacher, Aculbanya PS, Apac Sr. Joyce Anakare CCT, Pajule Lacani, Kitgum PTC Kalebo Stephan Teacher, Angole-Wera PS Kitara Paul Komakech CCT Nimaro, Kitgum PTC Laber Toolit Victor Ben Head of Progream Community Mobilization, Kitgum PTC Laboi Rose Regina Head Teacher, Pajimo Army PS Lalwery Mary Susan Inspector Kitgum Lochom Peter Logistics Officer, Pincer Associates Lomoro James Principal Inspector of Schools, ESA Martha Muhwezi Technical Advisor, FAWE/Uganda Martin Omago-Loican Commissioner, Dept of Special Needs Evaluation of REPLICA/UGANDA Conducted by School-to-School International for Creative Associates International, Inc. Final version April 2007 p. 72 of 85 Education, Guidance & Counseling (SNEGC) Martin Opolot M&E Specialist, UNITY project Mary Atyang DHT, Aculbanya, Apac Mkoola Samuel CCT Layibi, Gulu Munduru Rose CCT Lukome, Gulu Noel Iso Deputy Principal Preserve, Soroti PTC Oballim Christopyer SEO Kitgum Ochen Florence Senior Woman Teacher, Aculbanya PS, Apac Ocheng Vincent Ocen DEO Gulu Ochola Kerosin DPO Kitgum PTC Rev. Ochola Richard M. DPP Odwilo Simon CT Principal/DPO Soroti PTC Ojok Anthony Joe Senior Man Teacher, Aculbanya PS, Apac r Okellayot Damsasco CCT Pajimo, Kitgum PTC Okello Patrick Ginyakol Deputy Principal Officer, Core PTC, Gulu Okitoi Joseph Chairperson Scripture Union, Soroti PTC Okumu John Bismarck CCT GEM, Kitgum PTC Okuonziru Hellen Torach DIS Kitgum Olana Simon Head Teacher, Abera PS, Gulu Olla Jean Baptiste CCT Abera, Gulu Ololo Graphes Principal tutor/Patron writers’ club, Soroti PTC Oluku Thomas Tutor, Guidance and Counseling, PTC Loro Omunyokol Eunice (Mrs) Head Teacher, Angole-Wera PS, Amuria district Onekalit Maurice CCT Lira Palwo, Kitgum PTC Ongom Alex CCT Akworo, Kitgum PTC Ongom Ronald Campus Minister, PTC Loro Oony James President, PTC Loro Opio Omana Principal, PTC Loro Opus Florence DHT, Angole-Wera PS Otim-Ebong David CCT Dicwinyi, Kitgum PTC Otim Willie Senior Man Teacher, Angole-Wera PS Patrick Bananuka TDMS Outreach Coordinator Renuka Pillay Chief of Party, UNITY Project (formerly BEPS III/REPLICA) Susan Mubbala Coordinator, REPLICA activities, FAWE/Uganda Tom Duku Finance and Administration Manager, Pincer Associates, Pincer Associates Victor Avasi Training and Research, Pincer Associates Evaluation of REPLICA/UGANDA Conducted by School-to-School International for Creative Associates International, Inc. Final version April 2007 p. 73 of 85 Wanjisi Davis CCT Alero, Gulu Wegulo Malinga Lewis Tutor, Soroti PTC Evaluation of REPLICA/UGANDA Conducted by School-to-School International for Creative Associates International, Inc. Final version April 2007 p. 74 of 85 ANNEX C: MATERIALS REVIEWED Author & date unknown. Performing Arts and Learning in Schools (PALS): Final report on primary teachers based training in all REPLICA clustered schools in the north and northeastern Uganda. Author and date unknown. Report on support supervision and monitoring of model schools in Lango and Acholi Sub-regions. Author and date unknown. Report on the support supervision and monitoring of model schools in Gulu, Kitgum and Pader disticts. Author and date unknown (2006?) Soroti Primary Teachers' College Catchment Area Field Report on REPLICA model schools peer support supervision, 29th May to 1st June 2006. Author unknown. (2006) Report on the review meeting for Guidance & Counseling and REPLICA Programme: Reviewing the stutus of the progress of school-based implementation. May. BEPS (2005) A resource book for primary school guidance and counseling: Therapeutic play. USAID/MoES. BEPS (2005) Basic training for school service providers: Facilitator's guide. USAID/MoES. BEPS (2005) Guidance and counselling for primary schools: Teacher's handbook. USAID/MoES. BEPS (2005) Guidance and counselling for Primary Teacher Colleges: Tutor's guide. USAID/MoES. BEPS (2006) Uganda Project Education Assistance/BEPS: Final Report. Creative Associates International, Inc. August. BEPS (May 2006?) End of BEPS III Project Report. Cox, Sharon (2006a) Community Integration Programme: Final Report. July. Cox, Sharon (2006b) Primary Teachers' Colleges Community Integration Programme assessment report, March-April 2006. May. Cox, Sharon (2006c) Review of the CIP programme in five guidance and counselling districts. March-April 2006. May. Cox, Sharon (2006d) Review of the CIP in the guidance and counseling programme. Amuria, Katakiw, Lira and Ntungamo. 13th-23rd March, 2006. Cox, Sharon (2006e) Review of the REPLICA programme in Amuria, Apac, Katakwi, Lira and Soroti. March 13-17, 2006. Evaluation of REPLICA/UGANDA Conducted by School-to-School International for Creative Associates International, Inc. Final version April 2007 p. 75 of 85 FAWE (2005) Inception report to BEPS III Project of the MoES. FAWE (2006) The basics for the promotion of girls' education: A handbook for guiding action and mobilizing support. BEPS/REPLICA/USAID. FAWE (2006c) Quarterly report, 1st Quarter. REPLICA/Uganda. March. FAWE (2006c) Quarterly report, 2nd Quarter (January-March 2006). REPLICA/Uganda. March. FAWE (date unknown) Girls' education: Training guide. FAWE (2006a) Report on the findings of the baseline survey for the project to promote education in the conflict affected districts of Amolator, Amuria, Apac, Gulu, Kaberamaido, Katakwi, Kitgum, Lira, Pader, and Soroti. April. FAWE (2006b) Implementation report of the Girls’ Education component of the REPLICA project in the conflict affected districts of Kitgum, Pader, Gulu, Lira, Amolator, Abac, Soroti, Kaberamaido, Amuria and Kataqkwi. July. Gulu Core PTC (2006) The Kwiri-Kwaro. Vol. 1, No. 1. REPLICA. Harvey, Stephen et al (2005) Module 1: Leadership in education. Education Management Strengthening Initiative: Managing for quality, 2nd edition. MoES/USAID/UPHOLD/BEPS SUPER Project. Harvey, Stephen et al (2005) Module 2: Managing school improvement. Education Management Strengthening Initiative: Managing for quality, 2nd edition. MoES/USAID/UPHOLD/BEPS SUPER Project. Harvey, Stephen (2005) Module 3: Managing school finances. Education Management Strengthening Initiative: Managing for quality, 2nd edition. MoES/USAID/UPHOLD/BEPS SUPER Project. Harvey, Barbara & Dorothy Aanyu-Angura (2005) Module 4: Managing the school curriculum. Education Management Strengthening Initiative: Managing for quality, 2nd edition. MoES/USAID/UPHOLD/BEPS SUPER Project. Kitgum Core PTC (2006) The Kit-Deyo. Vol. 1, No. 1. REPLICA. Martha Bragin (January 2004) Education for all in the conflict zones of Uganda: Opportunities, Challenges and a Way Forward. (Available from Creative Associates International, Inc.) MoES (2005) National conceptual framework for psychosocial care in education: Priority needs, curriculum guide and materials. National Conceptual framework of psychosocial care in education. June. Draft. MoES (2005b) REPLICA Programme: Programme Scope, 2005/2006. Contact person: George Opiro. Evaluation of REPLICA/UGANDA Conducted by School-to-School International for Creative Associates International, Inc. Final version April 2007 p. 76 of 85 MoES (2005c) Report of the regional consensus building workshop held in Lira Hotel on the 12th of August 2005. Pincer Associates/REPLICA Program. MoES (2005d) "Opening a dialogue": A conference on peace building and conflict resolution held on the 24th March 1006, Soroti Hotel. MoES (2006) Guidelines on roles and responsibilities of stakeholders in the implementation of guidance and counselling in educational institutions. BEPS/USAID. MoES (2006) Teacher's guide to school talking environment. Wirefred George Opiro. USAID-BEPS III. June. MoES (date unknown) Psychosocial care in education: Essential services in schools. USAID/BEPS. Mutto, Milton et al (2006) Leadership and Governance: A handbook for school managers. Pincer Associates/REPLICA/MoES. Opiro, Wirefred George (2005) Capacity development for care and support: Supporting teachaers and caregivers to do their job effectively. Forum for Southern African Ministers of Education, Ezulwini Sun, Mbabane, Swaziland, 12-16 September 2005. PowerPoint presentation. Opiro, Wirefred George (2005) Psychosocial care in education: Essential services in school. Training kit for school service providers. Opiro, Wirefred George (2006a) Psychosocial trauma management skills: Teacher's handbook. 2nd edition. USAID/BEPS/MoES. Opiro, Wirefred George (2006b) REPLICA status report. MoES/USAID/Creative Associtates. July. Pillay, Renuka (2006) REPLICA: Integrating diverse voices into education programming for post-conflict development. BEPS/Uganda, October. Pincer Associates (2005?) Peace education, psychosocial care and leadership and management improvement in conflict affected primary schools: A baseline survey for the replica program. Kampala, Uganda: REPLICA? Pincer Associates (2006) Performance report for Pincer activities on peace education, leadership & management and psychosocial care & support. Second quarterly report. Jan-Mar 2006. Pincer Associates (date unknown) Revitalizing Edication, Participation And Learning In Conflict Affected Areas (REPLICA Program) Final Technical Report. REPLICA (2005a) Feedback meeting between the MoES, REPLICA subcontractors and BEPS III Project. Draft report. November 8-9. REPLICA (2005b) REPLICA Program-MoES: An inception report for implementation in 8 of the 18 most affected districts. Pincer Associates. Evaluation of REPLICA/UGANDA Conducted by School-to-School International for Creative Associates International, Inc. Final version April 2007 p. 77 of 85 REPLICA (2005c) Report of the national consensus building workshop, 4th August, Kampala. Pincer Associates. REPLICA (2006a) Community integration: Facilitator's guide. USAID/BEPS/MoES. REPLICA (2006b) Community integration: Programme brochure. USAID/BEPS/MoES. REPLICA (2006c) Community integration: Programme toolkit. USAID/BEPS/MoES. REPLICA (2006d) Report on the Ministry of Education and Sports Guidance and Counseling advocacy event held in Gulu district on 6th and 7th July 2006. REPLICA (2006e) Report on the cluster based teacher treaining for model schools in Soroti, Lira and Gulu, 9th-17th Dec 2005 and 26th Jan - 4th Feb 2006. Pincer Associates. REPLICA (date unknown) Capacity buidling for training in peace education, psychosocial care and leadership and management: The REPLICA programme. Report of the training of trainers conducted between 11th-15th of October 2005 in Lira and Soroti. Pincer Associates. REPLICA (date unknown) Peace education, psychosocial care and leadership and management improvement in conflict affected primary schools: a baseline survey for the REPLICA program. Final Report. Pincer Associates. REPLICA (date unknown) Peace education, psychosocial care and leadership and management improvement in conflict affected primary schools: a baseline survey for the REPLICA program. Final Report. Pincer Associates. REPLICA (date unknown) Report on public engagement meetings in Gulu, Kitgum and Pader districts at Gulu and Kitgum town council halls under the REPLICA program. 8th & 12th April 2006. Soroti Core PTC (2006) The Solot. Vol. 1, No. 1. REPLICA. Team Line (2005) Performing Arts and Learning in Schools (PALS): Progress report for implementation of Phase 1 of PALS activities, August-December 2005. REPLICA project, Uganda. December. Team Line (2006) Performing Arts and Learning in Schools (PALS): Progress report for implementation of Phase 2 of PALS activities, January-March 2006. REPLICA project, Uganda. Teamline (2006) Performing Arts and Learning in Schools (PALS): Final report for PALS activities. July. USAID (date unknown) SO8 Indicator table, Baseline & targets, 2003-2006.