THE MID-TERM EVALUATION OF USAID/UGANDA’S UNITY PROGRAM REPORT FINAL REPORT December, 2008 This document was produced for review by the United States Agency for International Development (USAID/Uganda). It was prepared by The Mitchell Group, Inc. for the Uganda Monitoring and Evaluation Management Services, herein referred to as UMEMS, number 617-C-00-08-00012-00 2 | The Mid-Term Evaluation of USAID/Uganda’s UNITY Program, December 2008 Acronyms AIDS Acquired Immune Deficiency Syndrome BEPS Basic Education Policy Support CAII Creative Associates International, Inc. CC Coordinating Centre CCT Coordinating Centre Tutor CRT Community Resource Team C-TEP Certificate in Teacher Education Proficiency CTO Cognizant Technical Officer (CTO DEC District Education Committee DEO District Education Officer DIS District Inspector of Schools ECD Early Childhood Development EFAG Education Foreign Assistance Group EMIS Education Management and Information Systems EPD Educational Planning Department ESSP Education Sector Strategic Plan EUPEC Enhancement of Universal Primary Education in the Community FAWE Forum for African Women Educationalist G&C Guidance and Counseling GDP Gross Domestic Product HIV Human Immune Virus HT Head Teacher IQC Indefinite Quantity Contract KYU Kyambogo University LOE Level of Effort LOI Language of Instruction M&E Monitoring and Evaluation MOES Ministry of Education and Sports MRCU Madrasa Resource Centre-Uganda MT Master Trainers NCDC National Curriculum Development Centre NDP National Development Plan NGO Non Government Organization NTB Non Text Book PALS Performance Arts and Learning in Schools PIASCY Presidential Initiative on AIDS Strategy for Communication to Youth PPET Post Primary Education and Training PRDP Peace Recovery and Development Plan PS Permanent Secretary PTA Parent Teachers Association PTC Primary Teachers College QEI Quality Enhancement Initiative RCC Regional Coordinating Center REPLICA Revitalization of Education Participation and Learning in Conflict Areas 3 | The Mid-Term Evaluation of USAID/Uganda’s UNITY Program, December 2008 SDP School Development Plan SMC School Management Committee SMT Senior Male Teacher SNE Special Needs Education SO Strategic Objective STF Straight Talk Foundation SWT Senior Woman Teacher TC Thematic Curriculum TDMS Teachers Development Management System TED Teacher Education Department TEWG Teacher Education Working Group TMG The Mitchell Group UMEMS Uganda Monitoring and Evaluation Management Services UNITY Uganda Initiative for TDMS and PIASCY UNATO Uganda National Teachers’ Union UNEB Uganda National Examination Board UNICEF United Nations Children Fund UPE Universal Primary Education UPHOLD Uganda Program for Human and Holistic Development USAID United States Agency for International Development USE Universal Secondary Education 4 | The Mid-Term Evaluation of USAID/Uganda’s UNITY Program, December 2008 Acknowledgements This midterm evaluation of the Uganda Initiative for TDMS and PIASCY (UNITY) project and the report of those findings was carried out by a six-person team consisting of: David P. Evans (Team Leader), Senior Education Consultant; Deborah Kasente, Education Consultant, Gender Studies, Makerere University; Robinah Kyeyune, Education Consultant, Teacher Education, Makerere University; Deborah Nakyejwe, Field Researcher; Geoffrey Opira, Field Reseacher; and Nyanzi Sulaiman, Financial Analyst. The six-person team was hired under a contract administered by the Mitchell Group (TMG) has an office in Uganda called Uganda Monitoring and Evaluation Management Services (UMEMS). UMEMS provided invaluable technical, administrative and logistic support to the evaluation team; especially UMEMS staff members Nestore Jalobo, Operations Manager; Caroline Kasabiti, Team Associate; Collins Obote, Project Accountant; and Patricia Rainey, Chief of Party. The team gratefully acknowledges the assistance provided by: • United States Agency for International Development (USAID)/Uganda, especially: Thomas LeBlanc, Senior Education Officer; Sarah Mayanja, UNITY Cognizant Technical Officer (CTO); and Rhona Walusimbi, Monitoring and Evaluation Specialist; • The UNITY project staff, including: Patrick Bananuka, Institutional Advisor; Alice Ibale, HIV/AIDS Manager; Florence Kanyike, Deputy Chief of Party; Vicky Kasiimire, Finance Manager; Polly Kiguli, Finance and Administrative Officer; Martin Opolot, Monitoring and Evaluation Specialist; Renuka Pillay, Chief of Party; Scholastica Tiguryera , Basic Education Manager, and Dickson Turyareeba, Research and Policy Assistant; • Representatives of the Ministry of Education and Sports (MOES), especially Byamugisha Albert, Assistant Commissioner, Education Planning, MOES and MOES Coordinator of the UNITY program; • Representatives of the international donor community; • Local government officials in each of the eight districts visited; and • Community leaders, members of the school committees, students and parents, and particularly to the supervisors, principals and teachers in all the schools visited. To all of those people and to any others who contributed to the provision of information to the team, thank you. Without their valuable assistance the evaluation could not have been completed (Annex C lists key people interviewed). Any errors are regrettable but, of course, the sole responsibility of the team. 5 | The Mid-Term Evaluation of USAID/Uganda’s UNITY Program, December 2008 Table of Contents Acknowledgements ................................................................................................................................................... 4 I. EXECUTIVE SUMMARY ....................................................................................................... 7 I. INTRODUCTION ................................................................................................................. 19 A. Evaluation Objectives and Scope ....................................................................................................................... 19 B. Methodology ...................................................................................................................................................... 19 C. Constraints in Undertaking the Evaluation ......................................................................................................... 21 D. Structure of the Report ....................................................................................................................................... 22 II. BACKGROUND .................................................................................................................... 23 A. Education in Uganda .......................................................................................................................................... 23 B. Overview of USAID Education Strategy in Uganda .......................................................................................... 24 C. Description of UNITY Program ......................................................................................................................... 24 D. Overview of MOES Education Strategy ............................................................................................................ 26 E. Relevance of UNITY Strategy to GOU Strategy ................................................................................................ 26 III. FINDINGS ............................................................................................................................ 27 A. Overview ............................................................................................................................................................ 27 B. Objective 1: Improved Professional Development of Teachers and Administrators at the Primary Level, Both in Pre- and In-Service ............................................................................................................................................. 27 1. C-TEP Program .......................................................................................................................................... 28 2. Creating Regional Centers ......................................................................................................................... 30 3. Reforming the Cascade Approach to Training ........................................................................................... 30 4. Developing Centers of Excellence ............................................................................................................. 30 5. Strengthening the Tripartite System of Support to Teachers ..................................................................... 34 6. Provision of Materials ................................................................................................................................ 34 7. Thematic Curriculum (TC)......................................................................................................................... 35 8. Expanding REPLICA ................................................................................................................................. 37 C. Objective 2: Expand Implementation of PIASCY ............................................................................................. 40 D. Objective 3: Increase Parental and Community Participation ............................................................................ 42 E. Objective 4: Monitor and Assess Effectiveness of Policy Implementation ........................................................ 44 F. Grants ................................................................................................................................................................. 45 IV. LESSONS LEARNED........................................................................................................ 46 V. COST-EFFECTIVENESS, SUSTAINABILITY AND REPLICATION ....................... 48 VI. RECOMMENDATIONS BY OBJECTIVE ..................................................................... 52 6 | The Mid-Term Evaluation of USAID/Uganda’s UNITY Program, December 2008 ANNEXES ................................................................................................................................... 56 Appendix A: Selected list of documents consulted ................................................................................................. 56 Appendix B: List of Institutions Visited by the Evaluation Team .......................................................................... 58 Appendix C: List of Individuals Interviewed .......................................................................................................... 59 Appendix D: Classroom Checklist .......................................................................................................................... 68 Appendix E: Anonymous Questionaire .................................................................................................................. 70 Appendix F: Analysis of Data from the Questionnaire ........................................................................................... 73 Appendix G: Interview Guide for Senior Woman / Man Teachers ........................................................................ 84 Appendix H: Focus Group Questionnaire: PTC and CCTs – Tutors ...................................................................... 85 Appendix I: Interview guide for DEO’s, DIS’s/Education committee ................................................................... 86 Appendix J : List of activities in Objective 1 .......................................................................................................... 87 7 | The Mid-Term Evaluation of USAID/Uganda’s UNITY Program, December 2008 I. EXECUTIVE SUMMARY I. Background The purpose of the evaluation is to assess the UNITY programs’ strengths and weaknesses to assist USAID in making informed management decisions regarding future education programming. The evaluation team used six basic methods to obtain its findings, including: reviewing project documents; observation of UNITY-assisted schools, head teachers, and teachers; targeted questions and in-depth interviews with individuals knowledgeable about the UNITY project; targeted questions in focus group discussions; anonymous questionnaires administered to CCT tutors; and classroom checklists. There were several constraints to conducting the UNITY midterm evaluation, including: access to some key UNITY reports or analyses was not available because they were not complete at the time of the evaluation; a limited window for field work, given the complexity and geographic spread of UNITY; and a small sample of schools not allowing as in-depth an analysis as would have been preferred, given the very complex project design. II. Findings Overall, the UNITY program has made a strong start implementing a very complex program and has a good chance to make a substantial impact on a number of different educational areas in the medium- to long-term. For example, UNITY appears to have revitalized the pre- and in-service TDMS system and, if some issues surrounding transportation and improving the transfer of knowledge from tutor to teacher can be solved, a positive impact will be felt for many years. To date, however, UNITY has had but a small impact on the school- level and/or on the quality of education. USAID and UNITY should consider focusing on a limited number of interventions that are comparative advantages of the UNITY program and that are focused on school and pupil-level improvements, such as making the cascade system of teacher training work, putting materials in the hands of teachers, developing different models to assist teachers implement thematic curriculum, increasing participation of parents and communities, and addressing educational policy or guidelines that might inhibit the UNITY focus. UNITY and USAID should also consider limiting its focus on only a few districts in the North. Specific Findings UNITY is Well Positioned to Assist the GOU The UNITY program is well aligned with the Government of Uganda (GOU) education policy and the MOES strategic plans. UNITY’s management approach, which stresses flexibility and responsiveness to the MOES, is one of the most striking aspects of the program. 8 | The Mid-Term Evaluation of USAID/Uganda’s UNITY Program, December 2008 A. Findings by UNITY Objective Objective One: Improving Professional Development Revitalized TDMS Despite substantial challenges, UNITY has revitalized the TDMS system and upgraded the skills of Primary Teachers College (PTCs) faculty and the Coordinating Center Tutors (CCTs) who are the backbone of the TDMS and provide the pre-service and in-service professional development and support to Uganda’s approximately 124,000 primary school teachers. Other aspects of the proposed approach to revitalize the TDMS system including creating Regional Centers to re-cluster PTCs into a more viable network, reforming the cascade approach to training, developing Centers of Excellence to foster local innovation, and to strengthen support to teachers are largely bold promises or yet undone. Improved Instructional Methods and Materials to Teachers and Schools UNITY plays a vital role in the development and distribution of significant amounts of teachers’ manuals and learning materials for pupils in support of the implementation of the thematic curriculum reform in primary grades 1-3 (P1- P3). In addition, UNITY, directly or with its partners and sub￾contractors, develops and distributes new - or old materials developed under previous USAID￾funded programs (SUPER, Uganda Program for Human and Holistic Development (UPHOLD) and Basic Education Policy Support (BEPS)) - in support of a variety of programs. By and large, the materials produced by UNITY are excellent and the area of materials development is a major strength of the UNITY program. Challenges facing implementation of Thematic Curriculum Considering that the introduction of thematic curricula is a major shift in basic education and that the reform is only two years old, schools seem to have made good progress and teachers report that pupils are learning faster in local languages than in English. Nonetheless, teachers reported substantial issues with the program stemming, in part, from the design of a “one size fits all” approach when schools have vastly different circumstances. For example in the North few teachers were applying even basic learner-centered teaching methods and the large number of pupils in most P1 and P2 classes (often more than 120 children) and the subsequent overcrowding of the classroom were seen as a major impediment to effectively implementing the reform. Expanded REPLICA Aside from supplementary teaching materials, the Revitalization of Education Participation and Learning in Conflict Areas (REPLICA) program is largely amorphous and its effects are not easily seen. The reason why REPLICA is not easily seen is complex, but some factors are that 1) most of the efforts to encourage community participation have been at the district and sub-county level rather than at the school level, 2) evidence of life skills in primary schools is very difficult to observe, 3) the evaluation team went to only a few REPLICA schools, and 4) some of the activities claimed by REPLICA are also attributed to other programs and organizations. Nevertheless, nearly all District Education Officers (DEOs), District Inspectors (DIS), PTC and CCT tutors, head teachers, and teachers thought that REPLICA was a good program that had made a modest contribution to their district or schools. Most interviewees also applauded REPLICA for its non-educational achievements such as facilitating dialogue among political leaders at the district level. At the same time, however, many observers noted that the REPLICA scale up of the BEPS pilot program was poorly 9 | The Mid-Term Evaluation of USAID/Uganda’s UNITY Program, December 2008 managed and “moved too far too fast.” USAID and UNITY should consider doing a more detailed independent evaluation of REPLICA before expanding or providing further funding. Objective Two: Expanding Implementation of Presidential Initiative on AIDS Strategy for Communication to Youth (PIASCY) Expanded PIASCY Given Uganda’s recent history with Human Immune Virus/Acquired Immune Deficiency Syndrome (HIV/AIDS) and the subsequent huge loss of people, nearly everyone seems to embrace PIASCY as an imperative. Moreover, everyone interviewed thought it is a good program and that introducing HIV/AIDS messages in primary schools is appropriate and that the school is the right place to deliver the messages. Although there was little hard evidence presented, most observers claim that PIASCY has had impacts such as fewer girls dropping out of school, fewer girls becoming pregnant and more girls completing school, more awareness in the community about HIV/AIDS and less stigmatization of HIV/AIDS victims. Objective Three: Increasing Parental and Community Participation in Education Parental and Community Participation in Education In all but a very few schools, School Management Committee (SMC) or District Education Committee (DEC) members were present for interview and most appeared to be involved in the management and governance of the school or the district. School Management Committee members and parents and Parent Teacher Associations (PTAs) maintained that they provided input into school development plans, reviewed school budgets, oversaw teacher performance, and very occasionally assisted in developing additional revenue streams or in-kind assistance to assist in the implementation of the school development plan. Most did not appear confident of their roles and responsibilities and some of the SMC or DEC members appeared to see their role as perfunctory. Almost all of the SMC and DEC members as well as the DEOs, DISs, tutors, head teachers, teachers and many other observers claim that the Universal Primary Education (UPE) declaration and statements made by politicians in support of UPE have killed parental involvement in schools. Many parents say they will not contribute to primary education because the government has said it is free and the government has promised to pay all costs. Those close to working with the community claim that a better articulation of roles and responsibilities is desperately needed to overcome these misconceptions. By contrast, most central government officials claim that the articulation of roles and responsibilities is clear and that “our people just do not want to pay.” In any case, there is very little real parental or community involvement in schools and Ugandan children’s education is suffering because of it. Objective Four: Implementing Education Policy Agenda Educational Policy Agenda The aim of the education policy component of UNITY, working with the MOES, is to: 1) identify current policy issues, formulate policies and develop action plans to implement the policy agenda; and 2) provide technical assistance to EDP’s Monitoring and Evaluation Unit in monitoring policy and program implementation. While UNITY has had some notable policy successes (Gulu Regional Summit), it appears that there are a myriad of policy issues that have not been addressed and are vital to the success of the MOES and UNITY. Part of the fault may lie with USAID, which assigned to this component an estimated level of 10 | The Mid-Term Evaluation of USAID/Uganda’s UNITY Program, December 2008 effort of only 5 percent of the total project effort. USAID should consider making this a larger Level of Effort (LOE). Objective Five: Grants and Private Sector Contributions Grants The intent of the grants program was to support sustained school- and community-based initiatives that address local issues. The idea was to make two tiers of grants: one to individual schools (SMCs) and another to local Non-Government Organizations (NGOs) that are already working with schools and communities to improve pupils’ attendance and attract and retain students, especially girls, throughout the primary cycle. Grants were to be made in the four thematic areas of PIASCY, literacy, quality improvement, and improved school-community relations. Although grants have been made to NGOs (Forum for African Women Educationalist (FAWE), Madrasa Resource Center), it appears that no grants have been made to schools. B. Other Relevant Findings Poor Learning Environment/Health Concerns of Pupils Although schools in Uganda are generally better, and better maintained, than schools in many other African countries, most schools offer generally poor learning environments. In addition to massive overcrowding (pupil/classroom ratios of 100 or more to one), classrooms frequently do not have - or have only a few - desks forcing many children to sit on the floor. Rooms are dark with walls soiled from use, and classrooms offer few displays of learning aids and materials, learning corners, books, or student work. Indeed, few classrooms above P2 have any materials of any kind displayed. Most schools visited in both the North and the South had relatively large populations of apparently unhealthy children. Many of the children seemed to be suffering from ring worm, possible intestinal problems, sever chronic coughs, vitamin deficiency, and many were obviously very poor. USAID should consider developing a “healthy schools” program. District and School Management With few exceptions, district officials and school head teachers appeared to be bright, committed officers interested in improving the quality of education in Uganda. Generally, district officials and head teachers were very articulate in describing the short- and long-term needs and priorities of the district or school even if those priorities appeared to reflect the needs of the educational establishment (top priority was infrastructure development-like, staff housing). Some schools had developed School Development Plans (SDP), but most had not and none of the SDPs focused directly on school quality. Seldom did one find any unanimity about district or school priorities and most objectives were unattainable and/or broad platitudes. It appeared that most would have benefited from realistic, time-phased strategic plans that had incremental but obtainable objectives. Generally, there was a lack of evidence of school accountability and transparency (e. g., displayed district or school budgets) and all head teachers and most district officials complained of woefully late (sometimes eight months into the school year) disbursements of UPE funds. A few head teachers appeared to exhibit good pedagogical and administrative leadership and a good knowledge of teacher evaluation processes and school development needs. At the same time, other head teachers appear to accept the status quo as a given, did not see obvious problems, and did not seem to provide dynamic leadership. Head teachers play a key role in any school system and the competency of head teachers and their relationships with teachers, students, SMC’s and parents 11 | The Mid-Term Evaluation of USAID/Uganda’s UNITY Program, December 2008 contribute considerably to program understanding, clearer procedures and enhanced implementation. In addition, it appears that some sub-district and district managers offer lack￾luster support to the schools and could profit from a better definition of their roles and responsibilities as well as training in management and leadership. Teaching Methods A very few teachers currently lead classes using well-thought out active￾learning exercises and techniques but most appear to be learning the thematic curriculum and active-learning methods recently introduced. It is somewhat premature to access the recent training of tutors, but the cascade model for delivering pedagogical messages to teachers does not seem to be working well and many teachers, head teachers and administrators suggested greater use of a whole school approach to training. Teachers in a few schools have made strides toward improved and diversified teaching methods but, for the most part, the progress in improving teaching methods is incomplete and has yet to reach its potential in most settings. Some classrooms have arranged desks in clusters and use some very basic active learning techniques but most teaching observed was uninspired “chalk ‘n talk” lecture and many teachers appeared to be demoralized. Even though it may be more expensive, perhaps greater emphasis should be placed on training using a whole school or whole community approach, depending on the training objective. Design Flaw? It appears that, by and large, the UNITY management and its prime contractor, Creative Associates International, Inc. (CAII), are doing a good job of carrying out the Scope of Work (SOW) originally outlined by USAID. That approach emphasized focusing on improving the professional development of teachers and administrators at the primary level through revitalizing the pre- and in-service TDMS system. Although the revitalization of the TDMS may be a necessary condition for improving primary education in the long-run, in the short-run it is mostly improving services for the intermediaries (tutors) within the MOES rather than on improving services to the children or teachers at the school level. Moreover, given the transportation and workload issues facing tutors, it is difficult to see much payoff within the remaining one-year life of the project for the ultimate outcome of program, which is “improved quality of education.” USAID may wish to consider a greater focus on school-level interventions. School Reading Program and Reading Skills Need Much More Attention One of the primary objectives of the Thematic Curriculum (TC) reform is to enhance literacy skills. The evaluation team found few primary schools with books which were age and grade appropriate for children and we did not observe any students reading a book for pleasure or research. The libraries in most schools were abysmal and there were no “learning corners.” Only one library had pupils in it. Without a dramatic improvement in the reading comprehension abilities of Ugandan children, the improvement of teaching and learning in all grades and subjects remains seriously at risk. International research is unequivocal that reading with comprehension explains up to 70% of children’s ability in other subjects and is the highest predictor of success in the job market. Without fundamental reading comprehension skills, and reading programs missing from a large majority of primary schools, the achievement of UNITY objectives, and for that matter the whole educational system, remains at serious risk. 12 | The Mid-Term Evaluation of USAID/Uganda’s UNITY Program, December 2008 Road Maps One of the strengths and hallmarks of UNITY is its ability to respond quickly in a flexible manner to its two masters: USAID and MOES. Nevertheless, a universal criticism of both the Certificate in Teacher Education Proficiency (C-TEP) and REPLICA programs is the somewhat hectic, often last-minute work schedule. Tutors claimed that C-TEP course requirements changed often and implementation partners claimed they had no road map of where they were expected to go and were consulted about implementation issues very late, if at all. Lack of Communication Most senior level staff in the MOES, despite being very supportive of UNITY, believes that communication and consultation about UNITY programs is a problem. The apparent dearth of consultation has programmatic and planning as well as symbolic ramifications. Although the UNITY program is implemented through the MOES working groups, the program directions and decisions are made by a smaller program management committee. The UNITY management committee consists of the USAID officers, the UNITY Program Manager and the Assistant Commissioner for Education Planning. Initially, the Assistant Commissioner has not been vested with the authority to make decisions for the Ministry, which might explain some of the confusion. But, more needs to be done to increase consultation and buy-in from MOES staff. In a similar vein, some implementation partners (subcontractors) would like greater consultation and communication about directions of the program. Up-Front Funding All the subcontractors of UNITY complained about the requirement that they provide up-front funding for their activities before they can be reimbursed for their efforts. Each of the subcontractors found the practice of requiring up-front funding onerous and a strain on their ability to implement their other ongoing programs. Expansion and Replication The general tendency in developmental assistance has been to replicate too rapidly and too shallowly. Managers and operatives in national capitals “plan” the inputs, outputs and timing which may serve political objectives or to them seem reasonable, though they are frequently exaggerated for political expediency and by “target inflation.” Interviews and candid conversations among UNITY staff affirm that program expansion and replication has been unduly quick and large before they were truly ready. They and some Ugandan counterparts believe it may be much preferred to fully develop pilot programs and “deepen quality” in present areas before moving into adjoining districts. What Can Uganda Afford? The GOU apparently invests about 7% of its Gross Domestic Product (GDP) in education. Investing 5% of GDP is generally thought to be good, so Uganda’s LOE for financing education is seen to be very good. Within the MOES budget, primary education apparently gains a fair share of the resources and the spending on categories is reasonable. On the other hand, it appears that the MOES spends from all sources about US $35 per primary education pupil, which is very low by world standards. It appears then that while the overall LOE made by the GOU to fund education is very good in relative (percent of GDP) terms, it is still low in absolute ($ spent per child) terms. This suggests that USAID, MOES, and UNITY must constantly ask the question, what can Uganda afford to maintain in the absence of donor funding? In this light, the TDMS system appears ripe for scrutiny and rationalization. 13 | The Mid-Term Evaluation of USAID/Uganda’s UNITY Program, December 2008 Regional Findings: The evaluation collected questionnaires from 112 tutors in the seven districts of Iganga, Soroti, Oyam, Gulu, Kabale, Mbarara, and Mpigi. The tutors were asked to complete the questionnaire anonymously and told that the researchers were hoping that they would provide candid answers. Three types of questions were presented in the anonymous questionnaire, including: 1) the respondents were asked their opinion or experience using a Likert Scale on simple questions like, “How effective were the workshops in improving your ability to train teachers in new skills?”; 2) respondents were asked to rank areas that they found more - and those they found less - useful; and 3) respondents were asked to supply and rank a list of areas they found most useful or less useful in the UNITY program. Overall, the results reinforced the findings collected in the in￾depth and focus group interviews and showed that the tutors were quite positive about the impact of UNITY, with all tutors ranking the C-TEP course very or modestly useful. II. Lessons Learned Limit Overall Objectives UNITY is a very complex program with a lot of moving parts. The first objective, professional development of teachers and administrators, has seven or eight different sub-objectives alone, depending on how you count. For ease of management, and in the interests of limiting its management units, USAID has combined a number of somewhat desperate set of elements into one project but that has just shifted the management burden to UNITY. Moreover, USAID and UNITY apparently agreed to reduce some regional positions in the management structure originally proposed by the contractor potentially further complicating the problem. If USAID expects to gain some project impact, UNITY should consider focusing on doing a few things well rather than spreading limited resources too thinly. Moreover, UNITY probably needs to focus on a more manageable, high-impact set of interventions. Two Masters The UNITY program is fully embedded within the MOES allowing it to respond directly to MOES concerns in a flexible manner. While flexibility is a strength of the project, the UNITY project has objectives, a performance plan, outputs and impacts that it must satisfy under the USAID contract. On the one hand, the UNITY project is expected to respond flexibly to needs and whims, while, on the other hand, it is expected to achieve very ambitious targets. It appears that UNITY management is frequently required to answer to two masters, potentially muddling project objectives and making implementation difficult and confusing. Under the circumstances there probably is no reasonable way around UNITY responding to two entities, but both USAID and the MOES should ensure that they have the same objectives and that they are well articulated in writing so that confusion is minimized. Project Strategy: Well-Articulated or Sum-of-the-Parts? In some areas, particularly in the C￾TEP and REPLICA programs, UNITY does not appear to have a clear, well thought out road map. It could be argued that UNITY is a complex set of interventions that don’t always lend themselves to strategic planning or that following a blueprint might limit opportunities. Nevertheless, the lack of a strategy or road map has created difficulties for both clients and partners, and reduced the effectiveness of some programs. 14 | The Mid-Term Evaluation of USAID/Uganda’s UNITY Program, December 2008 Value of a Well Sequenced Coordinated Approach Some UNITY project elements do not appear to have a well sequenced, coordinated approach, while others appear to have achieved those goals. When UNITY managed to provide a well sequenced and coordinated approach (e.g., the pilot program for REPLICA) it was relatively successful. By contrast, when programs did not receive a well sequenced and coordinated approach (e.g., CCT tutor training), less progress was made and more problems have surfaced. Focus, Focus, Focus It’s said that the secret to real estate investment is: location, location, location. It very well may be that focus, focus, focus is the key to achieving program impact. The UNITY project has spent the first half of its life developing one of the more complex project designs. Given that solving all of Uganda’s educational problems is beyond the manageable interests of the UNITY program, it would appear that the UNITY program over the remaining life-of-project (or should it be extended) should be modified to consolidate the gains made to date and/or work to guarantee that the best aspects of the program can be replicated by the GOU. Start Small and Build on Success Slowly It appears that the REPLICA pilot program was very successful because it started small and the pilot was manageable. By contrast, the effort to bring REPLICA to scale apparently has been less successful and fraught with problems, despite valiant efforts. UNITY should resist efforts to expand programs too fast or to too many recipients. Limited School-Level Change Perhaps, UNITY’s most significant outcomes to date have been in improving the professionalism of the TDMS system. But, given large tutor to client ratios and widespread transportation problems for tutors, little improvement can be seen in teaching methodologies at the school level. Moreover, in many schools the learning environments are poor, teachers are demoralized, and parents and communities are not meaningfully involved in their children’s education. In any education system, changing teaching practices and/or revitalizing primary education will always be a significant challenge as these changes are difficult to achieve and it will take longer to produce measurable impacts than the other types of interventions. Nevertheless, the UNITY approach may be too indirect to achieve its performance measures and perhaps UNITY should refocus itself on more school-level interventions. Teacher Training and Materials One of the most consistent findings across all districts has been the message from tutors and others that: 1) the training covered too many subjects in too little time, 2) the training did not model good principles of adult education and 3) trainees need more time and mentoring to fully assimilate the training. A similar finding repeated by many is that scores of the recently instituted reforms require learning materials, which are largely absent in schools. Given these findings, perhaps UNITY should refocus its efforts on addressing just one of these issues? Service Delivery or Pilot Project? The UNITY project provides a modest amount of funding to assist the MOES but purports to addressing a large number of educational issues facing Uganda, including all the primary school children in the country. If the UNITY project were to devote all its resources to the targeted seven million primary school students, each student would receive about US $3 in services. Given the financial realities, should USAID and UNITY stop thinking about the project as a service delivery program and redirect the orientation of the program to one that focuses on developing pilot activities for replication and sustainability by the MOES? 15 | The Mid-Term Evaluation of USAID/Uganda’s UNITY Program, December 2008 III. Recommendations by Finding UNITY is Well-Positioned to Assist the GOU • Continue to stress flexibility and responsiveness in UNITY but develop a broader consultation within MOES and among UNITY partners and develop road maps and strategic plans for all activities ensuring that everyone knows their role and responsibilities and what is coming next. Objective One: Professional Development Revitalized TDMS • Review the proposed process but vest Kyambogo University’s Department of Teacher Education with the authority to determine who fails or passes the C-TEP course and receives the certificate; • Work with the MOES and/or other GOU Ministries such as the Ministry of Public Service to determine the salary and promotion implications of passing the C-TEP course; • Revisit the structure and delivery mechanisms of the C-TEP course; • Work with Kyambogo University to identify candidates and possible funding for C-TEP II course; • Develop and implement measures to reform cascade approach to training; and • Drop programs that appear to be nothing more than bold promises (regional centers, Centers of Excellence). Improved Instructional Methods and Materials to Teachers • UNITY has developed some excellent materials but these materials are largely not in the hands of teachers and pupils. UNITY should exploit its strength in this area and ensure that these fine materials are at least in the hands of teachers. • Devise systems where materials are not available for resale on the parallel market. Challenges Facing Implementation of TC • Recognize that problems in the schools impacting TC are not going away in the short to medium term and develop methodologies to solve problems of schools; • Redouble efforts to improve reading instruction and practice and make it central to TC reform; and • Consider addressing the following issues only in UNITY-assisted schools: 1. Class size over 100 in the North, indifference or resistance in the South; 2. Limited materials in Local Languages everywhere; 3. Re-train or remedial training for all teachers in classroom management, continuous assessment, and materials development; 4. Train a critical mass of teachers or practice whole school training; 5. Experiment with regional or district models reflecting special needs of area; 6. Re-evaluate continuous assessment models; and 16 | The Mid-Term Evaluation of USAID/Uganda’s UNITY Program, December 2008 7. Provide teachers with good materials. Expanded REPLICA • Most people interviewed claimed that REPLICA, although a good program, expanded rapidly beyond the capacity for it to be managed effectively. Moreover, it appears that REPLICA has spent much of its efforts at the district and sub-county levels and has not addressed the school level, at least in terms of community mobilization. Although there appears to be a need for expanding the program from 1,500 schools to the 4,000 schools in all the 40 Peace Recovery and Development Plan (PRDP) districts, it appears than REPLICA has much work left undone. Although it does not appear that REPLICA expansion is warranted, any expansion of the program should be carefully considered and done in a phased, manageable roll-out. Objective Two: HIV/AIDS Mitigation Expanded PIASCY • Uganda’s HIV/AIDS mitigation program aimed at primary schools is an imperative that seems to be working well and the next logical step outlined in the UNITY Technical Approach appears to be to extend activities to the Post Primary Education Training (PPET) level and out-of-school children. The evaluation team was not able to review PPET in a junior secondary setting, owing to relatively recent introduction. Given the difficulties in addressing HIV/AIDS issues with out-of-school youth worldwide, UNITY should move very cautiously, if at all, in those areas. Nevertheless, given the success in primary schools, it appears that the expansion of PPET in junior secondary schools should proceed albeit at a cautious pace, especially given the different client group to be serviced. • PIASCY should not abandon efforts in the primary school sector. It appears that there are a number of things that could be revisited or new areas that could be addressed, including monitoring the progress of HIV/AIDS mitigation in primary schools; training teachers who lack confidence to better address PIASCY issues; and improving community participation to support HIV/AIDS orphans. Objective Three: Community Participation Parental and Community Participation in Education • Virtually everyone interviewed maintained that the lack of parental and community participation in primary schools is a serious and limiting factor to achieving improved education and UNITY should work in partnership with DEOs and SMCs to enhance participation and improve school governance; • To overcome the current malaise and inertia, UNITY should consider focusing on a limited number of districts (perhaps as part of the Quality Enhancement Initiative (QEI)) and experimenting with different ways to stimulate participation; and • Some areas for further investigation and experimentation include: 1. Best-practices developed under the UPHOLD program; 2. Training district managers in education management and development of realistic strategic plans; 17 | The Mid-Term Evaluation of USAID/Uganda’s UNITY Program, December 2008 3. Training head teachers in a whole school approach (along with teachers, community members and parents) in school management and the development of time-phased, realistically obtainable, school strategic plans; 4. Providing very limited small grants focused on improving school quality to SMCs to support the school strategic plan; and 5. Promoting visits between schools with good community participation and those without. Objective Four: Educational Policy Educational Policy • UNITY should pursue a more active role in policy analyses, especially in regard to educational policies that impact its scope of work; and • UNITY should consider promoting regional policy discussions among DEOs and education teams (e.g. Gulu Summit). Objective Five: Grants Grants • UNITY should consider use of small grant mechanisms as it was originally intended to support school-level interventions as part of an effort to stimulate parental and community involvement in schools. Currently, the funds intended for school-level interventions have been slated for motorcycle procurement. The motorcycles will improve the mobility of the CCTs and, hence, facilitate the dissemination of improved teaching methods. Small grants to communities should stimulate further parental involvement in schools. The trade-offs in the use of these small grants funds for motorcycles or community grants should be carefully reviewed by USAID. Other Issues Poor Learning Environment • See parental and community participation above; and • UNITY, MOES and/or USAID should consider addressing the health issues of pupils in primary schools, even in a limited way, as well as mounting a “healthy schools” program. District and School Management • UNITY should focus on a limited number of districts to improve the quality of district and school management (perhaps part of the QEI initiative) focused on: 1. Training district managers in education management and development of realistic strategic plans, 2. Use of Education Management and Information Systems (EMIS); 3. Overcoming bottlenecks in the system; 4. Making the TDMS cascade system work; 5. Human resource management; 6. School governance and accountability; 7. Improving working relationships within the district education teams; and 18 | The Mid-Term Evaluation of USAID/Uganda’s UNITY Program, December 2008 8. Methods to enhance community participation. Teaching Methods • Within the same limited districts (above), UNITY should develop a whole school approach to training teachers and providing them with the key materials needed to ensure success in a variety of MOES initiatives, e.g., TC, reading, active learning, improved classroom management. Design Flaw • Having provided the necessary inputs to the MOES to revitalize its TDMS, UNITY should now focus on school-level interventions. Road Maps • Without losing its flexibility and responsiveness, UNITY should develop a road map of future directions that spell-out the roles and responsibilities of all key players. Communication • UNITY should take care to consult widely with MOES staff and its partners and subcontractors. Expansion and Replication • Given the limited USAID budget for education (US $6.5 million/yr) and the myriad of MOES needs, UNITY should move from a program of service delivery for the MOES to a program with limited objectives aimed at developing activities focused on school-level improvements in educational quality. • UNITY’s significant contribution to revitalizing the TDMS system should not be lost and residual activities to making the system work better should be undertaken, including reforming the cascade system of training and possibly facilitating transportation (purchase motorcycles) for tutors in targeted districts, and addressing thorny policy issues like, improving timely flows of funds to schools, streamlining school governance and human resource issues, and linking teacher performance to employment IV. Cross-Cutting Recommendations • Work to consolidate the gains in the most effective programs and ruthlessly jettison least effective aspects of UNITY; • Develop high-quality manuals and materials, lessons learned, policy briefs, and best practices to turn over to the GOU; • Expand role with Ugandan universities and teachers colleges to incorporate C-TEP programs, active learning methods and school-based management into pre-service programs; and • Publish and distribute lessons learned and best practices to all UNITY assisted schools. 19 | The Mid-Term Evaluation of USAID/Uganda’s UNITY Program, December 2008 I. INTRODUCTION A. Evaluation Objectives and Scope This report is the midterm evaluation of USAID/Uganda’s UNITY Project. The UNITY project is the cornerstone of the US Government’s assistance to Ugandan’s education sector and is designed to directly respond to some of the Government of Uganda’s (GOUs) highest priorities in the sector, including: “primary pupils mastering basic literacy, numeracy, and basic life skills”, “strengthen the teaching force” and “strengthen the capacity of the Ministry – its agencies and institutions – to provide leadership and management.” The purpose of the evaluation is to assess the programmatic effectiveness of the UNITY program and garner lessons learned that would benefit USAID, the MOES, UNITY and their partners for future education programming. The objectives of this evaluation are three-fold: first, to examine what aspects of the program are most and least effective and how effectively the program is contributing to the improvement of educational quality in Uganda; second, to provide information to help guide USAID and its implementation partners to make programming decisions for the remaining portion of the program; and third, to help USAID focus on implementation gaps and emerging issues including recommending approaches that will lead to successful replication and likely sustainable outcomes. B. Methodology The evaluation was conducted in November and December 2008 by a team of national and international consultants. Field work was led by the independent consultants but guided by the very able staff of the UNITY program. For example, UNITY provided the evaluation team with lists of schools within districts that had been assisted but, the evaluation team selected the schools to visit. In addition, the PTCs in each district coordinated the district visits rather than UNITY staff so there could be no real or apparent conflict of interest in what the evaluation team saw. The evaluation team used seven basic methods to obtain its findings, including: • Review of project documents, evaluation findings carried out by UNITY or from similar programs, and research and background studies in education (see Annex A for a list of documents consulted); • Observation of UNITY assisted schools, principals, teachers and other educators (see Annex B for a list of schools visited); • Interviews with individuals and officials knowledgeable about the UNITY program (see Annex C for a list of individuals interviewed); • In-depth interviews with key informants very knowledgeable about the UNITY program • Targeted questions in focus group discussions with single and mixed groups of stakeholders; • Anonymous questionnaires administered to tutors (see Annex E for a copy of the anonymous questionnaire); and 20 | The Mid-Term Evaluation of USAID/Uganda’s UNITY Program, December 2008 • Checklists for teaching, learning, and physical conditions to focus observations on the conditions in schools (see Annex D for the classroom checklist). Stakeholders and beneficiaries interviewed included: • MOES officials at the national and district levels; • UNITY staff members at the national and district levels; • UNITY sub-contractor managers and administrators; • National Curriculum Development Center (NCDC); • USAID and other donor (Irish, United Nations Children Fund (UNICEF)) staff; • Kyamboyo University’s Department of Teacher Education; • Elected District Administrators (LC-5) • District Inspector of Schools (DIS); • District Education Offices (DEOs); • Primary Teachers Colleges (PTCs); • Coordinating Centers for Tutors (CCTs); • District Education Committees (DECs); • School principals (Head Teachers); • Teachers; • School Management Committee (SMCs) members; • Parents and parents’ groups; • Non-governmental organization (NGO) members; • Students; and • Members of the community and religious leaders. Within these stakeholder groups, the evaluation team regarded some stakeholders as having more direct knowledge of UNITY than others and focused more attention on those groups. The UNITY program has four major objectives and some of the stakeholders groups have been major beneficiaries of the assistance provided in each major objective. Since those stakeholders may have more insight into the UNITY program some stakeholder groups received special attention in particular areas. The emphasis was: • Objective 1: Professional Development - PTCs, CCTs, DEOs, head teachers • Objective 2: Expand PIASCY - Teachers, Head Teachers, Senior Male and Female Teachers, PTAs and Students • Objective 3: Increased Parental Participation - Parents and PTAs, SMCs, Head Teachers • Objective 4: Education Policy Agenda – District and MOES staff. USAID/Uganda provided the evaluation team with a list of 36 key questions in the initial SOW to guide the investigation. The evaluation team used these questions to develop illustrative interview guides, an anonymous questionnaire, and a work plan. The interview guides were used by the evaluation team as checklists to prompt recall of USAID’s interests but, while the questions were used occasionally during the interview, generally the interviewer asked a series of specific questions to probe the more general areas of the key questions. For example, when probing the area of the appropriateness of the REPLICA model for addressing the educational needs in the post-conflict context the interviewers asked questions like, What do you know about 21 | The Mid-Term Evaluation of USAID/Uganda’s UNITY Program, December 2008 the REPLICA model? What do you think are the educational needs in post-conflict situations? How well do you think the REPLICA model has addressed those needs? What, if any, additional areas need to be addressed in the REPLICA model to improve and/or replicate the model in other areas? A total of 12 schools were visited in seven provinces (Kibale, Mbarara, Mpigi, Iganga, Soroti, Oyam, and Gulu) and Kampala, thereby getting a small sample of schools in the southwest, central, east, and north. The 12 schools represent less than 1% of the total primary schools in the country. The evaluation team interviewed hundreds of stakeholders in these schools and districts, many in focus groups, making it impractical to publish all the names of the persons interviewed. C. Constraints in Undertaking the Evaluation There were several constraints to the evaluation of UNITY. First, due to the timing of the evaluation the team did not always have access to current information. The evaluation was conducted before the second year the UNITY Annual Report was completed. Since the Annual Report is normally the report in which the program contractor summarizes the data and self￾examines its performance over the past year, without that information, the evaluation team found it difficult to analyze the UNITY program. The evaluation team, however, did receive the second year UNITY Annual Report after the evaluation was conducted but before the final report was completed. Despite the extra work, much of the information in the Annual report was incorporated into the final report. In addition, although baseline data were collected prior to commencement of many UNITY program interventions, most of the comparisons between the base year and data collected (mostly in the second year) were not available for review. Given the relatively slow changes one would expect between an intervention and most social and education data, one might expect it to be premature to see any significant changes in data like student performance at this early date. The test developed by UNITY comparing program data with baseline data, however, show modest improvement in student achievement in literacy and math due to the thematic curriculum reform in P1 and P2. Although the evaluation team saw only preliminary snippets of the analysis and much more review and analysis needs to be done, the results appear to demonstrate the program’s positive impact on the quality of education. Second, given the scale and geographic spread of the UNITY project and the limited window of time to conduct the field work, there was insufficient time for the evaluation team to undertake a detailed, in-depth analysis of classroom behavior or of some specific programs and the time spent in each school was sufficient to form impressions only. The presence of outsiders frequently disrupted the classroom and its observation, although lengthy at times, was somewhat distorted. Moreover, the limited window of time made it difficult to interview all stakeholders in every location and some key observers of the UNITY program were unavailable for interview. Third, the sample of schools selected was small (less than 1% of the UNITY schools) and could not be completely random owing to the logistics of visiting schools when visits had to be conducted within the few hours daily when classes are in session. In addition, schools were visited near the end-of-the-school year when schools were preparing for exams and teachers and 22 | The Mid-Term Evaluation of USAID/Uganda’s UNITY Program, December 2008 students were preoccupied. Although a variety of urban and rural schools were visited, the evaluation team did not visit enough schools to enable us to draw many conclusions about regional or district differences. Some programs like, REPLICA, “model schools,” “centers of excellence,” guidance and counseling programs, early childhood development, and girl’s education, were difficult to examine, given the limited opportunities to review them and the somewhat amorphous nature of some of the activities. In addition, schools were obviously expecting the evaluation team and some schools had taken care to be very prepared for our visit. These constraints were taken into account by the team in evaluating the data and forming conclusions. D. Structure of the Report The report is organized in six main sections. Section II provides background information about the USAID/Uganda education strategy and interventions, a brief description of the UNITY programs, as well as a review of GOU’s education policy. The next Section outlines the evaluation findings (accomplishments, strengths and weaknesses, and lessons learned from each of the four UNITY objectives: professional development of the MOES; expanded implementation of PIASCY (HIV/AIDS mitigation); increased parental and community participation in education; and implemented education policy agenda. The fourth section summarizes the implications and lessons learned from the findings. Section V reviews the issues surrounding the replication and sustainability of the UNITY interventions. The report finishes with Section VI which outlines strategies and approaches for UNITY in the remainder of the project. 23 | The Mid-Term Evaluation of USAID/Uganda’s UNITY Program, December 2008 II. BACKGROUND A. Education in Uganda Over the last ten years Uganda’s school system has made some remarkable progress, despite several formidable challenges. The success in Uganda’s education sector can be attributed to the GOU’s prioritization of education as a key investment area and by the partnership between the various stakeholders including the Education Foreign Assistance Group (EFAG), local government, and NGOs. Some of the notable successes have been that primary school enrollment has nearly tripled from about 2.5 million in 1997 to about 7.4 million in 2007, HIV infection rates have been reduced, and government and NGOs have been able to deliver education services to many conflict affected populations. The rapid enrollment resulting from the implementation of the UPE policy in 1997 exerted huge pressure on the GOU’s available resources, especially classrooms, teachers, instructional materials and teacher’s housing, particularly in rural areas. As primary enrollment increased the quality of primary education plummeted as key inputs (teachers, materials) could not keep pace. Although gross indicators of educational quality have improved (pupil/teacher ratio of 52 to 1, pupil/classroom ratio of 79 to 1, pupil/textbook ratio of 2 to 1), these indicators are high by many standards and there are significant disparities between districts, within districts, across income groups and by gender. Indicators in Northern Uganda, for example, are well below the national averages and the evaluation team found pupil/teacher ratios and pupil/classroom ratios well over 100 to 1 and materials of any kind were virtually non-existent. By contrast, ratios in the central and southern regions were often below the national averages. The GOU’s commitment to education is strong. Uganda allocates over 7% of its GDP to the education sector. This percentage exceeds the average of 6 percent of GDP that OECD countries spend on education and is well above the 5% recommended by the World Bank. Moreover, Uganda allocates a substantial and above average portion of the education budget on primary education. On the other hand, Uganda is a poor country and the absolute level of funds spent of each child (unit cost of primary education) is only about US $35 per pupil. The recent implementation of Universal Secondary Education (USE), which was introduced in 2007 to expand access to secondary education as a measure to consolidate and sustain the gains of the UPE program, has placed added strain on the budget for primary education. In addition, the policy of increasing post-primary enrollments at current cost levels would require unsustainably large increases in education expenditures that are unlikely to occur, even in the short-run. Despite the GOU’s financial commitment to education, significant problems in the primary education system continue to exist, particularly in the North, including low completion rates; high dropout rates even with a policy of automatic promotion; high teacher attrition rates; low teacher qualifications; high rates of absenteeism of head teachers, teachers and pupils; poor classroom methodologies; weak capacity for school inspection; and poor educational quality, where Uganda ranks low in international testing, particularly in literacy, science and math. Improving educational quality is a key educational challenge and, given the magnitude of the problem, the GOU has initiated a decentralization process in the hope of spreading the 24 | The Mid-Term Evaluation of USAID/Uganda’s UNITY Program, December 2008 governance, financial, and managerial responsibility for improving education across different stakeholders. The GOU is also attempting to strengthen accountability mechanisms. Under decentralization, SMCs, DECs and district governments are expected to hold schools and teachers accountable but the system of performance appraisal rests with the district service commission and has been implemented inconsistently, if at all, across districts. Unfortunately, district governments are ill prepared to manage the approximately 124,000 primary school teachers. B. Overview of USAID Education Strategy in Uganda Within this setting, USAID/Uganda’s Strategic Objective (SO) is: Investing in People to carry out the goal of fostering a healther, better educated and more productive population. Intermediate results for this SO are: 1) improve the effective use of basic social sector services; 2) increase capacity in the public and private sectors to sustain the delivery of quality social services; and 3) strengthen the policy environment and social support for the delivery and use of these social services. The strategy statement further articulates: “Education is essential for Uganda to become more competitive in the global economy and girls’ education is key to lowering population growth. USAID’s program will continue to focus on improving the quality of basic education, including support to in-service and pre-service teacher training, providing technical assistance for curriculum and materials development (with a new focus on local languages for early primary), peace education in conflict-affected northern Uganda, and providing supplies and equipment to schools in conflict-affected areas.” Some of the major results expected from these investments are: • Enhanced quality of 7 million primary school students; • Better teaching performance as a result of training 50,000 teachers through in-service teacher training and 14,000 teachers trained in pre-service; and • Improved management capacity of 15,000 primary school administrators and district education officials. C. Description of UNITY Program UNITY is the cornerstone and the only activity of USAID/Uganda’s education program. The UNITY project has four central components and one cross-cutting component. The four central components are 1) professional development; 2) expanded implementation of PIASCY; 3) increased parental and community participation in education; and 4) implemented education policy agenda. The grants component to NGOs and schools was intended to enhance the four main components. The first UNITY component, Professional Development of Teachers, was expected to consume 45% of the programs LOE and works through existing structures of the MOES to improve the skills of teachers and those who support them. To accomplish this objective, UNITY undertook a series of major activities and minor (in terms of LOE, not importance) activities. The major activities undertaken by UNITY more fully elaborated in the findings section and description of activities include defining a strategy to improve professional development by re-clustering the existing PTCs into new Regional Coordinating Centers (RCC) of the TDMS and developing the RCC into a “Center of Excellence” fostering innovation in a particular thematic area. UNITY 25 | The Mid-Term Evaluation of USAID/Uganda’s UNITY Program, December 2008 also was to develop a certification for tutors, strengthen the cascade approach to training, support the introduction of the new thematic curriculum reform in P1- P3 through developing and distributing instructional materials to teachers and schools, expand a program (REPLICA) of six integrated topics for primary schools in the conflict and poor areas in the North, and strengthen the administrative support function of the TDMS through the certification of head teachers as well as developing guidance and workshops for district inspectors. Some of the minor activities include expand approaches to improving girls’ education, improving Special Needs Education (SNE), and developing a series of materials and manuals for tutors mentoring teachers, self￾instructional manuals for head teachers, SMCs, and inspectors, books on guidance and counseling, and magazines and newsletters aimed at primary teachers. The second UNITY component, with a LOE of 30%, entailed expanding the implementation of PIASCY. Although previous programs had delivered HIV/AIDS information to thousands of primary schools, thousands more primary school pupils and teachers remained to be informed and counseled, especially in the North, and the huge increase in primary school leavers now entering post-primary institutions lack HIV/AIDS information, guidance from teachers, and positive role models. To address these concerns, UNITY undertook three main activities: 1) expanded and deepened PIASCY in primary schools; 2) expanded PIASCY in northern Uganda working with a variety of partners; and 3) implemented a PIASCY program in junior secondary schools. UNITY’s third component (LOE of 20%) seeks to increase the participation of parents and community members in schools by 1) strengthening the participation parents and community members in community dialogues and in SMCs; and delivering information to parents and communities on school-related issues. The former focused on improving developing “model schools” where community members will design and implement School Development Plans (SDP), encouraging SMCs to improve school and teacher performance and improving accountability, and, using components of REPLICA like performing arts to sensitize and motivate school staff, students, parents, and community members on school matters. The later involves using print and mass media to deliver messages designed to inform and motivate parents and communities on how schools are expected to educate their children. The fourth component of UNITY (LOE of 5%) entails two activities: 1) working with the MOES to identify current policy issues, formulate policies, and implement an agenda that supports the UNITY project; and 2) providing technical assistance to the monitoring and evaluation (M&E) unit of the MOES. The grants program (no LOE) was intended to support “sustained school-community-based initiatives that address local issues.” Two tiers of grants were to be made: one to individual schools and SMCs and another to local NGOs that were working with schools and communities to improve pupil attendance and attract and retain pupils, especially girls, in primary education. The grants were to be made in the thematic areas of PIASCY, literacy, quality improvement, and improved school-community relations. 26 | The Mid-Term Evaluation of USAID/Uganda’s UNITY Program, December 2008 D. Overview of MOES Education Strategy In March 2005, the GOU published its Education Sector Strategic Plan (ESSP) for the MOES covering the fiscal years 2004/5 to 2014/5. The MOES in November 2008 provided a supplement to the ESSP entitled the National Development Plan (NDP), 2009- 2015. According to these publications, the highest priorities of the MOES are: • Increasing and improving equitable access to quality education at all levels; • Improving the quality and relevance of education at all levels; and • Improving the effectiveness and efficiency in delivery of the educational services. • Within these three broad goals, the MOES intends to undertake a series of strategies and interventions. Although there are scores of strategies and interventions outlined in the ESSP and NDP, those that directly relate to UNITY include: • Lowering social-cultural barriers to girls’ attendance; • Expanding and improving primary school facilities; • Improving the instructional processes that lead to increased students’ achievement of literacy, numeracy and basic life skills; • Implementing a TC reform through use of local languages in lower primary (P1-P3 with P4 as a transitional class to English), giving more school time to literacy and numeracy; • Developing grade-level tests of literacy and numeracy to carry out regular annual assessment of progress in pupils’ learning outcomes; • Implementing a QEI in 12 districts; • Strengthening the teaching force by appointing mentor teachers in every school to create a more efficient link between CCTs and teachers; • Providing in-service support to teachers, head teachers and school communities; • Rehabilitating, equipping, stocking, staffing, and maintaining existing PTCs; • Using pre-primary and early childhood development programs to prepare children for primary school; • Continuing to decentralize authority, financing and management of education services; • Tasking CCTs and DEOs to mobilize communities to support schools and monitor school performance; • Strengthening the MOES and the districts to provide leadership and management in school services; and • Ensuring quality assurance and accountability. E. Relevance of UNITY Strategy to GOU Strategy The UNITY program is well aligned with the GOU decentralization policy and with the strategic plans of MOES. Each of the four UNITY components has been designed with GOU policy in mind and UNITY components have changed as GOU policy has changed. The UNITY activities that particularly support the GOU policy and strategies are: increasing the involvement of local communities; improving the governance and accountability of district and schools; improving the learning environments and instructional materials in schools; and upgrading the quality and qualifications of teachers The UNITY program by design and implementation strongly supports the GOU initiatives in education. 27 | The Mid-Term Evaluation of USAID/Uganda’s UNITY Program, December 2008 III. FINDINGS A. Overview UNITY is a three-year Task Order under the USAID ABE-LINK Indefinite Quantity Contract (IQC) in which the firm CAII, with a number of subcontractors, was contracted to implement the four program objectives: professional development, expanded implementation of PIASCY, increased parental and community participation, and an implemented education policy agenda. The UNITY project began on November 7, 2006 and is scheduled to terminate on November 6, 2009. Much of the UNITY project activities, however, are a continuation of interventions initiated under the BEPS project, which was also implemented by CAII. The UNITY project, like the BEPS project, is implemented through the existing MOES structures and its nine working groups, whose chairperson is chosen by the Permanent Secretary (PS), MOES. Each UNITY staff member takes the lead on specific program areas, working with the MOES working group. The members of the working groups provide technical guidance and policy direction to project implementation and the working group chairperson provides quarterly progress reports to the PS. Working within the MOES and applying the UNITY team member’s or specific consultant’s technical expertise and by utilizing MOES working group structures, UNITY aims to strengthen the MOES and ensure that the UNITY interventions are sustainable and owned by the MOES. Within this context, the UNITY project’s progress and strengths and weaknesses will be view separately through the lens of each of the four program objectives. B. Objective 1: Improved Professional Development of Teachers and Administrators at the Primary Level, Both in Pre- and In-Service The first objective of UNITY is designed to strengthen the TDMS and improve the quality of teacher education at the Primary Teacher Colleges (PTC) and at in-service Coordination Centers (CCs) and schools. UNITY works in partnership with the Teacher Education Working Group (TEWG) and the Teacher Education Department (TED) to implement this program. The first objective comprises three core activities: the C-TEP course, the TC initiative, and the REPLICA program. A series of smaller sub-activities and interventions also supports the objective. Objective 1 is a very complex set of activities and has at least 62 actions that are slated to be accomplished in 3 years (please see Table 1, Appendix J). This would imply at least 20 actions every year at the rate of at least 2 activities being completed every month, on the average, just for one objective. While this offers a comprehensive menu and entry points for different players in the education system to benefit, the activities appear to be too many to be achieved in the time available. Moreover, the sequencing of the activities in Objective 1 has not been clearly spelled out, making it difficult to understand how the various activities fit together to create improvement in professional development. As a result, the evaluation focused on those activities that appeared to be key to the program. 28 | The Mid-Term Evaluation of USAID/Uganda’s UNITY Program, December 2008 1. C-TEP Program The overall goal of the C-TEP program is to improve the proficiency and professionalism in primary teacher education and strengthen the TDMS delivery of in-service support to primary school teachers. In a sense, the purpose of the C-TEP program is to re-tool, renew and upgrade teacher trainers and educators in the TDMS system with relevant and new skills, pedagogy and classroom methods and practices. To do this, the C-TEP training program focused on five areas: • Enhanced teacher trainers’ knowledge about current trends in education; • Sharpen teacher trainers’ skills in student-centered learning and teaching methods; • Established the use of Reflective Practice as a tool for teacher trainers and primary school teachers; • Developed and re-established appropriate attitudes among teacher trainers and educators; and • Initiated Action Research methods to improve the quality of education in PTCs and primary schools. The C-TEP program, which was developed and implemented over the first two years of the UNITY project, had a number of notable outputs, including: • Providing education leadership and management training (C-TEP) to 34 lead facilitators, 150 Master Trainers, and 1,030 PTC and CCT tutors; • 57 Master Trainers were trained on assessment protocols; and • The C-TEP course has been reviewed and approved by Kyambogo University (KYU) and 1,030 C-TEP course participants were assessed. These outputs have made a strong first step toward creating the framework and the critical mass of human resources to revitalize the TDMS system. Moreover, virtually everyone who took the C-TEP course and was interviewed, and all those who took the C-TEP course and answered the anonymous questionnaire, gave the course high marks and heaped high praise on the content of the program. Many course participants noted that C-TEP reminded them of forgotten practices and introduced them to new skills but, above all else, re-established professionalism, leadership, and teamwork skills within the TDMS system that will serve the system well for years to come. Despite the strong endorsement of the C-TEP course content by the vast majority of course participants, there was substantial criticism of the organization and management of the C-TEP program. Many of those interviewed complained of an unclear overall direction in the program, shifting course requirements, instructors who were often unfamiliar with the course material, assignments that could not be carried out because resource materials were not available, and no or very limited feedback on assignments submitted. Apparently, the link between UNITY and KYU, which was responsible for implementing the C-TEP program, was weak. KYU staff reported having had “only minimal involvement in the planning of the course” although they “were supposed to implement it”. Although KYU was charged with awarding the certificate for C-TEP, and the C-TEP participants’ assignments were marked at the university, at the time of this evaluation KYU had not released the results of C-TEP participants’ evaluation months after the C-TEP program was completed. Some of these problems are to be expected from a program under pressure to develop and implement C-TEP rapidly but any future applications of the C- 29 | The Mid-Term Evaluation of USAID/Uganda’s UNITY Program, December 2008 TEP course should carefully review these institutional arrangements and the criticisms of the program and take steps to resolve them. The assessment team found a number of other issues than should be addressed. First, UNITY’s strategy to improve professional development is not entirely clear. The strategy obviously entails strengthening the skills of teacher trainers but, it is not clear how the implemented activities for professional development were planned to fit together in a whole. It is not clear, for instance, how the C-TEP training for teacher educators was linked with REPLICA and/or the guidance and counseling and special needs education components. Perhaps more importantly, the UNITY technical approach maintains that it will develop “a strategy for improving the professional development of primary teachers”, but there does not seem to have been a real focus on the primary school teacher who is depended on to effect learning in the classroom. The professional development activities have been largely structured to upgrade the skills of teacher trainers and district supervisors, but little direct support is offered for the classroom teacher. UNITY has certainly made good progress toward revitalizing the TDMS, working through parts of the system to build capacity. However, some parts of the system are not working as effectively as others and need greater input to make them more effective. For example, the teacher training component is working better at the PTCs than it is at KYU and the cascade training approach of training CCTs (who, in turn, are expected to train head teachers who, in turn, are expected to train classroom teachers) is not yet achieving results at the classroom level. UNITY depends on the CCTs at the PTCs to provide mentoring support to teachers in their coordinating centres as a means of strengthening teacher performance. However, some district education teams observed that they are left out of the implementation chain since the CCTs deal directly with schools and do not communicate to the education offices about their activities. This creates an information and coordination gap between district custodians of basic education policies on the one hand and the school level implementers on the other. As one DEO put it in an interview, “the CCTs have the biological data on schools and we have the technical data” and he suggested that any primary education activity that is not owned by the district is very unlikely to succeed. The assessment found that TDMS structures have not been strengthened enough to impact the system, especially for results at the classroom level. The assessment team found that there are far too few CCTs in centres and districts that many of the districts are too large for them to reach all the teachers meaningfully, especially since they are poorly facilitated and often do not have transportation. Some CCTs reported that instead of going out to all the teachers in their schools, they go to some of the schools near their colleges and invite the rest to meet them at the college. Since many teachers do not have transportation either, many miss out on the support they could have obtained. Indeed, there is considerable concern at the MOES about the TDMS’s performance, with some officers questioning the effectiveness of the CCTs. Other MOES officials interviewed explained that the lack of training for the DIS is a critical gap in the system, pointing out that DISs are also classroom teachers and without such training they cannot be a dependable source of instructional support. As a member of the Monitoring and Evaluation Working Group observed, 30 | The Mid-Term Evaluation of USAID/Uganda’s UNITY Program, December 2008 “I am very interested in the reported gaps in classroom instruction. There have been three projects – SUPER, BEPS and now UNITY – all focused on improving teaching. If there has not been any improvement, perhaps there are too many activities and a focus on the classroom teacher ought to be considered.” Working with the MOES, UNITY has made a number of other efforts to improve the TDMS, including the creation of regional centers, developing centers of excellence, reforming the cascade model of training and strengthening support for teachers. 2. Creating Regional Centers In an effort to continue rationalizing the PTC system, UNITY is in the process of working with the MOES to recluster PTCs into a network served by regional centres. The centres would be selected PTCs, which would serve to deliver training and other support for CCTs and other tutors, as well as develop and distribute materials, in a language region. The main way UNITY has assisted this effort has been through providing the C-TEP training program to all PTC tutors. In is hoped that the regional centers will assist the MOES to roll out major national level interventions. To date, the centers have done well in supporting the implementation of the thematic curriculum and the C-TEP training apparently has enabled CCTs to provide some support for classroom teachers. Nevertheless, the PTC system probably needs to undergo further scrutiny and consolidation and careful review of the centers needs to be done to ensure that they have a meaningful role in the new rationalized PTC system before more investments are made to upgrade the centers. 3. Reforming the Cascade Approach to Training UNITY has made some progress toward reforming the cascade model of training. The first step was the development of a cadre of Master Trainers (MT) who were trained to deliver professional development programs to tutors. The tutors will, in turn, mentor classroom teachers in an effort to improve teaching and learning effectiveness. This process is an adaptation of the successful model employed by Aga Khan University and it is hoped that it will affect more intensive support at each level of the cascade. To date, there is evidence that the professional groups at the upper levels of the cascade – some DEOs and DISs, PTC principals as well as PTC tutors – have been more supportive since they participated in the C-TEP course. On the other hand, the professionals at the classroom level have not been as well supported and many teachers cite the lack of support from the CCT as a major weakness in the cascade approach. 4. Developing Centers of Excellence The UNITY project was designed to support for the regional centers in the North as well as establish others in other regions that would be institutionalized by the project with support from the MOES. These Centers of Excellence, listed below, include the older centers established by BEPS, and new centers established by UNITY. The centers are listed along with their assigned area of excellence, as well as the team’s assessment of the achievements made in this regard. 31 | The Mid-Term Evaluation of USAID/Uganda’s UNITY Program, December 2008 Centre Expertise Supporting Partner Evaluation Soroti Leadership and Governance Pincer Group International Going by the data at the two schools visited, which were in the same neighborhood, there is mixed evidence of the college’s expertise or modeling of leadership. At one school, the administration was strong and exemplary and there was an enthusiastic team of teachers, despite the many obvious hardships surrounding the teaching and learning processes. At the other, the administration was weak, with reports of head teacher absenteeism and lack of communication and observation of a demoralized teaching staff. Perhaps expertise at the centre would be reflected at the school level, given the mentoring and coaching role of CCTs. Gulu Peace Education Pincer Group International, Gulu Peace Institute No expertise was exhibited at the college. The resource room at the college had a few charts with statements discouraging child labor and others encouraging teamwork and peaceful resolution of conflict. But there was no display to suggest any Peace Education approaches or activities. Besides, in the tutors’ focus group whether at the college or at school or in the community, tutors reported unresolved discord among staff over cases of student indiscipline. “But if peace education is to be taught in schools, it should start with us and the students here when we are training them. Student teachers should be models also. For example, sometimes when you get students in the wrong … like a student escaping, instead of counseling the student, you are told off. So you are also forced to give up. You keep quiet and do your own things”. The testimony of a staff team that is split over a central issue like discipline is not a positive indication of a college modeling peace education. Kitgum Guidance and Counseling MOES, GC Department of Not in evaluation sample Loro Girls’ Education and GC FAWE The reported achievements that relate to support for girls are not easily attributable solely to girls’ education in the sense that UNITY 32 | The Mid-Term Evaluation of USAID/Uganda’s UNITY Program, December 2008 and its subcontractors describe it as a separate function of the professional development component. For instance, the Senior Woman Teacher at Loro Primary School runs a very active guidance and counseling office where girls are supported to make affordable feminine sanitary wear from basic materials: cotton wool, cotton cloth, plastic paper. She, however, did not make any specific reference to support from the PTC. Rather, she attributed support to volunteer parents and the school administration which supplies materials for the pads, a mattress, lesus and knickers besides pain killers, and time for GC activities. Iganga Special Needs Education XXX A modest demonstration of investment towards support for inclusive education is observable at Bishop Willis Core PTC in Iganga. The SNE centre at Bishop Willis Demonstration School and the activities in it were reported by the teachers to be supported by the PTC and UNITY. UNITY has provided brailed PIASCY handbooks and the TC document. Some real needs of the centre are not met and the teaching of children with visual impairment still meets with acute constraints. For instance, the brailers are in disrepair and may remain so for a very long time since the only source of correction is reported to be in Britain. There is an acute shortage of brailing paper that is so constant that “… children sometimes sit in class without writing”. The portable brailer that was described as functional is actually idle because it is too small to take A4 size paper. This provision therefore cannot ease the observed constraints. Besides, data at NCDC indicates that the SNE teachers are not trained in the delivery of the curriculum, due to lack of funding: UNITY was ready to provide USh 50,000,000 but NCDC’s budget is 132,000,000 and they have no money. The training gap further complicates the SNE situation. 33 | The Mid-Term Evaluation of USAID/Uganda’s UNITY Program, December 2008 Altogether, the principle of inclusion seems not to have been fully incorporated into the UPE scenario. The evaluation team concluded that for a universal primary education system advocating inclusion and implementing a skills-based curriculum focusing on literacy, numeracy and life skills development, these are real barriers to learning for the vision impaired children. Kabulasoke TC EUPEC The ECD center at Kabulasoke has a rich display of local materials for lower primary school children that were developed by students at the PTC. The Deputy Principal felt that the resource room had been effective in attracting neighboring schools and communities. It was also claimed that students from the PTC who teach P1 and P2 have performed well during practice teaching because they use participatory methods in the classroom. Nakaseke PIASCY CAII Not in evaluation sample Kibuli ECD, Muslim Education Madrasa Resource Centre Not in evaluation sample 34 | The Mid-Term Evaluation of USAID/Uganda’s UNITY Program, December 2008 As outlined above, the levels of any observable implementation efforts are very modest at best and varied substantially from center to center. It was the evaluation team’s conclusion that the centers of excellence are more dream than reality and that much more will need to be done to make the centers live up to their image. 5. Strengthening the Tripartite System of Support to Teachers The tripartite system of support includes the teacher education component represented by the teacher education division and KYU of the MOES; the district education supervision component in the offices of DES, the DIS and the DEO; and the head teachers’ association and Uganda National Teachers’ Union (UNATU). Although UNITY has worked with some of these bodies, the targets that the project identified at the beginning have not yet been met. For instance, the project has not made much observable progress towards achievement of the following key targets: • Deliver training to 15,000 head teachers; • Orient 250 DISs to their range of responsibilities; • Strengthen SMCs - many SMCs do not know their roles and therefore do not perform well; and • Organize training with DISs and CCTs to improve working relationships. As a result, it appears that much more work needs to be completed before the tripartite system of support to teachers is fully operational. 6. Provision of Materials The UNITY program has been able to produce a wide variety of materials to support the objectives of the UNITY program as well as supplement the efforts of the MOES. UNITY through its work with National Curriculum Development Centre (NCDC), KYU, Uganda National Examination Board (UNEB) and the 23 core PTCs developed assessment guidelines for the P1 TC and published and distributed 20,000 copies of implementation guidelines to all primary schools and PTCs. By the same means, UNITY has produced libraries for 45 PTCs and distributed books and NCDC’s teacher’s guide, TC and NTB materials including flash cards and charts to a variety of PTCs and schools. Other materials distributed by UNITY include those intended to support various professional development activities. For example, to support REPLICA, the UNITY project distributed 44,670 copies of the peace education teachers’ guide and 56,400 copies of the lower primary learners’ book; 6,929 copies of the leadership and governance book; 16,073 copies of the psycho-social support and care book; 7,070 printed copies of the girls’ education book; 9,070 copies of a book for PALS. UNITY has also distributed HIV/AIDS readers for P4 grades and higher grades and supported Straight Talk Foundation to distribute Teacher Talk, Farm Talk, Tree Talk and Young Talk to appropriate audiences in project schools. UNITY, in support of SNE, has printed copies of PIASCY handbooks and the thematic curriculum in brail. Without doubt, the development and distribution of materials under the UNITY program is very successful and perhaps the most impressive of UNITY’s many outputs. Most of the materials produced by UNITY (or BEPS and re-printed by UNITY) are excellent. By and large, the 35 | The Mid-Term Evaluation of USAID/Uganda’s UNITY Program, December 2008 materials are appropriate for the intended school level and to children and teachers’ needs. In a educational system suffering from a dearth of learning and instructional materials, the UNITY program is an outstanding example of good practice and quality materials and USAID and the MOES should consider maximizing the benefit of UNITY’s comparative advantage in this area. 7. Thematic Curriculum (TC) Uganda has taken a bold step by embarking on a TC reform. It is a bold undertaking because it has required full-scale reorientation of teacher trainers and teachers as well as school administrators and the development of both the curriculum and the support materials, much of which have to be published in local languages. Although NCDC led the process of rolling out the TC reform, UNITY has provided assistance to NCDC in a variety of ways too numerous to fully capture in this report. The objective of the TC reform is to reduce the primary education curriculum from 10 subjects into a limited number of themes. The hope is that the focus on themes will enable teachers to integrate literacy and numercy into most lessons and thereby increase children’s interest and performance in language and math. The reform also uses local language as the medium of instruction in the first three grades (P1-P3) and treats English as a subject. Other features of the TC reform are that the reform introduces Continuous Assessment as a means of evaluating children’s performance, focuses on using child-centered learning approaches, and attempts to use locally-made materials in the classroom. To date, the TC has been introduced nationally into the first two grades (P1 and P2) and will be introduced into grade 3 (P3) in 2009. Within the reform, P4 is regarded as the transition year to teaching in English and the curriculum for P5-P7 will be modified to streamline content and harmonize child-centered approaches with teaching competencies. Within this context, UNITY has worked with the NCDC to support the thematic curriculum reform in a variety of ways. First, UNITY has provided NCDC with a three-person technical assistance team who have skills in literacy, numeracy and life skills and has supported the Ugandan curriculum writing teams in a variety of areas including everything from the design of the themes, the review and editing of the materials, the pre-testing and pilots, and the assessment of the implementation of the TC. Second, UNITY has worked with the TEDs and the regional PTCs to train nearly 40,000 P2 teachers in thematic curriculum. Third, UNITY worked closely with the Department of SNE to adapt the TC to all the special needs sectors. Fourth, UNITY designed a longitudinal study of students to measure the results of the TC implementation on student achievement. As the time of this evaluation, the full results of measuring the performance of children was not available but, early partial results appeared to show that the TC reform was yielding substantial learning gains in children. UNITY has had some notable successes in support of the classroom implementation of the TC. The TC distributed by NCDC were available in all schools visited and were being used by teachers together with the teacher’s guide. Most P1 and P2 teachers had UNITY-inspired materials displayed in their classrooms, a practice that is absent in upper classes. A few teachers had some materials that they had generated to top up those provided by UNITY, MOES and UNICEF, demonstrating resourcefulness and creativity. 36 | The Mid-Term Evaluation of USAID/Uganda’s UNITY Program, December 2008 In all the schools visited by the evaluation team, local languages were the language of instruction (LOI) and children demonstrated an understanding of instructions and questions as well as examples of the content. Virtually all teachers, head teachers, and district education officials reported greater understanding of content by children using local languages. Most teachers demonstrate reasonably effective management of large classes and were sensitive to the needs of children. They know when levels of concentration drop and introduce alternative activity to break the monotony of their talk. Despite the generally good discipline maintained by the teachers, very few teachers practiced child-centered methods in the classroom. Despite these generally positive findings about the implementation of the TC, there were a number of constraints and issues observed. The training for head teachers and teachers, scheduled for five days but often reduced to three days due to circumstances both at the training centers and in the school and college calendar, was felt to be too short and inadequate for exposure to the conceptual and pedagogical issues in the curriculum. One basic constraint on implementation, therefore, is the lack of confidence to face the demands of the curriculum, described by an education officer in Soroti as “a lack of grounding in the principles of the curriculum which are not taught at college”. Other observers interviewed claimed that the development and use of materials in classroom delivery, and classroom management and assessment were also areas where additional training should be offered. A second problem highlighted by many head teachers was the problem of high turnover of thematic curriculum trained teachers. Many claimed that the absence or transfer of teachers trained to deliver the thematic curriculum is disruptive to the implementation of the program. Most head teachers suggested that all the teachers in every school (whole school approach) should be trained to create an adequate supply of TC trained teachers in a school and a critical mass of teachers within a district. Third, most teachers interviewed claimed that there are inadequate materials to support local language instruction. Except in the Mpigi district, the evaluation team did not find any local language teacher’s resources to support interpretation of the curriculum. Moreover, children do not have any local language materials to learn from. In the words of an education officer in Soroti, part of the issue is “a premature introduction of the curriculum – and a very late arrival of teachers’ and children’s books”. The quality of the teacher’s guide for the TC is another constraint. Everywhere that the evaluation team observed classroom teaching, the guide had long fallen apart evidently not because the guide was old but because of poor quality binding. Unfortunately, this is the one document that teachers have available for reference on the curriculum. Instructional materials such as NCDC’s charts and UNICEF’s flash cards and others generated by teachers were displayed in six of the eight classrooms where TC was observed in schools in the North and East. However, in all our observations, teachers taught with no reference to any display materials. Indeed, in most cases teachers had most materials like cards and charts tucked away in a cupboard in the corner where they were kept quite new from lack of use. It would seem that teachers have understood the requirement for displaying materials in the classroom to mean simply that. They recognize no connection between the curriculum content and the content 37 | The Mid-Term Evaluation of USAID/Uganda’s UNITY Program, December 2008 of the materials, even where the latter is evidently relevant to the sections of their lessons. Lack of information constrains distribution of some otherwise useful materials. For instance, while a school in Loro had a plentiful supply of Straight Talk literature, one of those visited in Soroti had very little and another other none at all, the reason being that teachers did not know how to get it. The DPO’s officer explanation that they were supposed to collect copies from the district education office seemed to be news to the teachers. Teachers observed and interviewed in the evaluation reported difficulties in translating the curriculum content into local language both in preparation and in the classroom lesson. Such difficulties were described, for instance, in graphic detail by teachers in a Gulu urban school who said that they have no Acholi Luo equivalent for the scientific concepts of insects, which is strictly a living thing with a head, a thorax and an abdomen with six legs. The teachers feel that their lack of capacity for explaining concepts in the local language is complicated by the absence of learners’ materials that could have supported these learning concepts. Classroom management has become a very demanding aspect of TC implementation especially in the urban North and the East where class size is double or triple the number of 50-55 assumed in the curriculum design. The difficulties are especially marked in Soroti and Gulu districts. Teachers and DISs and DEOs feel that it is very difficult for teachers to implement learner￾centered teaching, which is compounded by the lengthy lessons that are broken down into numerous parts that require the teacher to switch from one skill or aspect to another several times. Partly because of large class size, but also because they know little else, the teachers predominantly use the lecture mode, employing repetition and question-answer techniques to deliver lesson content. In all eight lessons seen on Iganga, Gulu, Soroti and Loro, classroom talk was limited to answering the teachers’ questions and reporting unacceptable behavior. Finally, another constraint on the implementation of TC is in the area of assessment. Although teachers explain that they are trying hard to comply with the requirement for continuous assessment in the TC, they report that they encounter great difficulties. Especially because of the large class size in the North and East but also because they have not had adequate training and exposure and parents prefer norm-referenced approaches to assessment, teachers describe various difficulties with continuous assessment. They find it impossible to perform continuous assessment meaningfully in large classes of 90 or larger found in the North and East where children are restless once the teacher turns his/her attention to anything other than directing the whole class. In addition, they find it difficult to report continuous assessment results to parents who either do not understand the value of this practice, given that they feel that in the long run their children are going to be exposed to the norm-referenced approaches. 8. Expanding REPLICA REPLICA is a package of six integrated components focused on the key and unique educational challenges in the conflict-affected areas in North and Northeastern Uganda. The six components of REPLICA are: 1) peace education; 2) leadership and governance; 3) psychosocial care and support, 4) community integration and participation; 5) promotion of girls’ education; and 6) performing arts and learning in schools. REPLICA was to be expanded from a pilot program affecting 30 schools to cover all 1,700 government primary schools in 13 districts (Gulu, Amuru, Kitgum, Pader, Lira, Apac, Dokolo, Amolatar, Oyam, Kaberamaido, Soroti, Amuria, and 38 | The Mid-Term Evaluation of USAID/Uganda’s UNITY Program, December 2008 Katakwi). Four PTCs (Soroti, Loro, Gulu, and Kitgum) were also included as core PTCs in the program. The expansion of REPLICA was subcontracted to two groups. Pincer Group International was subcontracted to deliver psycho-social support and care, peace education, guidance and counseling, use of performing arts in learning and school leadership and governance. Pincer was expected by USAID to top-up the efforts of the UNITY project. The second group contracted to work with the Pincer Group was the Forum for African Women in Education (FAWE). FAWE was subcontracted to deliver support for promoting girls’ education and post-primary PIASCY in selected districts. Some of the major accomplishments of REPLICA (Pincer/FAWE) have been: • Conducting “public engagement” in all REPLICA districts; • Helping to develop education ordinces in the REPLICA districts; • Conducting the Peace Recovery and Development Program (PRDP) Summit; • Supporting the Development of Centers of Excellence in the four core PTCs; • Providing school-based support supervision and monitoring; • Providing school-level training in the promotion of girls’ education, especially to improve retention, performance, and completion; and • Holding public engagements and collaborating with community members to raise awareness and build support for girls’ education. In addition to the outputs above, one of the major successes of the REPLICA/UNITY program appear to be the use of materials. REPLICA has distributed materials including the peace education books for teachers and learners, a leadership and governance book, a girls’ education book, a psycho-social support and care book, and a Performance Arts and Learning in Schools (PALS) book. Most of the materials are well received by teachers. For example, the peace education books are in active use as a part of the teachers’ manuals package in the Soroti and Iganga classrooms. Teachers claim that they refer to it for advice on how to handle problems with children with peaceful relations issues. Children in P2 at one school in Soroti are allowed to borrow copies of the book and many carried them in their improvised school bags. The teacher explained that they read the book at home with siblings. We observed that the copies also served as writing support in the classroom where children sat on the floor and had to write in their laps. The schools in Gulu district were not using the peace materials actively, although in one school copies were available in the head teacher’s office and, in the focus group discussion teachers, indicated that there were peace clubs in the school. Loro teachers had the book in class and maintained that they took tips from it for lessons on teaching peace. Beyond the availability of the project materials, it was difficult for the evaluators to observe REPLICA’s presence on the ground. There can be any number of reasons posited for the apparent absence of REPLICA. First, many of the REPLICA activities are somewhat ephemeral and it is difficult to see evidence of their application, especially ex post. Second, this evaluation visited a random and limited number of schools (five schools) in PRDP areas and those schools may not have been the best examples of the REPLICA program. Third, as the REPLICA subcontractor pointed out, programs tend to merge into each other and teachers and others interviewed may not clearly differentiate REPLICA from other programs. Fourth, many of the 39 | The Mid-Term Evaluation of USAID/Uganda’s UNITY Program, December 2008 REPLICA activities are not different from what teachers are regularly trained to do in the delivery of the curriculum or what they are charged to do as Senior Women’s Teachers (SWTs) or Senior Men’s Teachers (SMTs) by the MOES. Finally, the REPLICA program coordinator and his team explained that they have been focused on implementing the program at the district and sub-county level and not yet at the school level. For these and other reasons, the evaluation team believes that a much more in-depth evaluation of the REPLICA program should be undertaken, especially if USAID or the MOES intends to invest additional resources into the REPLICA program. The interviews we conducted with teachers, head teachers, and district officials suggest that the REPLICA program has achieved mixed results. Teachers find it difficult to integrate peace education in the curriculum and claim that the lessons are long and difficult to deliver in the classroom. In Gulu town, teachers felt that the ideas introduced by REPLICA “… were not really new”. They also pointed out that they were not implementing the ideas partly because of a lack of support from the REPLICA team, which seemed to them to “prefer to run the program in the district schools because they think these are the ones that have been affected by the war, and yet we too in town have been equally affected since people used to flock here for safety.” However, the district school teachers did not seem to use the peace education books actively and, although they made reference to the presence of peace education clubs at the school, they could not describe the membership or the club activities. In the area of psychosocial care and support, there also is a mixed picture. Nearly all schools have small signs throughout the school providing positive messages about a variety of topics including peaceful resolution of issues, HIV/AIDS, social conduct between the sexes, and girls’ education. Some teachers have been trained to deliver guidance and counseling (G&C) at school levels with the function placed particularly in the hands of SWTs and SMTs. The G&C function is quite effective in some schools, with the SWTs/SMTs allocated a room and material provisions for supporting especially girls who have extra challenges, like managing menstruation. In a Soroti school a SWT explained that since the time of training in 2007, they “…have been able to solve many problems especially because the children interact freely with the teachers. When a child has any problem, he goes to any teacher that he feels like. Also the children interact amongst themselves. Teachers also help the school and the parents to come together if there is any problem.” In addition, the school had a G&C room with some provisions for a girls’ changing room, a mattress, an anonymous box for suggestions and questions and a pot of drinking water. The head teacher pointed out the school’s peace tree, an orange tree that he planted himself near the head teacher’s house. But at another Soroti school, teachers in the focus group discussion all felt that REPLICA is a useful program but they described gaps in their skills that make the implementation of REPLICA programs difficult. For example, they indicated training needs for implementing G&C, promotion of girls’ education and support of SNE. The SWT here 40 | The Mid-Term Evaluation of USAID/Uganda’s UNITY Program, December 2008 described her challenges including the lack of provision for space to conduct G&C activities and the identification of children with special problems. Education supervisors at the district level attribute the implementation challenges to what was called “too much come and go” with the training “too little and too isolated”. An informant argued, “You can’t train someone once and affect an individual. There should be training at the start of a program, in the middle for coordination, and towards the end to equip them for departure and independent implementation and training of colleagues.” In spite of the observed implementation challenges, most LCV chairpersons hold a positive perception of REPLICA, teachers, DEOs and head teachers, all of who wish it to continue. Presumably, this is in part because of the REPLICA dialogue at the district level, which has resulted in recognition of the value REPLICA and its focus on peaceful conflict resolution. By all accounts the Inaugural Regional Forum on Education (the Gulu Summit) funded by UNITY was a resounding success. UNITY and REPLICA worked with the MOES and local government leadership in joint planning for the PRDP Summit, whose purpose was to publicize the “education renaissance” in Northern Uganda and develop a blue print for funding educational interventions in the PRDP districts. With an impressive turnout of 286 district stakeholders, including the Ugandan Prime Minister, Ministers of State, and the President of the Republic, participating in the summit, UNITY was able to gain substantial commitment to support educational reconstruction and development for the PRDP districts. C. Objective 2: Expand Implementation of PIASCY The PIASCY program is a strategy initiated by the GOU aimed at mitigating the spread and impact of HIV/AIDS in the primary and post-primary education pupils. UNITY is providing assistance to the MOES to expand the implementation of PIASCY nationwide by assisting in the delivery of HIV/AIDS training to teachers and facilitating the procurement of HIV materials and readers for almost 10,000 primary schools. The program aim is to contribute to a generation of people with knowledge and facts for HIV/AIDS prevention and positive living. The program provides customized information that is age-appropriate for students in upper primary and lower secondary. The PIASCY component of UNITY has achieved some remarkable outputs. In the Post Primary sub-sector, UNITY, working with the MOES, focuses on students that are, or are approaching, adolescence and has: • Printed more than 85,000 PIASCY student handbooks on HIV/AIDS; • Provided about 22,000 teachers guides; • Printed more than 82,000 G&C materials and teachers’ resource books; and • Provided training for a core team of 40 MTs, who in turn trained more than 1,800 head teachers and teachers. At the primary level, UNITY, taking over from another USAID project (UPHOLD) that concluded, has: • Provided more than 59,000 HIV Reader Kits (each Kit contains 12 assorted story books) for nearly 10,000 primary schools in 64 districts; and 41 | The Mid-Term Evaluation of USAID/Uganda’s UNITY Program, December 2008 • Trained more than 9,770 teachers on how to integrate the reader content into the school curriculum. At the school level, implementation of PIASCY activities has been left to senior women and senior men who are also teachers with a full teaching load. Other teachers do not really see PIASCY as their responsibility. The main sensitization avenues used by most schools are school assemblies, message posted around the school and “open days”. HIV/AIDS awareness has been dramatically increased and, as a result, the entire population of schools we visited has embraced PIASCY messages. Virtually all schools actively disseminate HIV/AIDS prevention messages within the school compounds, offices and classrooms. The effort to communicate HIV/AIDS messages is seen as so critical that in one Soroti school, where metallic plates in the compound were stolen, the administration decided to post PIASCY messages written in chalk on slates that are hung in trees during the day and taken into the office for the night. Some of the effects of the PIASCY program noted are that children and teachers are reported to be much more aware of HIV/AIDS; to share knowledge on issues; to open up on their status, and not to stigmatize affected peers. Teachers claim that the PIASCY handbooks - an output of the BEPS III Project and disseminated by UPHOLD - are a source of tremendous support, based on the accuracy of the content. They argue that the perceived authority of the books equips them with the confidence they need to communicate messages on sexuality and sexual behavior to the children in spite of the traditional perception of these as taboo subjects. All the senior women and men interviewed appreciate the handbooks for teachers because they are factual and well organized. Many claims are made about the impact of PIASCY that the evaluation team could not verify in the limited time of the evaluation. For example, it is said that fewer girls are getting pregnant and risky sexual behavior has reduced among pupils and even teachers. Some have also claimed that the PIASCY guidance and counseling has improved pupils’ discipline. SWTs also report fewer early or forced marriages, fewer dropouts due to pregnancy, and increased retention of girls in school. No statistical evidence was available for the evaluation team to verify these claims. Nevertheless, there is a strong and overwhelming conviction among both male and female teachers that these claims are true and that the PIASCY program has resulted in the desired change in knowledge, attitudes, and behavior in schools and, to some extent, in the associated communities. Overall, the PIASCY program appeared to be a well-run program making a substantial impact. There were, however, some issues raised that may need to be addressed. Not all the teachers have been trained in the implementation of PIASCY. Therefore some feel a lack of confidence in addressing HIV/AIDS issues. Given the limitations in teaching of reading in schools, including the absence of school reading programs, the HIV/AIDS readers are not actively read although teachers are aware of their availability in school. Head teachers of some of the schools we visited complained that they did not have any funds allocated to PIASCY so they could not initiate the environment for specific activities that they would have liked. Moreover, teachers report that parents’ knowledge and support to PIASCY are minimal and require more attention. Some community members vandalize PIASCY materials because they need the boards and slates or metallic plates to make hoes or chairs, and they do not appreciate the importance of such 42 | The Mid-Term Evaluation of USAID/Uganda’s UNITY Program, December 2008 materials to the school community. Some parents do not model safe behavior. Where these issues exist they lead to weak school-community partnership and weak HIV/AIDS mitigation. The SWTs and SMTs recommended a review of the PIASCY handbooks to make them more comprehensive. For example, the books assume that primary school children are not sexually active, that they are not exposed to HIV and that they do not suffer from AIDS. They would like more guidance on caring for the HIV/AIDS affected and keeping schools safe for children who live with infected peers and/or who need their support. A number of senior women and men said they had to deal with children who were infected with HIV at birth and whose status has been revealed by their parents or guardians on condition that they keep it secret. Some CCTs pointed out that training only senior women and senior men teachers may need to be revised because the practice was placing a heavier load on them than they can handle efficiently. They recommended that all teachers be trained so that ownership and responsibility of implementing PIASCY activities expands to all teachers and the whole school. They also suggested that PIASCY messages be integrated in the games pupils play to reinforce such messages through fun. Sensitization of parents on children’s rights, especially for the girls, was also considered an urgent need for supporting school-based activities for mitigating HIV/AIDS. D. Objective 3: Increase Parental and Community Participation Parental and community participation in education in Uganda is weak. To re-establish and strengthen the partnership between community/parents and schools, UNITY has designed a multi-pronged strategy to support community integration in education. These approaches include: public engagement and community dialogue, consultative meetings with stakeholders, using performing arts as a tool for communication and advocacy, and use of multimedia to inform parents and communities of their roles and responsibilities in education. In addition, UNITY maintains that it has mainstreamed community involvement in all its activities, especially REPLICA, Early Childhood Development (ECD). As seen earlier, one of the six components of REPLICA is community integration and participation. The REPLICA component involves a set of wide-ranging public engagement activities with a myriad of local stakeholders at the district and county level. REPLICA claims “significant milestones” as a result of those efforts, including: 1) the development of county by￾laws (Educational Ordinances) to regulate stakeholder participation and involvement in education; and 2) public engagement in the PRDP districts, which culminated in the PRDP Summit in Gulu that developed a strategic plan and blue print for funding education in the north. In the area of ECD, UNITY, through the Madrasa Resource Center (MRC), is working with SMCs and Community Resource Teams (CRTs) to involve parents and community members in identifying, planning and implementing ECD activities. MRC trained nearly 500 SMC members and more than 1,500 parents in support of ECD. UNITY has also engaged parents and community members in the districts of Kitgum and Masaka districts by using MTs to train SMC and PTA members on their roles and responsibilities, used community dialogues to increase community participation in schools, and improved school accountability through training and community dialogues. 43 | The Mid-Term Evaluation of USAID/Uganda’s UNITY Program, December 2008 Overall, the evaluation team saw very little real parental or community involvement in schools. There were, however, some notable successes. One of the most effective community involvement activities observed in the Western and Central regions of Uganda was the MRC efforts to put up a materials resource centre to support ECD at Kabulasoke PTC. The ECD resource centre at Kabulasoke PTC has a rich display of local materials, for lower primary and nursery children, that have been developed by students at the PTC who were trained by MRC. The Deputy Principal in charge of outreach at Kabulasoke felt that the resource room has been effective in attracting the attention of neighboring schools and communities that have come to learn how to make some of the local materials on display. It was also said that students from the college who teach P1 and P2 have performed well during teaching practice because they use participatory methods in class. During focus group discussions with SMCs and parents in the Central and Western regions, the groups articulated their roles, including: • Working to establish and keep good relationship between the teachers and parents; • Sensitizing fellow parents about meeting pupils’ needs, such as supplying scholastic materials and packing food for those in lower primary; • Making sure programs run smoothly in the school; • Implementing school programs, e.g. working to increase the number of pupils; • budgeting for the school; and • Mobilizing money and monitoring school discipline, teaching and the status of latrines. They noted, however, that their enthusiasm is often dampened by high ranking politicians who discourage parents from contributing to their children’s learning in UPE schools. In the North and East of Uganda, the evaluation team found that efforts for increasing community participation in education have had very limited impact at the school level. Teachers, SMCs and school officials, appreciate the contribution of some parents/guardians who, despite their poverty and their own lack of education, respond to the appeal to support their children’s schooling. But, they note that this group is small and that most parents are not aware of – or ignore- their roles and responsibilities. The teachers, SMC members, and school officials report a lack of cooperation among the majority of parents and guardians. While some parents agree to contribute stationery and a little money (USh 500 - 1,500) towards feeding, the majority adamantly refuse to contribute, often encouraged by politicians to simply send their children to school. Observation in classrooms and on the school compound and discussion with teachers reveals that some children report to school sick and hungry, or barely clothed. At one school in Soroti, children in a P1 class were all visibly infested with ring worm. Some had malaria as did a number in a P2 class in Iganga. While many children exhibited great efforts in obtaining paper or books and pencils – some writing with two inch long pencils on tiny shabby pieces of paper - a few report without any writing material, and are obviously not ready to learn. The conditions are observably worse in schools where the leadership is weak. Given that efforts for increasing community participation in education were generally reported to have led to low levels of change, people interviewed made various recommendations for 44 | The Mid-Term Evaluation of USAID/Uganda’s UNITY Program, December 2008 increasing parental and community participation. They stressed the need for a clear policy regarding feeding and caring for pupils at school so that parents know what they are supposed to contribute in school, observing that the USh 500 given to each individual pupil by Government is not enough to take care of the child at school. They also recommended enforcement of timely release of UPE funds since the irregular disbursement is one of the reasons why head teachers find it difficult to manage and plan the school year and why teachers are discouraged. They recommended gender balance in the posting of teachers and that in classes with large numbers of students there should be at least two teachers per class. In addition, they pointed out the need for workshops and social marketing programs for parents about their responsibility for educating their children and for monitoring the performance of schools. Both school officials and SMC members underlined the need to curb political pronouncements and interference in schools that has the effect of thwarting parental and community involvement in schools. E. Objective 4: Monitor and Assess Effectiveness of Policy Implementation In the area of policy implementation the UNITY project supports the MOES Educational Planning Department (EPD) in policy development, policy implementation, and policy tracking. UNITY works with EPD to assess the impact of the ESSP and then develop policy to address the issues that need resolution. According to recent UNITY documents, the priority policies currently under review, implementation and tracking are: Local Language and TC; Gender; Professional Development and Schemes of Work; ECD, and G&C. In collaboration with EPD, UNITY provided technical and budget support to track the implementation of three policies: TC, Automatic Promotion, and Local Language policy. The purpose of the policy tracking was to: assess the efficiency of each policy, seek stakeholder perceptions about each policy, identify gaps and inhibiting factors impinging on successful policy implementation, and provide practical policy recommendations to enable top management at MOES to make informed decisions in these areas. Inexplicitly, UNITY did not provide this tracking report to the evaluation team, even though the work was apparently carried out and finalized six months ago. We understand, however, that the findings of the study reinforce the findings of this evaluation. For example, the findings of the tracking study found that there are a number of systemic issues affecting the implementation of the TC reform, including: inadequate learning and teaching materials, high pupil/teacher ratios, complexities applying the continuous assessment techniques, difficulties translating English into Local Languages, and inadequate training of teachers. Moreover, the study showed that the policy of automatic promotion is in direct contradiction to the competency-based approach of the TC reform and that there were inadequate local language reference materials and a limited number of local language instructors. All of these findings are exactly those we found in our assessment six months later. Perhaps, a lesson learned here is that policy reform and implementation is difficult and that it is an important issue that needs constant attention and it would seem that UNITY should redouble its efforts to put good policy in place. It is evident that the education leadership at the district level and in some schools is very good. DEOs, DISs, Municipal Education Officials and some head teachers know what is good for basic education, are aware of the local circumstances in 45 | The Mid-Term Evaluation of USAID/Uganda’s UNITY Program, December 2008 their districts and are very articulate on government policy as well as on the input required to improve primary education delivery. But some adverse forces are working against pronounced primary education policy and good sense. While the various roles of different stakeholders were clearly documented as early as 1996 and repeatedly in subsequent years, education supervisors at the district report “politics has overtaken education, with the technocrats being shut down”. Prior to the introduction of the policy of UPE, communities worked to ensure learner-friendly school environments and supported the school leadership to supply children’s and schools’ needs – feeding, stationery, latrines, and staff accommodations. However, many of those interviewed argued that politicians blind their constituents to their children’s educational needs and the importance of parental and community participation in schools by claiming that “the government will do it all.” F. Grants Although not an Objective as such, the original UNITY strategy maintained that UNITY would manage a $2 million small grants program. The intent of the grants program was to support sustained school- and community-based initiatives that address local issues. The grants program was to result in a large number of communities engaged with schools. The idea was to make two tiers of grants: one to individual schools (SMCs) and another to local Non Government Organizations (NGOs) that are already working with schools and communities to improve pupil attendance and attract and retain pupils, especially girls, throughout the primary cycle. Grants were to be made in the four thematic areas of PIASCY, literacy, quality improvement, and improved school-community relations. Although grants have been made to NGOs (FAWE, Muslim Resource Center), it appears that a decision was taken early on in the project to abandon the grants program because it was seen as a time-consuming, convoluted process that might yield little and because the MOES experience with a similar small grants program was seen as unsuccessful. As a result, no grants have been made to schools under the UNITY program. Without second-guessing the decision to abandon the small grants to schools program, it is important to note some important results of that decision. First, very little of the UNITY project components provide any direct assistance to schools. Almost all the UNITY program components are one or two levels removed from the school. For example, nearly all the activities in component one (professional development) are aimed at the MOES personnel who do not work in the classroom (tutors, head teachers, district officials, SMCs). To be sure, the UNITY professional development activities have created the framework to assist classroom teachers and, in the medium- to long-term classroom teachers will benefit from the assistance but, UNITY is doing little to assist the largely demoralized teaching corps. Second, the aim of the small grants program was to provide some start-up money to engage parents and communities in their schools. The other techniques that UNITY is utilizing to engage communities, such as multimedia campaigns, public engagement and community dialogue meetings, using performing arts as a tool for communication, and consultative meetings with stakeholders, just do not seem to be having much of an impact. When small grants are combined with some of these other techniques and community leaders are trained with teachers so that there is a common purpose and strategy, dramatic changes can occur and be sustained in schools. Given the poor results observed in community involvement to date, perhaps UNITY, MOES and USAID should revisit the use of small grants. 46 | The Mid-Term Evaluation of USAID/Uganda’s UNITY Program, December 2008 IV. LESSONS LEARNED Limit Overall Objectives: UNITY is a very complex program with a lot of moving parts. The first objective, professional development of teachers and administrators, has seven or eight different sub-objectives alone, depending on how you count. For ease of management, and in the interests of limiting its management units, USAID has combined a number of somewhat disparate set of elements into one project but that has just shifted the management burden to UNITY. Moreover, USAID apparently reduced the management structure of UNITY originally proposed by the prime contractor potentially further complicating the problem. If USAID expects to gain some project impact, UNITY should consider focusing on doing a few things well rather than spreading limited resources too thinly. Moreover, UNITY probably needs focus on a more manageable, high-impact set of interventions. Two Masters: The UNITY program is fully embedded within the MOES allowing it to respond directly to MOES concerns in a flexible manner. While flexibility is a strength of the project, the UNITY project has objectives, a performance plan, and outputs and impacts that it must satisfy under the USAID contract. On the one hand, the UNITY project is expected to respond flexibly to needs and whims, while, on the other hand, it is expected to achieve very ambitious targets. It appears that UNITY management is frequently required to answer to two masters, potentially muddling project objectives and making implementation difficult and confusing. Project Strategy: Well-Articulated or Sum-of-the-Parts? In some areas, particularly in the C-TEP and REPLICA programs, UNITY does not appear to have a clear, well thought out road map. On the other hand, UNITY is a complex set of interventions that don’t always lend themselves to strategic planning. Nevertheless, the lack of a strategy or road map has created difficulties for clients and partners and reduced the effectiveness of some programs. Value of a Well Sequenced, Coordinated Approach. Some UNITY project elements do not appear to have a well sequenced, coordinated approach, while others appear to have achieved those goals. When UNITY managed to provide a well sequenced and coordinated approach (pilot program for REPLICA) it was relatively successful. By contrast, when programs did not receive a well sequenced and coordinated approach (CCT tutor training), less progress has been made and more problems have surfaced. Focus, Focus, Focus. It is said that the secret to real estate investment is: location, location, location. It very well may be that focus, focus, focus is the key to achieving program impact. The UNITY project has spent its first half developing one of the more complex project designs. Given that solving all of Uganda’s educational problems is beyond the manageable interests of the UNITY program, it would appear that the UNITY program over the remaining life-of-project (or should it be extended) should be modified to consolidate the gains made to date and/or work to guarantee that the best aspects of the program can be replicated by the GOU. Start Small and Build on Success Slowly. It appears that the REPLICA pilot program was very successful because it started small and the pilot was manageable. By contrast, the effort to bring 47 | The Mid-Term Evaluation of USAID/Uganda’s UNITY Program, December 2008 REPLICA to scale apparently has been less successful and fraught with problems, despite valiant efforts. UNITY should resist efforts to expand programs too fast or to too many recipients. Limited School-Level Change Perhaps UNITY’s most significant outcomes to date have been in improving the professionalism of the TDMS system. But, given large tutor to client ratios and widespread transportation problems for tutors, little improvement can be seen in teaching methodologies at the school level. Moreover, in many schools the learning environments are poor, teachers are de-moralized, and parents and communities are not meaningfully involved in their children’s education. In any education system, changing teaching practices and/or revitalizing primary education will always be a significant challenge as these changes are difficult to achieve and it will take longer to produce measurable impacts than the other types of interventions. Nevertheless, the UNITY approach may be too indirect to achieve its performance measures and perhaps UNITY should refocus itself on more school-level interventions? Teacher Training and Materials One of the most consistent findings across all districts has been the message from tutors and others that: 1) the training covered too many subjects in too little time, 2) the training did not model good principles of adult education and 3) trainees need more time and mentoring to fully assimilate the training. A similar finding repeated by many is that scores of the recently instituted reforms require learning materials, which are largely absent in schools. Given these findings, perhaps UNITY should refocus its efforts on addressing just one (or possibly both) of these issues? Service Delivery or Pilot Project? The UNITY project provides a modest amount of funding to assist the MOES but purports to addressing a large number of educational issues facing Uganda, including all the primary school children in the country. If the UNITY project were to devote all its resources to the targeted 7 million primary school students, each student would receive about US $3 in services. Given the financial realities, should USAID and UNITY stop thinking about the project as a service delivery program and redirect the orientation of the program to one that focuses on developing pilot activities for replication and sustainability by the MOES? 48 | The Mid-Term Evaluation of USAID/Uganda’s UNITY Program, December 2008 V. COST-EFFECTIVENESS, SUSTAINABILITY AND REPLICATION USAID raised three questions which were of special interest and that are addressed in no other part of the report. The three questions were: • Was the project implemented in a cost-effective manner? • What specific program elements are sustainable and how will they be sustained? • Does the project posses the organizational capacity to scale-up the program and expand to all PRDP defined districts? Since these questions are to some degree or another related to each other, each question is answered here in turn. Cost-Effectiveness The question, “Was the project implemented in a cost-effective manner?” has conjured up a variety of notions about different analyses USAID might want ranging from benefit/cost analysis, to the social rate of return of the investment, to the cost-effectiveness of various investment alternatives. In discussions with USAID, it was decided to keep it simple and provide an appraisal of weather or not the UNITY project has been implemented in an effective manner, especially in regard to costs. Despite the guidance from USAID, initially we attempted to conduct an analysis of the cost￾effectiveness of the UNITY expenditures, i.e., compare the unit costs of UNITY outputs with the unit costs of comparable outputs from other programs. Although we contacted a number of organizations (FAWE, Irish Embassy, UNICEF, Straight Talk Foundation) we quickly found that the data available was not comparable to UNITY data or was impossible to collect. Thwarted, we returned to the USAID question and attempted to determine whether or not UNITY is investing its funds wisely. The UNITY program is relatively complex with four major objectives but for ease of calculation we selected two main outputs to analyze: materials and training. After carefully establishing, with the help of UNITY staff, the actual costs incurred for each of the variety of materials produced and training programs delivered, we calculated the cost per unit of output for each item. In general, we found that the UNITY costs were very reasonable and similar to costs that might be incurred by any contractor, worldwide. For example, the daily cost of training a person in the C-TEP program was $82, while it was $71 in the REPLICA program, $181 in the PIASCY (PPET) program, and $28 in the FAWE program. An important consideration, however, is the duration and venue of the training program. For example, the PIASCY training was a 3-day program, making the total cost of the training per participant $543, while the C-TEP program was a 30-day program, making the total cost of training a tutor in the C-TEP program $2,460. 49 | The Mid-Term Evaluation of USAID/Uganda’s UNITY Program, December 2008 We found much the same thing in the area of materials production, that is, UNITY costs were very low-cost. For example, the majority of materials were produced for less than $2 per publication and only a few publications (C-TEP Book of Readings, C-TEP Training manual, and the HIV Readers) cost more than $5. Sustainability The question asked by USAID is, “What specific program elements are sustainable and how will they be sustained?” For the purposes of this review, we will ague that there are at least two kinds of elements of sustainability: financial sustainability and institutional sustainability. We will define financial sustainability to mean that UNITY programs, approaches, and practices are implemented by stakeholders using their own resources, while we will define institutional sustainability to mean the capacity to continue the education reforms embodied within UNITY beyond the life of the program. It appears all components of the UNITY program are committed to developing an institutionally sustainable program. There are three basic methods by which UNITY expects to develop institutional sustainability. The first method for achieving institutional sustainability is to create a cadre of human resources within the education system that can carry out the educational reforms. UNITY has been able to embed within the system (districts, counties, schools, clusters, communities, civil society) a body of well-trained personnel. For example, as of December, 2008, UNITY maintains that it has trained more than 1,000 tutors, 17,000 pre-service teachers (in TC), 100,000 teachers, and 19,000 officials and administrators. Although these groups of people have been trained for varying lengths of time, the majority of them have become agents of change for aspects such as, TC, accountability and transparency within the school system, or active-learning methods. A second way that UNITY expects to institutionalize and sustain the education reforms is to develop and leave behind key elements of the UNITY program. By the end of the project, UNITY expects to leave behind a raft of strategic plans; improved systems and tools for districts and schools, training modules; user-friendly manuals; and teacher training institutions with better training techniques and methods. Perhaps, UNITY’s third, and key, strategy in regard to institutional sustainability is to work through and within the institutional structure of the MOES. Working within the decentralized TDMS and the MOES working groups, and offering the UNITY team’s technical expertise and the occasional use of consultants, UNITY provides system-strengthening support that should ensure ownership and sustainability. All three of these elements of institutional sustainability appear to be in place and working to ensure that the capacity is in place to continue the UNITY education reforms beyond the life of the program. 50 | The Mid-Term Evaluation of USAID/Uganda’s UNITY Program, December 2008 If the outlook for institutional sustainability is good, the question of whether or not the UNITY program is financially sustainable is in doubt. In a school system which has very limited resources, we found very few cases of district governments sharing costs with UNITY or making plans to expand the very good UNITY activities using their own resources. There was one case where it was claimed that a district in the Northeast had invited REPLICA to present a program in the district and may have shared costs but, the details of that program were not available. Moreover, some programs like, the introduction of TC, have already experienced cut-backs in the length of training offered teachers or the amount of local language materials offered schools and teachers, suggesting that even high-priority activities may get short-shrift. The policy decision to mandate UPE and more recently USE will only exacerbate the financial situation for the MOES and make the sustainability of UNITY even more doubtful. Finally, although UNITY’s costs appear to be within acceptable ranges, the total cost for some of the outputs may be beyond the ability or the interests of the MOES to maintain. For example, the C-TEP course costs an average of $2,460 per tutor and, given that all the currently employed tutors have taken the course, it does not seem likely that MOES will mount a second C-TEP training program in the near future. As a result, some of the higher cost UNITY activities may not be sustained financially by the MOES. Replication The question asked by USAID was: “Does the project posses the organizational capacity to scale-up the program and expand to all PRDP defined districts?” This is a complex question that requires a complex answer. To the first part of the question, “Does the project posses the organizational capacity” the answer is a resounding, yes. The UNITY project has shown that it has terrific organizational capacity and that it can accomplish a variety of complex tasks and many tasks at the same time. To the second part of the question, “Does the project possess the organizational capacity to scale￾up the program” the answer is a qualified, yes. This evaluation has shown that UNITY has achieved some excellent outputs and deserves a great deal of kudos for a job well-done, while on the other hand, the evaluation has shown that the UNITY program appears to have too many activities for any project to manage effectively and that the project’s strength of providing a flexible response is, at the same time, a liability that hampers it’s performance. If the intent of the question is to ask should the current set of UNITY activities be scaled-up, the answer is that the program should first determine what smaller set of activities are in its manageable interest before it considers a scale-up and if it does scale-up it should not be in all the current UNITY activities. To the third part of the question, “Does the project posses the organizational capacity to scale-up the program and expand to all PRDP defined districts?” the answer is, probably not. As the evaluation shows, the REPLICA program appears to have been expanded to too many districts too quickly. Freely acknowledging that it was unable to visit a sufficient number of REPLICA schools, the evaluation team found that the impact of the program at the school level was slight. Indeed, when presented with this observation the REPLICA team claimed that it had focused at the district and sub-county level rather than the school level and their job was incomplete. Under 51 | The Mid-Term Evaluation of USAID/Uganda’s UNITY Program, December 2008 the circumstances, the evaluation team recommends that USAID conduct an in-depth review of the REPLICA program before it considers expansion and that, in any case, expansion to all PRDP defined districts would probably repeat the mistakes of the past and should not be undertaken without a major increase in resources and/or a redesign of the program and its management. 52 | The Mid-Term Evaluation of USAID/Uganda’s UNITY Program, December 2008 VI. RECOMMENDATIONS BY OBJECTIVE UNITY is well-positioned to assist the GOU • Continue to stress flexibility and responsiveness in UNITY but develop a broader consultation within MOES and among UNITY partners and develop road maps and strategic plans for all activities ensuring that everyone knows their role and responsibilities and what is coming next. Objective One: Professional Development Revitalized TDMS • Review the proposed process but vest Kyambogo University’s Department of Teacher Education with the authority to determine who fails or passes the C-TEP course and receives the certificate; • Work with the MOES to determine the salary and promotion implications of passing the C-TEP course; • Revisit the structure and delivery mechanisms of the C-TEP course; • Work with Kyambogo University to identify candidates and possible funding for C-TEP II course; • Develop and implement measures to reform cascade approach to training; • Drop programs that appear to be nothing more than bold promises (regional centers, Centers of Excellence). Improved Instructional Methods and Materials to Teachers • UNITY has developed some excellent materials but, these materials are largely not in the hands of teachers and pupils. UNITY should exploit its strength in this area and ensure that these fine materials are at least in the hands of teachers; and • Devise systems where materials are not available for resale on the parallel market. Challenges Facing Implementation of TC • Recognize that problems in the schools impacting TC are not going away in the short- to medium term and develop methodologies to solve problems of schools; • Redouble efforts to improve reading instruction and practice and make it central to the TC reform; and • Only in UNITY-assisted schools consider addressing following issues: - Class size over 100 in the North, indifference or resistance in the South; - Limited materials in Local Languages everywhere; - Remedial training for all teachers in classroom management, continuous assessment, and materials development; - Train a critical mass of teachers or practice whole school training; - Experiment with regional or district models reflecting special needs of area; - Re-evaluate continuous assessment models; and - Provide teachers with good materials. 53 | The Mid-Term Evaluation of USAID/Uganda’s UNITY Program, December 2008 Expanded REPLICA • Most people interviewed claimed that REPLICA, although a good program, expanded rapidly beyond the capacity to manage it effectively. Moreover, it appears that REPLICA spent much of its efforts at the district and sub-county levels and has not addressed the school level, at least in term of community mobilization. Although there appears to be a need for expanding the program from 1,500 schools to the 4,000 schools in all the 40 PRDP districts, it appears than REPLICA has much work left undone. Although it does not appear that REPLICA expansion is warranted, any expansion of the program should be carefully considered and done in a phased, manageable roll-out. Objective Two: HIV/AIDS Mitigation Expanded PIASCY • Uganda’s HIV/AIDS mitigation program aimed at primary schools is an imperative that seems to be working well and the next logical step outlined in the UNITY Technical Approach appears to be to extend activities to the post-primary level and out-of-school children (PPET). The evaluation team was not able to review PPET in a junior secondary setting, owing to relatively recent introduction. Given the difficulties in addressing HIV/AIDS issues with out-of-school youth worldwide, UNITY should move very cautiously, if at all, in those areas. Nevertheless, given the success at the primary school level, it appears that the expansion of PPET in junior secondary schools should proceed albeit at a cautious pace, especially given the different client group to be serviced. • PIASCY should not abandon efforts in the primary school sector. It appears that there are a number of things that could be revisited or new areas that could be addressed, including monitoring the progress of HIV/AIDS mitigation in primary schools; training teachers who lack confidence to better address PIASCY issues; and improve community participation to support HIV/AIDS orphans. Objective Three: Community Participation Parental and Community Participation in Education • Virtually everyone interviewed maintained that the lack of parental and community participation in primary schools is a serious and limiting factor to achieving improved education and that UNITY should work in partnership with DEO and SMCs to enhance participation and improve school governance; • To overcome the current malaise and inertia, UNITY should consider focusing on a limited number of districts (perhaps as part of the QEI initiative) and experimenting with different ways to stimulate participation; and • Some areas for further investigation and experimentation include: - Best-practices developed under the UPHOLD program; - Training district managers in education management and development of realistic strategic plans; - Training head teachers in a whole school approach (along with teachers, community members and parents) in school management and the development of time-phased, realistically obtainable, school strategic plans; 54 | The Mid-Term Evaluation of USAID/Uganda’s UNITY Program, December 2008 - Providing very limited small grants focused on improving school quality to SMCs to support the school strategic plan; and - Promoting visits between schools with good community participation and those without. Objective Four: Educational Policy Educational Policy • UNITY should pursue a more active role in policy analyses, especially in regard to educational policies that impact its scope of work; and • UNITY should consider promoting regional policy discussions among DEOs and education teams (e.g., Gulu summit). Objective Five: Grants Grants • UNITY should consider use of a small grant mechanism as it was originally intended to support school-level interventions as part of an effort to stimulate parental and community involvement in schools. Currently, the funds intended for school-level interventions have been slated for motorcycle procurement. The motorcycles will improve the mobility of the CCTs and, hence, facilitate the dissemination of improved teaching methods. Small grants to communities should stimulate further parental involvement in schools. The trade-offs in the use of these small grants funds for motorcycles or community grants should be carefully reviewed by USAID. Other Issues Poor Learning Environment • See parental and community participation above; and • UNITY, MOES and/or USAID should consider addressing the health issues of pupils in primary schools, even in a limited way, as well as mounting a “healthy schools” program. District and School Management • UNITY should focus on a limited number of districts to improve the quality of district and school management (perhaps part of the QEI initiative) focused on: - Training district managers in education management and development of realistic strategic plans, - Use of EMIS; - Overcoming bottlenecks in the system, - Making the TDMS cascade system work, - Human resource management, - School governance and accountability, - Improving working relationships within the district education teams, and - Methods to enhance community participation. 55 | The Mid-Term Evaluation of USAID/Uganda’s UNITY Program, December 2008 Teaching Methods • Within the same limited districts (above), UNITY should develop a whole school approach to training teachers and providing them with the key materials needed to ensure success in a variety of MOES initiatives e.g., TC, reading, active learning, improved classroom management. Design Flaw • Having provided the necessary inputs to the MOES to revitalize its TDMS, UNITY should now focus on school-level interventions. Road Maps • Without losing its flexibility and responsiveness, UNITY should develop a road map of future directions that spell-out the roles and responsibilities of all key players. Communication • UNITY should take care to consult widely with MOES staff and its partners and sub￾contractors. Expansion and Replication • Given the limited USAID budget for education ($6.5 million/yr) and the myriad of MOES needs, UNITY should move from a program of service delivery for the MOES to a program with limited objectives aimed at developing activities focused on school-level improvements in educational quality; and • UNITY’s significant contribution to revitalizing the TDMS system should not be lost and residual activities to making the system work better should be undertaken. These include reforming the cascade system of training and facilitating transportation (motorcycle purchase) for tutors in targeted districts, and addressing thorny policy issues like, improving timely flows of funds to schools, and streamlining school governance and human resource issues, and linking teacher performance to employment. Cross-Cutting Recommendations • Work to consolidate the gains in the most effective programs and ruthlessly jettison least effective aspects of UNITY; • Develop high-quality manuals and materials, lessons learned, policy briefs, and best practices for turn over to GOU; • Expand Ugandan universities and teachers colleges’ role to incorporate C-TEP programs, active learning methods and school-based management into pre-service programs; and • Publish and distribute lessons learned and best practices to all UNITY-assisted schools. 56 | The Mid-Term Evaluation of USAID/Uganda’s UNITY Program, December 2008 ANNEXES Appendix A: Selected list of documents consulted ABE-LINK UGANDA, Uganda Initiative for TDMS and PIASCY (UNITY) Project, Annual Performance Report December 14th 2007; Creative Associates Inc. ABE-LINK UGANDA, Uganda Initiative for TDMS and PIASCY (UNITY) Project, Quarterly Report VI, May-August 2008; Creative Associates Inc. ABE-LINK UGANDA, Uganda Initiative for TDMS and PIASCY (UNITY) Project, Quarterly Report V, January-April 2008; Creative Associates Inc. ABE-LINK UGANDA, Uganda Initiative for TDMS and PIASCY (UNITY) Project, Quarterly Report III, May-August 2007; Creative Associates Inc. ABE-LINK UGANDA, Uganda Initiative for TDMS and PIASCY (UNITY) Project Quarterly Report II, January-April 2007; Creative Associates Inc. ABE-LINK UGANDA, Uganda Initiative for TDMS and PIASCY (UNITY) Project Quarterly Report I November 2006-January 2007; Creative Associates Inc. Creative Associate Technical Approach and Management Plan and Staffing, October 25, 2006 ESA (2003) Educational Standards Agency Report of the National Inspection programme 2002/2003, Education Standards Agency (ESA) ESA/MoES (Not dated) Handbook for School Inspectors Education Standards Agency, Ministry of Education and Sports FAWE Uganda Chapter (June 2008), Final Report on REPLICA Program Activities for Lango, Acholi and Teso Sub-regions, Forum for African Women Educationalists, Bukoto Kampala Lister. S (29 September 2005) Uganda Country Report, Joint Evaluation of General Budget Support 1994-2004, International Development Department School of Public Policy University of Birmingham U.K. Mark Lynd, (April 2007); Evaluation of the REPLICA Project; School-to-School International MoES & UNITY, NORTH/REPLICA STRATEGY, USAID/UNITY, Ministry of Education and Sports/ The Uganda Initiative for TDMS and PIASCY MoES (April 2008); A formative Evaluation of REPLICA Project in Lango and Acholi Regions￾Final Draft; Ministry of Education and Sports 57 | The Mid-Term Evaluation of USAID/Uganda’s UNITY Program, December 2008 MoES (August, 2008), Guidelines on Policy roles and Responsibilities of Stakeholders in the implementation of Universal Primary Education (UPE) Ministry of Education and Sports MoES (January 2008), Education Sector/Thematic Working Paper (PEAP); Ministry of Education and Sports Pre-training Visit to Districts in Northern Eastern Uganda; Report on the REPLICA Program June 2007 Steffensen J (12 September 2005) Decentralization and PGBS in Uganda, Joint Evaluation of General Budget Support 1994-2004, International Development Department School of Public Policy University of Birmingham U.K. The Pincer Group (April 2008), REPLICA Program Draft Report, The Pincer Group International Limited Wandegeya Kampala. The Pincer Group (August 2008), REPLICA Program Report, The Pincer Group International Limited Wandegeya Kampala. The Pincer Group (March 2008), REPLICA Program Report, The Pincer Group International Limited Wandegeya Kampala. The Pincer Group (May – August 2007), REPLICA Program Progress Report, The Pincer Group International Limited Wandegeya Kampala. The Pincer Group (May 2008), REPLICA Program Report, The Pincer Group International Limited Wandegeya Kampala. The Pincer Group (September 2008), REPLICA Program Report, The Pincer Group International Limited Wandegeya Kampala. The Pincer Group 2007), What is “on the ground” regarding the REPLICA project in model school under LORO CORE PTC Catchment?” School Specific Profiles Year one of REPLICA Programme, The Pincer Group International Limited Wandegeya Kampala. TMG (October,2008);proposal for USAID/Uganda UMEMS Technical Proposal for the Evaluation of the CAII/UNITY Program; The Mitchell Group Inc. Unicef (September 2007) United Nations Girls’ Education Initiative – Making UNGEI Work, Lessons from Four African Countries, UNICEF Eastern and Southern Africa Regional Office Nairobi Kenya. UNITY/MoES (May 2007), Revitalizing Education, Participation and Learning in Conflict Areas – (REPLICA) Program Baseline Report, The Uganda Initiative for TDMS and PIASCY /Ministry of Education and Sports. 58 | The Mid-Term Evaluation of USAID/Uganda’s UNITY Program, December 2008 Appendix B: List of Institutions Visited by the Evaluation Team S/N Institution 1 United States Agency for International Development (USAID) 2 Irish Aid 3 United Nations Children Fund (UNICEF) 4 Department of Teachers Education 5 Department of Pre-primary and primary Education 6 Department of Secondary Education 7 Monitoring and Evaluation Department 8 Department of Special Needs Education 9 Planning Department 10 MADRASA Resource Centre 11 Straight Talk Foundation 12 Pincer Group International 13 Kyambogo University,Dept.of Teachers Education 14 Aga Khan University 15 National curriculum Development Centre (NCDC),Kyambogo 16 Education Standard Agency 17 Iganga District 18 Soroti District 19 Oyam District 20 Gulu District 21 Kabala District 22 Mbarara District 23 Mpigi District 24 Bishop Willis Core Primary Teacher’s College 25 Soroti Core primary Teacher’s College 26 Loro Core Primary Teachers College 27 Gulu Core primary Teachers College 28 Kabale-Bukinda Primary Teachers College 29 Bishop Stuart Core Primary Teachers College 30 Kabulasoke Core Primary Teacher College 31 Bishop Willis Demonstration Primary School 32 Bunyiro Moslem Primary School 33 Awoja Primary School 34 Angopet Primary School 35 Loro Primary School 36 Gulu public Primary School 37 Torchi Primary School 38 Kyanamira primary School 39 Nyakigugwe primary School 40 Nyakayojo primary School 41 Galatiya primary School 42 Bukalagi primary School 59 | The Mid-Term Evaluation of USAID/Uganda’s UNITY Program, December 2008 Appendix C: List of Individuals Interviewed S/N Name Institution Position 1 Tom Leblanc USAID Senior Education Advisor 2 Sarah Mayanja USAID CTO 3 Rhona Walusimbi USAID M&E Officer 4 Renuka Pilly UNITY COP 5 Florence Kanyike UNITY D COP 6 Patrick Bananuka UNITY Advisor 7 Scorastica Tiguryera UNITY Basic Education Manager 8 Alice Ibale UNITY In-charge HIV/AIDS 9 Martin Opolot UNITY M&E Officer 10 Kavuma Carol Agakhan University Lead trainer C-TEP 11 Cathy Watson Straight Talk Foundation Director 12 Sekalala Shafik Madrasa Resource Centre Director 13 Muto Milton Pincer Group International Team Leader 14 Hon.Ferua Andaman R Arua DLG Chairman LCV 15 Laura Keihas UNICEF Program Officer Education 16 Jessica Justine Ilomu IRISH AID Program Manager 17 Byamugisha K.Albert MoES Asst. Comm Planning 18 Oketcho Wimon C MoES Asst.Comm.S NE 19 Nkaada Daniel MoES Com PPE 20 Clive Kaddu Buyisi MoES Principal Educ Officer Sec. 21 Aguti Janet Florence MoES Principal Educ Officer TED 22 Kibenge David Aggrey MoES Principal Asst.Sec & PRO 23 Annet Mugisha MoES Education Officer TED 24 Kasimagwa MoES Education Officer TED 25 Margo O Sullivan MoES Moes Advisor 26 Resty Muziribi MoES Assistant Commissioner PPE 27 Cuthbert Mulyalya MoES SEP M&E 28 Joseph Muvawala MoES EPD 29 Sara Margiotta EFAG 30 Dues Monday IMU 31 Patricia Kibira MoES Ag FOB 32 Tim Kos 33 Lamex Omara A MoES AC/PES 34 Grace Abalo MoES EO/PES 35 Vicent Ssozi MoES AgP/STAT 36 Akakwasa Justus MoES AC 37 Dhizaala Moses MoES NPA 38 Sarah Namuli T MoEs AC/BE 39 George Kalibbala Netherlands CTO 40 Joseph Eilor MoES Planning 60 | The Mid-Term Evaluation of USAID/Uganda’s UNITY Program, December 2008 S/N Name Institution Position 41 Byabagambi Christine Kyanamira P/S Head Teacher 42 Mugisha Bernard Kyanamira P/S Senior Man Teacher 43 Tumusiime Joventa Kyanamira P/S Senior Woman Teacher 44 Akampa William Kyanamira P/S Teacher 45 Ahimbisibwe Agnes Kyanamira P/S Teacher 46 Nareeba Letiva Kyanamira P/S Teacher 47 Twikirize Fred Kyanamira P/S Teacher 48 Arigye Jacinta Kyanamira P/S Deputy Head Teacher 49 Nganyaki Vincent Kyanamira P/S Teacher 50 Twesiime Faith Kyanamira P/S Teacher 51 Tumukurate Denis Kyanamira P/S Teacher 52 Musinguzi Hope Kyanamira P/S Teacher 53 Arineitwe Christine Kyanamira P/S Teacher 54 Turyahabwe Angeline Kyanamira P/S Teacher 55 Atuhe Leocadia Kyanamira P/S Teacher 56 Asiimwe Jackline Kyanamira P/S P.2teacher 57 Kyomuhangi Lady Kyanamira P/S P1teacher 58 Munywanisa B .L. Kabale-Bukinda Tutor 59 Kadde Norah Kabale-Bukinda CCT(Tutor) 60 Kashana Nankunda .S. Kabale-Bukinda CCT 61 Basinguzi Japeth Kabale-Bukinda CCT 62 Mubangizi Methodius Kabale-Bukinda CCT 63 Musinguzi Christopher Kabale-Bukinda CCT 64 Gemera James Kabale-Bukinda CCT 65 Musiimenta .G.Willy Kabale-Bukinda CCT 66 Masiko Gad Kabale-Bukinda Tutor 67 Kasigazi Johnson Kabale-Bukinda CCT 68 Kaheeru John B (Fr.) Kabale-Bukinda CCT 69 Twinomugisha Jackson Kabale-Bukinda CCT 70 Mwebaze Asaph Kabale-Bukinda Dpo 71 Lwamafa Asaph Kabale-Bukinda Principal 72 Kabakyenga Eliaz Nyakigugwe P/S Headteacher 73 Bwesigye Tom Nyakigugwe P/S C/Man SMC 74 Beesiime John Nyakigugwe P/S C/Man PTA 75 Tirwakunda Bernard Nyakigugwe P/S V/C/Man PTA 76 Mrs. Mbareeba Pelegia Nyakigugwe P/S Member Sm 77 Kyokwijuka Peter Nyakigugwe P/S Member SMC 78 Tumuhimbise A.Odanta Nyakigungwe P/S Teacher 79 Tumushabe Joy Nyakigungwe P/S Teacher 80 Tumusiime Emily Nyakigungwe P/S S/W Teacher 81 Tindimwebwa Moses Nyakigungwe P/S Teacher 82 Bashaija Lazaro Nyakigungwe P/S Senior Male Teacher 83 Turyahikayo Frank Nyakigungwe P/S Teacher 84 Tindyebwa Robert Nyakigungwe P/S Teacher 61 | The Mid-Term Evaluation of USAID/Uganda’s UNITY Program, December 2008 S/N Name Institution Position 85 Kyarimpa Charity Nyakigungwe P/S Teacher 86 Atukunda Annet Nyakigungwe P/S Teacher 87 Akakikunda Pamella Nyakigungwe P/S Teacher 88 Aheirwe Joviah Nyakigungwe P/S Teacher 89 Baruugi Julius Nyakigungwe P/S Deputy Head Teacher 90 Mwesigwa Wycliffe Nyakigungwe P/S Teacher 91 Kyomukama Damson Nyakigungwe P/S Teacher 92 Orikiriza Benson Nyakigungwe P/S Teacher 93 Kagwa Josephat Nyakigungwe P/S Teacher 94 Tumusiime Emilly Nyakigungwe P/S P1. Teacher/S/W Teacher 95 Akakikunda Pamela Nyakigungwe P/S P.2 Teacher 96 Mpora Nathan Bishop Stuart Core PTC CCT 97 Basaaija Christopher Bishop Stuart Core PTC CCT 98 Mugisha Deus Bishop Stuart Core PTC CCT 99 Ayebaze Godfrey Bishop Stuart Core PTC Pre-Service 100 Ndibanoha Selgius Bishop Stuart Core PTC Pre-Service 101 Kwikiriza Joan Bishop Stuart Core PTC CCT 102 Twesigye Ruth Bishop Stuart Core PTC Pre-Service 103 Ankunda Grace Bishop Stuart Core PTC Pre-Service 104 Magara Enoth Bishop Stuart Core PTC CCT 105 Mbabazi Jovuletim Bishop Stuart Core PTC Tutor 106 Kariyo .N.Baker Bishop Stuart PTC CCT 107 Mwesigwa Ricky Bishop Stuart PTC Tutor 108 Nyerwanire Joseph Bishop Stuart PTC CCT 109 Atuhamye Bernard Bishop Stuart PTC Tutor 110 Turyagyenda Centrlo Bishop Stuart PTC Tutor 111 Tumulamye Fred. K. Bishop Stuart PTC Tutor 112 Batwire John Bishop Stuart PTC CCT 113 Mbabazi Gaddie Bishop Stuart PTC CCT 114 Mubakye Sam Bishop Stuart PTC CCT 115 Kamugisha Norah Mbarara DLG Councilor Member SSC 116 Magezi Agnes Mbarara DLG Councilor Member SSC 117 Tumusiime Deus Mbarara DLG DEO Mbarara 118 Tumusiime Deziderio Mbarara DLG Senior Education Officer 119 Kabarema Adonia K Mbarara DLG Sec Social Services 120 Arinaitwe K. Kururagire Mbarara DLG Councilor GSC Member 121 Mbabazi Edward Mbarara DLG DIS 122 Mbabazi Joy Kiconco Nyakayojo I P/S P.2 Teacher/S/W Teacher 123 Mushabe Aggrey Nyakayojo P/S Teacher 124 Tumuhimbise Alex Nyakayojo P/S Teacher 125 Asiimwe Osbert Nyakayojo P/S Teacher 126 Komucunguzi Bonny Nyakayojo P/S Teacher 127 Kemigisha Todozia Nyakayojo P/S P.1 Teacher 128 Tumukunde Patience Nyakayojo P/S Teacher 62 | The Mid-Term Evaluation of USAID/Uganda’s UNITY Program, December 2008 S/N Name Institution Position 129 Ziryampa Erasmus Nyakayojo P/S Deputy Head Teacher 130 Rwahusha Eliezer Nyakayojo P/S S/M Teacher 131 Turyakira Godfrey Nyakayojo P/S Head Teacher 132 Bavekuno .M.Kyeswa Mpigi DLG DIStrict Councilor 133 Kinene Billy Mpigi DLG DIStrict Councilor 134 Ssempigga Warale Mpigi D/Lc D/C Kyegonza 135 Mukwaya .J.Letakemu Mpigi DLG Councilor Kalamba 136 Baingana Vincent Mpigi DLG Chairman Educ.Committee 137 Ndagire Jascent Mpigi DLG Mpigi DEO 138 Kalyango Godfrey Mpigi Inspector of School 139 Rev. Elijah .N.Kabuye Galatiya P/S Member Foundation Body 140 Livingistoni Bbosa Galatiya P/S Member PTA 141 M A Kabuye Galatiya P/S Member SMC 142 Veronic Miyonga Galatiya P/S Member SMC 143 Alice Bosa Galatiya P/S Member PTA 144 Mugambe Minsusera Galatiya P/S Chairman PTA 145 Kibirango .K.Anthony Galatiya P/S Member SMC 146 Mayanja Christopher Galatiya P/S LC1 Chair Person 147 Ssali Samuel Galatiya P/S Headteacher 148 Giradesi Nakyanzi Galatiya P/S Member PTA 149 Nabadda Violet Galatiya P/S P.1 Teacher 150 Nankabirwa Christine Galatiya P/S P.2 Teacher/S/W Teacher 151 Imaret Leonard Bukalagi P/S Vice/Chairman PTA 152 Matovu Teopista Bukalagi P/S SMC Member 153 Kasumba Aloysius Bukalagi P/S Member PTA 154 Mary Luyiga Bukalagi P/S Member PTA 155 Gorret Akiiza Bukalagi P/S Member P.T.A 156 Bro. Fabian Bahemuka Bukalagi P/S Head Teacher 157 Bukenya Augustine Bukalagi P/S Teachers’ Representative 158 Kakinda Magdalene Bukalagi P/S S.W Teacher 159 Kibirige Micheal Bukalagi P/S S.M Teacher 160 Nakawunde Clotilda Bukalagi P/S P.1 Teacher 161 Nabbanja Resty Bukalagi P/S P.2 Teacher 162 Ndyabahika Web Kabulasoke Core PTC Deputy Principal Outreach 163 Ndyahisyahe James Kabulasoke Core PTC CCT 164 Musoke Kyeyune Kabulasoke Core PTC CCT 165 Banura A Faith Kabulasoke Core PTC CCT 166 Gakyaro Emmanuel Kabulasoke Core PTC CCT 167 Mutaaya Marion Kabulasoke Core PTC CCT 168 Nakate Milly Kabulasoke Core PTC CCT 169 Kaganda M Ignatius Kabulasoke Core PTC CCT 170 Balazewa Henry Gabula Kabulasoke Core PTC CCT 171 Muddibo R Dickens Kabulasoke Core PTC CCT 172 Agobe Pollbert Kabulasoke Core PTC CCT 63 | The Mid-Term Evaluation of USAID/Uganda’s UNITY Program, December 2008 S/N Name Institution Position 173 Okello Ogola .P. Iganga DLG D.E.O 174 Kasada Baker Iganga DLG Senior Education Officer 175 Kibira Daniel Iganga DLG C/M Education Committee 176 Kadeebo Fred Iganga DLG Member Educ.Committee 177 Magumba Fredrick Iganga DLG Member Educ.Committee 178 Steven Kalya Bishop Williss P.T.C Principal 179 Onyait James Bishop Willis P.T.C Deputy Principal Outreach 180 Adupa James Bishop Willis P.T.C CCT 181 Oojambo Peter Bishop Willis P.T.C Pre-Service Tutor 182 Nafuna M.Sarah Bishop Willis P.T.C CCT 183 Tumwine Jimmy Bishop Willis P.T.C Pre-Service Tutor 184 Mukenye Ahamed .M. Bishop Willis P.T.C CCT 185 Okwanga Peter Bishop Willis P.T.C CCT 186 Aciro Betty Bishop Willis P.T.C Pre-Service Tutor 187 Nambafu Rose Nasolo Bishop Willis P.T.C Pre-Service Tutor 189 Acom Regina Bishop Willis P.T.C Pre-Service Tutor 190 Okitoi John Bishop Willis P.T.C Pre-Service Tutor 191 Naigaga Rosset Mary Bishop Willis P.T.C CCT 192 Odwory Fred M Bishop Willis P.T.C Deputy Principal 193 Odingi-Wajala Wilson Bishop Willis P.T.C CCT 194 Wambi Moses Bishop Willis P.T.C CCT 195 Kalifa Wilson John Bishop Willis P.T.C CCT 197 Kimono Lucia Norah Bishop Willis P.T.C CCT 198 Maina Peter Bishop Willis P.T.C CCT 199 Kyakulaga G.Samuel Bishop Willis P/S Vice Chairman P.T.A 200 Kibuuka J.K Bishop Willis P/S Member SMC 201 Mafabi Peter Bishop Willis .P/S Member P.T.A 202 Kataike Jenipher Bishop Willis P/S Member P.T.A 203 Mpaata Andrew Bishop Willis .P/S Headteacher 204 Kakuuma Dianah Bishop Willis .P/S Member P.T.A 205 Obbo Richard Bishop Willis .P/S Deputy H/M 206 Nyiro Wilberforce Bishop Willis .P/S Deputy H/M Special Needs 207 Mark Loweli Binyiro P/S PCV 208 Wambi Moses Bunyiro P/S CCT 209 Nkangazi Samuel Bunyiro P/S Chairman Water Project 210 Baaziba Steven Bunyiro P/S Member P.T.A 211 Hamidu Abone .W. Bunyiro P/S Member P.T.A 212 Ismailwakasalo Bunyiro P/S Chairman P.TA 213 Drugobije Rose Bunyiro P/S Member P.T.A 214 Birungi Musa Bunyiro P/S Member P.T.A 215 Okubu Peter Aejo Soroti DLG DIS 216 Etoyu M.Oumo Soroti DLG D.E.O 217 Muhammed Nasser Soroti DLG Member Educ.Committee 218 Apwoyo Mary Soroti DLG Member Educ. Committee 64 | The Mid-Term Evaluation of USAID/Uganda’s UNITY Program, December 2008 S/N Name Institution Position 219 Opit Joseph Okojo Soroti DLG C/M Education Committee 220 Jane Alum Soroti DLG Member Educ.Committee 221 Anyumel Beatrice Soroti DLG Secretary For Education 222 Angoari John Micheal Soroti Core P.T.C CCT 223 Orwang Micheal .R. Soroti Core P.T.C CCT 224 Enyimu Joseph Soroti Core P.T.C Pre-service Tutor 225 Asiba Francis Soroti Core P.T.C CCT 226 Aisu Noel Soroti Core P.T.C Pre-service 227 Epulu John Micheal Soroti Core P.T.C Pre-service 228 Aisu Grace Soroti Core P.T.C Pre-service 229 Amuron Frances Egabu Soroti Core P.T.C CCT 230 Amongin Tokei Jane Soroti Core P.T.C Pre-service 231 Tino Emmie Dorothy Soroti Core P.T.C Pre-Service 232 Etapu Ogweta Joseph Soroti Core P.T.C Pre-service 233 Wegulo Malinga Lewis Soroti Core P.T.C Pre-Service 234 Elasu John Mackay Soroti Core P.T.C Pre-Service 235 Isina Christine Soroti Core P.T.C Pre-Service 236 Ololo Graphes Soroti Core P.T.C Pre-Service 237 Ogworo James Soroti Core P.T.C CCT 238 Emuron William Soroti Core P.T.C Depupty Principal 239 Ojangole John Max Soroti Core P.T.C CCT 240 Asio Grace Soroti Core P.T.C CCT 241 Arionget Stella Soroti Core P.T.C Pre-Service 242 Okalebo Angela Soroti Core P.T.C Pre-Service 243 Ejolu Patrick Okolong Soroti Core P.T.C CCT 244 Epau .A. David Soroti Core P.T.C CCT 245 Oluka Vincent Soroti Core P.T.C Principal 246 Odwilo Simon Soroti Core P.T.C Deputy Principal Outreach 247 Emero Omoding George Awoja Primary School Headteacher 248 Ecedu James Awoja Primary School Member P.T.A 249 Aboko Immaculate Awoja Primary School Teacher P.4 250 Akello A.Florence Awoja Primary School Teacher P.2 251 Auta Francis Awoja Primary School Teacher P.5 252 Amoncun Joyce Mary Awoja Primary School Teacher P.3 253 Akol Florence Awoja Primary School Teacher P.2 254 Acen Teddy Awoja Primary School Teacher P.5 255 Adulai Grace Awoja Primary School Teacher P.4 256 Aurut Francis Awoja Primary School Teacher P.6 257 Okiror Richard Awoja Primary School Teacher P.7 258 Okello John Micheal Awoja Primary School Teacher P5 259 Aipo Janet Louise Awoja Primary School Deputy Headteacher 260 Agaret Emmanuel Awoja Primary School Teacher P.6 261 Opio Gervas Awoja Primary School Teacher P.7 262 Asekenye Mary Francis Awoja Primary School Teacher P.1 65 | The Mid-Term Evaluation of USAID/Uganda’s UNITY Program, December 2008 S/N Name Institution Position 263 Emong James Fidel Awoja Primary School Teacher P.1 264 Ejoku Perpetua Awoja Primary School Teacher P.1 265 Amuron Joyce Awoja Primary School Teacher P.3 266 Okipi Joseph Charles Angopet Primary School Head Teacher 267 Opolot Raphael Angopet Primary School C/M P.T.A 268 Okise Vincent Angopet Primary School Member SMC 269 Odele Richard Angopet Primary School Member SMC 270 Ayoe Rose Angopet Primary School Member SMC 271 Aiso Magdalena Angopet Primary School Member P.T.A 272 Oreija John Micheal Angopet Primary School Vice Chairman P.T.A 273 Amolo Julius Angopet Primary School Member SMC 274 Willibarido Opit Angopet Primary School Treasurer P.T.A 275 Okiror Micheal Angopet Primary School Teacher P.7 276 Nume Stella Angopet Primary School Teacher P.1 277 Akello Jane Florence Angopet Primary School Teacher P.2 278 Openya Simon Peter Angopet Primary School Teacher P.5 279 Naizuli Juliet Angopet Primary School Teacher P.4 280 Atoke Harriet Angopet Primary School Teacher P.6 281 Awor Florence Grace Angopet Primary School Teacher P.1 282 Esangu Stephen Angopet Primary School Teacher P.2 283 Okello Norman Oyam DLG D.E.O 284 Owani Jacob Oyam DLG D.I.S 285 Umar Betty Oyam DLG Member Educ.Committee 286 Owor Robert Oyam DLG Member Educ.Committee. 287 Okello Olwa Francis Oyam DLG Member Educ.Committee 288 Okori Robert Milton Loro Core P.T.C CCT 289 Awio Francis Loro Core P.T.C CCT 290 Odongo Dickens .L. Loro Core P.T.C CCT 291 Okello Bosco Loro Core P.T.C Pre-Service 292 Ogwang Simon Parry Loro Core P.T.C Pre-Service 293 Adimola Clint Loro Core P.T.C CCT 294 Biringo Bosco Loro Core P.T.C CCT 295 Ogal Isaac Loro Core P.T.C CCT 296 Ogwal Antero Loro Core P.T.C CCT 297 Okello Nicholas Loro Core P.T.C CCT 298 Okello Denis Loro Core P.T.C CCT 299 Ojok Francis Ocen Loro Core P.T.C CCT 300 Renge Geoffrey Loro Core P.T.C CCT 301 Okuma Tom Loro Primary School Vice Chair Person SMC 302 Owiny Adea William Loro Primary School Head Teacher 303 Aale Harriet Loro Primary School Teacher 304 Okullo Kenneth Reigan Loro Primary School Teacher 305 Acen Lilian Loro Primary School Teacher 306 Ongom Samuel Loro Primary School Teacher 66 | The Mid-Term Evaluation of USAID/Uganda’s UNITY Program, December 2008 S/N Name Institution Position 307 Agwe Gas Loro Primary School Teacher 308 Ogwal Christopher Loro Primary School Teacher 309 Okello Denis Dandus Loro Primary School Teacher 310 Okello Tom Loro Primary School Teacher 311 Ngole Lawrence Loro Primary School Teacher 312 Ojok Moses Loro Primary School Teacher 313 Ocen Louis Loro Primary School Teacher 314 Ekwang Vincent Blair Loro Primary School Teacher 315 Opong James Ayo Loro Primary School Teacher 316 Ojok Agnes Jane Loro Primary School Teacher 317 Eyuta Robson Daniel Loro Primary School Teacher 318 Okeng Albino Loro Primary School Teacher 319 Wana David Akwar Loro Primary School Teacher 320 Owani Moses Loro Primary School Teacher 321 Opio Martin Loro Primary School Teacher 322 Aloro Jimmy Brown Loro Primary School Teacher 323 Obwoya Richard Loro Primary School Teacher 324 Bua Adea Daniel Loro Primary School Teacher 325 Ojok Francis Loro Primary School Deputy Head Tecaher 326 Awidi Esther Loro Primary School Teacher 327 Okello Benard Bosco Loro Primary School Teacher 328 Rev.Ocheng Vincent Gulu DLG DEO 329 Ocii Santo Gulu DLG Inspector Of Schools 330 Omona Darlington Gulu Municipal MEO –Admin 331 Apolo Josephine .A. Gulu Core P.T.C Pre-Service 332 Aide Micheal Gulu Core P.T.C Pre-Service 333 Etapu Emaju Isaac Gulu Core P.T.C CCT 334 Ocan Pons Donatian Gulu Core P.T.C CCT 335 Wanjisi Davis Gulu Core P.T.C CCT 336 Adyero Grace Gulu Core P.T.C CCT 337 Were Abrahams Gulu Core P.T.C Principal 338 Omara Okoya Kaziro Gulu Core P.T.C Deputy Principal Outreach 339 Kizito Mayanza Gulu Core P.T.C CCT 340 Okot Pius Alok Gulu Core P.T.C Pre-Service 341 Acayo Okello Martina Gulu Core P.T.C CCT 342 Tedeo Wallance Gulu Core P.T.C CCT 343 Lacwec Joel Gulu Core P.T.C Pre-Service 344 Nyangwen Alwodo C Gulu Core P.T.C Pre-Service 345 Br.Julius Onyango Gulu Core P.T.C Pre-Service 346 Okwangacon Lawrence Gulu Core P.T.C CCT 347 Apiyo Caroline Gulu Public School Member SMC 348 Amoo .V.Okullo Gulu Public School School Head Teacher 349 Wathum Christopher Gulu Public School Teacher 350 Moro Wilfred Gulu Public School Teacher 67 | The Mid-Term Evaluation of USAID/Uganda’s UNITY Program, December 2008 S/N Name Institution Position 351 Lajul Donas Franklin Gulu Public School Teacher 352 Adee Monica Lomoro Gulu Public School Teacher 353 Lanyero Lilly Rose Gulu Public School Teacher 354 Aguti Deborah Gulu Public School Teacher 355 Akumu. L.Opobo Gulu Public School Teacher 356 Onyango Julius Gulu Public School Teacher 357 Okwanga Lambert Gulu Public School Teacher 358 Odong Denise Miles Gulu Public School Teacher 359 Olanyakene Mark Torchi Primary School Head Teacher 360 Ayella Andrew Torchi Primary School Member P.T.A 361 Lanyero Norah Torchi Primary School Member P.T.A 362 Omona Emmanuel Torchi Primary School Chairperson P.T.A 363 Obita Nicholas Torchi Primary School Member SMC 364 Akello Eromina Torchi Primary School Member P.T.A 365 Aneno Josephine Torchi Primary School Member S.M.C 366 Tokwiny Felix Torchi Primary School Member S.M.C 367 Olweny Charles Torchi Primary School Member SMC 368 Okello Paul Torchi Primary School Member P.T.A 68 | The Mid-Term Evaluation of USAID/Uganda’s UNITY Program, December 2008 Appendix D: Classroom Checklist School Observation Checklist Objective: To assess the school’s current learning environment according to the technical approach/outputs/indicators listed in the UNITY documentation Activity or area yes no Comments Classroom Setting 1. DISplayed learning aids are in good condition 2. DISplayed learning aids are appropriate for grade level 3. Student work (papers, paintings etc) is DISplayed 4. Every child has a seat and something to write on (like, desk) 5. Desks are arranged in clusters 6. Learning corners can be found (math, reading) Teaching Learning Process 7. Teacher is present in classroom 8. Teacher lectures/talks no more than 50% of class period 9. Teacher has students actively demonstrate understanding of lesson 10. Teacher question students at least 1-2 times per lesson to check on understanding 11. Teacher encourages students to answer questions 12. Teacher can provide a copy of prepared lesson plan 13. Teacher manages the learning activity & class Student Participation 14. Most students paying attention Students are “on task” 90% of time 15. Students ask / initiate questions to initiate DIScussions with teacher at least 1-2 times per lesson 16. All participate actively engaged in group activity 17. Students relate learning to their experience / real life & demonstrate understanding / learning has taken place 18. Girls are called on and participate as much as the boys 19. Teacher has thematic curriculum materials in P1 & P2 20. Teacher has REPLICA materials 69 | The Mid-Term Evaluation of USAID/Uganda’s UNITY Program, December 2008 21. Teacher has PIASCY (HIV/AIDS) materials 22. Teacher and pupils have SNE materials Parental and CommUNITY Involvement 23. Evidence of commUNITY involvement in schools (commUNITY school improvement projects, SMC available, commUNITY leaders involved in school) 24. Parents involved in preparing children for learning 25. Parents knowledgeable about school programs like REPLICA/PIASCY General Comments: 70 | The Mid-Term Evaluation of USAID/Uganda’s UNITY Program, December 2008 Appendix E: Anonymous Questionaire Anonymous Questionnaire: Tutors District:_____________________________ This questionnaire is mostly about CTEP and is intended for pre-service and in-service tutors. If you do not have enough information and/or can not answer the question, please leave it blank. Please tick the answer that best matches your opinion, experience, or understanding. 1. How many times were you a participant in UNITY training workshops? 0 time 1 times 2 times 3 times 4 times 5 times 2. How effective were the workshops in improving your ability to train teachers in new skills? Very effective Modestly effective No opinion Less effective Not effective 3. How often do you provide mentoring services to head teachers/teachers? 0 times a week 1 times a week 2 times a week 3 times a week 4 or more times a week 4. How effective has the CTEP program been for upgrading the skills of head teachers/teachers? Very effective Modestly effective No Opinion Less effective Not effective 5. To what extent are you applying the new skills in the classroom? Very often often sometimes Rarely never 6. How effective are the HIV/AIDs readers/materials in mitigating HIV/AIDs? Very effective Modestly effective No Opinion Less effective Not effective 7. How effective has REPLICA programming been in enabling teachers to provide the educational needs of children (Question for Northern areas and CCTs). Very effective Modestly effective No Opinion Less effective Not effective 8. How effective has the primary thematic curriculum reform been for improving pupils learning achievement? 71 | The Mid-Term Evaluation of USAID/Uganda’s UNITY Program, December 2008 Very effective Modestly effective No Opinion Less effective Not effective 9. Please rank the areas of primary thematic curriculum that you have found to be most useful in improving the quality learning? _____________________________________________________________________ ________________________________________________________ 10. How useful are the materials provided by UNITY aimed at improving literacy? Very useful Modestly useful No Opinion Less useful Not useful 11. How useful are the materials provided by UNITY aimed at improving numeracy? Very useful Modestly useful No Opinion Less useful Not useful 12. Overall, how useful has been the educational information you have received from UNITY? Very Useful Modestly useful Neutral effect Less useful Not useful 13. Please list the assistance you have received from UNITY in order of its usefulness in facilitating your job responsibilities. Most Useful Least Useful _______________________ ________________________ _______________________ ________________________ 14. Please indicate the degree of impact the UNITY program has had on your schools (pupils). Area Great effect Some effect Neutral effect Little effect No effect Enrollment Attendance Retention Pupil achievement (performance) Others (specify) 15. Considering your involvement in the UNITY program, please list below any suggestions for improving the program and/or any additional areas that need attention. 72 | The Mid-Term Evaluation of USAID/Uganda’s UNITY Program, December 2008 ______________________________________________________________________________ ______________________________________________________________________________ ________________________________________________________ 73 | The Mid-Term Evaluation of USAID/Uganda’s UNITY Program, December 2008 Appendix F: Analysis of Data from the Questionnaire Synopsis of findings from anonymous questionnaires a) Majority of tutors who attended UNITY workshop found them very effective b) 67% of the tutors found C-TEP course to be modestly effective in up-grading skills of teachers and 61% often apply the new skills in PTC classroom. c) Most tutors found HIV/AIDS readers/materials provided by UNITY to be effective in mitigating HIV/AIDS d) Tutors in Northern and Eastern Uganda found REPLICA program to be modestly effective in enabling teachers provide education needs for children. e) Most tutors found the thematic curriculum reform effective in improving pupils learning achievement. f) Most tutors found local language as medium of instruction and continuous assessment to be the most useful components of the thematic curriculum improving quality learning. g) Tutors found materials and information provided by UNITY to be very useful most especially at improving literacy and numeracy. h) Most tutors noted that UNITY program had some degree of effect on pupil’s enrolment, attendance, retention and achievement but little impact on parental involvement. i) Majority of tutors suggested provision of transport means to CCTs and community mobilization as ways of improving the UNITY program. 74 | The Mid-Term Evaluation of USAID/Uganda’s UNITY Program, December 2008 PERCENTAGE VIEWS FROM THE TUTORS ADMINISTERED ANONYMOUS QUESTIONAIRES FROM NORTHERN,EASTERN,CENTRAL AND WESTERN REGIONAL DISTRICTS OF GULU,OYAM,SOROTI,IGANGA,MPIGI,KABALE AND MBARARA No.of times participated in UNITY workshop 2times 1% 3times 38% 4times 27% 5times 34% 2times 3times 4times 5times Effectiveness of the workshop in improving ability to train teachers very effective 65% modestly effective 34% No opinion 1% very effective modestly effective No opinion 75 | The Mid-Term Evaluation of USAID/Uganda’s UNITY Program, December 2008 How often do you provide mentoring services 0times 8% 1time aweek 14% 2times a week 3times a week 25% 21% 4 or moretimes 32% 0times 1time aweek 2times a week 3times a week 4 or moretimes Effectiveness of C-TEP in upgrading skills of Head/teachers very effective 24% modestly effective 67% No opinion 6% Not effective 3% very effective modestly effective No opinion Not effective 76 | The Mid-Term Evaluation of USAID/Uganda’s UNITY Program, December 2008 The extent of applying newskill in Classroom Very often 33% Often 61% Sometimes 3% Rarely 3% Never 0% Very often Often Sometimes Rarely Never Effectiveness of HIV readers/materials in mitigating HIV/AIDS Very effective 41% modestly effective 56% No opinion 3% Very effective modestly effective No opinion 77 | The Mid-Term Evaluation of USAID/Uganda’s UNITY Program, December 2008 Effectiveness of REPLICA in enabling teachers provide for children need Very effective 16% modestly effective 79% Not effective 5% Very effective modestly effective Not effective Effectiveness of Thematic Curicullum in improving pupils learning achievement Very effective 53% modestly effective 43% Less effective 4% Very effective modestly effective Less effective 78 | The Mid-Term Evaluation of USAID/Uganda’s UNITY Program, December 2008 Areas of Thematic Curicullum considered most useful Numeracy&numera cy 23% local language Thematic approach 26% 13% continous assesment 24% local materials 9% class tchr syst 5% Numeracy&numeracy local language Thematic approach continous assesment local materials class tchr syst Usefulness of materials provided by UNITY in promoting literacy Very useful 71% Modestly useful 28% No opinion 1% Very useful Modestly useful No opinion 79 | The Mid-Term Evaluation of USAID/Uganda’s UNITY Program, December 2008 Usefulness of materials provided by UNITY in promoting numeracy Very useful 63% Modestly useful 33% No opinion 3% Less useful 1% Very useful Modestly useful No opinion Less useful Overall usefulness of information provided by UNITY Very useful 82% Modestly useful 18% Very useful Modestly useful 80 | The Mid-Term Evaluation of USAID/Uganda’s UNITY Program, December 2008 Degree of impact of UNITY program on pupils Enrolment Great effect 60% Some effect 34% No comments 6% Great effect Some effect No comments Degree of impact of UNITY program on pupils Attendance Great effect 57% Some effect 37% No comments 6% Great effect Some effect No comments 81 | The Mid-Term Evaluation of USAID/Uganda’s UNITY Program, December 2008 Degree of impact of UNITY program on pupils Retention Great effect 24% Some effect 62% Neutral effect 7% No comments 7% Great effect Some effect Neutral effect No comments Degree of impact of UNITY program on Pupil achievement Great effect 33% Some effect 55% Neutral effect 3% No comments 9% Great effect Some effect Neutral effect No comments 82 | The Mid-Term Evaluation of USAID/Uganda’s UNITY Program, December 2008 Degree of impact of UNITY program on parental Involment Great effect 6% Some effect 1% No comments 93% Great effect Some effect No comments Degree of impact of UNITY program on Behavioral change Great effect 7% Some effect 4% No comments 89% Great effect Some effect No comments 83 | The Mid-Term Evaluation of USAID/Uganda’s UNITY Program, December 2008 Sugestions for improving the UNITY program transport facilitation 48% more materials 17% more training 15% community involvement 20% transport facilitation more materials more training community involvement 84 | The Mid-Term Evaluation of USAID/Uganda’s UNITY Program, December 2008 Appendix G: Interview Guide for Senior Woman / Man Teachers 1. Name of school…………………………District………………………………… 2. Senior Teacher’s subject area ………………………...,………………………… 3. Brief description of responsibilities…………………………………………….. ………………………………………………………………………………….. 4. Do you receive Kids Time newsletter? YES……NO……Language …………… 5. Comment on the appropriateness of Kids Time newsletter ……………………………………………………………………………………………… ……………………………………………………………………………… 6. Explain how children and teachers use Kid Time newsletter. ……………………………………………………………………………………………… ……………………………………………………………………………… 7. What materials do you have on HIV/AIDS prevention and related issues?*Indicate source. Ask to see materials. ……………………………………………………………………………………………… ……………………………………………………………………………… 8. Comment on the usefulness of the materials you are provided with to use in your office/work ……………………………………………………………………………………………… ……………………………………………………………………………… 9. In what ways has UNITY supported you to perform your responsibilities? ……………………………………………………………………………………………… ……………………………………………………………………………… 10. What significant change has come about in the work of SWT/SMT since the UNITY project started here? ……………………………………………………………………………………………… ……………………………………………………………………………… 11. What more should UNITY do to support your work as SWT/SMT? ……………………………………………………………………………………………… ……………………………………………………………………………… 12. What challenges do you meet as a SWT/SMT? ……………………………………………………………………………………………… ……………………………………………………………………………… 13. How do you overcome the above challenges? 85 | The Mid-Term Evaluation of USAID/Uganda’s UNITY Program, December 2008 ……………………………………………………………………………………………… ……………………………………………………………………………… Appendix H: Focus Group Questionnaire: PTC and CCTs – Tutors Objective: To determine how appropriate and effective the professional development activities in teacher training has been under the UNITY program 1. What training programs have you participated in under the UNITY program? 2. What was the content including subject and methods of the C-TEP training program? 3. Given your responsibilities, how effective do you think the training was to improve your skills? 4. What have you done in schools to apply the training you have received? 5. What do you think you have achieved since you have been trained under the UNITY program? 6. How effective was the training to meet your professional needs? 7. Are teachers applying the skills you have taught them…if no, why not? 8. How do pre-service tutors apply the UNITY training in their program? 9. Considering the needs of primary education in Uganda, what additional professional development needs do you think UNITY should address? 10. Do you have any other concerns or issues you wish to share with the evaluation team? 86 | The Mid-Term Evaluation of USAID/Uganda’s UNITY Program, December 2008 Appendix I: Interview guide for DEO’s, DIS’s/Education committee 1. How have MOES/UNITY activities helped you to better manage education services/and support schools and teachers in your district? 2. In what ways does the regional centre contribute to the improvement of primary education in your district? 3. What is your experience with the implementation of the following; Thematic Curriculum/REPLICA/PIASCY in your district? 4. What aspects of UNITY support is your district able to continue independently? 5. What roles are you playing to roll-out/expand Unity programs in your district? 6. What actions is the district Education Committees taking to facilitate implementation of Unity programs in your district? 7. Are there any specific changes in the education sector at the school level that you can say are a result of Unity programs in your district? 8. Are there any education-related best practices in your district that you wish to recommend to other district? 9. What lessons have you learnt from the design and implementation of Unity in your district? 87 | The Mid-Term Evaluation of USAID/Uganda’s UNITY Program, December 2008 Appendix J : List of activities in Objective 1 Table 1: Objective 1 (Professional Development) Stated Activities and Sub-activities A B C D Core Activity Define a strategy to improve professional development. Revitilise the TDMS. Deliver improved instructional methods and materials to teachers and schools. Strengthen programs for Schools and leaning centers in the North Sub-Activities A1. Develop a strategy for primary teachers that is firmly grounded in the structures and programmes of the MOES and that supports USAID’s objectives. B1. Create regional centres. B2. Reform the cascade approach to training. B3.Develop centres of excellence to foster local innovations B4. Strengthen the tri￾partite system of support to teachers C1. Develop and implement the thematic curriculum nationwide. C2. Develop new materials. C3. Print and distribute existing materials. C4. Deliver support to girls and children with special needs. D1. Expand REPLICA activities. D2. Distribute REPLICA materials. D3. Strengthen centres of excellence. D4. Pilot an accelerated learning program. D5. Adjust REPLICA to adapt to changing conditions. Sub-Sub￾Activities A1.1 The COP will share the proposed strategy with the Teacher Effectiveness Working Group. A1.2 The working group will take its strategy to the departments of Education Planning, Teacher Education and Monitoring and Evaluation Working Group for Approval. A1.3 Creative will work with EPD to establish a strategic integrated action plan. B1.1 UNITY will work with the MOES/ TE to re￾cluster PTCs into a network served by Regional Centres. B1.2 Regional centers will roll-out national level initiatives, respond to local needs and support the implementation of the Thematic Curriculum. B1.3 At the end of the first year the effectiveness of Regional Centres will be reviewed by MOES and USAID. B1.4 The project will avail 5 Regional Centres with IT equipment and software for developing materials, Audio-visual facilities for instruction and model resource rooms. B1.5 UNITY will recommend placement of 5 Regional Advisors – these advisors will build C1.1 In year 1, UNITY will help roll out P1 curriculum in 9 local language regions, test P2 and develop materials for P3 and P4 (4 actions). C1.2 In year 2 UNITY will roll-out P2, Test P3 and P4 and develop materials for P5,6 and 7 (6 actions) C2.1 UNITY will develop new materials for tutors and head teachers C2.2 The project will work with regional centres in the desk publishing of local language materials. C2.3 Will work with NCDC and KYU to produce potable libraries. C2.4 Materials by Straight Talk 88 | The Mid-Term Evaluation of USAID/Uganda’s UNITY Program, December 2008 capacity of Regional Centre staff, coordinate training and oversee development of local language materials. B2.1 AKF will work with KYU to revamp its certificate course. B2.2 The best of the certified trainer will become national-level master trainers B2.3 AKU staff will use the revamped course to train at least 6 master trainers in each region. B2.4 Training of tutors will take place in the region. B2.5 EUPEC will directly train 60 selected core tutors in training of trainers skills- each tutor will receive 2 weeks of training. B2.6 AKU will help KYU to bring into alignment pre-service and in-service curricula and materials. B3.1 The project will continue to support centres of excellence formed by BEPS in the North. B3.2 The project will develop centres of excellence in other regions of the country. B3.3 Creative will help the centres of excellence to develop, test and package innovations as training modules and materials. B3. 4 Creative will strengthen links among the centres and institutions of higher learning to reduce dependence on foreign expertise. B4.1 UNITY will work with relevant multiple players to strengthen management of curricula and enhanced quality in schools. B4.2 UNITY will cooperate with the Dutch Foundation will continue to be published and be adapted to schools in the North and for SNE. C3.1 UNITY will contract for printing and distribution of: Thematic Curriculum materials, a monitoring manual for CCTs and teachers, a manual for SMCs, course materials for head teachers, a guidance manual for inspectors, Teacher Talk, Thema, a set of books on guidance and counseling (8 actions). C4.1 Will extend FAWE’s to improving girl’s education. C4.2 Will assist Kampala Pharmaceutical Industries to deliver free devices to hearing impaired children. C4.3 Will expand programs piloted in the North e.g. Guidance and counseling to support disadvantaged children in other areas. 89 | The Mid-Term Evaluation of USAID/Uganda’s UNITY Program, December 2008 and Irish teams to support KYU’s development of a training course for head teachers. B4.3 UNITY will work with TDMS to deliver workshops to 15,000 head teachers. B4.4 The project will orient about 250 DISs and strengthen School Management Committees B4.5 The project will print and distribute materials developed by BEPS for ESA and use these for a training workshop. B4.6 The project will take several measures to strengthen district office’s capacity to manage education services. Estimated total actions 4 25 29 4