LIBERIA TEACHER TRAINING PROGRAM PHASE II i MID-TERM ASSESSMENT OF THE LIBERIA TEACHER TRAINING PROGRAM PHASE II LIBERIA MONITORING & EVALUATION PROGRAMS (L-MEP) CONTRACT NO. 669-C-00-10-00181-00 MID-TERM EVALUATION November 2013 This publication is made possible by the generous support of the American people through the United States Agency for International Development (USAID/Liberia) under the Liberia Monitoring and Evaluation Program (L-MEP), Con-tract Number 669-C-00-10-00181-00. The author’s views expressed in this publication do not necessarily reflect the views of the United States Agency for International Development or the United States Government. LIBERIA TEACHER TRAINING PROGRAM PHASE II ii This assessment report is dedicated to the memory of Dr. Frank Schorn, who died suddenly during the investigation. LIBERIA TEACHER TRAINING PROGRAM PHASE II iii TABLE OF CONTENTS Acknowledgements .................................................................................................................................................................... v Acronyms and Abbreviations....................................................................................................................................................vi Executive Summary ................................................................................................................................................................... 1 Background .............................................................................................................................................................................. 1 Findings by Result Area ............................................................................................................................................................ 2 Lessons Learned....................................................................................................................................................................... 2 Service Delivery in a Capacity Void .......................................................................................................................................... 3 Focus and Concentrate ............................................................................................................................................................. 3 Conclusion ............................................................................................................................................................................... 4 I. Introduction ....................................................................................................................................................................... 5 A. Purpose of the Assessment.......................................................................................................................................... 5 B. Methodology and Approach ........................................................................................................................................ 6 C. Constraints to Undertaking the Assessment................................................................................................................. 7 D. Structure of the Report................................................................................................................................................ 8 II. Background........................................................................................................................................................................ 9 A. Development Environment and Challenges................................................................................................................. 9 B. Overview of Education in Liberia..............................................................................................................................10 C. Government of Liberia Education Plans, Priorities and Strategies..............................................................................13 D. Congruence of USAID/Liberia’s Country Development Cooperation Strategy (CDCS) with GOL Strategies............14 E. Overview of USAID/Liberia’s Basic Education Strategy ...........................................................................................16 F. Overview of Liberia Teacher Training Program (LTTP) ............................................................................................18 III. Findings .......................................................................................................................................................................21 A. RESULT 1 (R1): MOE, CEO, DEO and RTTI Capacity Strengthened to Plan, Manage, and Monitor Educational Services ...............................................................................................................................................................................................21 1. Accomplishments...........................................................................................................................................................21 2. Assessment by Sub-Result ..............................................................................................................................................23 B. Result 2: Improved Teacher Policy and Procedures for Teacher Recruitment, Training, Deployment, and Career Development..........................................................................................................................................................................41 1. Accomplishments..........................................................................................................................................................41 2. Assessment by Sub-Result ..............................................................................................................................................43 C. Result 3: Improved Teacher Training Programs and Reading/Math Delivery Systems........................................................46 1. Accomplishments...........................................................................................................................................................46 2. Assessment by Sub-Results.............................................................................................................................................47 IV. Design and Management of the Cooperative Agreement ........................................................................................51 V. Lessons Learned and Recommendations ......................................................................................................................55 A. Lessons Learned. ...........................................................................................................................................................55 B. Recommendations..............................................................................................................................................................59 LIBERIA TEACHER TRAINING PROGRAM PHASE II iv VI. Final Thoughts ............................................................................................................................................................62 VII. Annexes........................................................................................................................................................................64 Annex 1: List of Documents Consulted ..................................................................................................................................64 Annex 2: List of Individuals Consulted by Organization .........................................................................................................67 Annex 3: List of Provinces, Districts and Schools Visited .......................................................................................................76 Annex 4: Institutional Management and Education Administration Checklist .........................................................................78 Annex 5: Illustrative Interview Guides....................................................................................................................................93 Annex 6: Implementers Questionnaire Guide .......................................................................................................................103 Annex 7: Observational Tools ..............................................................................................................................................122 Annex 8: Surveys and Scorecards..........................................................................................................................................149 LIBERIA TEACHER TRAINING PROGRAM PHASE II v ACKNOWLEDGEMENTS This external assessment was undertaken by the USAID Liberian Monitoring and Evaluation Program (L￾MEP) between June and October 2013 by a six person team: Mark T. Bassie, Education Evaluation and Mon￾itoring Specialist; John C. Dalton, Senior Institutional Development Specialist; David P. Evans, Team Leader and Senior Education Evaluation Specialist; Mariama Koon, Logistics Coordinator; Frank R. Schorn, Senior Teacher Training Specialist; and John Y. Sumo, Senior Liberian Education Expert. The team was ably assist￾ed in the field evaluations by Romelus Amnon, Prince Bawoh, Moses Borlay, Ted Johnson, Patience Suah, and Sayku Waritay, all employed by the Liberia Teacher Training Project (LTTP). Over the course of the assessment, the team met and worked with many people who provided enthusiastic support to our efforts and we wish to thank them all, even if we cannot name them here. We wish to give special thanks, however, to the following people: • Etmonia David Tarpeh, Minister of Education, Republic of Liberia • Dr. Julia Richards, Education Team Leader, USAID/Liberia • Michael V. Blundell, Chief of Party, Liberia Teacher Training Program (LTTP) • Casey McHugh, Education Program Support Specialist, USAID/Liberia • Dr. Albert B. Coleman, Senior Policy Advisor, Ministry of Education, Republic of Liberia • Dr. Walter N. Phillips, Associate Director Policy, Planning and Systems, LTTP • James Z. Whawhen, Chief of Party, Liberia Monitoring and Evaluation Program (L-MEP) • George K. Werner, Director General of the Civil Service, Civil Service Agency, Republic of Liberia • E. Othello Gongar, Former Minister of Education and Commissioner, Governance Commission, Republic of Liberia • Trokon B, Wayne, Reading First and Math Team Leader, LTTP • Advertus O. Wright, Director, Zorzor Rural Teacher Training Institute, Ministry of Education, Republic of Liberia • Precious Dennis, Director, Kakata Rural Teacher Training Institute, Ministry of Education, Republic of Liberia • Reverend Quie, Director, Webbo Rural Teacher Training Institute, Ministry of Education, Republic of Liberia LIBERIA TEACHER TRAINING PROGRAM PHASE II vi ACRONYMS AND ABBREVIATIONS AED Academy for Educational Development AfT Agenda for Transformation through Action AM Assistant Minister AMFA Assistant Minister for Fiscal Affairs AMTE Assistant Minister Teachers’ Education AYP Advancing Youth Project BFA Bureau of Fiscal Affairs BHR Bureau of Human Resources BP Bureau of Procurement BTE Bureau of Teacher Education CD Country Director CDCS Country Development Cooperation Strategy CEB/CSB County Education Board/County School Board CEO County Education Officer COP Chief of Party CA Cooperative Agreement CS Civil Service CSA Civil Service Agency DEO District Education Officer DM Deputy Minister DMI Deputy Minister for Instruction ECE Early Childhood Education ECCD Early Child Care Development EGMA Early Grade Math Assessment EGRA Early Grade Reading Assessment EMIS Education Management Information System EU European Union FGD Focus Group Discussions FTI Fast Track Initiative LIBERIA TEACHER TRAINING PROGRAM PHASE II vii GC Governance Commission GDP Gross Domestic Product GEMS Governance and Economic Management Support Project GNI Gross National Income GNP Gross National Product GOAL Girls Opportunities to Access Learning GOL Government of Liberia GPE Global Partnerships for Education HICD Human and Institutional Capacity Development HR Human Resources ICT Information and Communication Technology IRC International Rescue Committee IFESH International Foundation for Education & Self Help IFMIS Integrated Financial Management Information System INGO’s International Non-Governmental Organizations IP Implementing Partners ISTT In-Service Teacher Trainers KRTTI Kakata Rural Teacher Training Institute L-MEP Liberia Monitoring and Evaluation Program LTTP Liberian Teacher Training Program MCID Mississippi Consortium for International Development M&E Monitoring & Evaluation MIA Ministry of Internal Affairs MOE Ministry of Education MOF Ministry of Finance NCAE National Council for Adult Education NCHE National Commission for High Education NEAB National Education Advisory Board NGOs Non-Governmental Organizations OD Organizational Development PFM Public Finance Management PFMRAF Public Finance Management Risk Assessment Framework LIBERIA TEACHER TRAINING PROGRAM PHASE II viii PMEP Performance Monitoring and Evaluation Program PMP Performance Management Program PRS Poverty Reduction Strategy PSTT Pre-Service Teacher Trainers PTA Parent Teachers Association RTI Research Triangle Institute RTTIs Rural Teacher Training Institutes SIP School Improvement Plan SMB School Management Board SMC School Management Committee SOW Scope of Work SOP Standard Operating Procedure STTA Short Term Technical Assistant TBD To Be Determined TTs Teacher Trainers TTIs Teacher Training Institutes UL University of Liberia UN United Nations UNDP United Nations Development Program UNICEF United Nations International Children’s Education Fund USAID United States Agency for International Development VSAT Very Small Aperture Terminal WB World Bank WRTTI Webbo Rural Teacher Training Institute ZRTTI Zorzor Rural Teacher Training Institute LIBERIA TEACHER TRAINING PROGRAM PHASE II 1 EXECUTIVE SUMMARY BACKGROUND This is a report on the mid-term assessment of the Liberia Teacher Training Program Phase II (LTTP II). The LTTP II is a five-year project that focuses on three areas (components): (i) strengthening the institutional capacity, policymaking and systems of the Ministry of Education (MOE), particularly those systems necessary to enable teachers to provide quality services; (ii) supporting pre-service and in-service teacher training and creating a reliable, transparent system for teacher recruitment, certification, promotion and compensation; and (iii) support to the national plan to ensure all children are reading by grade 3 and introducing an early grade reading and math curricula in a selected sample of schools. The LTTP II is implemented under a Co￾operative Agreement with FHI 360 and the Research Triangle Institute (RTI) International. FHI 360 is re￾sponsible for components 1 and 2, as well as pre- and in-service aspects of component 3, while RTI works on early grade reading and math aspects of component 31. The purpose of this mid-term assessment is three fold: 1) to inform USAID/Liberia, its implementing part￾ners, and other stakeholders about how well the program’s activities are contributing to the expected results, 2) to provide guidance to the implementing partners for effective operation of the project to achieve those results over the remaining life of the project, as well as 3) to inform future USAID investments in education in Liberia. To achieve these purposes and objectives, the assessment team used four methods to gather information, including a review of a variety of relevant documents, one-on-one interviews with key stakeholders, focus group discussions, and classroom observation. The assessment was conducted in five counties in Liberia (Bong, Lofa, Nimba, Montserrado, Margibi) over a four week period from June 17th to July 14, 2013. In to￾tal, the assessment team visited 14 schools and interviewed more than 300 people. The key results expected to be achieved under the LTTP project are: • Result 1: “MOE, CEO, DEO and RTTI capacity strengthened to plan, manage and monitor educa￾tional services.” • Result 2: “Improved teacher policy and procedures for teacher recruitment, training, deployment and career development.” • Result 3: “Improved teacher training programs and reading/math delivery systems.” For ease of presentation, the assessment’s findings are organized under each of the three main results intend￾ed for the LTTP program. 1 RTI International leads the design and implementation of early grade reading and math intervention in grades 1, 2, and 3. While RTI Interna￾tional provides support (by providing materials and training to instructors) to both in-service and pre-service components that are implemented by FHI 360, RTI International has limited decision making with respect to design, activities, or management of either pre- or in-service approaches. Furthermore, RTI International has only been given a budget to cover labor-related costs; all other costs required for the implementation of the project, e.g., materials, workshop costs, are managed by FHI 360. LIBERIA TEACHER TRAINING PROGRAM PHASE II 2 FINDINGS BY RESULT AREA Overall, the LTTP has made progress toward its proposed results in all three of the program result areas. Over the first three years, the program was able to make good progress during periods where there was con￾tinuity at the Minister and Deputy/Assistant Minister level at the Ministry of Education (MOE), but the fre￾quent turnover of top staff within the MOE often thwarted progress. Recently, the LTTP appears to be large￾ly stymied by internal capacity constraints and the inability or indifference of the MOE to implement program reforms. Under Result 1, “MOE, CEO, DEO and RTTI capacity strengthened to plan, manage and monitor educa￾tional services,” the LTTP has made good progress in helping the MOE to develop the blueprint for institu￾tional reform (Education Reform Act of 2011), developing and expanding the EMIS system, and introducing some needed aspects of personnel and payroll reform. Moreover, the LTTP has provided capacity building training to officers at the MOE’s central office, as well as for staff at the level of County Education Officers (CEOs), District Education Officers (DEOs) and Rural Teacher Training Institutes (RTTIs). Despite this progress, the MOE still has much to do to improve its ability to prioritize and operationalize planning and financing educational services, integrate and manage teacher development and career planning within the hu￾man resource management system, and implement the Education Reform Act, which was heralded as the way forward for fundamental changes in educational management and service delivery in Liberia. Under Result 2, “Improved teacher policy and procedures for teacher recruitment, training, deployment and career development,” the LTTP has taken preliminary steps to improve policy and procedures for teacher recruitment, training, deployment and career development by developing a series of “White Papers” and anal￾yses outlining policy options aimed at providing the foundation for the MOE to improve teacher policy and procedures. Unfortunately, the turnover of Ministers, Deputy Ministers and Assistant Ministers has thwarted adoption of the suggested policy and procedures, making for relatively little progress. In Result 3, “Improved teacher training programs and reading/math delivery systems”, the LTTP has made some improvements in teacher training programs, and has trained 1,046 of teachers in pre-service programs and in-service programs. In addition, the LTTP has made a strong start in improving reading and math deliv￾ery through the training of teachers in reading and math methods, and introduced new training and learning materials in about 800 schools. Many teachers, however, appear to be only partially trained and most teach￾ers, principals and even the coaches hired to assist the teachers assert that they need further training. Moreo￾ver, the lack of counterpart resources from the MOE may suggest that the program might be unsustainable. LESSONS LEARNED Capacity Constraints at the MOE Are Overwhelming: The MOE is almost bereft of systems and Standard Operating Procedures (SOP). Aside from the standard Government-wide financial management system, most other internal management systems, including the planning, procurement, human resource, monitoring, evaluation, and professional development systems essential for MOE administration are weak or non￾existent. Budgetary Constraints Severely Limit MOE Performance and Requires Strategic Thinking: Liberia invests one of the lowest percent of GDP (2.8%) in the world to education and ranks 147 out of 167 countries. Only four so-called “failed states” are ranked lower in Africa. Liberia’s neighbors invest considerably more in edu￾cation (Guinea 3.1%, Sierra Leone 3.6%, and Cote D’Ivoire 4.6%) and the current funding level is insufficient to keep pace with population growth. The MOE needs to become much more strategic in its operations, and LIBERIA TEACHER TRAINING PROGRAM PHASE II 3 it needs to develop an implementable strategic plan with realistic objectives and targets based on an accurate and realistic costing of needs. The Government should consider guaranteeing a specific percentage of nation￾al revenues – not a percentage of either gross revenues, including external funds, or total estimated expendi￾tures - to support education, not only as a demonstration of its commitment to national human capital devel￾opment, but also to insure that donor funding actually increases educational access and quality. Donor Financing Drives the Education Agenda: Given the lack of budgetary resources and a weak strategic plan that is little more than a “wish list,” the MOE exerts limited influence in determining the future direc￾tions of education in Liberia. In FY 2013 donor funding in education was approximately $US 59 million, which is greater than Government of Liberia’s (GOL) resources provided to the MOE ($US 45.5). With such meager resources, the MOE is eager to accept almost whatever is offered by donors, even if it not aligned with MOE priorities. The level of funding allocated to the MOE by the Government is insufficient to sup￾port effective learning environments or to sustain donor investments. Consequently, much of the potential impact of donor support may be lost. Centralized Administration: Although the MOE has begun the process of decentralization, very little real progress has been made and the Ministry is still very centralized. The MOE has developed an organizational chart and staffing patterns for both the central MOE and decentralized staff at the county level and five key officers (accounting, human resources, monitoring and evaluation, procurement, and planning) have been assigned to each of the fifteen county CEOs. In addition, data on schools and school staffing is being col￾lected and analyzed for better, evidence-based decision making. Despite this apparent progress, the decentral￾ization process has been largely ignored by senior MOE managers, and there does not appear to be a com￾mitment to improving the management of educational services through implementation of the Education Reform Act. SERVICE DELIVERY IN A CAPACITY VOID A compelling case can be made to improve the quality of education by improving the capacity and perfor￾mance of teachers, improve the methods and materials used to teach early grade reading and math. Unless this is done, another generation of young Liberians will grow up with very low levels of literacy, numeracy and other skills. Notwithstanding that compelling case, it is clear that the capacity of the MOE to capitalize on the LTTP’s resources and implement an improved service delivery program was overestimated. Conse￾quently, progress in future service delivery programs will be similarly jeopardized until the MOE strengthens its institutional capacity and planning, management, human resource, and financial systems. FOCUS AND CONCENTRATE Unless the MOE makes a determined effort to fix the broken systems and put in place guidelines, operating procedures and working systems (many tools have been developed by the LTTP, but await review and ap￾proval by the MOE), much of the work of the LTTP has done will be stalled on a number of key fronts. Both the MOE and the LTTP implementers need to focus and concentrate on institutional strengthening and putting systems in place. Reexamine Expansion of Service Delivery Until the MOE is Strong Enough to Absorb the Assistance: Alt￾hough the early grade reading and math program appears to have made a strong start and is widely heralded by teachers and administrators, the LTTP staff implementing the program, as well as most teachers, and school administrators, suggest that the program is spread too thinly. The lack of MOE counterpart funds, LIBERIA TEACHER TRAINING PROGRAM PHASE II 4 along with other factors cited above, suggests that the program may not be sustainable in the medium-term. USAID/Liberia and the LTTP implementers may want to reexamine the programs’ output targets, hold back on the planned expansion, and use the savings to reinforce the gains already made. White Elephants: The MOE and USAID need to undertake a serious review of teacher training institutions in Liberia. The three rural, residential RTTIs in Liberia are high-cost models for delivering teacher training in one of the poorest countries in the world, with an underfunded MOE that is struggling to provide basic edu￾cation services of any kind. All three institutions appear to be over-staffed with a teaching faculty that is re￾ported to be lackluster, at best. Moreover, it is not clear that the institutions are providing the correct balance of pre- and in-service training; the correct mix of pedagogy and subject matter content; or the right length of training, given their students’ needs. Data for What? : The Education Management Information System (EMIS) has made great strides and pro￾vided significant information that has begun to influence MOE planning and decision-making. At the same time, the EMIS has very little information about issues such as equity, quality and relevance and largely re￾ports data without much policy analysis. The EMIS could be most useful for developing evidence-based de￾cision-making within the MOE but, given the MOE’s lack of financial resources to provide basic services, it seems unlikely that the EMIS data will lead to a significant change in the MOE’s decision-making in the near￾to medium-term. The LTTP should continue the strong start it has made in the EMIS and work to improve the analytical capacity of the MOE, but needs the strong support and commitment from senior MOE offi￾cials. CONCLUSION The MOE and the staff of the two organizations responsible for implementing the LTTP project, i.e., FHI 360 and RTI, are to be commended for their diligence and commitment to supporting education reform in Liberia despite the high turnover in senior management at the MOE. There are many daunting challenges to improving education and teacher training in Liberia and the assessment team hopes that this report will help to highlight some of the efforts that need to be made to improve delivery of education services for the chil￾dren of Liberia. LIBERIA TEACHER TRAINING PROGRAM PHASE II 5 I. INTRODUCTION A. PURPOSE OF THE ASSESSMENT This is a report on the mid-term assessment of the Liberia Teacher Training Program Phase II (LTTP II). The LTTP II is a five-year project that focuses on three areas (components): (i) strengthening the institutional capacity, policymaking and systems of the Ministry of Education (MOE), particularly those systems necessary to enable teachers to provide quality services; (ii) supporting pre-service and in-service teacher training and creating a reliable, transparent system for teacher recruitment, certification, promotion and compensation; and (iii) support to the national plan to ensure all children are reading by grade 3, and introducing an early grade reading and math curricula in a selected sample of schools. The LTTP II is implemented under a Co￾operative Agreement with FHI 360 and the Research Triangle Institute (RTI) International. FHI 360 is re￾sponsible for components 1 and 2, as well as pre- and in-service aspects of component 3, while RTI works on early grade reading and math aspects of component 3. The purpose of this mid-term assessment is three fold: 1) to inform USAID/Liberia, its implementing part￾ners, and other stakeholders about how well the program activities are contributing to the expected results; 2) to provide guidance to the implementing partners for effective operation of the project to achieve those re￾sults over the remaining life of the project; and 3) to inform future USAID investments in education in Libe￾ria. More specifically, the mid-term assessment will attempt to achieve the following objectives: • Assess implementation progress and constraints, estimate the probability of achieving target results and determine possible causes for success (failure) to reach those targets; • Identify any deficiencies in the program design and recommend possible solutions to over￾come those deficiencies; • Assess the effectiveness of the LTTP approaches, activities and delivery systems and man￾agement structure and determine whether or not this program, or aspects of the program, can or should be replicated in future interventions; • Assess the potential for the Ministry of Education (MOE) to sustain the various aspects of the teacher training program; • Identify the major data, research or knowledge gaps on teacher training in Liberia and pro￾vide recommendations on what type of studies should be undertaken to inform future de￾sign of education programs in the areas of: 1) teacher training and professional development; 2) early grade reading and literacy; 3) systems strengthening and Human and Institutional Capacity Development (HICD); 4) education policy reform; and 5) decentralization of the MOE. • Incorporate a broad political economy contextual analysis of the LTTP project, with special emphasis on MOE payroll reform, educational access, educational quality, and de￾concentration and decentralization within the MOE; and • Provide specific, actionable recommendations regarding the nature and scope of possible fu￾ture interventions in the basic education sector, including but not limited to: early grade reading, teacher training, teacher management and professional development, information LIBERIA TEACHER TRAINING PROGRAM PHASE II 6 systems, and HICD. This analysis of possible future interventions will identify which inter￾ventions appear to be most appropriate and feasible, which interventions might best build on past efforts, and which interventions might show less promise and should be avoided. To achieve these main objectives, the assessment team was guided by the 78 USAID-generated assessment questions outlined in Section III of the Scope of Work (SOW) (Please see Annex 1 for the SOW) B. METHODOLOGY AND APPROACH To achieve these purposes and objectives, the assessment team used four methods to gather information, including: Document Review: A document review was conducted of all relevant LTTP I and II documents as well as other contextual information that could inform the team on key issues. USAID/Liberia provided the team with the materials to inform the assessment. In addition to the documents provided by USAID/Liberia, the assessment team reviewed documents provided by the MOE, other donor, as well as academic research and technical papers on selected topics such as early grade reading, decentralization, or HICD. Moreover, the assessment team carefully reviewed other documents such as the Government of Liberia’s (GOL) develop￾ment strategies, the USAID Country Development Cooperation Strategy (CDCS), the Results Framework and the Performance Monitoring and Evaluation Plan (PMEP) of USAID/Liberia with specific emphasis on its education program, and USAID agency-wide documents such as the Evaluation Policy and education strategies (a complete list of documents can be seen in Annex 2). One-on-one Interviews with Key Stakeholders: The assessment team conducted extensive one-on-one inter￾views with key stakeholders at the national, county, district, and school levels. In greater Monrovia, the team interviewed key personnel at the LTTP office, including the implementing partners staff; MOE staff at the central ministry level, including the Minister and her chief advisors, the Deputy and Assistant Ministers, and some key mid-level operational personnel; education experts at key donor agencies, such as the European Union, United Nations (UN) agencies, and other bilateral donors. Away from the capital, the assessment team interviewed County Education Officers, and key staff, including the county officers responsible for planning, accounting, human resource management, monitoring and evaluation, and procurement; the Dis￾trict Education Officers and staff; administrators and teacher trainers at the Rural Teacher Training Institutes (RTTIs); and school principals, teachers, parents, and community members. (A complete list of Key Inform￾ants is contained in Annex 3). Focus Group Discussions: Focus Group Discussions (FGD) were conducted, most notably when the oppor￾tunity presented itself to obtain information from large numbers of people at one time, or when one-on-one interviews with some of the direct and indirect beneficiaries were not possible. Focus group discussions were held with GOL and MOE administrators, pre-service teacher trainees at the RTTIs, teacher trainers at RTTIs, DEOs, teachers, and civil society members. At the focus group meetings the assessment team used a smaller set of questions derived from the overall questions developed for that specific group, e.g., a focus group questionnaire guide for teacher trainers was taken from the larger set of questions developed for teacher trainers in general. Field Observations: Field observations were made by visiting county and district offices and schools. Wher￾ever possible, the assessment team members observed classroom teaching practices. We inspected schools and classrooms and explored with administrators, principals, and teachers current methods for managing LIBERIA TEACHER TRAINING PROGRAM PHASE II 7 teaching and learning and the importance given to reading in the system. Samples of checklists used for schools, teachers, teacher trainers, are attached in Annex 5. The evaluation team used a semi-structured questionnaire for each of the client groups to guide the interviews and a scorecard/checklist to identify gaps in MOE systems and procedures. (Annex 6) The questionnaires were used to guide the interviewer and were valuable references to ensure that all USAID’s questions were answered. This was especially true since the assessment team members often needed to split into smaller teams and all team members, including the Liberian team members, were expected to interview a cross￾section of stakeholders. The interviewers also asked more in-depth questions as the interviewee and time permitted. The scorecard/checklist related to the MOE’s management and systems capacity was compiled based on interviews with senior managers at the MOE and inputs from the LTTP consultants. The field as￾sessment was conducted in five of the counties in Liberia (i.e., Bong, Lofa, Nimba, Montserrado, Margibi) and included two of the three RTTIs. It was carried out over a four week period between June 17th and July 14, 3013. The assessment team visited a total of 14 schools and interviewed more than 300 individuals. C. CONSTRAINTS TO UNDERTAKING THE ASSESSMENT The assessment team experienced some constraints in carrying out the assessment. First, half way through the assessment one of the three expatriate investigators on the team, Dr. Frank Schorn, passed away. Conse￾quently, the assessment team was able to gain only a small portion of his insight on the issues related to teacher training. It must be assumed that the report would have been enriched by his sustained involvement. Fortunately, the late Dr. Schorn had been teamed with an astute Liberian observer of the education sector and the team leader participated in most of the interviews led by Dr. Schorn. Moreover, the team held exten￾sive discussions daily and Dr. Schorn’s findings and views were well-known to all of us on the assessment team. The school year in Liberia closes at the end of June, with the last week of the school year reserved for exami￾nations. Since schools only were in session during the first few weeks of the assessment, classroom observa￾tion was possible for only about ten days. The push to conduct classroom observations and interviews with school personnel before schools closed meant that the assessment team had not completed the interviews with the LTTP II central staff prior to the school visits. Although it is not likely that these premature visits to the schools and the rural counties introduced bias into the investigation, it is possible that the assessment team missed some potentially important points. On the other hand, the first-hand observations and inter￾views with school personnel implementers possibly allowed the assessment team to probe more deeply when interviewing the LTTP II central staff upon the team’s return from the field. The very presence of the assessment team at the schools introduced some bias in the observations. Moreo￾ver, the conversations with principals, teachers, and community members were at times truncated and some￾what stilted by the chaos surrounding the visit. Nevertheless, observations of school facilities; books, learn￾ing and teaching materials; teachers performing in the classroom; conversations with teachers, principals, coaches, CEOs and DEOs; as well as the shortcomings in financial support for classroom education, did al￾low the team to form sound impressions about the management of teaching and learning in the system. Given the nature of the assessment, i.e., to learn to what extent the project’s objectives are on track, the as￾sessment team selected a purposeful sample of key officials for interviews. As a result, the sample is by its nature not random and cannot yield statistically significant objective results. LIBERIA TEACHER TRAINING PROGRAM PHASE II 8 Since the road surfaces in Liberia are often rutted and/or dirt, and the distances between settlements are of￾ten large, the team spent an inordinate amount of time in four-wheel drive vehicles traveling from site to site. As a result, the sample of schools is relatively small and the assessment team was forced to limit its interview topics. However, despite these constraints the assessment team has confidence that the findings represent a true picture of current education management practices in Liberia. D. STRUCTURE OF THE REPORT The next section of the report and the Background section, reviews the development challenges in Liberia and for the Government of Liberia (GOL) and outlines the GOL’s strategies and USAID/Liberia’s efforts to address those challenges. Section three of the report reviews the findings of the assessment, which are orga￾nized according to objectives, the three result areas, as well as program management and design issues. The fourth section reviews the lessons learned and conclusions from the assessment. Finally, the fifth section of￾fers some recommendations for future programming for the remaining life of the program, as well as for sus￾taining LTTP accomplishments in institutional strengthening, capacity building and teacher training. LIBERIA TEACHER TRAINING PROGRAM PHASE II 9 II. BACKGROUND A. DEVELOPMENT ENVIRONMENT AND CHALLENGES Liberia, the oldest republic in Africa, is one of the poorest and least developed countries in the world. Nearly any comparison of poverty indicators or human development factors place Liberia near the bottom of sub￾Saharan African Countries. For example, the Human Development Index for 2012, a composite statistic of life expectancy, education, and income, places Liberia 174 out of 186 countries.2 The Government of Liberia (GOL) estimated that nearly 64 percent of Liberians live below the poverty line and 48 percent live in ex￾treme poverty.3 It is estimated that more than 40 percent of the population has a food consumption pattern that cannot sustain a healthy life4 and life expectance at birth is less than 58 years, placing Liberia 197 out of 223 countries in the world. Poverty and under-development are not the only challenges Liberia faces. Many years of minority rule and inequitable distribution of resources resulted in a civil war that lasted from 1989 to 2003. The period was one of widespread destruction and great hardship with at least 200,000 Liberians dead5 , more than a million dis￾placed to refugee camps in neighboring countries, families shattered, basic infrastructure destroyed including roads, water and electricity supply, schools and health clinics. Social, political, and government systems at all levels were disrupted. With 16 different officially recognized ethnic groups, and a deep division in access to basic services between urban and rural areas, there are many unhealed divisions and the social fabric is torn by ethnicity, religion, geography and history. Skilled Liberians left the country in droves, and there is report￾edly widespread cronyism, corruption, economic mismanagement, and almost a total lack of operating sys￾tems. GDP fell by 90 percent between 1987 and 1995. Even on the eve of elections for a new government in 2005, average income was only one-sixth what it had been in 1979. Over the last ten years, there have been substantial achievements and progress, but the gains that have been made are fragile. Much more needs to be done to sustain the accomplishments, and there are still many daunting challenges. Measures of nutrition and health status and access to basic services all show Liberia’s low starting point and how far the country has to go to meet its development goal of becoming a middle income country6 by 2030. Liberia’s Gross National Income (GNI) in 2011 was estimated to be $2657per capita and only two countries in the world had lower per capita GNI. One major challenge is to improve the capacity of the government at all levels in all Ministries. Currently, there is a thin veneer of skilled managers and techni￾cians who are capable of planning, implementing, managing, and monitoring complex investment programs. Many senior and mid-level officials fled the country and, while some have returned from the Liberian Diaspo￾ra, many have not. In addition, the country lost a generation during the war to poor and interrupted educa￾tion, leaving a dearth of qualified people working at the middle ranks. Exacerbating the challenge to effective government are the fault lines that fracture the country along ethnic and religious lines, between the “haves” and “have-nots,” between the urban and rural residents, and between 2 United Nations Development Report, Human Development Index (HDI), March, 2013 3 Government of Liberia, Poverty Reduction Strategy, 2007. p 16 4 The State of Food and Nutrition Survey in Liberia, Comprehensive Food Security and Nutrition Survey, 2010, p 2 5 Other sources give estimates up to 350,000 people. US State Department Background Note on Liberia 6 Defined by the Liberians as a per capita income of more than US $1,000 7 United States Agency for International Development, Country Development Cooperation Strategy, 2012. p 11 LIBERIA TEACHER TRAINING PROGRAM PHASE II 10 the descendants of repatriated ex-slaves from the U.S. and indigenous Liberians. While these divisions have been part of the country’s past, they were heightened by the war and added layers of distrust and suspicion, which makes it more difficult to restore stability to the country. The government must convince those that have been marginalized in the past that there are new standards of inclusion and fairness in access to services like education, despite the severe financial limitations of the Government. The government must also confront the legacies of Liberia’s past approach to governance and service deliv￾ery, including a dismal record of performance and pervasive corruption. Liberia is a country where historically the country’s revenues and modest wealth benefited the powerful few and where patronage networks created powerful systems dedicated to inertia. Within this environment, the Liberian state often has weak legitimacy and may be seen as having limited interest in creating responsive, accountable, transparent government. More than 61 percent of the Liberian population is under 24 years old and the median age of the population is 17.9 years.8 The impact of the civil war on the Liberian youth cohort was substantial with many exposed to the horrors of war, the erosion of social values, institutions and their communities, and limited opportunities for education. Without adequate education or social or job skills, youth unemployment and underemploy￾ment remain high, creating a drag on the economy and making it difficult for the government to finance the volume of needed basic services. For example, the explosion in school-aged children and the small number of adult wage earners makes it difficult for the GOL to meet its fundamental obligation of providing adequate resources for education. Unfortunately, it appears that whenever donors have provided supplementary fund￾ing, the Government has reduced its own commitment. These, and many other challenges, suggest that there is a limited window of opportunity for the current gov￾ernment to move ahead to implement the reforms urgently needed and there is a need to show real results in the face of these daunting challenges. At the same time, the current development setting in Liberia is one of high-risk and the GOL must show the political will to effectively implement critical reforms or the opportuni￾ties will be lost. B. OVERVIEW OF EDUCATION IN LIBERIA The MOE under the constitution and laws of Liberia is charged with the responsibility to manage and super￾vise all schools and higher institutions of learning. There have been many changes in the Liberia education sector over the past five decades due to the enormous problems that have beset the country. The policy mak￾ers have reorganized and expanded the mission of education through the Education Act of 1972, which was repealed and replaced by the Education Act of 2001, which itself was recently amended, then replaced by the new Education Reform Act of 2011. The Education Reform Act of 2011 reorganized the MOE’s internal structure, established three key building blocks for reform, the so-called centers for excellence in education management, curriculum development and certification (of teachers) and accreditation (of institutions), and adopted a decentralization mandate. Af￾ter two years, the promise of reform has not been matched by any sustainable implementation actions. The organization structure of the Ministry of Education is depicted below. 8 CIA, World Fact Book, 2013 LIBERIA TEACHER TRAINING PROGRAM PHASE II 11 The Ministry is headed by a Minister who is assisted by three Deputy Ministers and eight Assistant Ministers, all are politically appointed and in charge of the following Departments and Bureaus:  Department of Administration o Bureau of Fiscal Affairs and Human Resources o Bureau of Administration  Department of Instruction o Bureau of Early Childhood o Bureau of Basic and Secondary Education o Bureau of Teacher Education o Bureau of Science, Technical, Vocational and Special Education o Bureau of Student Personnel Services  Department of Planning, Research and Development o Bureau of Planning, Research and Development  Policy Unit  Research (EMIS related) Unit  Planning and Reporting Unit LIBERIA TEACHER TRAINING PROGRAM PHASE II 12  External Financing/Donor Coordination Unit  Planning Unit In addition to these line departments, the MOE also has other staff positions such as Internal Auditor, Coun￾selor, and Scholarship Coordinator. While the new structure accurately reflects the service delivery mandates of the Education Reform Act, it is also worth noting that the bureaus are each headed by an Assistant Minister. The Minister is also assisted in performing the mandated duties by officials in the counties and districts around the country. In terms of education management, the counties are managed by 15 County Education Officers (CEOs) and further sub-divided into 106 districts that are administered by District Education Offic￾ers (DEOs). The Liberian Education System has several divisions as outlined below by the MOE:  Early Childhood Development (ECD) or Pre-primary education which starts at about age three to five years of age and includes nursery school and kindergarten.  Primary education (Grades 1 to 6) officially starts at age 6 and is available in public, private, reli￾gious/mission-sponsored, and community based schools.  Junior secondary schooling (grades 7- 9) starts at 12 years. The system has combined primary and junior secondary education levels to form “Basic Education”, which cover the first nine year period. After the completion of basic education, a student is expected to be sufficiently prepared for continuation of advanced education, to undertake skills training, or to get a job.  Senior secondary is three years of senior high schooling (grades 10 -12) starting at age 15 and ending at 17 years.  Tertiary level institutions include: Community/Junior Colleges, which offer baccalaureate and associ￾ate degrees; teacher training colleges; universities; and vocational and technical institutions. Teacher Education: Given the death, retirement or emigration of teachers during the civil war, there is a need for trained teachers at all levels throughout the country. Training to upgrade the teaching force is being ac￾commodated through a combination of in-service and pre-service training. Primary school teachers are trained in both pre-service and in-service programs, as appropriate. For example, the in-service teacher train￾ing program provides intensive remedial training for uncertified teachers who have been teaching without formal preparation. Upon successful completion of the in-service training program they receive a so-called “C” certificate, a formal accreditation by the MOE for primary school teachers. Training for teachers at the lower secondary level (“B” certificate for grades 7 –9) has not been re-introduced by Government since the war. Education and training for secondary level teachers (“AA” certificate, bachelor and advanced degrees) are offered at the country’s few colleges and universities such as the University of Liberia, Cuttington Uni￾versity, William V.S. Tubman University, Grand Bassa Community College, as well as faith based institutions around the country. The government has three Rural Teacher Training Institutes (RTTI) located at Zorzor, Kakata and Webbo that are entrusted with implementing the pre-service “C” certificate teacher training pro￾gram. These RTTIs provide the same C Certificate to raw recruits with no prior exposure to educational prin￾ciples or any classroom experience, since the MOE provides for current teachers who complete the in-service LIBERIA TEACHER TRAINING PROGRAM PHASE II 13 training discussed above. This report highlights the weakness of this approach and suggests possible alterna￾tives for consideration by Government. C. GOVERNMENT OF LIBERIA EDUCATION PLANS, PRIORITIES AND STRATEGIES The 14 year civil war was debilitating and touched every sphere of life in the country and the education sector was among the hardest hit. As a result, the Government of Liberia (GOL) has identified education as a major national development priority. The Ministry of Education (MOE) has undertaken a series of specific education policy instruments to guide the revitalization process. In addition, Government-wide planning documents include strategies for the social sector, which encompasses education. Pertinent among the new policy instruments are the “Education Reform Act” of 2011, “Agenda for Transformation through Action” (AfT) and the “Education Sector Plan”. All of these doc￾uments contain details of the education sector’s development plans and priorities, as well as the strategies re￾quired for achieving them. Based on these documents, the priorities of the MOE are: Management and Financing the Education System: Under this priority area, the Ministry of Education intends to develop the institutional capacity for management at the central, regional and local levels, and to increase planning, budgeting, and management capacity. The Government also plans to increase accountability and transparency throughout the education system, and to strengthen the decentralized system of governance. Strengthen Human Resource Planning, Management and Development: MOE will develop and utilize an effective human resources management strategy and error-free database, including (i) reviewing, creating and distributing job descriptions to all MOE personnel; (ii) conduct a skills audit to ensure that personnel have the necessary skills and qualifications to undertake their jobs; (iii) provide training, as needed, to enable per￾sonnel to perform their roles and responsibilities efficiently and effectively; (iv) develop and utilize a perfor￾mance appraisal system; and (v) develop and utilize an effective system of monitoring, supervision and re￾porting. Prepare a Cadre of Trained and Certified Primary School Teachers: The Ministry of Education plans to pro￾vide competent, well-qualified and motivated teachers for all educational institutions. To achieve this goal, the Ministry has the following specific policy objectives: (i) provide adequate training and professional develop￾ment programs for teachers from the pre-primary to tertiary levels; (ii) put in place arrangements and a framework that result in teachers becoming motivated and supported to carry out their responsibilities; (iii) make provisions for the deployment of teachers to rural and under-served areas; (iv) increase the number of females in the teaching profession; and (v) improve efficiency in teacher management. Improve the Accountability and Capacity of MOE: The Ministry of Education intends to develop monitor￾ing and supervisory mechanisms, procedures and practices that will provide useful management information to guide the MOE’s actions and policies. The specific objectives under this initiative include: (i) strengthen the capacities of District Education Offices and County Education Offices in monitoring and supervision, including classroom assessment of student learning; (ii) strengthen record keeping capacity at the school/community level; (iii) decide on a set of outcome indicators for monitoring progress towards educa￾tion sector program objectives and outline mechanisms for monitoring plan progress; (iv) improve the Edu￾cation Management Information System (EMIS), the analysis and use of EMIS data; and (v) develop a decen￾tralization policy and strategy with built-in transparency and accountability mechanisms. LIBERIA TEACHER TRAINING PROGRAM PHASE II 14 D. CONGRUENCE OF USAID/LIBERIA’S COUNTRY DEVELOPMENT COOPERATION STRATEGY (CDCS) WITH GOL STRATEGIES Liberia’s medium-term development strategy, the Agenda for Transformation through Action (AfT) as well as the National Vision 2030, aims for Liberia to become a middle income country by 2030 and to increase inclusive￾ness for a more equal and just society.9 Accordingly, USAID/Liberia’s strategic goal is: “Strengthened Libe￾rian Institutions Positioned to Drive Inclusive Economic Growth and Poverty Reduction.” The development hypothesis underlying this goal is that in order to achieve long-term development progress, Liberia must es￾tablish and strengthen inclusive, indigenous institutions that enable it to effectively mobilize its own resources and efficiently manage those resources for development purposes. To achieve this strategic goal, USAID/Liberia has established four development objectives (DO):  DO 1 More effective, accountable, and inclusive governance  DO 2 Sustained, market-driven economic growth to reduce poverty  DO 3 Improved health status of Liberians  DO 4 Better educated Liberians Insofar as progress toward one of these DOs can be inhibited by failure in the other DOs, these four DOs are seen to be mutually reinforcing and progress must be made in all four DOs, if the strategic goal is to be achieved. In addition, Human and Institutional Capacity Development (HICD) is a core, crosscutting strate￾gic priority in the USAID/Liberia portfolio. As a result, USAID/Liberia is pursuing a two track approach: 1) build and strengthen human and institutional capacity to sustain development progress and, in the shorter run; and 2) improve access to critical goods and services needed to achieve the development goal. Several other features of the USAID/Liberia’s Country Development Cooperation Strategy (CDCS), which was approved in May 2013, are important to note. First, to foster more equitable growth and improve the delivery of government services, a key strategic decision is to support the GOL’s aim of incremental decen￾tralization to the county and district levels. Second, in line with the GOL’s phased decentralization strategy, USAID assistance will focus in six highly–populated counties (Lofa, Bong, Nimba, Grand Bassa, Margibi and Montserrado), which represent 75 percent of Liberia’s population and are seen to be located in Liberia’s key economic development corridor. Third, the strategy will aim to lock in achievements and make them irre￾versible, in part by empowering civil society organizations as accountability watchdogs. Since women and girls are key actors in the development process, but have been particularly disadvantaged in access to basic services, USAID/Liberia has a strategic focus on addressing gender and other inequities. Finally, USAID/Liberia aims to focus its efforts in such a way to reduce harmful environmental impacts. 9 Government of Iberia, Agenda for Transformation through Action, p 36 LIBERIA TEACHER TRAINING PROGRAM PHASE II 15 USAID/Liberia’s Results Framework is shown below. Figure 1. Results Framework for USAID Assistance to Liberia, 2013-2017 GOAL: Strengthened Liberian Institutions Positioned to Drive Inclusive Economic Growth and Poverty Reduction DO 1: More effective, accountable, and inclusive governance DO 2: Sustained, market-driven economic growth to reduce poverty DO 3: Improved health status of Liberians DO 4: Better educated Liberians IR 1.1: Public resources managed more transparently and accountably IR 1.3: Strengthened capacity to implement election processes IR 1.4: Civil society and media exercise their advocacy and oversight roles IR 1.2: Improved policies, models and providers increase access to justice IR 2.1: Food Security Enhanced IR 2.2: Natural Resources Managed Sustainably IR 2.3: Enabling environment supports private enterprise growth IR 3.1: Increased utilization of quality services IR 3.2: More responsive services through effective health system decentralization IR 3.3: Increased financial sustainability of services IR 3.4: Increased safe water supply IR 4.1: Improved basic education opportunities IR 4.2: Improved higher education opportunities LIBERIA TEACHER TRAINING PROGRAM PHASE II 16 E. OVERVIEW OF USAID/LIBERIA’S BASIC EDUCATION STRATEGY The quality of educational services delivered and the level of learning attained in Liberia are very low. Ap￾proximately half of all adults in Liberia are illiterate,10 and more than half of all girls who reach the sixth grade are unable to read and write.11 Thirty percent of the population has never attended school, with women (52 percent) being three times more likely than men (18 percent) to have no formal education at all.12 The median years of education for the population between the ages of 15-49 years are low, with 1.6 years for women and 5.8 years for men.13 Many factors help explain the low educational achievement in Liberia. First, Liberia’s civil war left the educa￾tion system in ruins. During the war, more than 30 percent of public schools were destroyed or severely dam￾aged; 75 percent of the students had no desks or chairs, and even with low enrollment levels, only one stu￾dent in 27 had access to a textbook.14 Administrators and teachers were killed or fled the country, leaving schools with untrained and unqualified teachers or no staff at all. The human and institutional capacity of the government to manage the education system was severely diminished as people fled and/or the systems and operating procedures were lost. An entire generation of children – now youth 10 to 24 years old - had badly disrupted schooling, if any at all. Second, almost ten years after the end of the war: many school facilities are still poor; most teachers are poor￾ly qualified to teach and have limited subject matter knowledge; teaching and learning materials are almost non-existent aside from special-purpose donor-financed materials; and principals are largely untrained. The operational staff at the Ministry of Education, from the central to the local level, have limited vision and ca￾pacity; lack data and analysis for decision-making and the allocation of resources; and work in an overly cen￾tralized, apathetic, unproductive bureaucracy bloated by political patronage, including ghost teachers and ad￾ministrators. In addition, many elementary school children start late and classrooms often have a majority of children who are overage15. Late enrollment is a particular problem for girls, who tend to drop out of school when they reach puberty. Of every 100 students that start school, only 60 finish six grades. Third, the amount spent on education in Liberia is 1.9 % of GDP.16 According to the UNDP’s International Human Development Indicators, Liberia allocated 1.9% of GDP to education in 2013, which placed it at 150th out of 194 countries on the list. This decline from an already low base should be a matter of the great￾est concern to the Government of Liberia, as well as to the donors, which seem to be supplanting not sup￾plementing, Liberia’s commitments to education. Even in 2008 Liberia ranked considerably below its neigh￾bors (Guinea spends 3.1 %, Sierra Leone spends 4.3 %, Cote d’Ivoire spends 4.6 %, and Ghana 5.4 %). The World Bank estimates that 3.2 percent of GDP is the minimum needed to just maintain a national education system, so the amount Liberia spends means that it is falling behind year-by-year. By comparison, Zambia (1.3%) and the failed or fragile states of Somalia (0.4%, the Democratic Republic of the Congo (2.5 %) and the Central African Republic (1.3 %) are among those African states that allocate less than Liberia to educa￾tion. 10 Core Welfare Indicators Questionnaire (CWIQ), Liberia Health Service, 2007. 11 Ibid. 12 World Bank (2010), Liberia Education Country Status Report 13 Liberia Demographic and Health Survey (LISGIS), 2008 14 Ibid 15 Government of Liberia, Educational Statistics for the Republic of Liberia, 2013 16 World Bank, Public Spending on Education, 2013 LIBERIA TEACHER TRAINING PROGRAM PHASE II 17 Within this milieu, USAID/Liberia’s education development objective is: “Better Educated Liberians.” The development hypothesis underlying USAID’s education program is that by increasing the number of trained teachers in Liberian schools; improving teachers’ and pupils’ access to modern teaching and learning re￾sources and tools; improving instruction in basic skill areas like reading and math; and systematically strength￾ening the governance and management of Liberia’s education system and the Ministry of Education, more children and youth will attend and complete primary school or alternative basic education programs. As a result, they will succeed not only at higher levels of schooling, but also will improve their health results, in￾crease their economic productivity and employability, increase their participation in the democratic process, and improve their living standards. USAID/Liberia is taking a two-track approach to its basic education objective: 1), supporting direct service delivery interventions to boost educational access and outcomes in selected counties in the shorter-term, while 2) building the human and institutional capacity of the MOE and the educational system in general – at the central and decentralized levels – to deliver more effective and efficient educational services on a sus￾tained nation-wide basis over the longer-term. Direct, short-term interventions include dissemination of im￾proved curricula; provision of teaching and learning materials and other pedagogical resources and tools; teacher training and faculty strengthening; and public outreach and stakeholder engagement to promote in￾creased enrollment, retention and improved reading skills and learning outcomes, particularly for girls. Medi￾um- and longer-term capacity development work in this sector engages key partners and stakeholders in teachers’ professional development, information management and curricular reforms, and improved institu￾tional decision-making, planning and policy development, and monitoring and evaluation capabilities. The Mission’s programs in basic education prioritize early grade reading and youth education/ livelihood in￾terventions in the six counties targeted for USAID support. Education sector policy and systems reforms, including information and resource management, are national in scope. The principal beneficiaries of USAID’s education program in Liberia are primary school children, over-age and out-of-school youth and young adults (including ex-combatants), teachers in training, and administrators within Liberia’s educational system, at the central, county and community levels. In all of these categories, USAID assistance particularly seeks to engage girls and women, and promote their increased participation in and contributions to the development of Liberia’s educational system. The Mission’s results framework for the basic education sub-sector is: Intermediate Result 4.1: “Improved Basic Education Opportunities.” There are three sub-intermediate results, including: Sub IR 4.1.1 - Improved quality Sub IR 4.1.2 - Increased equitable access Sub IR 4.1.3 – Strengthened human and institutional capacity There are currently three core basic education programs that support the Intermediate and Sub-IRs, includ￾ing:  Liberia Teacher Training Program (LTTP) II: Provides pre‐service and in‐service teacher training to prepare a cadre of trained and certified primary school teachers; supporting the national plan to en- LIBERIA TEACHER TRAINING PROGRAM PHASE II 18 sure all children are reading by Grade 3; strengthening the policy, systems and institutional capacity necessary to support teachers and improve educational services  Girls’ Opportunities to Access Learning (MCC‐GOAL): Field tests intervention packages to deter￾mine which has the greatest potential for increasing enrollment, attendance, retention, and primary school completion rates for girls in targeted schools.  Advancing Youth Program (AYP): Enhances governmental capacity to provide increased equitable access to quality alternative basic education (ABE) services, social and leadership development and livelihoods for youth with limited or no literacy skills F. OVERVIEW OF LIBERIA TEACHER TRAINING PROGRAM (LTTP) The Liberia Teacher Training Program Phase I started in 2006 implemented by the Academy for Educational Development (AED) as the prime contractor, along with two sub-grantees –the Mississippi Consortium for International Development (MCID) and the International Rescue Committee (IRC). The program was planned for three years, beginning in November 2006, and was extended to end on May 31, 2010 to facilitate a smooth transition to the LTTP Phase II. The key objectives of the program were to support the MOE in developing a strategy and competency framework to reform the teacher education system; strengthen the overall teacher professional development system at the Teachers College/University of Liberia, as well as in the three national Rural Teacher Training Institutes through: (i) staff upgrading, curriculum reform, and mate￾rials development; (ii) training teachers in residential and field-based programs; (iii) improving training facili￾ties at the RTTIs; and (iv) strengthening school level management and community support for education through support to the PTAs and school principals. The LTTP II is a follow-on of the LTTP Phase I, and is implemented by FHI 360 in partnership with RTI International, to provide support both at the central MOE and in the five counties in the USAID develop￾ment corridor (Montserrado, Bong, Margibi, Nimba, and Lofa). The overarching goal of the LTTP II is to enhance students’ learning in general, and reading proficiency in particular, and to establish a functioning pro￾fessional development system for teachers and to strengthen the capacity of the MOE to manage such a sys￾tem. The interventions target reforms in three areas: (1) policies, systems and capacity development of the central MOE and lower levels; (2) teacher policies and pre- and in-service teacher training and professional development; and (3) curriculum standards, materials and testing for reading and mathematics skills in early grades 1-3. LIBERIA TEACHER TRAINING PROGRAM PHASE II 19 Goal 1: Improved Reading Skills for Liberian children by 2015 Goal 3: Increased equitable access for Liberian children by 2015 Objective 1: Institutional Capacity strength￾ened to provide educational services R1: MOE, CEO, DEO and RTTI Capacity strengthened to plan, man￾age and monitor educational services SR1.1: Critical MOEMOE systems are strengthened to guarantee equitable access to quality education services. SR1.2: Education Quality Monitoring and Instructional supervision strengthened at CEO, DEO and school levels SR1.4: Improved communication of changes and progress in educational development R2: Improved teacher policy and proce￾dures for teacher recruitment, training, deployment and career development SR 2.1: Teacher qualifications, recruitment, training and deployment guidelines devel￾oped and used SR 2.2: Teacher career structure, growth and incentive policy established SR2.3: Efficient and transparent accredita￾tion and examination system established R3: Improved teacher training programs and reading/math delivery systems SR3:1: A national standards-based model for early grade reading and math devel￾oped and implemented. SR3.2: In-service Teacher Education pro￾gram strengthened with emphasis on reading and math SR3.4: Strengthened National University delivery system to provide high quality courses in teacher education, including reading and math Objective 1: Improved teacher effectiveness in the classroom, especially in reading and math Liberia Teacher Training Program Result Framework January 2013 SR3.3: Pre-service Teacher preparation program strengthened with emphasis on reading and math SR 1.3: Policy and programmatic decisions are based on data from an information management system, policy analyses and research LIBERIA TEACHER TRAINING PROGRAM PHASE II 20 The objectives of the LTTP17 are: (a) to promote teacher effectiveness leading to improved reading skills for Liberian children by 2015; and (b) strength institutional capacity leading to increased equitable access for Li￾berian children by 2015. These two objectives are inextricably related and activities in one domain have to take cognizance of the impact on the other. This mutually reinforcing and interactive effect is expressed in the new LTTP Results Framework. The key results expected to be achieved under the revised the LTTP Results Framework are the following:  Result 1: MOE, CEO, DEO and RTTI capacity strengthened to plan, manage and monitor educa￾tional services.  Result 2: Improved teacher policy and procedures for teacher recruitment, training, deployment and career development.  Result 3: Improved teacher training programs and reading/math delivery systems 17 As outlined in Modification 8 to the Cooperative Agreement. LIBERIA TEACHER TRAINING PROGRAM PHASE II 21 III. FINDINGS For ease of presentation, this section on the assessment findings are organized under each of the three main results intended for the LTTP program. A. RESULT 1 (R1): MOE, CEO, DEO AND RTTI CAPACITY STRENGTHENED TO PLAN, MANAGE, AND MONITOR EDUCATIONAL SERVICES 1. ACCOMPLISHMENTS This planned result (R1) is the foundation of the LTTP II project. Accomplishments in Result 1, admittedly difficult to achieve in such a challenging institutional environment, would build a sustainable foundation for the MOE to “plan, manage and monitor educational services”, while also creating the basis for progress and results in (R2) and (R3), the two other LTTP II project components. Initially, the three planned results had been treated by the implementing partners, FHI 360 and RTI, as somewhat independent areas for intervention and treatment. This appears to have been a consequence of the way that USAID and the implementing partners had designed the project, as reflected in several project doc￾uments such as the Cooperative Agreement (CA), the Performance Management Plan (PMP), and the Five Year Action Plan. However, Modification 8 to the CA, approved in January, 2013 was an attempt to create a more integrated and interconnected approach to project implementation, in which project activities under R-1 would support the attainment of outputs and results related to professional development and career ad￾vancement policies for teachers and administrators (R 2), as well as C-certificate teacher training and curricu￾lum integration for early grade reading and math (R 3). A Framework for the Assessment of Capacity was prepared and is included in the annexes as a “scorecard” on MOE management and administration. It depicts a weak institution lacking essential systems. Key priorities for systems strengthening may be deduced from the scorecard. Even after refocusing the project through Mod 8, the LTTP II is still a very challenging project with a broad and diverse scope. To gain insight into how project activities were implemented, the assessment team used a customized institutional assessment framework to record what has been accomplished, assess the contribu￾tion of these accomplishments to achieving the LTTP-II’s goals, objectives, and results, and suggests the rela￾tive value of these activities for achieving the target result in R1, The assessment’s findings are intended to:  Help focus and concentrate the remaining project life span on critical systems  Weigh the merits of decentralized capacity development  Sharpen the focus on policy and evidenced based decision making  Link communication to the broader objective of community engagement, parental participation, and institutional cohesion by informing the base of the MOE pyramid about the impact and benefit of educational reforms LIBERIA TEACHER TRAINING PROGRAM PHASE II 22 To assess progress and interim results under R1, the team employed a customized institutional assessment framework to determine whether and to what extent the accomplishments of the project were building the capacity of the MOE to plan, manage, and monitor educational services. The method used for this institutional assessment is called the “The 8-S Approach”. It is based on the follow￾ing eight factors: 1. Statutory Basis: laws, regulations, decrees, policies, etc. 2. Service Mandates: based on the Education Reform Act of 2011, what are the specific service com￾ponents that are delivered by the Ministry? 3. Systems: what are the specific procedures or processes used to plan, manage, monitor, finance, and administer each of the services? 4. Structure: the official organization – including centralized departments and bureaus as well as decentralized entities, including CEO, DEO and schools – through which educational service are delivered and administered 5. Staff (Personnel): the estimated number, type, quality and readiness of the Ministry’s personnel to deliver the services within the defined structure using the various systems and procedures that are available to them 6. Standards: for items 2, 3, 4, 5 what are the performance standards, the output indicators, the means of evaluating access, quality, coverage and relevance? 7. Strategy: in light of the financial constraints within which the MOE operates, and considering the large coverage area and difficult access roads, how does the MOE – with guidance from the LTTP II project, use its limited resources most efficiently to deliver the defined services? 8. Synthesis: what is the operational process needed to collect data, analyze results, take corrective action as needed, and make policy and program decisions using EMIS as the primary tool for ev￾idence-based decision making? This approach is very much in line with USAID’s Human and Institutional Capacity Development (HICD) model, since the same “performance gap assessment” method is used to determine how the LTTP II went about identifying capacity constraints that inhibit MOE performance and then how (or if) it developed the means for addressing them. Since many of the preliminary analyses usually required by the HICD approach have already been done, it was possible for the assessment team to directly identify the performance gaps at MOE and make an assessment of how they are being addressed under the LTTP II. The results of the as￾sessment of accomplishments in these eight categories, i.e., the 8-S format, are provided in Table 1 at the end of this section. The observations, findings, conclusions and recommendation that follow are all grounded in the assessment methodology. Each of the sub-results (SRs) is portrayed and analyzed below, followed by a summary level assessment that is tied directly to R1 so as to provide the LTTP implementers and USAID with a cohesive summary and suggested areas for attention. Each section is introduced by a brief discussion of the topic being assessed, followed by a description of how each level in the education management hierarchy from MOE central office, through CEOs and DEOs, to individual schools has been affected by project activities. LIBERIA TEACHER TRAINING PROGRAM PHASE II 23 2. ASSESSMENT BY SUB-RESULT The following assessment framework presents the team’s findings and conclusions concerning each of the following four Sub Results (SR 1.1 – SR 1.4)  SR 1.1 Critical MOE systems are strengthened to guarantee equitable access to quality educa￾tional services  SR 1.2 Educational quality monitoring and instructional supervision strengthened at CEO, DEO, and school levels  SR 1.3 Policy and programmatic decisions are based on data from an information manage￾ment system, policy analyses and research  SR 1.4 Improved communication of changes and progress in educational development Each of these is discussed below: SR 1.1 Critical MOE systems are strengthened to guarantee equitable access to quality educational services The assessment team assumed that the “critical” systems to be addressed under in this Sub Result would logi￾cally encompass those used by MOE to plan, manage and monitor educational services at the central office and then to its decentralized service delivery units at the county, district and school level. The planning, man￾agement, and monitoring systems which the assessment team assumed to be critical to the attainment of R1 are:  Budgeting (especially on a program basis with defined results)  Human Resources (not merely for teachers, but for all MOE personnel)  Monitoring and Reporting  Consolidated Financial Reporting (to assure that both GOL and donor resources are identified and related to program purposes within a consistent framework)  Communication In-depth discussions with the implementing partner’s senior management team indicated that they under￾stood the relevance of these systems to R1, but that it did not define “managing” as encompassing, budgeting or consolidated financial reporting. To date the focus has been on human resources, largely on payroll trans￾parency issues that are being addressed through a biometric system; building the necessary conditions for monitoring and reporting by completing the first round of EMIS; and designing a website for improved communication by the MOE. In response to inquiries as to why the project had not addressed the program budgeting or financial reporting systems, the implementing partner indicated that work on these other critical systems was outside their purview. They explained that a type of “donor auction” occurs in which responsi￾bilities for specific areas of capacity building or systems strengthening are allocated to external entities, each with its own methodology and approach. For example, the Global Partnerships for Education (GPE) had some responsibility for financial and management systems design at MOE; the World Bank has staked its claim to Public Finance Management (PFM) reform, and that the EU had been engaged in strengthening the LIBERIA TEACHER TRAINING PROGRAM PHASE II 24 MOE’s planning and budgeting systems. The consequence of these actions is that the MOE systems reform is paralyzed.18 The following assessment follows a hierarchical approach, starting with the MOE’s central office and pro￾ceeding with a discussion of planning, managing and monitoring systems at each level. MOE Central Office: The MOE operates as a top down, command-and-control adhocracy. There are no reli￾able management systems in place to guide the delivery of educational services. It should be noted that man￾agement systems may be either manual or automated. Either can work well, depending on the environment and the rigor of execution and enforcement. But in the case of the MOE central office there are no docu￾mented, uniform systems for almost any aspect of institutional planning, management, or monitoring, wheth￾er done by hand or by computer. A management system is a set of interrelated steps and approvals, which when repeated in the same sequence, will always produce the same result. That definition captures the essence of a management system: repetitive, reliable, regular. At numerous meetings with the assessment team, the MOE central office management staff – at all levels - stated that there were no systems in place. The universal response to the team’s inquiries was, “We have no systems at MOE”. To confirm this, the team reviewed the 2012 Public Financial Management Risk Assessment Framework (PFMRAF) report for the MOE. Confirmation of our findings was emphatic:  “absence of written policies and procedures across multiple functional areas”;  “MOE not able to provide…written financial management policies and procedures, which should be used to guide the creation of an annual budget”;  “MOE does not have written policies or procedures guiding budget execution…”;  “MOE does not have internal policies and procedures…(related to)...procurement procedures”; and,  “HR and PAYROLL: Internal policies and procedures are not in place”. In addition to the above, the assessment team observed that the MOE’s financial administration also was un￾able to produce a consolidated report of its financial position, which combined Government appropriations and external funds. This represents a huge risk for fraud and corruption based on the possibility of double billings, misrepresentation of costs, padding expense claims, procurement fraud, cronyism and similar means of evading integrity. In reality, there are two MOE’s. One is funded by the Government, although very poorly, given the dire straits in Liberia), the other by the donors. For financial accounting purposes, much of the latter’s contribu￾tion is treated as a lump sum which is “off-budget”, but reflected in the overall planned expenditure frame￾work. One solution to this segregation of funds - a condition which creates resource allocation difficulties and a lack of transparency - is to create a global funding request to the MOF from the MOE based on a program budget structure, which not only satisfies the Integrated Financial Management Information System (IFMIS) budgetary classifications, but also provides more insight into what results are planned. This is a challenging but essential reform of a critical MOE management system. The IFMIS budget classification approach can 18 In fairness it should be noted that USAID and the LTTP II were waiting for the USAID Governance and Economic Management Support Pro￾gram (GEMS) to address the systems weaknesses within the MOE. GEMS has just started working with the MOE. LIBERIA TEACHER TRAINING PROGRAM PHASE II 25 identify what expenditures are authorized for the MOE, but it cannot provide any insight into what services are being provided, what is planned or actually accomplished through that expenditure. A program budget, with performance features such as results and progress monitoring milestones, is a possible solution, which the assessment team suggests should be a prime consideration for USAID’s future investments in the sector, whether during or after the LTTP II. Since R-2 and R-3 are underfunded and unable to achieve any traction within the MOE, this recommendation is critical to systems development that supports educational quality. The assessment team considers this as a priority intervention that is within the mandate of the LTTP contrac￾tor, and should be incorporated in the contractor’s work plans for the coming years. Alternatively, this re￾quirement may be addressed by USAID under another project, either GEMS or a new HICD initiative. Creating a program budget structure, then developing the procedures and forms to prepare a budget estimate on that basis, and also training a cadre of MOE personnel on how to make that system work will be a chal￾lenge. The team weighed this probability against the potential benefits. Because it is so critical to MOE plan￾ning, managing and monitoring, the assessment team recommends that the MOE begin a program budgeting development project. Such a system, once in place, could enable the MOE to maximize the impact of its total funding and also justify additional Government appropriations tied to specific program priorities and out￾comes such as teacher training and improved human resource management. This critical financial planning and management system should be introduced as soon as possible. Otherwise, the Ministry of Finance will continue to set education policy by default. County Level School Boards/CEOs: In terms of “critical systems” to plan, manage and monitor educational services, the County School Boards and the County Education Offices, as provided for under Chapter 4 of the Education Reform Act, are still under the guidance and control of the central office of the MOE, and will remain so until such time as a comprehensive decentralization plan has been devised, approved, tested and implemented. Five to ten years would be the likely timeframe for that scenario to play out. The public administration theory of the Education Reform Act is that the MOE central office would set policies for decentralized management of a county school system, which would be under the leadership of an ap￾pointed County School Board and led by a County Education Officer responsible for administration of the county system. The divisions between policy and management have a bearing on the implementation of the LTTP II project going forward and for USAID assistance to the sector in future years. The LTTP II has made a very big wager that the Education Reform Act is going to be implemented in a timely manner and in accord with the general parameters set out in the law. For example, Modification 8 proposes the following strategies that are closely linked to the law:  “Building capacity in strategic planning, management and monitoring and evaluation in the counties to support teacher development;  There will be a mass of trained staff at the CEO and DEO levels in strategic planning, moni￾toring and self-evaluation, as well as instructional supervision that is focused on reading and math; and  Train CEOs and DEOs in planning, management, and instructional leadership” Nevertheless, in terms of critical systems development, Modification 8 also still assumes: LIBERIA TEACHER TRAINING PROGRAM PHASE II 26  “Educational planning, management, monitoring and evaluation guidelines, standards and proce￾dures will be in place and used at all levels” The assessment team assumes that this latter activity can only be carried out by the central office of the MOE, with support from LTTP II, as appropriate. In light of the slow, apparently disinterested roll-out of the Education Reform Act by MOE leadership, the implementing partner – FHI 360 - might want to reconsider its emphasis on county-level technical assistance to support development of critical systems and instead con￾centrate on building central office capacity. As of now, decentralization has been shelved. SR 1.2 Educational quality monitoring and instructional supervision strengthened at CEO, DEO and school levels Unfortunately, this Sub Result doubles down on the original LTTP II decentralization wager. Whether this was based on the presumed presence and active leadership of the previous Minister and the former LTTP policy advisor, who were the architects of the Education Reform Act, is not clear. However, the current policy and management leadership at MOE does not seem to support the pursuit of these priorities. This has serious implications for achieving this SR and, by extension, R1 itself. The LTTP assumption for achieving this SR is that capacity for educational quality monitoring can be created at the level of counties, districts and schools. Based on visits to schools in four counties, and after meeting in the field with CEOs and DEOs, this is an overly optimistic premise, at least in the medium-term. In addition, as previously noted, the guidance, direction and oversight for educational quality monitoring at decentralized levels would likely reside centrally, not at the county or district level. Moreover, unless some logistical prob￾lems, such as the lack of transportation, are solved, it is unlikely that CEOs or DEOs could conduct quality monitoring over the remaining life of LTTP II. MOE Central Office: The assessment team found only limited evidence that the MOE central office is en￾gaged in developing any criteria or methods for monitoring quality of education, nor does a plan exist for developing the decentralized capacity for this activity. The basic functions of the Ministry are defined in Section 2.1.2 of the Education Reform Act. The commitment to quality education is listed several times in the enabling sections of the law, as follows: “The Ministry shall have the following functions: (a) Provide and ensure the provision of quality education to all citizens and residents without discrimina￾tion; (b) Reduce illiteracy by providing quality….education...at all levels for all citizens and residents without dis￾crimination; and, (c) Enhance the rights and uplift the status of all citizens and residents in society through well-designed and directed quality education programs” Not only is quality education enshrined in the law, it is clearly linked to broad societal objectives that are im￾portant in a post-conflict situation. In addition, the law also specifies the inputs to quality education (and here we assume the central office) is responsible to produce: LIBERIA TEACHER TRAINING PROGRAM PHASE II 27 “(j) Develop and implement uniform academic and administrative standards in the education sector through￾out the nation; (k) Develop and enforce a minimum level of learning requirement for each level of the education system be￾low …higher education; (m) Develop and facilitate the advancement of level-specific curricula, and guidelines for textbooks develop￾ment and/or acquisition and distribution; and (n) Develop and facilitate the expansion of an appropriate and level-specific teacher-training curriculum, in￾cluding guidelines, processes, standards and institutional arrangements.” Quality education, and its associated inputs, is the responsibility of the central Ministry. Based on this legal mandate, the MOE must define what constitutes and characterizes “quality” in each area of educational ser￾vice delivery and then assure that the instruments needed to measure, monitor and evaluate quality are creat￾ed, tested and used. While the EMIS has produced data that highlights the urgency of developing a definition of quality education and a means of determining its existence, there is currently no such definition of quality education, nor did the assessment team discover any entity engaged in this process. Consequently, monitoring educational quali￾ty – at any level – is impossible. Unless and until the Ministry acts to provide useful definitions of quality, the LTTP II cannot achieve that aspect of this SR within the life of LTTP II. CEOs/DEOs: With respect to “instructional supervision”, the long-standing practice is for CEOs and DEOs to be responsible for instructional supervision, although this term in the Liberian context probably should be downsized to “conduct periodic school visits”. Modification 8 commits the LTTP to the following:  Educational planning, management, monitoring and evaluation guidelines, standards and proce￾dures will be in place and used at all levels;  There will be a mass of trained CEOs and DEOs in strategic planning, monitoring and evalua￾tion, as well as instructional supervision that is focused on reading and math;  DEOs will undertake instructional supervision at least in the LTTP supported schools  Identify needs for …instructional leadership at MOE, CEO and DEO levels  Train CEOs and DEOs in…. instructional leadership. It is difficult to reconcile the mental images from visits to schools and Ministry offices with this rhetoric. In addition, the Education Reform Act defines the duties of CEOs and DEOs as, essentially, operations manag￾ers not as educators responsible for instruction supervision. The law states that the CEO will:  Be responsible for the operations of the school system in the county, including responsibility for the personnel of the system and the education programs in the county;  Prepare the budget; LIBERIA TEACHER TRAINING PROGRAM PHASE II 28  Supervise the implementation of all relevant national education policies, regulations, decisions, programs and development project through the DEO; and  Make regular reports This is an administrative position under the new law. It is a bureaucratic post. The law seems to convey to the County Board the authority for instructional supervision and instructional leadership in consultation with the Ministry central office. The CEO is their operational employee. The District Education Officers are mandated by Education Reform Act to, “supervise the implementation of all relevant national education policies, regulations, decisions, programs and development projects…located within the district, as directed by the CEO and the County School Board”. The assessment team loosely translated this mandate as one in which the DEO was responsible for providing instructional supervision, as intended by R 1.2. However, based on observations and meetings with the DEOs, it is unlikely that either (i) supervision takes place or (ii) that its focus is instructional. DEOs typically do not have the resources to carry out this task on a regular, scheduled basis. Limited to travel by motorcycle, which often lacks fuel, and unable to reach the far corners of their counties, the DEO’s ability to fulfill the mandate is severely compromised. LTTP Coaches: The only scheduled and reliable source of instructional supervision in MOE primary schools is being provided by the early grade reading and math coaches funded by the LTTP. Although limited to su￾pervision of those teachers who have received training in the EGRA and EGMA methods, field visits by the LTTP coaches represents a working model that the MOE could emulate. The assessment team did not ob￾serve enough of the coaching to render an opinion as to its “quality” nor relevance to instructional supervi￾sion. Nevertheless, the standard elements of formal and rigorous instructional supervision by the Early Grade Reading Assessment (EGRA) and Early Grade Math Assessment (EGMA) coaches were described to the observation team and appeared to be acceptable practice. In should be noted, however, that the coaches are few in number relative to the overall need for instructional supervision. Sixty eight coaches to cover 800 schools in five counties is still a demanding schedule. Any conclusions presented by LTTP about the efficacy of the methods used, cannot be verified without a more formal, objective and documented evaluation of im￾proved reading scores compared to the defined baseline. Although student progress cards were supposed to be created, the assessment team was informed by school principals that this documentation was not being maintained. Finally, instructional supervision also entails an evaluation of a trained teacher’s use of instructional material. In the case of math workbooks, only materials for level one have been distributed and then only to some schools. The planned distribution of level two and level three workbooks has been delayed, partly because it was determined that the students in Grades 2 and 3 were simply not ready for these more challenging materi￾als. In addition, some reading teachers indicated that they did not have enough workbooks or that the work￾books were under the control of the principal who treated them like typical Government inventory that be￾longed to the MOE, and not to the students. The LTTP should determine that the materials needed for EGRA and EGMA instruction are in place so that instructional supervision can be conducted by the LTTP and subsequently by the MOE. Schools: Although LTTP does not place a great deal of emphasis on developing the capacity of principals to act as instructional supervisors, the infrequent visits by LTTP coaches and the almost non-existent observa- LIBERIA TEACHER TRAINING PROGRAM PHASE II 29 tions by the DEOs, who are operating without the resources needed to perform this task, suggests the need to place more emphasis on school based management and the use of principals as instructional leaders. No changes in the SOW would be needed, simply a shift in emphasis based on the following existing commit￾ments, with commentary by the assessment team in italics:  Educational planning, management, monitoring and evaluation guidelines, standards and proce￾dures will be in place and used at all levels (scope for school principals)  Schools will develop their own School Improvement Plans (SIP) (an opportunity to train some principals on school based management and instructional supervision)  Develop an operational plan for the MOE to support….schools in planning and management (school based management could be emphasized)  Develop and print training materials in educational planning, management and instructional lead￾ership (with significant levels of training required, however) Since the status and duties of the position of CEO and DEO is still in flux and the incumbents to these posi￾tions are up for renewal or replacement, LTTP might consider a grass roots capacity building effort that starts with a small sample of schools and focuses on developing the capacity of the principals to be instructional leaders. Rural Teacher Training Institutes (RTTIs): The assessment team visited two of the three RTTIs and met in￾dependently in Monrovia with the Directors of all three of the RTTIs, i.e., Zorzor, Kakata and Webbo. In advance of these meetings, the team reviewed the PFMRAF’s to determine whether these institutions were “going concerns” from a business management or operational perspective. With respect to R1 only, the find￾ings are not very encouraging. Other sections of this assessment will address the threshold questions about these entities, i.e.; (i) should they exist at all; (ii) do they have the right mission; (iii) do they provide a valuable service relative to demand for qualified teachers; and (iv) can they address a different segment of the teacher professional development continuum? This assessment concentrates on the planned result under R1 and will address these three questions.  Has the LTTP II project built the capacity for the RTTIs to plan educational services?  Has the LTTP II project built the capacity for the RTTIs to manage educational services?  Has the LTTP II project built the capacity for the RTTIs to monitor educational services Planning: Since the inception of the LTTP II, extremely critical PFMRAF reports have been completed for each of the three RTTIs. These call into question the value of the proposed management intervention by the project, which is that RTTIs will be able to develop their strategic plans based on empirical data fed by EMIS. Although there is value in developing a strategic plan for the RTTIs in general, and for each specific RTTI, the assessment team believes that the MOE needs to undertake a thorough review of the role and purpose of these institutions before they embark on additional strategic planning. Currently, the RTTIs offer a nine￾month pre-service teacher training program of very questionable quality; a two-month residential in-service LIBERIA TEACHER TRAINING PROGRAM PHASE II 30 program, followed after nine months of off-site cluster training; then by final three weeks of testing prior to issuing a C-Certificate so that existing teachers can meet the Government requirements. The in-service train￾ing is aimed at providing basic remedial training for already-employed-but-uncertified teachers so as to im￾prove the quality of education. Any proposed strategic planning process should carefully consider the op￾tions for teacher training and the proposed role for the RTTIs in that scheme. Currently, there is no “strate￾gic” thinking being applied to the RTTIs in terms of alternative ways to achieve the desired result, given the resources being allocated to this objective. Managing: The assessment team focused on the efficiency of the RTTIs as measured by a rough ratio of graduates to total staff. In the case of Zorzor, for example, the team was informed by school administrators that in the current school year there were 265 students and 102 staff, of which only 12 were teacher trainers. In the case of Webbo, the Director informed the team there were 130 graduates with 92 staff of which 10 were teacher trainers. Similarly, the Director of the Kakata RTTI informed the assessment team that there were 255 graduates with a staff complement of approximately 100. These are hugely inefficient organizations, heavily over-staffed in non-core positions, e.g., cooks, gardeners, security. In addition, the assessment team was told that many of the teacher trainers were not well-qualified and that the RTTI’s were seen as a “dumping ground” for unproductive personnel. The assessment team was also informed that the RTTI’s were required to maintain “ghost workers” on the payroll. These factors sug￾gest a lack of a management ethic within the RTTIs. Indeed, some “insiders” within the RTTIs suggested that the MOE was disinterested in reforming the RTTIs. In this light, the inability of the RTTIs to obtain the basic financial support from the MOF needed to sustain a functioning teacher training institute seems to suggest that the Government itself places such low value on its operations that funding is diverted from them to more cost-effective activities. Under the current LTTP implementation approach, supporting pre-service training at the RTTIs is still a con￾tractual requirement. The assessment team has grave doubts about both the viability of the RTTIs and the efficacy of the strategy to sprinkle some teacher training on unqualified recruits, arm them with a C certificate, and then turn them loose on rural students. The team believes that a different approach and strategy, i.e., one which concentrates on in-service training rather than pre-service training might produce more cost efficien￾cies and better educational results. Monitoring: There is general agreement, even by the RTTI senior staff, that one of the fundamental failings of the RTTI pre-service and in-service training and certification programs is the lack of follow up with the graduates to determine if they are performing capably after being granted a C certificate. Follow-up monitor￾ing by the RTTI staff or MOE personnel would encourage the RTTIs to take greater responsibility for the quality of their programs. Currently, there probably are no personnel resources or basic means to carry out monitoring of the RTTI graduates, but this practice should be encouraged and supported by the LTTP II, thereby buttressing their own resource-limited field monitoring activities by addressing three dimensions of the EGRA and EGMA initiative: the relevance and utility of the materials and approach, the success of the students, and the quality of the pedagogy. SR1.3 Policy and programmatic decisions are based on data from an information management sys￾tem, policy analyses and research Education Management Information System: The LTTP II has provided a key input required to satisfy this SR, namely, the development of an EMIS that has the potential of motivating educational policies and policy- LIBERIA TEACHER TRAINING PROGRAM PHASE II 31 based decisions, as well as helping the MOE to make strategic program choices. The purpose of this planned result is to change the dynamic away from ad hoc, un-planned, and non-evidence-based decision-making pro￾cesses to ones that can be traced back to objective data and quality research. While acknowledging the great start made by EMIS, there is a further need to develop additional data concerning access, service quality, de￾ployment of personnel, school facilities and perhaps, most importantly, reliable unit cost estimates for the fundamental service to be delivered. The MOE, with assistance from the LTTP, might try to develop a standard cost estimate for a single classroom, with a qualified teacher, and a uniform allotment of furniture and learning materials for 35 children (to assure textbook -adequacy). Standard costing would enable a much more accurate picture of educational delivery costs and, thereby, provide the basis for budget preparation. However, these additions should in no way diminish the fundamental finding that the EMIS is an excellent tool for which LTTP should be justifiably proud. In fact, as emphasized later in this document, the team be￾lieves that providing more detailed analysis of the data to MOE decision makers could be a very important means of focusing attention on the alarming declines in enrollment of girls in primary schools. The anecdotal information that school officials had cited previously is now very clearly corroborated by a wealth of data produced by the EMIS. Policy Analyses and Research: The LTTP also assisted in the development of five research and policy￾oriented brochures aimed at “supporting the implementation of the Education Reform Act of 2011”. The five documents produced are entitled:  Policies for Reform  Liberia Education Administrative Guidelines  Monitoring Framework  County Monitoring Manual  County Evaluation Guidelines Since these volumes were the type of research and analyses that would support policy and programmatic deci￾sions as anticipated by SR 1.3, the assessment team reviewed these materials to see if they have had any im￾pact and if so, how much, and if not, why not. Although the volumes are very well written and represent academically sound research, as would be expected, they have not had a practical effect on the policy making or programming at MOE, nor were they very well connected to the rest of the LTTP II program. One of the key features of all five volumes is their apparent disconnect with any specific responsible actor within the Ministry. That is to say, whose job is it to take the material, convert it into actionable intelligence and then develop policy candidates for implementation? This is not clear. In addition, the two main instrumental texts related to (i) policy and (ii) regulation represent world-wide best practice models, rather than specific policies or regulations that could be adopted by the Ministry to the Liberian context. The three levels at which policy is needed are: 1. System-wide policies that relate to education as a public good 2. Entity policies that apply to all programs and activities of the MOE LIBERIA TEACHER TRAINING PROGRAM PHASE II 32 3. Operational policies that assure consistent treatment of all transactions To convert the five documents into actionable policy candidates at each of these levels, the assessment team believes that the Education Management Center19, supported by the LTTP which, after all, prepared the orig￾inal texts, should be empowered to develop a policy framework. Clear policies and documented systems will facilitate the transformation of the MOE. While some consideration should be given to policies related to decentralization, the absence of management support for this approach would suggest that other policy can￾didates should take precedence. One of the over-arching policy documents is the Education Reform Act itself. This represents a major accom￾plishment for which the LTTP II deserves kudos. The new law (i) rationalizes the central office and creates three new centers that can be used as in-house change agents in the areas of management, career develop￾ment, and curriculum design and implementation; (ii) introduces citizen decision making and oversight through County School Boards; and (iii) creates the capacity to consolidate and integrate donor funding with the Government’s own appropriations. Similar to the five documents identified above, however, much of the promised “reform” in the Education Reform Act is unrealized. It lacks both a champion, such as the former Minister who was its principal architect, and an engineer, i.e., someone who can create the structure that the architect intended and bring it to fruition. Once again the Education Management Center might be the think tank to make this happen. The Education Management Center could be the unit that is given wide latitude to craft policies and programs that will guide future development of a reformed MOE. SR 1.4 Improved communication of changes and progress in educational development Communication is poor throughout the educational establishment. Not only is vertical communication from the center to the periphery weak, irregular, and often non-existent, but also within the MOE central office itself. Horizontal communication between and among departments and bureaus at MOE headquarters is ex￾tremely ineffective. One measure of this poor communication is the fact that numerous MOE employees in￾formed the assessment team that they had never seen or read the new Education Reform Act. The Education Reform Act is intended to be the blueprint for future reform and is the legal framework within which they are supposed to act. The lack of infrastructure for effective communications between MOE headquarters and field offices, such as the CEOs and DEOs, is an enormous challenge. Internet usage as a percentage of the total population in Li￾beria are estimated at1.3% and even cell phone penetration rates were estimated at 43.75%, which is on the low end among African countries. Low penetration rates are more common in West Africa than in other re￾gional groupings.20. Electric power in rural areas is unavailable or unpredictable. The LTTP II approach of linking all County Education Offices through a combination of computers powered by solar panels/ batteries and a dedicated VPN connection from each CEO to the central office is still under development. Although the use of VSAT, satellite communications, and VPN systems is now a fairly common and tested practice in many developing countries, Liberia may prove to be the exception. This activity warrants close monitoring. 19 LTTP II should also ensure that the mandate of the Education Management Center is clear to develop the policy framework and empowered to implement the program. 20 Internet World, Statistics, 2012 LIBERIA TEACHER TRAINING PROGRAM PHASE II 33 The LTTP-II should be commended and congratulated for its efforts to use technology as a way to leap frog the current rural disenfranchisement brought about by a lack of internet services, land lines/cell towers, elec￾tricity and a host of other missing infrastructure that inhibits unified governance and cohesive management of Liberia’s education system. Nevertheless, the LTTP’s use of technology is a very fragile approach, especially in the context of the capacity challenges that exist. Creating ownership and a sustainability plan within MOE should be top priorities for the project. Central Office MOE: Based on interviews and observations, it seems that the central office is a collection of semi-independent islands operating without any organizing principles or management policies. Many of those interviewed had not read the new Education Reform Act, nor could they articulate what the “ministry as minis￾try” had as its overall priorities. As indicated above, the “two ministries” problem, i.e., one funded by the government and the other by donors; one traditional and bureaucratic, the other flexible and programmatic; one unable to meet basic standards, the other pushing for expansion into new ventures; all of which adds to the chaotic communication system. Operating in three locations during a transition to new, modern and mis￾sion-appropriate offices also has made communication difficult, to be sure. The assessment team assumes that the consolidation of offices will begin to pay dividends, but believes that the LTTP II needs to do much more in the area of building the MOE’s capacity for improved communications so that horizontal and verti￾cal communication is dramatically improved. For the LTTP, the relocation of MOE to 3rd Street creates the framework for improved communication among central office staff. An intranet will be established and all staff members will be expected to have indi￾vidual e-mail addresses. Security features also are being developed to assure the proper use of the new re￾sources. The new technology, however, will not overcome an MOE management mindset in which commu￾nication has a very low value. Senior managers need to lead by example and create an open, inclusive envi￾ronment in which public information is seen as an asset to be shared, not something to be used for personal gain. The LTTP needs to develop a demonstration-based, i.e., learning under supervision, phased roll-out plan to make communication and information management critical systems for decision making. CEOs/DEOs: Information about “changes and progress in educational development”, as required under the communication SR, never seems to reach the MOE field staff providing front line administrative and teach￾ing services in counties and districts. Based on interviews with these senior staff members, the only salient feature of the new Education Reform Act of which they seemed to be aware was the fact that their own person￾nel appointments were being reviewed, and that either the County School Board (as provided by law) or the MOE central office (based on allegations of corruption by County School Boards in the above-referenced appointment process) would decide their fate. The more fundamental purposes of the reform were of no ap￾parent importance to them. He LTTP had such a large stake in the success of the proposed reform that the assessment team assumed that more workshops and fora would have been conducted. That was not the case. One additional example of poor communication was the claim by a CEO that he was not consulted prior to the deployment of the five members of the “decentralization management team” – a key feature of the Educa￾tion Reform Ac and, therefore, he had no real ownership or interest in what they were doing. He dismissed them as mere ”kindergarteners”. In that same county, the County Supervisors advised the team that the new County Education Board was none of her business and, further, that there was no consultation with her, the “authorized representative of the President”, as there should have been. Changing the mindset of the old re￾gime will take more than words on paper. LIBERIA TEACHER TRAINING PROGRAM PHASE II 34 Schools: At the bottom of the information chain, sits the 1,523 primary school principals who actually man￾age the delivery of “education” to the approximately 250,000 enrolled students21. School principals are work￾ing under very difficult conditions, with limited (practically no) resources, scant remuneration and nothing in the way of incentives, rewards or recognition. Communication with the MOE county or central office man￾agement is via the District Education Officer, who may come once in a month, once in a quarter, or, more often, not at all. The principals do submit a monthly report using the Principals Monthly Reporting Template, which includes an assessment of the number of books and other material needed in the school. It seems to be forwarded to parts unknown. Some developing countries, such as Mozambique, have addressed similar communication challenges to those faced in Liberia. School officials there have overcome these obstacles through the use of technology, includ￾ing placing cell phone towers on school grounds to enable text and telephone links. On a more basic level, each school is provided with a bulletin board which is affixed to a wall easily accessible to parents, students and faculty members. The principal and parents in Mozambique are expected, and sometimes are motivated to do it by small grants, to post important and current information on the bulletin board. Minutes of PTA meetings are placed there, as are, by regulation, a register containing all expenditures from the school grant. These bulletin boards are nothing fancy, but they are glassed in, locked and serve a very useful communica￾tion purpose in the absence of printed media or other communication resources. Finally, the assessment team determined that PTAs are active in several schools and some, especially those in urban areas where business activities create some sort of local economy, actually contribute modest, but much appreciated funding to schools. Mobilizing parents to support education reforms is absolutely critical. They – the parents - need to know that “changes and progress in educational development” have the prospect of offering a better future for their children. Leaving parents uninformed and in isolation is a poor strategy. The LTTP should work with the new Education Management Center to develop the policies, procedures and pro￾gram outlines by which the MOE will both engage existing PTAs and encourage the formation of new ones. This might be especially important for linking parents with the reading program. For the reading and math initiative to be successful and sustainable, the Government and the MOE must provide its own resources (via program budgeting) to support community empowerment, create trust and foster partnership. The Table on the following page provides a summary of the accomplishments of the LTTP II project and the team’s assessment of whether these outputs meet the requirements of the Cooperative Agreement. It is fo￾cused on R1, the basic capacity building result that the LTTP project is meant to achieve. Following the table is a list of items that are either in process, but not yet completed, or have not yet begun and the team’s rec￾ommendations about how to add value to them or, in some cases, drop them from further consideration. 21 Education Management Information System (EMIS) Annual Report 2012 LIBERIA TEACHER TRAINING PROGRAM PHASE II 35 Accomplished Assessment Statutory Basis 1. Education Reform Act of 2011 2. Education Reform Act im￾plementation regulations and policies brochures prepared 1. Adopted in August 2011 as a result of significant and valuable technical inputs and support from the LTTP. This is the blueprint for a future MOE, but has been relatively dormant since passage. 2. These documents contain broad textual commentary on education best practice, but are not actionable regulations or policies. A concerted effort is needed to focus MOE management on the need to develop and use policies at the level of the county, the MOE and for management decisions. Service Mandates 1. Training conducted in plan￾ning, management and evalu￾ation for Ministry and Coun￾ty officers 2. Reading has been made a priority in the Education Sec￾tor Plan (ESP) 1. It may be that the LTTP is too far ahead of the MOE and the Government on the issue of decen￾tralization. It seems to have allocated resources to training “County offices”, ignoring the risk that their employment may not be continued. 2. Reading is not yet fully adopted as a core subject for all primary schools, which places the EGRA invest￾ments at risk. There is a need for clear MOE policies on this matter. Systems 1. Data collection and reporting system have been introduced to support the EMIS and generate Statistical Abstract Bulletin. 2. Internet services at the new HQ have been provided to foster improved communica￾tion. 1. EMIS data collection process was an excellent effort that is well appreciated by the MOE. Based on field interviews, however, the meaning of the da￾ta/information for educational planning, manage￾ment or monitoring purposes seems not to have been widely shared, although the document itself was distributed at a LTTP workshop. The assessment team’s basic question, “Data for what?” per SR 1.3 remains unanswered 2. Modernization through central office computeriza￾tion is a key accomplishment, provided it leads to LIBERIA TEACHER TRAINING PROGRAM PHASE II 36 3. IN PROCESS County level technology being installed for management (comput￾ers/solar powered with bat￾tery backup) and communi￾cation (VSAT-based satellite communications) better decisions and results. Using computers for better management should be accelerated as a LTTP priority. 3. This is a technological solution that is very important ESPECIALLY when the MOE actually moves ahead with real decentralization through delegation. The assessment team did not have the opportunity to ob￾serve the installation nor to assess the capacity of the MOE to operate and sustain it after the LTTP is completed. The team did learn that the installation has not been accomplished beyond the central level. Structure 1. Central Office structures de￾fined and responsibilities more clearly delineated 2. Three Centers established per the new ERA 1. Although a bit “top heavy” with Assistant Ministers, the structure provides a workable foundation for planning, managing and monitoring educational ser￾vices. Some Organizational Development (OD) and HICD support would be useful. The new Centers have great potential, but are unutilized by the MOE. Senior management has not embraced the concept of an Education Management Center, nor taken steps to enable it by assigning key personnel to start pro￾gram and policy formulation. This is the only new Center established under the Education Reform Act. The other two centers: The Center for Curriculum Development and Research and The Center for Educational Accredita￾tion and Certification continue existing operations under new titles and administrative arrangements. 2. It appears that the LTTP anticipated more commit￾ment by the MOE to the Education Reform Act provisions related to decentralization. The 75 indi￾viduals who have been trained have not been regu￾larized under CSA or included in the Budget for LIBERIA TEACHER TRAINING PROGRAM PHASE II 37 3. Decentralized capacity sup￾ported by deployment of 75 professionals (5 in 15 coun￾ties) in the areas of planning, procurement, personnel, fi￾nance and M&E. 4. County School Boards estab￾lished 2013-2014. 3. Although the County School Boards have been es￾tablished, and individuals appointed, they have no resources and their authority is already being ques￾tioned by the central office. Some political resistance also was noted. Staff 1. Training and capacity build￾ing has been provided to the IT Unit as a result of an em￾bedded advisor who has rig￾orously developed a work ethic based on excellence and a modified “six sigma” ap￾proach to error free imple￾mentation. 2. IN PROCESS, Biometric card system for personnel control and better planning and management 3. County officers selected, ori￾entated and trained 1. The embedded advisor concept has produced more results than the alternative approach used by the LTTP, i.e., maintaining separation and being located in a different building. 2. The team commends the effort, but cannot charac￾terize it as a success until it is in operation. The theo￾ry is well understood. LIBERIA TEACHER TRAINING PROGRAM PHASE II 38 22 In practice, this standard is not met in most schools 3. The five person team should have included an in￾formation specialist given the large investment in technology. Perhaps the skill set of the M&E staff member could be redefined to include the needed competencies. Standards 1. Policies and guidelines for IT developed and approved 2. Reading taught 5 days/week in selected schools22 1. The team did not observe these in operation. 2. Still not adopted as policy by the MOE, however. The MOE has not changed its behavior. Strategy 1. Modification 8 tightened the focus of the project and cre￾ated linkages between and among result areas. 2. Project activities have shifted from support for “critical MOE systems” to plan, man￾age and monitor educational services to a decentralized approach with support based on county-level strategic planning, management and instructional supervision. 1. The assessment team believes that the LTTP is too broad and unfocused to be successful and sustaina￾ble. The project should provide the model for what it is trying to instill in the MOE. The key “condition precedent” to program implementation should be policies and systems that unify and guide the MOE to a realistic, child-centered approach implemented in a cost-effective, well managed manner. 2. It appears that the LTTP has made a big, risky and currently a losing bet on an MOE-led decentraliza￾tion model for Liberia. Under the former administra￾tion, led by a reform-minded Minister, momentum might have been sustained. That is no longer a valid assumption and the LTTP may need to rethink its approach and strategy in the context of the current reality. LIBERIA TEACHER TRAINING PROGRAM PHASE II 39 3. Donor –MOE coordination is asserted to have been im￾proved. 3. The authority granted to the Deputy Minister for Planning, Research and Development to be the “principal liaison…with development partners and the donor community”, combined with “responsibil￾ity for the preparation of the annual budget” and the mandate to act as the ”clearinghouse for educational development activities” - combined with the fact that the Center for Education Management (current￾ly an empty vessel) reports to the Deputy Minister - establishes the logical entry point for the LTTP at the Ministry of Education and should be the basis for the new strategy. Synthesis 1. Education Management In￾formation System (EMIS) contains valuable data about access. 1. There is no force for integrating programs and coor￾dinating the various initiatives that are pulling MOE in several worthy but, in the end, cancelling-out op￾tions. Since the MOE is so controlled by donors on the program side, e.g., Early Childhood Education in a Ministry which cannot offer basic education, the LTTP must try to use an expanded EMIS, – includ￾ing analyses produced by collecting data at the cen￾tral level on finances and personnel - and other inte￾grating tools to achieve some synthesis. 2. Policies, systems, standards and realistic strategies, including the annual or medium-term budget, need to be developed and enforced so that the Ministry is focused and concentrated on priorities. This is the challenge for LTTP, as well. For the sake of its own credibility, the LTTP should demonstrate the behav￾ior it is recommending to the MOE. Among the first steps would be to simplify its work program, concen￾trate on the central office of the MOE and build crit￾ical systems, improve its own internal M&E capacity and bring in long term (no STTAs) experts to help build the Education Management Center into a via￾ble think tank for the Ministry. LIBERIA TEACHER TRAINING PROGRAM PHASE II 40 Although much of what LTTP has done has potential to bring about change, none of its initiatives is one hundred percent complete or internalized by the MOE. There are many positive activities in process, but with respect to R1, the LTTP should allocate more time and resources to finalizing the on-going work, assure that it has been adopted by the MOE and integrated into the MOE’s standard operations. The LTTP has pro￾posed to the assessment team that it intends to undertake the following new and additional activities prior to 2015:  “Educational planning, management, monitoring and evaluation guidelines, standards and procedures in place and used at all levels;  MOE coordinates donor activities and resources better;  Adequate mass of trained staff at CEOs, DEOs with focus on reading and math;  Schools develop their School Improvement Plans (SIP)  DEOs undertake instructional supervision at least in the LTTP supported schools;  EMIS infrastructure and capacity developed from the school to the county level;  A standard annual data collection questionnaire used by schools to enter data from the school-based recording system;  CEOs and DEOs consolidate data received from schools to feed the MOE;  MOE process national level data and comprehensive national statistical abstract;  DEOs, CEOs, RTTIs and MOE expected to use EMIS generated data for planning, man￾agement and monitoring purposes” This agenda is way too broad and seems unrealizable and impractical. The next annual plan should address how to make the project’s activities more valuable by concentrating and focusing on priorities that are closely aligned with the MOE’s own efforts. LIBERIA TEACHER TRAINING PROGRAM PHASE II 41 B. RESULT 2: IMPROVED TEACHER POLICY AND PROCEDURES FOR TEACHER RECRUITMENT, TRAINING, DEPLOYMENT, AND CAREER DEVELOPMENT 1. ACCOMPLISHMENTS Under Result 2 of the LTTP, there are three sub-results including:  S-R 2.1: Teacher qualifications, recruitment, training, and deployment guidelines developed and used;  S-R 2.2:Teacher career structure, growth and incentive policy established; and  S-R 2.3: Efficient and transparent accreditation and examination systems established. This section will review the accomplishments to date to achieve these sub-results, and then the assessment team will review the findings in detail for each. Despite a flurry of activity under the LTTP II in the first half of the program to improve teacher policy and procedures in a variety of areas, overall progress toward achieving these results has been slow at best. It ap￾pears that the LTTP, working with consultants, has developed nearly a dozen papers, which might be consid￾ered guidelines, policies and procedures that directly or indirectly address the recruitment, training, deploy￾ment and career development. Although the papers were not reviewed in great depth, the consensus of the assessment team was that the papers were of “mixed” quality. In addition, other observers and people famil￾iar with the papers volunteered that the papers were of poor or mediocre quality. Whatever the quality, the papers were presented to the Ministry of Education, but the LTTP has received little feedback and the initia￾tive to support the MOE under this result has been largely stalled. Although the review, modification, valida￾tion and approval of guidelines and procedures by the Ministry of Education is vital for the LTTP to achieve Result 2, and even more important for progress in the Liberian education system, the lack of progress in Re￾sult 2 highlights some fundamental problems in the Ministry of Education, as described below: Contextual Factors Limiting Success The first two years of investments by the LTTP in improving teacher policies and procedures for teacher re￾cruitment, training, deployment and career development highlight the fact that international investments in the sector are not likely to produce the transformation and impact intended unless fundamental and systemic weaknesses in the Ministry of Education (and other GOL agencies) are addressed. Some of the key factors are: Uncertain Environment: Normally, some turnover in personnel is anticipated over the life of a program like the LTTP. Nevertheless, there is the assumption that most of the key personnel expected to share implemen￾tation responsibility will be in place for an extended period, and they would be empowered to make decisions. Over the three-year life of the LTTP II, the leadership (Minister and the politically appointed Deputy and Assistant Ministers) of the MOE has changed three times. Moreover, recently the CEOs and DEOs have been invited to reapply for their posts, the Directors of the RTTIs have faced uncertain financing for their institutions and their salaries have not been paid on a regular basis. The result is that there is little continuity and no institutional memory within the MOE. Just as officials get familiar with the issues, they are replaced LIBERIA TEACHER TRAINING PROGRAM PHASE II 42 with someone who must go through the same learning process. In addition, the constant shuffle of officials makes people insecure and risk averse, especially to decision-making that aims to transform the MOE. Finally, either the shuffle of officials or the political nature of many of the appointees results in a less than a whole-hearted commitment to the effort to transform education. Consequently, the absence of continuity in senior management positions creates a steep learning curve that pushes policy considerations aside. Attention to teacher recruitment, training, deployment and career devel￾opment has not been addressed, despite efforts by the LTTP, because there is no time for the MOE leaders to acquaint themselves with the depths and details of the issues, and then take the needed action. Lack of Vision and Inability to Prioritize and Operationalize Plans: Over the last few years, the MOE has developed and approved a ten-year Education Sector Plan, developed and approved the Education Reform Act of 2011 (with LTTP assistance), developed various short-term implementation strategies, and conducted a Joint Sector Review. Despite these accomplishments, the focus of the MOE is constantly changing, and few ideas are fully developed, analyzed, funded, or carried out to fruition. The MOE has developed and adopted so-called “strategic plans”, but they are mostly “wish lists” of desirable outcomes. There is no time-bound plan with strategic objectives and priorities where costs that are fully determined and feasible targets are out￾lined. There is no plan or set of priorities that would guide the rebuilding process for the education system in general, or the professional development of teachers. Every year the LTTP outlines a work plan for the com￾ing year, which maps out a mutually acceptable set of actions and milestones for action with the MOE. How￾ever, the work plans are then largely ignored by the MOE. For example, the MOE has not taken ownership of – or managed to create responsibility for action plans - teacher development, including recruitment, train￾ing, deployment and career development. Personnel and Payroll Issues Inhibit Education Reforms: The MOE has a broken and opaque human re￾source systems, despite the development of organization charts, a relatively clear description in the Education Reform Act of 2011 of the roles and responsibilities of most of the various units within the MOE, and a presentation of MOE staffing patterns prepared by the LTTP. The LTTP has worked to create a new dy￾namic and professional human resource management system based on practices and systems that deserve high praise. Nevertheless, confusion between the Civil Service Agency and the MOE, and the MOE’s lack of control over actual payment of personnel, i.e., the Ministry of Finance is responsible for payment, inhibit the MOE’s ability to understand and clean up its payroll. In theory, there are job descriptions and standard oper￾ating procedures for personnel, but in practice there is a lack of accountability for staff and very limited inter￾nal controls to reduce inefficiencies and curb fraud.23 Reportedly, the MOE has large numbers of ghost ad￾ministrators and teachers and, until recently, the MOE had no clear idea of the number of public primary schools. Without a payroll system that is devoid of ghost administrators and teachers, it is very difficult for the MOE to develop policies and programs to recruit, train, and deploy teachers or make informed decisions about how to support their professional development needs. Moreover, the lack of reliable data on the number of teachers in the system precludes any reliable demand and supply analysis of teachers and promotes poor planning for pre-service and in-service teacher training. Finally, payment for ghost teachers and the at￾tendant loss of resources, e.g., the Ministry of Finance estimates that $11 million of the MOE budget is lost annually to ghost worker payments24, could be used to deliver much needed services to the children of Libe￾ria. 23 The Global Corruption Report of 2013 ranks Liberia as the second worst corrupt country in its survey. 24 Interview with MOE Deputy Minister for Administration LIBERIA TEACHER TRAINING PROGRAM PHASE II 43 Insufficient Resources and Political Will: The resources allocated to the MOE are meager and frequently cut to the minimum by the Ministry of Finance with little regard for the impact of those cuts on the recruitment of quality teachers, teacher training or career development. Budgets for RTTIs, the main source of pre￾service and in service teacher training, were inadequate to support operations and in June 2010 and October 2011, the LTTP, at the USG’s request, provided support for food and fuel supplies for the RTTIs. The Cen￾tral MOE has not reached agreement on plans to make the RTTIs semi-autonomous, putting at risk the cur￾rent programming at the RTTIs and the sustainability of the institutions. Moreover, the quality of teaching and learning at the RTTIs is low, owing to a poorly trained faculty; have a very limited nine month pre-service program; and new program entrants have poor reading, math, and academic skills, i.e., some participants test at the 6th grade level. As long as the salary, working conditions and administration at the MOE remains apa￾thetic, it will be difficult to attract well-prepared people to the teaching profession. Heavily Centralized MOE: The MOE is a highly centralized ministry where nearly all decisions are made in Monrovia and all communication is one-way from top to bottom. Despite the Education Reform Act of 2011, which aims at decentralizing the Ministry, little effort has been made to implement the law or to empower the three Centers of Excellence. For example, the Center for Educational Accreditation could, among other things, “establish minimum requirements for the accreditation…and licensing of teachers”, which would be the first step toward implementing the roadmap for supporting teachers and developing quality education. In addition, the highly centralized nature of the MOE means that no decisions about teachers - recruiting, hiring, performance appraisal, promotion, dismissal, rewarding – are made at the local level. Parents and civil socie￾ty, which are the clients for MOE’s services, have no say in decisions that impact their families. Each of these weaknesses undermines the MOEs’ capacity to improve teachers’ qualifications and perfor￾mance, and have made the task of the LTTP program to assist with the development of policies and pro￾grams very difficult. The LTTP program has done many correct things, but has been thwarted by the MOE’s apathetic attitude and the poor systems. Given these findings, it may be timely to consider commissioning a study to examine other options for the development of teachers in Liberia. 2. ASSESSMENT BY SUB-RESULT Having considered the macro-level problems plaguing Result 2, a review of the sub-results and some of the detailed findings in those sub-results follows. Sub-result 2.1: Teacher qualifications, recruitment, training, and deployment guidelines developed and used Although there are areas for improvement, it appears that the LTTP has made a reasonable effort to assist and enable the MOE to improve the recruitment, training, and deployment of teachers in Liberia. The LTTP commissioned five reports25 on guidelines and studies focused directly or indirectly on these topics. These reports might have done a better job of reflecting the context of Liberia and, more importantly, the LTTP should have done more to develop ownership within the MOE. By the same token, however, the MOE needs to be actively involved with the LTTP as a partner in the education reform process. Although the reports were developed and passed on to the MOE, the MOE has not taken any action to review, modify, validate or approve the guidelines. It remains to be seen if, over the remaining life of the LTTP program, the MOE will 25 For Example, Naame Saaim, (2012). Development of Teacher Recruitment, Training, Deployment and Utilization Policy and Procedures, Mon￾rovia, Liberia LIBERIA TEACHER TRAINING PROGRAM PHASE II 44 move ahead in this important area. However, at a minimum, the LTTP should aggressively pursue some sort of agreement with the MOE’s upper management that would ensure more timely decisions. Sub-Result 2.2: Teacher career structure, growth and incentive policy established. Aside from the policy documents cited above, there does not seem to be any progress in this area. In fair￾ness, there is a kind of logical progression in the various sub-results. If no progress can be made on teacher qualifications and recruitment, and no progress can be made in accreditations and examinations, it is difficult to expect a teacher career structure to be put in place. The LTTP has introduced modern HR management practices, if sustained, will provide the foundation for designing a career advancement policy and related procedures. In addition, of course, teacher policies must incorporate the certifications currently offered through pre-service and in-service training. Sub-Result 2.3 Efficient and transparent accreditation and examination system established Much like sub-result 2.1, it appears that the LTTP has taken appropriate steps to initiate the development of this result, but for whatever reasons the MOE has not moved ahead in this area. The LTTP program com￾missioned a study26 by a prominent consultant in 2011 and presented the findings of that report to the MOE. It appears that study has also languished within the MOE without any action to review, modify, validate or approve the guidelines. Key staff informed the assessment team that they have been provided with little direc￾tion and no resources to proceed with career planning and advancement procedures. Given the lack of lead￾ership and resources, the LTTP II should consider withdrawing from this area. Other Findings for Result 2. Teacher Recruitment: Given the generally poor working conditions, relatively low pay, limited career ad￾vancement possibilities, a ministry with a reputation for highly centralized decision-making and uncertain and intermittent payment of salaries, it is difficult for the MOE to recruit qualified individuals to become teachers. Many of the applicants for pre-service training at the RTTIs were recruited by the CEOs and DEOs, and ap￾parently did not meet the entrance criteria established for the program. Many presented forged documents and were weeded out of the recruitment class, only to reappear in the incoming class once training com￾menced. Entrance examination results showed that many recruits were woefully ill-prepared for the training and did not have the knowledge or skills necessary to be a teacher. Examinations given after the nine month pre-service training program showed about one-third of the people who had matriculated through the pro￾gram had not passed the exam. One of the implications of these findings is that the pre-service teacher training program may be too short to provide adequate training to the ill-prepared individuals who are attracted and recruited to teaching. There are few teacher training programs that are less than two years in duration, and many are three years long, es￾pecially those that take in academically unskilled recruits. A second implication of the findings is that the MOE should move expeditiously to base the recruitment of teachers on merit. As long as a system is allowed to exist where DEOs and CEOs are able to influence the selection of proposed teacher recruits, the educa￾tion system will be seen as an employer of last resort and one where patronage and corruption are allowed. 26 Snyder, C.W., Jr. (2011) Accreditation in Liberia: Creation of the Center for Certification and Accreditation. Monrovia Liberia LIBERIA TEACHER TRAINING PROGRAM PHASE II 45 Teacher Training: The assessment team found that the number recruited for pre-service and in-service train￾ing were not based on any reliable estimates of the supply, demand, and unmet need for teacher training. It is apparent the three RTTIs produce a little less than 500 new teachers per year at the C-certificate level (prima￾ry school), while another group of a little less than 500 already employed teachers get upgraded to C￾certificate level through a six-week in-service training program, followed by a year of once-a-month Saturday classes and mentoring. Both the pre- and in-service programs produce certified teachers whose quality is sus￾pect, given the low starting point and lack of training in relevant classroom skills for most entrants. Moreo￾ver, of the approximately 11,500 primary school teachers in the system,27 it appears that the MOE is provid￾ing training annually for about 8 percent of the primary teacher corps. The MOE appears to be working under an old, immediate post-conflict paradigm where most teachers had fled the country, and those teachers that remained were unqualified and needed training of any duration. Given the finding that about one-third of the newly-trained teachers could not pass the exit exam, perhaps it is time for the MOE to consider a new paradigm for pre-service teacher training, one that provides longer (two years or more), and more in-depth training of the new recruits. Moreover, rural, residential pre-service training institutions may be the most expensive model for ensuring an adequate teaching supply in one of the poorest countries in the world. Some thought should be given to altering the mix of pre-service/in-service training offered by these three institutions and/or transforming some of the RTTI’s into more multi-purpose training institutions. At the very least, a study of the short- to long-run teacher training needs (demand and supply) of the country should be undertaken to ensure that scarce resources are not wasted. Teacher Deployment: The deployment of teachers, especially to rural areas, is a contentious issue in Liberia. Many young, newly trained teachers have little interest in serving away from the relatively bright lights of greater Monrovia, or at least the county seats. Working conditions in most rural schools are poor and the living accommodations may not be up to the standard expected. In addition, many newly trained teachers might be working in a relatively isolated professional environment, where many of their colleagues are indif￾ferent or absent for long periods, the principal cannot provide much academic leadership or support, and there are few teaching and learning materials. On the other hand, there are hundreds of unqualified teachers already in place in the schools, but their lack of skills is part of the problem of low quality education. A MOE policy on teacher career structure, growth and incentives is clearly needed and should be one of the priority policies of the MOE but, given the political nature of the decisions and the revolving door of deputy minis￾ters, it seems that the GOL will keep “kicking the can down the road” unless it creates a stable policy analysis capability inside the ministry, separated from day to day administrative duties, that has a mandate to address this issue. Career Development: Once a teacher is assigned to a school, the teacher has no outline of a career devel￾opment path within the MOE. The teachers can expect to stay in the same job at the same pay grade for the rest of their careers. The MOE has no career ladder or lattice, no career development strategy, and there is no articulation between in-service training, pre-service training, certification, licensing, job promotion, and career development. There is no information about how to improve as a teacher or succeed in the profession. Un￾like most school systems where teachers know that, if they take a series of in-service programs, they will re￾ceive higher pay or will be in-line for possible promotion to principal. Not surprisingly, the teaching profes￾sion in Liberia is largely apathetic and appears demoralized. Clearly, the design and implementation of a ca￾reer development system is sorely needed in Liberia. The LTTP should re-double its efforts to get the MOE 27 Government of Liberia, Education Statistics, 2012 LIBERIA TEACHER TRAINING PROGRAM PHASE II 46 to review the reports the LTTP has provided on this topic and map-out next steps for the development of a career development system in the MOE. As suggested above (see page 50), unless the MOE is fully commit￾ted to and actively engaged in this reform initiative, it may be time to withdraw support and concentrate on other priorities, jointly agreed upon. C. RESULT 3: IMPROVED TEACHER TRAINING PROGRAMS AND READING/MATH DELIVERY SYSTEMS 1. ACCOMPLISHMENTS Under Result 3 of the LTTP program, there are four sub-results:  S-R 3.1: A national standards-based model for early grade reading and math developed and imple￾mented;  S-R 3.2: In-service Teacher Education programmed strengthened with emphasis on reading and math  S-R 3.3: Pre-service teacher preparation program strengthened with emphasis on reading and math; and  S-R 3.4: Strengthened National University delivery system to provide high quality courses in teacher education, including reading and math. This section of the report will review the progress made to date under each sub-result. Sub-Result 3.1: A national standards-based model for early grade reading and math developed and implemented. The LTTP has made a strong start toward developing a standards-based model for early grade reading and math. The LTTP has implemented an early grade reading and math program in a first cohort of 800 schools in five of the most populated counties in Liberia (Bong, Lofa, Margibi, Monteserrado, and Nimba). The 800 schools represent about 49 percent of the public primary schools in the whole country, but roughly 70 per￾cent of the primary schools in the five counties.28 By the end of the LTTP close to 100% of the schools in these 5 counties will be supported. Given that the CDCS focuses USAID’s development work in those five counties, and USAID does not work in the other 10 counties in Liberia, the LTTP effort to develop a stand￾ards-based model in the 800 schools has been a huge undertaking and a major achievement, even if is not nation-wide. The challenge for sustainability of this initiative is to embed successful methodology in MOE policy and practice. MOE has promised to fund additional coaches under its own budget and to make these regular civil service positions so that they will be more or less permanent fixtures in the service delivery sys￾tem. However, this has not yet been formalized. Before any expansion of the program, there must be condi￾tionality that the MOE will take the steps necessary to sustain this USAID investment. 28 Government of Liberia, Ministry of Education, Education Statistics, 2012. The data in the MOE Education Statistics report is a product of the EMIS and a work in progress. Data in the report represent the best available, and it is thought to be reasonably accurate. It is possible that there has been under- or-over-reporting in some categories. LIBERIA TEACHER TRAINING PROGRAM PHASE II 47 2. ASSESSMENT BY SUB-RESULTS The assessment teams’ review of the reading and math program revealed a number of strengths, as well as several weaknesses in the program. On the positive side, both the early grade reading and math programs were widely acclaimed by teachers, principals and administrators in each of the five counties we visited. Classroom teachers especially extolled the merits of the program and were often eager to demonstrate their new skills. In Liberian school system, which has a dearth of materials and very limited in-service training op￾portunities, it was not difficult to gauge the enthusiasm of teachers for the math and reading programs. The early reading and math program provided them with the first training and materials they had received in years. It was clear, moreover, that the program lessons and materials were appreciated by teachers and that they regularly used them in the classroom. Another strength of the program, according to teachers and principals, was the once-a-month visits by the LTTP coaches. Most teachers and principals cited good support and helpful, constructive criticism from the coaches as an important part of the program’s success. Initially, the assessment team was not able to obtain the results from the comparison of the pre- and post-tests of reading achievement, but everyone claimed that children were learning to read faster and with more comprehension under the program. Subsequently the team was able to obtain performance data from the early grade reading assessments conducted by the LTTP at the end of the 2012-2013 school year, which is represented in the text box below. “On average, children in grades 1 through 3 in participating schools improved their achievement in all the reading skill areas in English that were tested at the end of the 2012–13 school year. Compared to their per￾formance in 2011, Grade 1 students on average almost tripled their ability to read familiar words, more than tripled their decoding skills, and were able to read text more than one-and-a-half times more fluently. Grade 2 and 3 students on average were 10 times better at decoding and had almost three times higher oral reading fluency than before the program. On average, students in all grades improved their listening and reading comprehension.” This is an enormous accomplishment for the LTTP project and augurs well for future expansion, provided that the MOE takes more of a leadership role. At present they are grateful observers of the LTTP. Creating – demanding – ownership by the MOE must be embedded into the LTTP work plan. On the weakness side, teachers and most principals maintained that they needed more training in reading and math to be effective in the classroom. Most teachers claimed to have received a one-week training program in reading instruction and one week in math instruction, but they felt too much material was covered too quickly for them to ab￾sorb the lessons. Some cited attending additional training in the subject areas, but also thought the length was too short and the meetings too infrequent to ensure that they were fully competent. Principals particularly thought they should receive more training in the subject areas so that they could better perform their role as the academic leader at the school. Interestingly, many of the coaches thought that they needed more training in reading and math pedagogy because, although they were college graduates with decent reading and math skills, they were not familiar with teaching those skills. The assessment team also recognized the need for the LTTP to make a fundamental trade-off between cov￾erage, i.e., assuring that many teachers get some introductory training, and intensive training for fewer teach￾ers, but with more depth and duration. The choice for coverage as the policy option may prove wise, if the MOE assumes ownership of the initiative and uses the foundation created by the LTTP as the basis for in￾service training. LIBERIA TEACHER TRAINING PROGRAM PHASE II 48 Most principals and some teachers suggested that the program should be expanded to the first six grades, a “whole school approach” because they noted that there were many overage children in the upper primary grades. They believed it was unrealistic to assume that all children would learn to read or do math in the first three grades. They also noted that the frequent turnover and absenteeism suggested the benefits of training the whole school, especially the principal who must frequently serve as a substitute teacher. Teachers also thought the programmed lessons and materials should be expanded for each grade in reading and in math. The program should not be launched until all the materials were available for all the teachers. They often cited an example of the math materials for grades 2 and 3 not being delivered and teachers, so the teachers used the programmed materials for grades 1 in grades 2 and 3. Many teachers thought that the mate￾rials for grade 1 should not be used in grades 2 and 3, in part, because of the dearth of materials available and, in part, because they had no curriculum to build on in the subsequent grades. Several administrators observed that the coaches’ rate of pay ($350/month), as required by previous MOE management, far exceeded regular teachers’ salary. They thought the program was too high-cost to be sus￾tained by the MOE. However, the team believes that attracting, training and retaining qualified college gradu￾ates to work as coaches warrants a salary commensurate with their educational attainment and skill set. Once again, however, the assessment team was concerned about sustainability, not because the salary for coaches was too high (many central office administrators and RTTI faculty are compensated at the same level), but because absent a career ladder and standards for promotion, the coaching corps could become just another dumping ground for unqualified individuals. The absence of systems and procedures to recruit, train and de￾ploy teachers has consequences throughout the ministry and jeopardizes Liberia’s avowed goal of quality edu￾cation. Although everyone made bold claims about the programs’ impact on improving children’s reading and math skills, no one could cite any hard evidence to support the assertion. Moreover, when the assessment team asked about assessing and tracking student performance, no one could effectively articulate the student moni￾toring system or rattle-off data from that system, suggesting that more work needs to be done in this area. For example, teachers need to regularly track each student’s progress toward the reading and math standards establish by the education system, but few teachers had any records on student performance. Overall, the impressions of the early grade reading and math program were positive, but there are a number of things to suggest that the program might not be sustainable, including:  Teachers, principals and coaches appear to need more training and lack confidence to carry on the program without continued assistance from the outside;  Teaching and learning materials appear to be well-done and widely used, but most thought there should be more materials for both reading and math;  Many thought the program should adopt a “whole school” approach; and  More attention should be paid to student assessments and tracking by everyone in the educa￾tion system Result 3.2: In-service Teacher Education program strengthened, with emphasis on reading and math LIBERIA TEACHER TRAINING PROGRAM PHASE II 49 It appears that LTTP has provided a considerable amount of in-service training to implement the first cohort of the early grade reading and math program. It is clear that the LTTP has worked with the in-service teacher training program, which offers a C-certificate to existing teachers, to incorporate training for teaching reading and math into the C-Certificate curriculum. However, it is not clear that this result has been achieved or, even, that it can be achieved or measured during the life of the LTTP project. One would guess that the in￾troduction of reading and math modules and reading and math materials into the in-service program at the RTTIs would have some impact on strengthening in-service teacher education. The discussions the assess￾ment team had with in-service teacher trainers suggested that the introduction of reading and math modules and materials were useful and strengthened the program, but the assessment team had no way to verify that impact. Moreover, a variety of observers noted that many of the in-service teacher trainers were very weak themselves in reading and math skills and needed upgrading. The team also felt that the period of post-conflict remedial in-service training needed to be concluded. De￾veloping a more rigorous training approach, consisting of both content and pedagogy, should be the com￾bined objective of the new centers for curriculum development and educational accreditation. Sub-Result 3.3: Pre-service teacher preparation program strengthened with emphasis on reading and math Our observation about this sub-result is much the same as that made in Sub-Result 3.2, namely it appears that the LTTP has attempted to strengthen the pre-service teacher preparation in math and reading, but that it is very difficult to determine the impact of that assistance on the pre-service program at the RTTIs. For exam￾ple, the LTTP reviewed and revised the pre-service curriculum, purchased materials in the content areas and trained demonstration school teachers in the use of the materials, but a review of the demonstration schools at one of the RTTIs showed no reading and math materials at the demonstration school or the library. Moreover, the discussion with pre-service students yielded very little useful information. The meetings with the pre-service students was a large, focus group discussion and the students wanted to vent about problems in the school rather than focus on the impact of the proposed improvements in the reading and math pro￾gram. Pre-service training has been dominated by two negative factors: (i) the need to recruit minimally qualified candidates, rush them through a superficial training program and then appoint them (with unmerited fanfare) as teachers in rural schools on an “anything is better than nothing” strategy; and (ii) the requirement to justify the existence and expense of the RTTIs by having them engaged in pre-service training exercises that are known to be deficient. Since there is no real justification for continuing an in-service program with such ap￾parent deficiencies, the Government of Liberia needs to confront reality and restructure pre-service training, either by increasing the use of Peace Corps Volunteers as rural teachers and teacher trainers, or increasing the flow of university graduates into the teaching profession. This shift would also eliminate the practice of offering teaching credentials, i.e., the C-Certificate to RTTI graduates who have not been properly trained. If the RTTIs are continued – for political or other reasons – they should only be authorized to offer “conditional” teaching credentials, which can only be recognized by the MOE after a period of 2-3 years of supervised teaching. Sub-Result 3.4: Strengthen National University Delivery system to provide high quality courses in teacher education, including reading and math LIBERIA TEACHER TRAINING PROGRAM PHASE II 50 The LTTP program has put a series of measures in place that are aimed at strengthening the national univer￾sity system, including faculty support to the RTTIs; introducing the reading and math modules to faculties of education; supporting female students with stipends to attend pre-service teacher training; introducing the universities to computers, books and other digital library resources; and providing scholarships for 19 Masters Degrees and 4 PhD programs at universities in Africa. Although at the mid-point of the LTTP, it is difficult to determine if the sum of the parts will result in the desired impact. It is clear that the LTTP has made ef￾forts to strengthen the National University Delivery system. LIBERIA TEACHER TRAINING PROGRAM PHASE II 51 IV.DESIGN AND MANAGEMENT OF THE COOPERATIVE AGREEMENT In this section of the report the assessment team looks at how the project was designed and organized to achieve the planned results and if the structure, systems, staffing and strategy used by the implementing part￾ner has affected the pace and output of the LTTP-II project. Based on these observations, the team offers suggestions as to how USAID and the grantee can work together to align the LTTP-II resources to capitalize on successful interim results, accelerate high-value activities, focus on key priorities, and achieve planned re￾sults. The LTTP implementing instrument is a Cooperative Agreement. Cooperative Agreements suggest that all partners to the program, i.e., the implementing partner, USAID, and the host-country, are in agree￾ment about the direction the program is to take and that each has the capacity to carry out their tasks in the work plan. It appears that the ability of the MOE to prioritize and implement is very weak and many basic systems are not in place. In addition, the implementer has not provided the strong technical direction needed, especially after the departure of key international experts. Under these circumstances, USAID’s ability to in￾fluence the program’s outcome is weaker than if a standard competitively bid contract had been employed. A Cooperative Agreement with a weak institution like the MOE may not be appropriate in the near and medi￾um term future. All Parties Over-Estimated the Capacity of the MOE: The Liberian Ministry of Education is characterized by frequent changes in leadership, weak educational planning and management, and lack of financial and human resources throughout. The Ministry is striving to redress the debilitating effects of past conflicts, while laying the foundation for a nation-wide education delivery system. Consequently, it is not in a position to say “no” to offers of development assistance by any donor even when it lacks the framework and capacity to carry out the duties expected of a true development partner. The LTTP is based on the principles of joint engagement, but currently the MOE lacks the capacity to work as a full partner with the LTTP. The LTTP needs to adjust its operating strategy to account for the poor capacity within the MOE. Outside of Monrovia, the Lack of Resources and the Near-chaotic Conditions are Major Barriers: Resource starved schools throughout rural Liberia lack basic educational materials. For some schools the an￾nual allotment of supplies consisted of little more than a box of chalk. Teachers are frequently out of class, often chasing their salaries in county capitals, and school management is weak and untrained. In addition, the basic infrastructure of roads and power shows the effects of the long civil war. Roads are barely passable making inspections infrequent and ineffective. It is very difficult to do service delivery in rural Liberia, par￾ticularly when the MOE brings no resources and little expertise to the program and Government funding is less than what is required. The Program’s Coverage and Scope of Work is Overly Broad: The LTTP is a wide-ranging program that is dedicated to institutional strengthening/capacity building targeted at teacher training. A second planned result is aimed at improved teacher certification training programs, while a third result is aimed at improved early LIBERIA TEACHER TRAINING PROGRAM PHASE II 52 grade reading and math instruction and reading/math delivery systems. It would appear that, similar to the MOE itself, the program is being pulled in different directions, thereby reducing the ability of the program’s management team to develop a realistic and targeted action plan which allocates time and resources to key priorities. Compounding this diffusion problem has been the tendency of the MOE to see the LTTP as an ATM, ready to disburse cash on demand to fill the voids created by poor MOE planning and lax administra￾tion. For example, in the first year of the program, when food or fuel was needed to bail out the RTTIs, the LTTP was called to the rescue. If a conference or meeting was planned by the MOE, the LTTP was request￾ed to pay the caterer. If documents were needed to be printed, the LTTP was a reliable source of funds. And finally, if laptops were deemed essential to employee performance – as in the case of the 75 members of the decentralization management team – the LTTP was asked to step up and promise delivery That promise is still pending. The assessment team observed that the implementing partner has achieved some impressive partial victories that have not been capitalized upon, such as EMIS, the Education Reform Act, and the initial progress on im￾proved teaching and learning of reading. The team believes that a principal reason why more progress has not been achieved is the overly-broad scope of the program, even for a 5-year $65 million follow-on award. This can be seen with dramatic impact by reviewing the list of activities in Modification 8. The team believes that USAID and the grantee should reduce the number of activities and agree to focus on a limited number of important accomplishments. Some of the Areas to be Addressed Are Much More Difficult than the Program Description Suggests: The policy and management improvements anticipated by the “words” in the program description are not corre￾lated with the amount of time and effort required to bring about the planned result. For example, R 2, “Im￾proved teacher policy and procedures for teacher recruitment, training, deployment and career development” is itself a multi-year, multi-million dollar project that would involve HR experts, computer specialists, man￾agement consultants and similar high level advisors. This is a highly challenging task, which requires the in￾volvement and complete dedication to this program by the Civil Service Agency, the Ministry of Finance, as well as skilled counterparts in the MOE’s Department of Administration. If USAID believes that the result is critical to the long term interests of both Governments, it should request the implementing partner to deploy the necessary resources and define a reasonable timeframe to accomplish this task. In addition, the imple￾menting partner should be very careful not to over-promise. At the time of the mid-term assessment, the implementing partners’ team was not staffed by the number and type of qualified international experts needed for an institutional capacity building program of this type, in this environment, with this client. The assessment team was advised that the implementing partners had insti￾tuted a policy to “Liberianize” the program by withdrawing international experts and replacing them with local professionals. While this is an admirable social policy, it does little to advance the objectives of the pro￾gram or to assure that the client, the MOE, will benefit from the implementing partners’ expertise. Accord￾ingly, the assessment team is recommending that the grantee, as part of its annual work plan modification exercise, consider carefully how to access long term resident experts who will work directly with their MOE counterparts, especially on the institutional strengthening priorities of R 1 and R 2 and R 3. Once the work plan is tightened up and focused on a few key outputs, the implementing partner should con￾sider two implementation mechanisms:  The assessment team has observed that the program’s activity, which has had the most im￾pact in terms of accomplishment and creating MOE ownership, were those where the grant￾ee’s advisor is co-located with MOE counterparts. These include EMIS, biometric cards, and LIBERIA TEACHER TRAINING PROGRAM PHASE II 53 computer networks to cite the best examples. Other activities, which are not co-located with the MOE, show progress, but not to the same extent and not with the same degree of own￾ership. The team recommends that staff and long term experts work side by side with their counterparts.  The grantees’ team does not have the long term resident experts in place to develop the pol￾icies and systems needed as the foundation for program implementation. STTAs are not recommended, since they are not fully vested or accountable for results, and they need to work side-by-side, not replace MOE staff. USAID and the implementing partner should consider establishing a consultant staffing policy that will assure the right skills at the right place for the rest of the program. The Selection of Targets for Capacity Building Includes a Few “White Elephants”: The assessment team be￾lieves that the Rural Teacher Training Institutes (RTTI) are a vestige of a by-gone time when rural, residential teacher colleges and training institutes were the only logical approach to preparing a workforce interested in working in remote areas of the country. That dynamic is no longer valid. If young aspirants had any doubt that the teaching profession is seen as a low prestige, dead-end career, their nine month residency at the RTTIs would confirm it. These institutions are poorly managed, expensive to operate and perhaps misaligned with the real needs of the MOE, which is for a greater concentration on in-service training. A study of the viability and role of these institutions is required, as is a supply and demand study to determine how many teachers are really needed, and can be absorbed on the Government payroll. In addition, the GOL needs to develop and implement a policy to remove unqualified teachers from the ranks of teachers. Work at the Operational Level is not Solely with Political Appointees: The LTTP-II has moved from a highly visible and valuable technical assistance resource to the prior Minister and his deputies to a program attempt￾ing to achieve some traction with a new set of senior MOE managers. Although it is a natural tendency to highly value contacts with decision makers, and of course that relationship is important. It may be just as es￾sential to work with the bureau directors and other technical staff to encourage ownership and sustainability. This is especially true once the MOE moves to appoint a Director and technical staff for the Education Man￾agement Center, which will serve as one of the key implementers over the life of the project. Sequencing is an Important Implementation Strategy in the 2013-2015 Period: As a follow-up to the above comment related to the excessive breadth of the program and the inclusion of too many activities, most of which have not even commenced in earnest, the assessment team also believes that sequencing is an im￾portant strategy for implementation. The following chart shows the link that the assessment team recom￾mends: LIBERIA TEACHER TRAINING PROGRAM PHASE II 54 Institutional strengthening of the MOE is the condition precedent for the LTTP to be considered a success. Without that achievement, all other investments by the program are unlikely to have any sustained impact. The challenge for LTTP II is to find the right entry point so that inputs to strengthen the MOE’s manage￾ment and staff can be sequenced in a way that quickly and efficiently supports the institution without depriv￾ing students of needed services. Model the Behavior that the Project is Advocating: One of the key recommendations that will be presented in Section IV is for the MOE (with help from LTTP) to focus on systems development, among which is a monitoring system that tracks progress toward key outputs and outcomes at defined intervals. This will be an important step to building both capacity and credibility. Using a performance-based program budget will ena￾ble performance to be measured to assure that the plans are on track and progressing toward the defined re￾sult. Most activities have two dimensions that require monitoring: (i) the input that is the instrument for sup￾porting MOE (time, money, deliverables), and (ii) the MOE’s actual outputs based on that support. That is one of the main purposes of benefit/cost analyses in a performance management framework. The current PMP, based on a work plan and activities that are cluttered with mini-events, needs to be sharpened and an internal performance monitoring system created to assure that the program’s own investments are being made in a timely manner. THEN ….. Objective 2/R2 and R3 - Improved teacher performance in the classroom, especially through reading and math, can be achieved by (i) improving teacher policy and procedure for teacher, recruitment, training, deployment and career develop￾ment, and (ii) improving teacher training programs and reading and math delivery systems IF ….. Objective 1/Result 1 - MOE institutional capacity to plan, manage, and mon￾itor educational services is achieved through (i) strengthening critical systems, (ii) decentralizing monitoring and supervision, (iii) improving evidence based decisions based on reliable data to set policies and (iv) communicating progress and results .. LIBERIA TEACHER TRAINING PROGRAM PHASE II 55 V. LESSONS LEARNED AND RECOMMENDATIONS A. LESSONS LEARNED.  Capacity Constraints at the MOE are Overwhelming: The MOE is almost bereft of systems and Standard Operating Procedures (SOP). Aside from the financial management system, which is only passable, the planning, procurement, human resource, monitoring and evaluation, and pro￾fessional development systems are weak or non-existent. There is little institutional memory due to revolving leadership at the top levels and most systems are broken or opaque (perhaps by de￾sign). The uncertain environment created by the constant turnover of key personnel results in a lack of vision and limited ability to prioritize and operationalize plans, and an apathetic, risk￾averse, demoralized and insecure workforce that are less than fully devoted to task of transform￾ing the education system.  Budgetary Constraints Severely Limit MOE Performance and Requires Strategic Thinking: Libe￾ria invests one of the lowest percent of GDP (2.8%) in the world (ranks 147 out of 167 coun￾tries) and only four failed states are ranked lower in Africa. Liberia’s neighbors invest considera￾bly more (Guinea 3.1%, Sierra Leone 3.6%, Cote d’ Ivoire 4.6%) and the current level is insuffi￾cient to keep pace with population growth. Of the $US 93 million budget request, the Ministry of Finance provided only $US 45.5 million, which was the same as the previous year. The $US 45.5 million leaves no money for any basics, let alone improvements in the MOE. As a conse￾quence the MOF is the defacto (and unqualified) decision maker for education policy. The MOE needs to become much more strategic, develop a strategic plan and a program budget with realis￾tic objectives and targets based on real costing of needs. Aside from a few individuals, however, the MOE probably does not have the experience, expertise or political will to carry out a strate￾gic planning process. Therefore, the LTTP project needs to engage the MOE’s senior manage￾ment in a process that will define objectives and lay the foundation for improved performance.  Donor Financing Drives the Education Agenda and Ministry of Finance may be Misinterpreting the Data: In FY 2013 donor financing in education was approximately $US 59 million, which is greater than the resources provided by the GOL ($US 45.5 million). With such meager re￾sources, the MOE is eager to grab whatever is offered by donors, even if not aligned with MOE priorities. Some donors try to be all things to all stakeholders and spread meager resources too thinly. The donors’ diffuse agendas may confuse the MOE, and may cause it to lose focus. With host country and donor resources greater than $US 100 million flowing into the education sys￾tem, the MOF may infer that MOE resource levels are adequate, even though there are no MOE counterpart funds. The MOE may be buying time with donor funds, hoping MOE personnel and payroll reforms that will soon kick in extra funds for the MOE. In any case, the level of re￾sources allocated to the MOE is insufficient to support an effective learning environment or sus￾tain donor investments with the consequence that much of the potential impact of donor sup￾port will be lost. LIBERIA TEACHER TRAINING PROGRAM PHASE II 56  Success in R 1 Will Lay the Foundation for Sustainable Change at the MOE: The team has conclud￾ed that the sustainability of the LTTP’s investments in the MOE is dependent on success under Re￾sult 1, “MOE, CEO, DEO and RTTI capacity strengthened to plan, manage and monitor education￾al services”. In this context, there has been considerable discussion about the entry point for MOE capacity building and the sequencing of support from the LTTP. There are those who argue that the central office of the MOE is unreliable and unfocused. Continuing to pour new wine into old bottles is a losing proposition, they contend. Those who hold this view believe that it would be preferable to make a quick jump to support for de-concentrated CEOs and DEOs. The LTTP project has already initiated some steps under this model by training 75 decentralized management team members and starting to deploy computers and satellite communication facilities to the counties. The assessment team sees this as a high risk approach, absent some clear commitment from the top management of the MOE. Moreover, further investment appears to require a financial agreement from the Ministry of Finance that decentralization is the preferred Government strategy and that funds will be provided to sustain it. The alternative approach, and the one recommended by the assessment team, is to use the Education Man￾agement Center (or a strong alternative) as the vehicle to initiate work on the policies, procedures, protocols and systems that will bind together the departments and bureaus of the Ministry into a cohesive organization. We believe that a “build central capacity first” strategy is correct and will produce results. We would not ordi￾narily endorse such a strategy, given the past failures at the MOE, but the new centers offer an opportunity for the LTTP to work in partnership with deployed staff from the MOE on a joint venture basis. A two year strategy, with three month short interval action plans, could be very successful. The LTTP would probably need to recruit one or more advisors to be posted in the MOE to provide leadership, perhaps on long term basis, and supplemented by STTAs. Systems development by the LTTP is proceeding and the outputs of the project will meet international best practice standards. It seems that a strong link between the program’s advisor and MOE counterparts has been established. The focus, up to now, has been on payroll transparency (biometric cards), data collection and reporting (EMIS). However, some of the other key systems that are essential to “planning, managing and monitoring” have not begun. The team believes that the next two years of the project should have a defensi￾ble strategy based on the joint MOE-LTTP resources available, and the specific hurdles to be overcome. Sim￾ilarly, developing an agreed program structure for the Ministry, as the basis for a reform to program budget￾ing in the future, would be very useful. Once that is in place, even if budgets are still on a line item basis, the MOE can at least develop and monitor performance under program categories. The team believes that this is important and should be done jointly at the Education Management Center, which should also develop policies for each of these initiatives so that they are embedded in the new man￾agement culture of the MOE. This can be slow and simple, but if not the LTTP, who? If not now, when?  Reading and Math Programs are Making Progress: The team is impressed by the work by the EGRA and EGMA consultants on the LTTP-II. The assessment team recommends reinforcing the training for the teachers already trained, and a go-slow approach to rolling-out of the second component. While this reinforcement is taking place, the LTTP needs to work from within at the Center of Cur￾riculum Development to develop a longer term roll-out strategy for training teachers in reading, and to institutionalize the reading and math program with the Center of Curriculum Development. LIBERIA TEACHER TRAINING PROGRAM PHASE II 57  Education Financing has been Deficient and Cynical: The Government of Liberia should establish a specific resource envelope tied to national tax revenues so that the Ministry can have a predictable flow of funds, and the donors’ contributions can be additive not an offset to the national commit￾ment to fund education.  Centralized Administration and the Old Order - MOE as a Model of Decentralization: The Gov￾ernment of Liberia, as a unitary state, has always been highly centralized with most, if not all deci￾sions, made in Monrovia by the powerful elite. The government passed the Education Reform Act in 2011 to map out a new path for a more decentralized MOE, one of the first GOL ministries to do so. An organizational chart and staffing patterns for the central MOE and decentralized staff at the county level have been developed and five key officers (accounting, human resources, monitoring and evaluation, procurement, and planning) have been assigned to the CEO in each county. In addi￾tion, data on schools and school staffing is being collected and analyzed for better, evidence-based decision making. Despite this apparent progress, the Minister of Education, who was the architect of the decentralization plan, was fired. Perhaps due to other demands, the current Minister of Educa￾tion is not focused on decentralization and the five key officers in each county are only temporary employees of the MOE and have no resources to carry out their mandate. In addition, the County Education Boards, which were set-up to monitor the MOE and the decentralization process, have been largely ignored. Although decentralization holds much promise for improving the management of the MOE, it appears that the roll-out was pre-mature and/or the old order is unwilling to relin￾quish power. On the other hand, the MOE is forging a path for much of the government and, if the MOE can succeed in decentralizing, it could be a model for the nation.  “Emergency” Certification of Existing Teachers has Served Its Purpose: The Government’s post￾conflict strategy of providing existing, uncertified teachers a pathway to legitimacy as bona fide teachers and public employees of the Ministry of Education was an expedient, but necessary ap￾proach. The absence during the civil war of any centralized certifying authority for educators created a large cohort of teachers throughout the country whose competence and capacity has never been tested. Their service, which was sustained over a fourteen year period, provided some semblance of education for many of the country’s youth. The emergency measure of issuing a C Certificate to those existing teachers who successfully complete an in-service training program has served the country well. However, it is time to call an end to this strategy. A new strategy is needed to regularize the cadre of State-certified teachers, and this could be the foundation upon which a more structured, professional system could be constructed. New policies and procedures must now be developed to assure the recruitment and promotion of bona fide educators.  Certification of New Teachers Under the Same Theory of “emergency” has Proven to be a Strategic Error. Creating a pre-service, induction training program for new recruits to the teaching profession has not proven to be successful. The policy of offering a C Certificate to raw recruits who complete a nine-month residential program at one of the three RTTIs is deeply flawed. Dumbing-down the standards for admission, achievement, graduation and placement has not only failed to address the need for qualified teachers for schools in rural Liberia, but it has simultaneously created a “false posi￾tive” for the RTTIs, the institutions responsible for preparing these teachers. A pre-service program for teacher training needs to be at least two years in duration, followed by a 2-3 year internship under supervision prior to granting any certificate that conveys a professional qualification. The institutions and policies needed for such a program do not now exist in Liberia. LIBERIA TEACHER TRAINING PROGRAM PHASE II 58 Among the strategic weaknesses observed by the team were:  Resource constraints for each of the RTTIs were not defined, leading to serious budget shortfalls for such basic commodities as food and materials to say nothing of personnel sala￾ries  The basic supply and demand analysis related to both absorptive capacity of the RTTI to en￾roll the number of students that were admitted or the placement of graduates in teaching po￾sitions was not conducted  Service Delivery in a Capacity Void. A compelling case can be made to ignore the entrenched inter￾ests at MOE headquarters in Monrovia and shift the important services needed to improve educa￾tion in Liberia to the county level. Unless concerted efforts are made to improve the quality of ed￾ucation by improving the capacity and performance of teachers and improving the methods and materials used to teach early grade reading and math, another generation of young Liberians will grow up with very low levels of literacy, numeracy and the other skills. Moreover, decentralization of an education system in the best of circumstances, and these circumstances surely are not ideal, is a difficult issue to undertake and it frequently is a medium- to long-term task. Notwithstanding that compelling case, it is clear that the capacity of the MOE to implement service delivery programs was overestimated and little progress can be made in service delivery of education programs unless the MOE can muster the political will to build its institutional capacity and systems, and other key players, especially the Ministry of Finance and the Civil Service Agency, can assist the MOE by supporting payroll reforms and providing financial support.  White Elephants. The three rural, residential RTTIs are high-cost models for delivering teacher training in one of the poorest countries in the world in an underfinanced MOE that is struggling to provide basic education services of any kind. All three institutions appear to be over-staffed with a teaching faculty that is reported to be lackluster. Moreover, it is not clear that the institutions are providing the correct balance of pre- and in-service training, the correct mix of pedagogy and sub￾ject matter content, or the right length of training, given their students’ needs. Despite the fact that USAID built the RTTIs and renovated them after the war, USAID should work with the MOE to reconsider the use of the institutions and the future of teacher training in the country.  Data for What? The EMIS has made great strides and provided significant information that has be￾gun to influence the MOE. The MOE, with LTTP support, has managed to produce an edition of the Educational Statistics Report, which provides very useful data on largely access information. For example, the data apparently show that as many as one-third of the schools previously thought to be in the system do not exist, perhaps adding to the controversy about ghost teachers, schools and corruption. Without question, the provision of data and some analysis in the Education Statis￾tics Report is a major accomplishment. At the same time, the EMIS has very little information about issues such as equity, quality and relevance and largely reports data without much analysis. Although the EMIS collects and presents data from the counties, little data is available at the coun￾ty level and almost no analysis is done of that data. If the MOE is serious about decentralization, the EMIS system needs to work with the counties to develop those data and analysis. Finally, the EMIS could be most useful for developing evidence-based decision-making within the MOE. However, given the lack of resources to provide basic services, let alone address capacity problems LIBERIA TEACHER TRAINING PROGRAM PHASE II 59 within the MOE, it seems unlikely that the EMIS data will lead the transformation of the MOE in the near- to middle-term. B. RECOMMENDATIONS Currently, the LTTP appears to be largely stymied by the indifference of the MOE to move ahead on a num￾ber of fronts. In Result 1 (MOE, CEO, DEO and RTTI Capacity strengthened to plan, manage and monitor services), the LTTP has made good progress in helping to develop the blueprint for institutional reform (Edu￾cation Reform Act of 2011), the EMIS system, and some aspects of personnel and payroll reform. Neverthe￾less, the MOE still limps along without much vision, without many systems in place, and little progress to￾ward implementing the new law. In Result 2 (Improved teacher policy and procedures for teacher recruit￾ment, training, deployment, and career development), the LTTP has taken preliminary steps to pursue this result, including developing a series of analyses necessary for the MOE to act, but the turnover of Ministers, Deputy Ministers and Assistant Ministers has thwarted any adoption of policy and procedures. In Result 3 (Improved teacher training programs and reading/math delivery systems), the LTTP has made marginal im￾provements in teacher training programs and made a strong, but possibly unsustainable start in improving reading and math delivery systems. One option might be to do nothing in the hopes that the new Minister and her team will get up to speed and soon move ahead expeditiously. Despite some positive rhetoric, however, more than a year has passed with little positive action from the MOE. Based on the track record of the MOE, one could anticipate that the LTTP, despite the valiant efforts of the LTTP staff, will end with minor successes but failure in the major areas. Limit Service Delivery Until the MOE is Strong Enough to Absorb the Assistance: Although the early grade reading and math program appears to have made a strong start and is widely heralded by teachers and admin￾istrators, the LTTP staff implementing the program, as well as most teachers, and school administrators, sug￾gest that the program is spread too thinly. Nearly everyone interviewed suggested that the training of early grade (1-3) teachers and principals was good and that the materials provided were terrific. They universally argued that more training was needed for the teachers, principals and coaches and that more reading and math materials should be provided. Many maintained that a “whole school” approach should be adopted for training teachers and providing teaching and learning, given the turnover of teachers and the presence of large numbers of over-age children with no reading and math skills in all primary grades. The lack of MOE coun￾terpart funds and MOE personnel who are only half-heartedly committed to the program, along with the other factors cited above, suggest that the program may not be sustainable in the medium-term. USAID/Liberia and the LTTP may want to reexamine the programs’ output targets and use the savings to reinforce the gains already made. Focus and Concentrate: The MOE currently is nearly impotent to deliver educational services in large part, because it has very few systems working. Nearly everything seems to be done in an ad hoc manner without regard to standard operating procedures. Unless the MOE makes a determined effort to fix the broken sys￾tems and put in place guidelines, operating procedures and working systems, using the many tools have been developed by LTTP, but are awaiting review and approval by the MOE, much of the work of the MOE will be completed slowly and the LTTP will be stalled on many fronts. Both the MOE and LTTP need to focus and concentrate on institutional strengthening and putting systems in place. LIBERIA TEACHER TRAINING PROGRAM PHASE II 60 Priority Actions Should be Defined and Implemented: The assessment team recommends the following ac￾tions for LTTP: Within Result 1 (MOE, CEO, DEO, and RTTI capacity strengthened to plan manage and monitor educa￾tional services) the LTTP should consider redoubling its efforts to develop systems within the MOE.  Assuming the LTTP will have some input in selecting and prioritizing the systems, the LTTP should work closely with the responsible units (Center for Education Management) to provide technical as￾sistance and technical resources so that it will have the capacity to carry out its new mission as soon as possible.  To ensure sustainability, the LTTP should employ the model of co-locating with the MOE unit for a substantial portion of the week.  If the MOE decides to continue the decentralization effort, the LTTP should continue to assist the decentralization effort by training and empowering the five-person units assigned to the CEO, with a possible emphasis on information management and systems maintenance. Within Result 2 (Improved teacher policy and procedures for teacher recruitment, training, deployment and career development), LTTP should continue to develop analysis about these topics but should consider adopting some new tactics. A demonstration approach to professional development would be the key deliver￾able under R- 2. The technical development work required for an “improved teacher policy and procedures for teacher re￾cruitment, training, deployment and career development” under R-2 has not been initiated, although LTTP has prepared some initial working papers for consideration by the MOE. The team is recommending that the LTTP begin to empower the Center for Certification, Accreditation and Licensing (or a strong alternative) so that it becomes the internal “consulting unit” on career planning for MOE’s teacher corps. As a starting point, the team suggests that the Center develop all of the requirements for career planning of primary school teachers and administrators. A strategy is needed to:  Identify all teachers assigned to primary schools (name, age, academic records, years in the class￾room, residency, training courses attended, year of certification, etc.)  Develop a policy on how to classify the teachers  Classify all teachers in this preliminary framework  If this is the documented supply, use EMIS data to help determine the demand  Develop demand data according to classification and by location This is just an example of the preliminary work needed to develop policy options regarding salaries, condi￾tions for advancement (e.g., years of service, performance evaluations.) At the present time R-2 is at a standstill. There appears to be only some payroll cleansing activity but nothing substantive is being done on professional development. While using MOE’s past efforts as a model for the rest of Government may seem to be a compliment, it has the risk of diverting resources outside the scope of the Agreement. LIBERIA TEACHER TRAINING PROGRAM PHASE II 61  First, once the MOE moves into its new building, some LTTP staff might consider co-locating in the building with the unit responsible for Human Resource policy and procedures.  Second, some of the analysis completed under the LTTP project appears to be overly academic and not very practical. Care should be given to develop manuals and procedure that are practical and un￾derstandable to administrators who have had no exposure to these topics. LTTP should consider var￾ious approaches to develop a better understanding of the importance of the analysis, manuals, and procedures and to ensuring that the systems and policies are implemented.  Finally, the LTTP program should consider developing a series of analyses that might assist the MOE to move ahead quickly in the area of teacher policy. For example, the LTTP might consider developing an estimate of the short-, medium- and long-term needs (supply and demands analysis) for teacher training or the LTTP should develop a paper on the optimal mix of pre- and in-service training, the optimum length of training needed for the client groups the RTTIs are serving, and what might be the most cost-effective methods for supplying Liberia the teachers it needs. Within Result 3 (Improved teacher training programs and reading and math delivery systems), LTTP should consider severely dampening implementation of the second cohort of schools set to receive the early grade reading and early grade math programs to a manageable number and work to consolidate the gains made in the first cohort. The second cohort might take in one-third to one-half of the schools originally envisioned and the saving could be used to ensure that the program in the first cohort break-though. The program aimed at reinforcing the gains in the first cohort should consider:  Since it is difficult to change teacher behavior and many teachers need more than one workshop to assimilate the information, LTTP should consider a series of training programs that build on the previous course work (reading workshop 101, followed by reading workshop 102, followed by read￾ing workshop 103, etc.).  LTTP should consider dropping deep rural schools that are especially difficult for coaches to get to on a regular basis.  Additional training in reading and math and school leadership modules should be considered for school principals.  A “whole school” approach (all teachers in the school trained) where the reading and math program is expanded for grades 4-6, insofar as there are many regular and over-age children in those grades who cannot read or do simple math.  Finally, the LTTP should consider expanding the programmed materials used in the reading and math program as they were very well received but most teachers expressed a need for more and var￾ied materials. LIBERIA TEACHER TRAINING PROGRAM PHASE II 62 VI.FINAL THOUGHTS The LTTP-II, its resident staff and the two organizations responsible for implementing the Cooperative Agreement are to be commended for their diligence and commitment to supporting education reform in Li￾beria, which is a daunting challenge. Among the noteworthy accomplishments are the following:  Education Reform Act of 2011 – The LTTP-II consultants provided policy and strategy guidance to the Ministry of Education in the development of a sound organizational structure, which sup￾ports both the core functions on the Ministry, and also established three “centers for excel￾lence” to serve as the platform for future reform and transformation of the organization;  Education Management Information Systems (EMIS) – to have completed this task at all is a major achievement. To have done it at such a high quality level reflects a competence and skill that must be commended. If sustained, the EMIS has the capacity to change the way that evi￾dence and data are used for educational decisions in the Ministry and Government.  Early Grade Reading - this effort has demonstrated that the Government’s policy goal of all children reading at the end of Grade 3 can be achieved with the right strategy and approach, and under conditions of continuous reinforcement. The companion piece – early grade math – has not advanced as quickly, but it seems to be as highly valued by teachers, suggesting the oppor￾tunity to redouble efforts in the coming two years. The assessment team also took away some lasting impressions of schools, teachers and principals in remote rural schools. Under-funded, under-serviced, and often unappreciated, the local educators are still striving to create a better future for Liberia’s children. Putting a human face on this crisis in educational service delivery has motivated the team to make some strong and provocative recommendations for the MOE, as well as the LTTP itself. On the other end of the spectrum, it appears that it might be the right time to seek high-level meetings be￾tween USAID (Ambassador, USAID/Liberia Mission Director, Chief of Education Office) and the GOL (President of the Country, Minister of Finance, Minister of Education) where a frank discussion might take place outlining the problems and seeking some assurance that some milestones will be achieved within the next year. Some of the milestones might include a portion of the nine milestones below:  A three-year strategic plan aimed at developing institutional capacity and/or implementation of the Education Reform Act of 2011 with realistic, fully costed targets;  The development of the Center of Excellence for Management within the MOE;  A consolidated MOE budget with MOF assurances for minimum funding levels set as a % of own￾source revenue. Additional donor support could then supplement the national investment not be a substitute for the lack of it;  A staffed and funded decentralization plan, if it is the decision of the MOE and USAID to proceed with that plan; LIBERIA TEACHER TRAINING PROGRAM PHASE II 63  A monitoring and evaluation unit using EMIS data to inform MOE policy at the central and county levels;  A revised or newly developed teacher certification and licensing scheme, at least for primary school teachers;  An analysis of the demand and supply of teachers and unmet needs for teacher training over the next ten years and a policy on how to deliver pre- and in-service training;  A policy analysis on how best to utilize the RTTIs;  A functioning early grade reading and math unit within the curriculum center of excellence It would seem that a direct expression of concern about the lack of progress in the MOE and requiring some compliance to milestones is an appropriate option, because the window for change in Liberia may be starting to close. It will be important for the MOE to have developed the institutional capacity to deliver educational services before the current national administration leaves office. LIBERIA TEACHER TRAINING PROGRAM PHASE II 64 VII. ANNEXES ANNEX 1: LIST OF DOCUMENTS CONSULTED 1. A Commitment to Making a Difference, Monrovia, Liberia – 2010 2. A Guide to Observation and Clinical Field Experience 3. A Qualitative Study on the Environment and Classroom Interactions in GOAL Case Study Schools, USAID/Liberia Monitoring and Evaluation Project, 2013) 4. An Act Repealing the Public Employment Law and Amending the Executive Law to Create a Civil Service Agency 5. Capacity to Improve Teaching and Learning and Achieve Transformational Reform in Liberia (LTTP report for MOE) 6. Decentralization and the MCSS Model (LTTP report for MOE) 7. Education Financing in Developing Counties: Level and Sources of Funds February 2002 8. Education Statistics for the Republic of Liberia (2012) 9. Educational Diagnosis through Monitoring, County Capacity in M&E for Decentralization (MOE publication prepared by LTTPII) 10. Females Teaching in Liberia (LTTP report for MOE) 11. From Closed Books to Open Doors West Africa’s Literacy Challenge 12. GOL Budget Framework paper FY 2013/14 13. Government of Liberia Agenda for Transformation 14. International Education Statistics 15. Joint Education Sector Review Report 2012 16. KRTTI Risk Rating Pooled by Criterion (Is this the KRTTI PFM Risk Assessment?) 17. Liberia Anti-Corruption Commission (LACC) IAACA 18. Liberia Education Country Status Report 19. Liberia Education Reform Act 2011 20. Liberia Governance Commission Stakeholder Survey on Education 21. LTTII: Project Year III, Quarter II Report 22. LTTP: English Skill Builders Topic 3 LIBERIA TEACHER TRAINING PROGRAM PHASE II 65 23. LTTP: Math Skill Builders Topic 5 24. LTTPII: Five Year Work Plan June 2010-September 2015 25. LTTPII: Associate Award Program Description 26. LTTPII: Budget II 27. LTTPII: Early Grade Mathematics Student Activity Book 28. LTTPII: Early Grade Mathematics Teacher Guide 29. LTTPII: Early Grade Reading in Liberia Decodable Stories 30. LTTPII: Final PMP 31. LTTPII: Institutional Management and Education Administration Assessment Checklist: LTTP 32. LTTPII: Interim Assessment of Performance of LTTP Students in Reading and Math 33. LTTPII: Liberia Teacher Training Program (LTTP) EMIS 34. LTTPII: Mid-Term Assessment of the Liberia Teacher Training Program March 2009 35. LTTPII: Principal Training Modules 36. LTTPII: Quarterly and Annual Reports 37. LTTPII: Reading First + Math Volume -2 Liberia Teacher Manual 38. LTTPII: Reading First +Math Student Report Card Manual 39. LTTPII: Reading First Volume -1 40. LTTPII: The Early Grade and Math Baseline Assessment Report 41. LTTPII: The Problem and Development Hypothesis 42. Ministry of Education, “Government of Liberia Education Sector Review Report” June 23-24, 2009. 43. Ministry of Education, “Liberia Education Administrative Regulations, Vol. 4.” Monrovia, Liberia 44. Ministry of Education, “Liberian Primary Education Recovery Program”, prepared for Fast Track In￾itiative, Monrovia, Liberia. March 30, 2007. 45. Ministry of Education, “Policy Framework for Liberia Education and Training Sector”, Monrovia, Liberia 2000. 46. Ministry of Foreign Affairs, “The Liberian Education Act 2011”, Monrovia, Liberia August 9, 2011. 47. MOE Evaluation Guidelines (Suggestions for Good Evaluation Practices MOE) (document prepared by LTTPII) 48. MOE Monitoring Framework (Institutionalization of Monitoring for Effective Management in the Education) (document prepared by LTTPII) 49. MOE Principal Monthly Reporting template LIBERIA TEACHER TRAINING PROGRAM PHASE II 66 50. MOE: Annual School Census Questionnaire 2013 51. MOE: County Monitoring Manual (Recommendation for Monitoring Practices) (document prepared by LTTPII) 52. MOE: Education Statistics for the Republic of Liberia, National Statistical Booklet 2012 53. MOE: EMIS Toolkit 2013, Government of the Republic of Liberia MOE (document prepared by LTTPII) 54. MOE: Liberia Education Administrative Regulations MOE (document prepared by LTTPII) 55. MOE: Liberia Education Sector Plan 2010-2020 56. MOE: Parent Teacher Association Guidelines for Trainers 57. MOE: Parent Teacher Association Operational Manual 58. MOE: Policies for Reform 59. MOE: Pre- Service ‘C’ Certificate Staff Policy Manual 60. MOE: Pre-service ‘’C’’ Certificate Curriculum 61. Performance Monitoring Plan (PMP) LTTP (same as #9) 62. Political Economy Analysis, USAID Liberia, 2013) 63. USAID/Liberia Country Development Cooperation Strategy 64. USAID/Liberia Education Briefing Book USAID Education Strategy/2011-2015 65. USAID/Liberia MOE PFMRAF Stage 2 Report 66. USAID/Liberia Teacher Training Program II (LTTP II), Enhancing Teacher Quality. Efficiency and Effectiveness. Monrovia, Liberia (brochure). 67. USAID/Liberia: PRMRAF Assessment Report of Ministry of Education 68. USAID’s Leadership in Public Financial Management LIBERIA TEACHER TRAINING PROGRAM PHASE II 67 ANNEX 2: LIST OF INDIVIDUALS CONSULTED BY ORGANIZATION No Name Sex Title Institution Date 1 LTTP Staff W. Teerix Beh M Program Coordinator LTTP/RTI 06/11/13 2 Alexander Kollie M Logistician LTTP/RTI 06/11/13 3 Hawa K. Idris F Sr. Finance Specialist LTTP/RTI 06/11/13 4 Romelus E. Amnon M Sr. Reading & Math Spe￾cialist LTTP/RTI 06/11/13 5 Geanjay G. Roberts F Assessment Specialist LTTP/RTI 06/11/13 6 Adolphus S. Toe M Finance Associate LTTP/RTI 06/11/13 7 Mulbah Howard M Sr. Program Manager LTTP 06/11/13 8 A.H. Nupput M C.A.S LTTP 06/11/13 9 George Idum Sam M Senior Finance Director LTTP 06/11/13 10 Kemo Sheriff M Sr. Finance Manager LTTP 06/11/13 11 Patrick K. Nagbe M Reading & math specialist LTTP 06/17/13 12 Prince Bawoh M Satellite Manager LTTP 06/17/13 14 Jacob K. Jallah M Coach LTTP 06/18/13 15 Enoch S. Oppong M Education Field Manager LTTP 06/19/13 16 Theresa E. M. Caesar F Reading & Math Specialist LTTP/RTI 06/19/13 17 V. Varnell Kiardu M Reading & Math Specialist LTTP/RTI 06/19/13 18 Ted Johnson M Satellite Manager LTTP 06/19/13 19 Patrick Sayon M Finance/Admin assistant LTTP 06/19/13 20 Walter Philips M Policy and IT Advisor LTTP 06/24/13 21 Michael Blundell M Chief of Party LTTP 06/25/13 22 Trokon B. Wayne M Team Leader for Reading LTTP/RTI 06/25/13 23 Patience Usuah F Program Coordinator LTTP/RTI 06/25/13 LIBERIA TEACHER TRAINING PROGRAM PHASE II 68 24 Sayku Waritay M Team Leader for Pre￾service LTTP 06/25/13 25 G. Benedict Kofa M Education Specialist LTTP 06/25/13 26 Mulbah Howard M Sr. Program Manager LTTP 06/25/13 27 Sam D. Davis M Sr. Program Manager LTTP 06/25/13 30 Delwlebo Tuowal M M&E Specialist LTTP 06/27/13 31 Emmanuel B. Morris M M&E Associate LTTP 06/27/13 32 Oretha Doe Jackson F Coach LTTP 06/19/13 33 Moses J. John M Field Education Officer LTTP 06/19/13 34 Medina Korda F Research Education Ana￾lyst/Education policy and system RTI International LTTP/RTI 07/09/13 35 Other USAID Projects: Ollie White F IFESH Country Director IFESH 07/08/13 36 Vicki Cooper M Chief of Party/GEMS GEMS 06/28/13 37 Lisa Deyo F Chief of Party/AIR AIR 06/27/13 38 Simon James M Former COP-Advancing Youth AYP 07/02/13 Ministry of Education (MOE) 39 Emmanuel Dahn M District Education Officer MOE 06/18/13 40 William Smith M Secretary MOE 06/18/13 41 Karweah Kwabrie M Planning Officer MOE 06/18/13 42 Felix G. Traves M Finance Officer MOE 06/18/13 43 J. Mayoud Toue M Procurement Officer MOE 06/18/13 44 Kwelleegbo G.S. Kapu M CEO/Gbarnga Bong MOE 06/20/13 45 Jackson S. Simgbeh M DEO / Gbarnga Bong MOE 06/20/13 46 Philip F. Mulbah M DEO/Salala Bong MOE 06/20/13 47 Bories B. Bantea M HR MOE 06/21/13 48 Stephen J Tooney M M&E Officer MOE 06/21/13 49 Stephenson Suah M Procurement Officer MOE 06/21/13 LIBERIA TEACHER TRAINING PROGRAM PHASE II 69 50 Felicia Doe Suma F Assistant Minister for Ear￾ly Childhood Education MOE 06/26/13 51 Dweh D. Miller M Program Officer, Early Childhood Education MOE 06/26/13 52 Edwin Tetteh M Former Deputy Minister for Administration MOE 06/26/13 53 Musu Dixon Badio F Former Assistant Minister for Teacher Education MOE 07/27/13 54 Hawa Goll Kotchi F Deputy Minister for In￾struction MOE 06/27/13 55 Albert Coleman M Senior Policy Advisor to the Minister MOE 06/27/13 56 Dr. Khalipha Bility M Deputy Minister, Planning and Research MOE 06/27/13 57 Shedrach Kerl M Director, Teachers Educa￾tion MOE 07/03/13 58 Patience King F Director, licenses & certi￾fication MOE 07/03/13 59 J. Fernarch Gbozuah M Asst. Director, Licenses & Certification MOE 07/03/13 60 Timothy Gaye M Former Assistant Minister for Fiscal Affairs MOE 07/03/13 61 Ruel Dempster M Consultant, Fin. Mgt/HRD MOE 07/03/13 62 Augustine Josiah M MOE Comptroller MOE 07/03/13 63 Ruth Mayor F Chief Accountant MOE 07/03/13 64 Wilhemina C. Versini F Deputy Comptroller MOE 07/03/13 65 Hon. Etmonia David Tarpeh F Minister of Education MOE 07/02/13 66 Moses Jackson M Assistant Minister for Teacher Education MOE 07/08/13 LIBERIA TEACHER TRAINING PROGRAM PHASE II 70 Ministry of Internal Affairs (MIA) 67 F.M.Kromah M Political Officer MIA 06/18/13 68 Reginald Mehn M County Inspector MIA 06/18/13 69 Christiana Dagadu F Superintendent Nimba County MIA 06/18/13 70 Hon Selena Polson Mappy F Superintendent Bong County MIA 06/21/13 71 Anthony B. Sheriff M Development Superinten￾dent Bong County MIA 06/21/13 Schools 72 Samuel King M Principal Combat Stress School 06/12/13 73 Johnny Sirleaf M Vice Principal Ann Sandell Police Academy School 06/12/13 74 Theresa Morris M Vice Principal for Aca￾demic Affairs Ann Sandell Police Academy School. 06/12/13 75 Muriel Best F Volunteer Ann Sandell Police Academy School 06/12/13 76 Victoria Paye F Teacher Ann Sandell Police Academy School 06/12/13 77 Lydia S. Someway F Teacher Ann Sandell Police Academy School 06/12/13 78 Samuel King M Principal Combat Stress School 06/12/13 79 Johnny Sirleaf M Vice Principal Ann Sandell Police Academy School 06/12/13 80 Theresa Morris M Vice Principal for Aca￾demic Affairs Ann Sandell Police Academy School 06/12/13 81 Muriel Best F Volunteer Ann Sandell Police Academy School 06/12/13 82 Davidlyn Z. W. Wright F Teacher Ann Sandell Police Academy School 06/12/13 83 Sadiatu Flowers F Teacher Ann Sandell Police Academy School 06/12/13 84 Victoria Thomason F Teacher Ann Sandell Police 06/12/13 LIBERIA TEACHER TRAINING PROGRAM PHASE II 71 Academy school 85 Leo Garswah M Teacher Ann Sandell Police Academy School 06/12/13 86 Thelma Worthmehn F Principal Jorbar Pub. School 06/12/13 87 Christopher Ampleton M Principal Geblon Pub. School Todee 06/13/13 88 Moses Dennis M Teacher Geblon Pub. School Todee 06/13/13 89 Boima Wordsworth M Principal Careysburg Pub. School 06/13/13 90 James F. Yarsia M Chief Advisor Salayea Public School 06/18/13 91 Joseph M. Folokulah M Vice Principal Salayea Public School 06/18/13 92 Kpadeh P. Flomo M Vice Principal Sharon Elementary School 06/18/13 93 James Y. Tokpa M Principal Sharon Elementary School 06/18/13 94 Emmanuel S. Yark￾pawolo M Teacher Sharon Elementary School 06/18/13 95 Dada S. Kankah M Teacher Gbapa Public School 06/19/13 96 Jerome Zoick M Principal Gbapa Public School 06/19/13 97 Betts K. Gan F Teacher Gbapa Public School 06/19/13 98 Jesse D. Morgbay M Teacher Gbapa Public School 06/19/13 99 Stephen Gbean M Principal Sanniquelay Extension School 06/19/13 100 Cecelia P. Tokpah F Registrar Sanniquelay Extension School 06/19/13 101 Albert L. Leukpah M Vice principal Sanniquelay Extension School 06/19/13 102 Clarice Saye F Teacher Sanniquelay Extension School 06/19/13 103 Fredrick E. Zianu M VP Assistant Sanniquelay Extension School 06/19/13 104 P. Emmanuel Zuma M Vice Principal George A Dumba 06/20/13 LIBERIA TEACHER TRAINING PROGRAM PHASE II 72 School 105 Robert Lorma M Registrar George A Dumba School 06/20/13 106 Stephen G. Dahn M Teacher George A Dumba School 06/20/13 107 Cooper Kpakakh M Teacher George A Dumba School 06/20/13 108 Rancy Keekpo M Teacher George A Dumba School 06/20/13 109 Fong Bembo M Teacher George A Dumba School 06/20/13 110 Saye Keliza M Principal Zogeseh Public School 06/20/13 111 George N. Goyemaniwoah M Teacher Zogeseh Public School 06/20/13 112 Nelly K. Swan F Teacher Zogeseh Public School 06/20/13 113 Victoria Yini f Teacher Zogeseh Public School 06/20/13 114 Arthur F. Taylor M Vice principal Jorkpemue Public School 06/21/13 115 Mrs. R. B. Lablah F Registrar Jorkpemue Public School 06/21/13 116 Wilson Garlematter M Principal Jorkpemue Public School 06/21/13 117 J. Kalokpo M Vice principal Dorothy Cooper Pub￾lic School 06/21/13 118 Irene G.M. Yorkollie F Teacher Dorothy Cooper Pub￾lic School 06/21/13 119 Stephen B. Frederick M Principal Dorothy Cooper Pub￾lic School 06/21/13 120 Mathew D. Wortor M Teacher Dorothy Cooper Pub￾lic School 06/21/13 121 Florence B. Gibson F Teacher Dorothy Cooper Pub￾lic School 06/21/13 LIBERIA TEACHER TRAINING PROGRAM PHASE II 73 122 Irene N. Yeate F Teacher Dorothy Cooper Pub￾lic School 06/21/13 United States Agency for International Development (USAID) 124 Kimberly Rosen F Acting Mission Director USAID 07/05/13 125 John Ellis M Program Officer USAID 07/05/13 Julia Richards F Education Team Leader USAID Casey McHugh F Education Program Spe￾cialist USAID 126 Mardea Nyumah F Education Specialist USAID 07/05/13 127 Miriam White F Education Specialist USAID 07/05/13 128 Paul Binkley M Education HICD Advisor USAID 07/05/13 Rural Teachers’ Training Institute (RTTI) 129 Lovo M. Subar F Human Resources ZRTTI 06/17/13 130 Stephen Bood M Secretary ZRTTI 06/17/13 131 Patrick S. Kassia M ICT specialist ZRTTI 06/17/13 132 Lovo M. Subar F Human Resources ZRTTI 06/17/13 133 Stephen Bood M Secretary ZRTTI 06/21/13 134 Dewaladou J. S. Gayvolor M Teacher/Trainer ZRTTI 06/21/13 135 Richard Wolobah M Teacher/Trainer ZRTTI 06/21/13 136 Blamoh Z. Fully M Teacher ZRTTI 06/21/13 137 Alonzo Harris M Teacher ZRTTI 06/21/13 138 Jenisiah E. K. Sulunteh F Teacher ZRTTI 06/21/13 139 Samuel W. Gbianhbei M Teacher ZRTTI 06/21/13 140 Dorminic Morlu M Trainer ZRTTI 06/21/13 141 Zola Robert F Trainer ZRTTI 06/21/13 141 Titus Gargar M Trainer ZRTTI 06/21/13 143 Jackson K. Dorbor M Trainer ZRTTI 06/21/13 144 Tarnue J. Gorvego M Registrar ZRTTI 06/21/13 LIBERIA TEACHER TRAINING PROGRAM PHASE II 74 145 Nathaniel S. Saryue M Teacher WRTTI 06/21/13 146 Clarence G. Padma M Teacher WRTTI 06/21/13 147 Jackson Torbor M Teacher WRTTI 06/21/13 148 Joseph W. Appleton M Instructor WRTTI 06/21/13 149 Kenneth Q. Nyepan M Actg AD WRTTI 06/21/13 150 J. CHA-CHA Nunu M Teacher WRTTI 06/21/13 151 Jorna W. Varmoma M Instructor WRTTI 06/21/13 152 Harrison Sangbe M Instructor WRTTI 06/21/13 153 Thomas Kan Nyepan M Instructor WRTTI 06/21/13 154 Precious Dennis F Director KRTTI 07/18/13 155 Jerome K. Nyan M Teacher KRTTI 06/21/13 156 Sylvanus P. Momoh M Dean KRTTI 06/21/13 157 Dewaladou J. S. Gayvolor M Teacher/trainer ZRTTI 06/21/13 158 Richard Wolobah M Teacher/Trainer ZRTTI 06/21/13 159 Blamoh Z. Fully M Teacher ZRTTI 06/21/13 160 Alonzo Harris M Teacher ZRTTI 06/21/13 161 Jenisiah E. K. Sulunteh F Teacher ZRTTI 06/21/13 162 Samuel W. Gbianhbei M Teacher ZRTTI 06/21/13 163 Dorminic Morlu M Trainer ZRTTI 06/21/13 164 Zola Robert F Trainer ZRTTI 06/21/13 165 Titus Gargar M Trainer ZRTTI 06/21/13 166 Jackson K. Dorbor M Trainer ZRTTI 06/21/13 167 Tarnue J. Gorvego M Registrar ZRTTI 06/21/13 168 Nathaniel S. Saryue M Teacher WRTTI 06/21/13 169 Clarence G. Padma M Teacher WRTTI 06/21/13 170 Jackson Torbor M Teacher WRTTI 06/21/13 171 Joseph W. Appleton M Instructor WRTTI 06/21/13 LIBERIA TEACHER TRAINING PROGRAM PHASE II 75 172 Kenneth Q. Nyepan M Actg AD WRTTI 06/21/13 173 J. CHA-CHA Nunu M Teacher WRTTI 06/21/13 174 Jorna W. Varmoma M Instructor WRTTI 06/21/13 175 Harrison Sangbe M Instructor WRTTI 06/21/13 176 Thomas Kan Nyepan M Instructor WRTTI 06/21/13 177 Jerome K. Nyan M Teacher KRTTI 06/21/13 178 Sylvanus P. Momoh M Dean KRTTI 06/21/13 179 J. Gertrude Smart F Principal KRTTI 06/21/13 180 Mitchel l F. Kolobah M Teacher KRTTI 06/21/13 181 J. Gertrude Smart F Principal/Teacher KRTTI 06/21/13 182 Rev. Quie M Director WRTTI 07/09/13 183 Kenneth Q. Nyepon M Academic Dean WRTTI 07/09/13 184 Abraham Kromah M Business Manager WRTTI 07/09/13 185 Rev. Quie M Director WRTTI 07/09/13 Donor Agencies 186 Bernard Batidzirai M Education Specialist, UNICEF UNICEF 07/08/13 187 Fazlul Haque M Deputy Representative & Rep UNICEF 07/08/13 188 Juan Casanova M Head of Operation, EU European Union 07/15/13 Others 189 George Werner M CSA Director Civil Service Agency 06/28/13 190 E. Othello Gongar M Commissioner Governance Commis￾sion 07/02/13 LIBERIA TEACHER TRAINING PROGRAM PHASE II 76 ANNEX 3: LIST OF PROVINCES, DISTRICTS AND SCHOOLS VISITED No Provinces Districts Schools 1 Montserrado County Soul Clinic/Paynesville Combat Stress School 2 Montserrado County Police Academy/Paynesville Road-Ann Sandell School 3 Montserrado County Joe bar/Paynesville Paynesville Community School 4 Montserrado County Todee Geblum Public School 5 Montserrado County Careysburg Careysburg Public School 6 Lofa County Zorzor ZRTTI Demonstration school 7 Lofa County Salayea Sharon Memorial Elementary School 8 Lofa County Salayea Salayea Public School 9 Nimba County Sanniquelie-Mah Gbapa Public School 10 Nimba county Sanniquelie-Mah New Sanniquelay Extension School 11 Nmiba County Ganta Zogeseh Public School 12 Nimba County Ganta George A. Dumba School 13 Bong county Jorquelleh Jorkpenmue Public School 14 Bong County Jorquelleh Dorothy Cooper Public School LIBERIA TEACHER TRAINING PROGRAM PHASE II 77 No Provinces Districts Schools 1 Montserrado County Soul Clinic/Paynesville Combat Stress School 2 Montserado County Police Academy/Paynesville Road-Ann Sandell School 3 Montserado County Joe bar/Paynesville Paynesville Community School 4 Montserado County Todee Geblum Public School 5 Montserado County Careysburge Careysburge Public School 6 Lofa County Zorzor ZRTTI Demonstration school 7 Lofa County Salayea Sharon Memorial Elementary School 8 Lofa County Salayea Salayea Public School 9 Nimba County Sanniquelie-Mah Gbapa Public School 10 Nimba County Sanniquelie-Mah New Sanniquelay Extension School 11 Nmiba County Ganta Zogeseh Public School 12 Nimba County Ganta George A. Dumba School 13 Bong county Jorquelleh Jorkpenmue Public School 14 Bong County Jorquelleh Dorothy Cooper Public School LIBERIA TEACHER TRAINING PROGRAM PHASE II 78 ANNEX 4: INSTITUTIONAL MANAGEMENT AND EDUCATION ADMINISTRATION CHECKLIST Liberia Teacher Training Program II ASSESSMENT METHOD: The results presented below are based on findings and conclusions drawn from interviews and technical re￾ports concerning policies, systems and institutional capacity of the MOE to plan, manage, monitor and report on pre-service and in-service teacher training so as to “prepare a cadre of trained and certified primary school teachers”. It encompasses assessments of implementation processes and systems, outcome and impact, as well as sustainability and cost effectiveness. Areas Assessed vis a vis Best Practice Exists Works Effectively Exists Needs Improvement Does Not Exist. Needed Does Not Exist. Not Needed STATUTES AND REGULATIONS 1. The new Education Act is clear concerning the lawful authority and responsibility of the Ministry to man￾age and deliver Teacher Training (TT) 2. Regulations and procedures have been developed and distributed by the Ministry to centralized and decentral￾ized (RTTIs) units covering service standards, performance indicators, management reports and other admin￾istrative processes for TT. 3 Delegation of authority and respon￾sibility for TT program management is clearly defined and issued through Ministry regulations published/posted conspicuously. (ACCOUNTABILITY) 4. Policies regarding administration of programs for pre-service and in￾service TT programs are clear, formal- LIBERIA TEACHER TRAINING PROGRAM PHASE II 79 Areas Assessed vis a vis Best Practice Exists Works Effectively Exists Needs Improvement Does Not Exist. Needed Does Not Exist. Not Needed ized, published/posted, and uniformly applied 5. Budget policies and guidelines for preparing TT cost estimates for all units are clear, formalized, pub￾lished/posted, and uniformly applied 6. Accounting policies and procedures for administration of TT funds are clear, formalized, published/posted and uniformly applied. 7. The public’s right to have access to information is assured and embedded in the culture and practice of the Min￾istry, i.e., someone has specific author￾ity to release public documents, sub￾ject to specific conditions only, and there is evidence of such. 8. The Minister or similar official is responsible, by law, for informing the public of the Ministry’s progress and results for educational access, quality and for TT progress, e.g., certified teachers are counted/plan versus actu￾al data is available. TOTALS 2 3 3 SERVICES 1. Service priorities, standards and performance indicators for delivering pre-service and in-service TT have been developed through a consultative process that includes administrators, teachers and, as appropriate, commu￾nity groups. LIBERIA TEACHER TRAINING PROGRAM PHASE II 80 Areas Assessed vis a vis Best Practice Exists Works Effectively Exists Needs Improvement Does Not Exist. Needed Does Not Exist. Not Needed 2. Planned results, expressed in quanti￾tative terms (outputs and planned out￾comes), have been defined with ap￾propriate measures and indicators, adopted and published. (TRANSPARENCY) 3. The commitment of the Ministry to student-focused, quality services is communicated from senior manage￾ment and administrators to all em￾ployees and parents 4. Teachers and administrators are trained, monitored, and recognized for delivering quality services. Classroom monitoring for post-TT is a formal￾ized internal service with an estab￾lished budget and clear methods for reporting results. 5. Service standards, i.e., the learning objectives for students which are sus￾ceptible to parental monitoring are posted at all schools and any other service delivery location where teach￾ers engage students in learning. 6. Service units (what is actually deliv￾ered) are recorded, reported to man￾agement, and reconciled with fees col￾lected. (VALUE FOR MONEY) TOTALS 1 4 1 STANDARDS 1. Education services, both for inter￾nal service (such as TT) and external services such as classroom perfor￾mance are standardized and expressed LIBERIA TEACHER TRAINING PROGRAM PHASE II 81 Areas Assessed vis a vis Best Practice Exists Works Effectively Exists Needs Improvement Does Not Exist. Needed Does Not Exist. Not Needed in quantifiable and measurable terms. (TRANSPARENCY) 2. Performance standards are moni￾tored and reported by the MOE and adjusted at least annually. (ACCOUNTABILITY) 3. The unit cost of all services is calcu￾lated; submitted to the MOE’s man￾agement; and made part of the justifi￾cation for the annual budget. (TRANSPARENCY & VALUE FOR MONEY) 4. Services delivered (# of units) and unit cost data are reported to the MOE’s senior management as well as to other authorities, e.g., President, Parliament, Ministry of Finance. Pro￾ject specific, quantitative performance data for the LTTP is captured and reported to USAID and other donors in the sector. (OPENNESS) 5. Services are delivered in the most cost-efficient and effective manner – as measured by available data - con￾sidering the financial condition of the Government and the needs of the population. 6. Service standards are transparent, i.e., administrators, teachers, know what they are required to do to obtain TT service and what they will receive in terms of quality, efficiency and timeliness. NOTE: Bonus payments (seating allowances, travel reimburse￾ment, salary enhancements will be LIBERIA TEACHER TRAINING PROGRAM PHASE II 82 Areas Assessed vis a vis Best Practice Exists Works Effectively Exists Needs Improvement Does Not Exist. Needed Does Not Exist. Not Needed reviewed from a political economy perspective.) (EQUITY) TOTALS 6 STRUCTURE 1. The MOE organizational structure (or the authority to create one or adapt to new conditions) is defined in the Education Act and is appropriate to the TT service delivery require￾ments. NOTE: Attention will be paid to issues of decentralization (whether by delegation or through de￾concentration). Other sections below will also be used for this assessment. 2. The MOE’s organizational structure – specifically for TT - is documented, including decentralized offices and RTTI’s, copies of the chart (“organo￾gram/org chart”) are posted at all TT service delivery locations and are also available by request to parents, do￾nors, community groups, etc. 3. The TT services to be delivered are clearly linked to the MOE’s organiza￾tional structure to facilitate responsi￾ble, accountable management and to establish the basis for sustainability of TT services and improved learning. SUSTAINABILITY LIBERIA TEACHER TRAINING PROGRAM PHASE II 83 Areas Assessed vis a vis Best Practice Exists Works Effectively Exists Needs Improvement Does Not Exist. Needed Does Not Exist. Not Needed 4. Vertical and horizontal reporting relationships for TT services within the established MOE hierarchy are clearly defined. NOTE: Internal re￾ports on LTTP progress and perfor￾mance will be reviewed as a measure of internal ownership and capacity for oversight. (PERFORMANCE MANAGEMENT and COMMUNICATION) TOTALS 2 2 SYSTEMS (GENERAL) 1. In general, all systems that affect the LTTP and the MOE’s TT programs are documented (in manuals), suscep￾tible to tests for transparency and are linked to performance management. (TRANSPARENCY) 2. Systems require the least number of steps for TT oriented transactions (EFFICIENCY) to create incentives for teacher enrolment and participa￾tion. 3. Whenever possible TT service de￾livery and performance tracking sys￾tems are computerized and operated by skilled employees. 4. ICT is used for two-way infor￾mation sharing and communications (horizontally and vertically) with a pol￾icy and strategy in place at the MOE to build and integrate this capacity within the TT service delivery system. Performance reports on LTTP are LIBERIA TEACHER TRAINING PROGRAM PHASE II 84 Areas Assessed vis a vis Best Practice Exists Works Effectively Exists Needs Improvement Does Not Exist. Needed Does Not Exist. Not Needed posted on a MOE web site. NOTE: a project website was initiated by the contractor, but is intended for transfer to the MOE and may be considered a sustainable investment. TOTALS 4 Integrity 1. Internal mechanisms exist to enable conscientious employees to inform management about corrupt behavior and actions of managers and peers. REPORTING 2. Annual consultations are held be￾tween the MOE and USAID (and other donors) to identify areas where joint efforts may reduce the incidence of fraud, waste, abuse, or vulnerability to petty corruption. TOTALS 1 1 Planning 1. Annual MOE plans, including for TT, are results-based, i.e., they are de￾signed to accomplish pre-defined ob￾jectives and are susceptible to periodic monitoring. Key milestones will be defined. The Five Year LTTP Action Plan is reviewed and jointly modified (MOE and LTTP contractor) not less frequently than annually. 2. Annual TT plans prepared by the MOE indicate who/what office will be responsible for achieving defined results under each MOE and LTTP LIBERIA TEACHER TRAINING PROGRAM PHASE II 85 Areas Assessed vis a vis Best Practice Exists Works Effectively Exists Needs Improvement Does Not Exist. Needed Does Not Exist. Not Needed component.(ACCOUNTABILITY) 3. Annual plans will include the MOE’s own resource commitments for TT as well as donor funding, i.e., LTTP will be fully integrated; planned results and resource commitments are fully integrated programmatically, but enable resource tracking for each funding source. (OWNERSHIP) 4. Each activity/task in the LTTP an￾nual plan has a named individual(s) who is responsible for its implementa￾tion and is accountable for the results. TOTALS 3 1 Budgeting 1. The annual education action plan and budget includes cost estimates for priority activities and funding com￾mitment by the MOE and all donors, i.e., activities are not contingent on uncommitted sources; sources and uses of funds are defined. 2. The MOE’s annual budget submis￾sion has an “Activity” called Teacher Training (TT) for which resources are requested by the Minister. This is aimed at promoting ownership and commitment and assuring that execu￾tive and legislative “support” is matched with appropriations and re￾leases of funds. (TRANSPARENCY). 3. The MOE’s annual budget submis￾sion, including TT, is reviewed with key external partners and customers LIBERIA TEACHER TRAINING PROGRAM PHASE II 86 Areas Assessed vis a vis Best Practice Exists Works Effectively Exists Needs Improvement Does Not Exist. Needed Does Not Exist. Not Needed concurrent with Parliamentary review, or sooner. 4. The approved MOE budget is pub￾lished/posted so that approved fund￾ing and service levels (adjusted by the MOE, if necessary) are communicated to all employees. The budget is public and can be downloaded from website 5. A copy of the total MOE “gross” budget, including all outside sources of funding, is provided to all partners, donors, and grantors. TOTALS 5 Communications 1. Central MOE management shares information about the policies, chal￾lenges, and successes of the MOE with other managers and staff through internal communication systems. To the extent possible, technology will be used to reduce costs of transportation. (OPENNESS) 2. Brochures, handouts and other in￾formation-sharing materials related to quality, access, teacher performance, parental opportunities for engage￾ment, etc. are prepared and distributed throughout the country. TOTALS 1 1 Accounting Controls 1. “Budget-to-actual” financial reports for funds controlled by the MOE, its key units/programs and for externally LIBERIA TEACHER TRAINING PROGRAM PHASE II 87 Areas Assessed vis a vis Best Practice Exists Works Effectively Exists Needs Improvement Does Not Exist. Needed Does Not Exist. Not Needed funded/internally accounted for pro￾jects are prepared monthly and sub￾mitted to top management. 2. All expenditures are consistent with and recorded against an approved budget activity. 3. Authority to expend funds and the actual approving documents are trans￾parent and subject to co-signature controls, i.e., no single individual can authorize and expenditure, sign a check or approve any commitment document, including the Minister as appropriate. 4. All revenues are documented, i.e., verified by receipts, posted daily, and reported monthly to the appropriate manager. 5. Irregularities in either the process or the amount of any individual transac￾tion are documented by the senior financial officer and communicated to the MOE Internal Auditor. 6. Accounting records, including all source documents, are maintained on￾site for a minimum of three years. Ap￾propriate space and facilities are made available for this task. NOT ASSESSED (see PFMRAF) TOTALS 2 3 Grants/Donor Coordination 1. Donor funds are subject to the same controls and procedures as other LIBERIA TEACHER TRAINING PROGRAM PHASE II 88 Areas Assessed vis a vis Best Practice Exists Works Effectively Exists Needs Improvement Does Not Exist. Needed Does Not Exist. Not Needed MOE funds. When donor funds are used to top-up salaries or pay seating or transportation allowances they are subject to specific policies and author￾ization by delegated authority. 2. A written policy requires that donor programs supplement and support the core mission and strategy of the MOE. (VALUE FOR MONEY) 3. Programs and activities funded by donors are implemented transparently and according to contract. 4. Travel, seating allowances, training opportunities, and other benefits are invested in the organization by allocat￾ing them to individuals who will return the most benefit to the MOE. Specific policies are adopted to assure that fairness, equity and transparency at￾tend to all such expenditures. 5. Financial records of donor-funded projects are posted/published as part of an overall, integrated report cover￾ing all funds spent from whatever source. (TRANSPARENCY) TOTALS 5 Management Reporting 1. The LTTP project manager (inside the MOE) is provided with monthly reports that track progress toward de￾fined performance goals or planned results. (OVERSIGHT) LIBERIA TEACHER TRAINING PROGRAM PHASE II 89 Areas Assessed vis a vis Best Practice Exists Works Effectively Exists Needs Improvement Does Not Exist. Needed Does Not Exist. Not Needed 2. Reports are quantitative, using graphs and charts with specific ex￾planatory text that will enable MOE officials to understand what is being achieved, what activities are on/ahead of schedule and what activities are either behind schedule or not con￾forming to the project contract. 3. The Minister and other senior man￾agers meet at least quarterly to review LTTP progress reports, specifically for TT and to propose corrective action, as required. 4. MOE quarterly progress reports, encompassing the LTTP project, are published/posted and provided to donor partners to facilitate coordina￾tion. 5. Annual statistical and progress re￾ports are published by the MOE. TOTALS 1 4 Procurement 1. All purchases of goods and services by the MOE are made through a competitive process, including bulk purchases (framework contracts) competitive shopping, and competi￾tive sealed bids, when appropriate. NOT ASSESSED (see PFMRAF) 2. A policy exists, and is rigorously implemented, concerning conflicts of interest, nepotism and cronyism. MOE has adopted policies regarding anti-corruption. (INTEGRITY) LIBERIA TEACHER TRAINING PROGRAM PHASE II 90 Areas Assessed vis a vis Best Practice Exists Works Effectively Exists Needs Improvement Does Not Exist. Needed Does Not Exist. Not Needed 3. The lowest, responsive bidder is always selected by the MOE to deliver the goods or provide the services. Evaluations of all bids are by commit￾tee and the selection is justified in writing. NOT ASSESSED (see PFMRAF) 4. Past performance records of all bidders and vendors is maintained by MOE and used for determining “re￾sponsible” bidders. NOT ASSESSED (see PFMRAF) 5. Vendors with poor past perfor￾mance with respect to fiduciary issues are blacklisted. NOT ASSESSED (see PFMRAF) TOTALS 1 Human Resources/Personnel Management 1. The MOE has a HR policy that is consistent with national law, but em￾phasizes open, advertised, competi￾tive, and merit based approaches to appointment and promotion. Teachers are actively recruited under a revolving solicitation to increase the number of teachers available for induction/pre￾service training. 2. The MOE only promotes on the basis of merit based on the actual and potential contribution of the appoin￾tee to achieving MOE goals for TT. 3. Appointments to supervisor posi￾tion or above are considered, reviewed and approved by a committee estab￾lished for that purpose and are audited annually as part of the standard, inter￾nal audit to assure compliance with LIBERIA TEACHER TRAINING PROGRAM PHASE II 91 Areas Assessed vis a vis Best Practice Exists Works Effectively Exists Needs Improvement Does Not Exist. Needed Does Not Exist. Not Needed appointments (by direct entry or in￾ternal promotion) for all such posi￾tions. 4. Performance messages, rewards, commendations, “celebrations” and similar recognition programs exist and are supported by the MOE Minister and senior management. NOTE: Recognition within the TT project facilitates sustainability. 4 Auditing 1. The MOE’s Internal Auditor regu￾larly conducts internal audits of TT projects, however funded, to identify financial irregularities and manage￾ment problems. 2. Audit findings are reviewed by top management and then pub￾lished/posted conspicuously to inform all staff of systems deficiencies and management’s commitment to address them. NOT ASSESSED (see PFMRAF) 3. Internal audits conform to generally accepted international auditing stand￾ards, or equivalent. NOT ASSESSED (see PFMRAF) TOTALS 1 STAFF 1. MOE job descriptions, containing clearly defined duties and reporting relationships, exist for all TT related positions. Performance standards are attached to the job description in or- LIBERIA TEACHER TRAINING PROGRAM PHASE II 92 Areas Assessed vis a vis Best Practice Exists Works Effectively Exists Needs Improvement Does Not Exist. Needed Does Not Exist. Not Needed der to foster a “results” orientation. 2. Management decisions on person￾nel are documented and published in a gazette or similar. Staff movements are posted on the web, e.g., hiring, retirements, maternity leave, resigna￾tions, promotions, etc., if available or in a publication widely distributed and available to all staff. OPENNESS 3. Staffing levels in the TT project are formally and uniformly assessed and regularly and adjusted according to demand, service requirements and guidance by LTTP specialists assisting with capacity building efforts. 4. MOE staff assigned to an organiza￾tional unit report to and are accounta￾ble to the head of that unit. (ACCOUNTABILITY / NON￾INTERFERENCE) 5. Working time is reported accurately and openly (either a time clock or timesheets verified by management) NOT ASSESSED (see PFMRAF) 6. Performance for the TT program in MOE is appraised formally, openly, and accurately according to a uniform system approved by top management and within the oversight of an em￾ployee committee. 7. Champions are identified and re￾warded for their commitment and per￾formance. (SUSTAINABILITY) TOTALS 1 5 LIBERIA TEACHER TRAINING PROGRAM PHASE II 93 ANNEX 5: ILLUSTRATIVE INTERVIEW GUIDES Illustrative Questionnaire Guide COP Implementing Partners Objective: to obtain a candid self-appraisal of the success and failures of the program to date, to understand the constraints and problems the IP faces and to help map out a strategy that will lead to success in the sec￾ond half of the program. 1) How long have you served as COP of the program (or director of the sub-component)? 2) What are the constraints and challenges you have faced in implementing this program? 3) What modifications have you had to make to overcome the challenges? 4) In what areas are you on-track to meet your objectives and results? 5) In what areas are you NOT on-track to meet your objectives and results? 6) What areas need to be strengthened to help you meet your objectives and results? 7) What steps are you taking to ensure sustainability of the program’s interventions? 8) How effectively have the LTTP components engaged the MOE at the national and local level? Has the MOE assumed ownership of the LTTP components? 9) How has the project integrated gender strategies into its activities? 10) What are the programmatic or organizational gaps that hinder the success of the program and how can these be addressed in the second half of the program? 11) What are the lessons learned from the implementation of LTTP so far? LIBERIA TEACHER TRAINING PROGRAM PHASE II 94 12) Are stakeholders satisfied with the program and what areas do they think need improvement? 13) How would you change the implementation of the program, given what you have seen so far? 14) How can this assessment help you to achieve success in the program? LIBERIA TEACHER TRAINING PROGRAM PHASE II 95 Illustrative Questionnaire Guide Reading and Math Specialists: Implementing Partners Objective: To determine what are the challenges, constraints and progress in the reading and math compo￾nents of the LTTP and to determine what can be done to overcome any problems in the future. 1. What do you see as the main objectives of the reading (math) component? 2. What have been the main strategies used to achieve these objectives? 3. What are the constraints and challenges you have faced in implementing this component? 4. What modifications have you had to make to try to overcome the challenges? 5. In what areas are you on-track to meet your objectives and results? 6. In what areas are you NOT on-track to meet your objectives and results? 7. What areas need to be strengthened to help you meet your objectives and results? 8. What steps are you taking to ensure sustainability of the component’s interventions? 9. To what extent is there ownership in the MOE for improving early grade reading (or math) instruc￾tion? 10. How receptive has the MOE been to the LTTP intervention design in reading and math? 11. How could USAID assistance most effectively contribute to improved reading (math) skills in the fu￾ture? 12. How has the program integrated gender strategies into its activities? LIBERIA TEACHER TRAINING PROGRAM PHASE II 96 13. What are the programmatic or organizational gaps that hinder the success of the program and how can these be addressed in the second half of the project? 14. What are the lessons learned from the implementation of the LTTP reading (math) program so far? LIBERIA TEACHER TRAINING PROGRAM PHASE II 97 Illustrative Questionnaire Guide Minister of Education Objective: to determine the extent of MOE ownership and commitment to the LTTP. 1. How long have you been Minister of Education in Liberia? 2. What are the main challenges you face? 3. Have you seen the LTTP in operation? 4. What are the strengths and weaknesses of the LTTP and how can the program better serve the MOE and its challenges? 5. How effectively have the LTTP components engaged the MOE at the national and local levels? 6. What are the programmatic or organizational gaps that hinder the success of the program and how can these be addressed in the second half of the project? 7. What are the lessons learned from the implementation of the LTTP so far? 8. Are MOE staff satisfied with the training, and support they have received from the LTTP? 9. What is the MOE view on the appropriateness, relevance, and impact of the LTTP in general and of the early grade reading program? 10. What steps are you taking to ensure sustainability of the project interventions? 11. What are the main lessons learned from you education investments in Liberia? LIBERIA TEACHER TRAINING PROGRAM PHASE II 98 Illustrative Questionnaire Guide (did you interview NGOs?) NGOs Objective: To learn about NGOs experience working in the education sector and with the MOE, especially in the areas where the LTTP works. 1. What are the objectives of the NGO, especially in education? 2. How long has the NGO been working in Liberia? 3. What are the main components of your education program in Liberia? 4. What are the main challenges you face in the implementation of your program? 5. What modifications have you had to make to try to overcome the challenges? 6. In what areas are you on-track to meet your objectives and results? 7. In what areas are you NOT on-track to meet your objectives and results? 8. How effectively have the components of your program engaged the MOE at the national and local levels? Has the MOE assumed ownership of the components? LIBERIA TEACHER TRAINING PROGRAM PHASE II 99 Illustrative Questionnaire Guide Donors Objective: To determine donor perspectives on the MOE, the LTTP and its components and to determine future donor plans. 1) What are the current objectives of the donor’s portfolio in education? 2) What are the main areas and components of the donor’s portfolio? 3) What are the main challenges in working with the MOE and in education in Liberia? 4) What modifications have you had to make to try to overcome the challenges? 5) In what areas are you on-track to meet your objectives and results? 6) In what areas are you NOT on-track to meet your objectives and results? 7) How much do you know about USAID’s LTTP? Have you had any direct contact with the program and, if yes, what are your impressions? 8) How do you rate the prospects for sustainability of your projects in Liberia? 9) What are the likely impacts of donor investments in education in Liberia? 10) How effective have your programs gender strategies been and what are your best practices? LIBERIA TEACHER TRAINING PROGRAM PHASE II 100 Illustrative Questionnaire Guide County Education Boards/ School Management Boards Objective: To gain a civil society perspective on the LTTP, the workings of the MOE, and the effectiveness of education service delivery. 1) What is the role of the education board (school management committee)? Does this include the PTAs? 2) What are the major challenges the education board (school management committee) is facing? Is the education board the County Education Board? 3) What involvement have you had with the LTTP? 4) Which components of the LTTP have you had some experience with? 5) What has the impact of the training teachers in the school had on the learning performance of the children? 6) What impact can be seen on the reading or math performance of the children? 7) What areas does the county (district or school) need to assist it to improve education? 8) Looking to the future, what areas of assistance do you think are needed to improve education in Li￾beria, especially in reading and math achievement? LIBERIA TEACHER TRAINING PROGRAM PHASE II 101 Illustrative Questionnaire Guide Teacher Trainer: RTTI Objective: To determine if the teacher training component in the RTTIs is on track to meet its objectives and gain insight about how to improve implementation in the second half of the program or in future teacher training programs. 1) What is your position in the RTTI and what role have you played in the LTTP? 2) What were the objectives of the LTTP when it started? 3) What kinds of activities has the project brought to the RTTI? 4) How well has the project achieved its objectives at the half way point? 5) What changes would you recommend to improve the project’s effectiveness? 6) How effective was the LTTP design to the challenges of the Liberian education context? Are there any major obstacles that must be overcome and, if yes, how would you address those obstacles? 7) How effective is the LTTP in delivering and addressing a professional development program for Li￾berian educators? 8) What activities could strengthen the program and why? 9) What are the best practices and successes (failures) that could be replicated (avoided)? 10) What are the constraints, challenges and lessons learned from the LTTP implementation so far? 11) How sustainable are the different components? Are there areas that should be phased out? How cost-effective is the approach? LIBERIA TEACHER TRAINING PROGRAM PHASE II 102 12) How can USAID best assist the Liberian education system to improve reading instruction and deliv￾ery systems? 13) How effective has been the LTTP’s approach to increasing women’s access and participation to the teaching field? What more needs to be done? 14) How effective is the teacher training programs at the pre-service and in-service levels? What needs to be done to improve the articulation between the two levels? 15) How effective is the LTTP approach to school-based teacher training in early grade reading and math? Please explain? 16) What are the early outcomes of the LTTP interventions with teachers on teacher knowledge, skills, attitudes or student performance? 17) Are the RTTIs and/or MOE staff satisfied with the quality of the training, activities and support they received? LIBERIA TEACHER TRAINING PROGRAM PHASE II 103 ANNEX 6: IMPLEMENTERS QUESTIONNAIRE GUIDE IMPLEMENTING PARTNERS: Director of Teacher Training Objective: To determine the quality and relevance of training provided to improve equity, efficiency and ac￾cess by increasing teacher effectiveness through a well-planned, implemented and evaluated pre and in-service programs. 1. What are the goals, scope, outcomes of the pre-service and in- service training programs: number and types of participants, range of activities? This includes new skills and competencies acquired. 2. How were training needs determined? 3. What were the objectives and what skills and competences were gained and applied by the partici￾pants? 4. What feedback has been provided by the participants and Ministry? How were they involved in the design and all phase of the program? 5. What follow-up was provided by whom? 6. Are there any indicators of improved teaching performance? 7. How has the MOE capacity to plan, implement, document and evaluate in-service and pre-service teacher education been strengthened? 8. Is there a comprehensive teacher training plan? 9. How have the trainers been selected, trained, monitored and evaluated? 10. What are the training priorities for the next phase of the project? LIBERIA TEACHER TRAINING PROGRAM PHASE II 104 11. How have the coaches/Mentors been trained and deployed. What are the indicators of their effec￾tiveness? 12. What is the quality of the instructional materials and how were they developed and disseminated? 13. What incentives were provided to participate in the professional development activities? 14. What are the projects greatest accomplishments? 15. What are the challenges for the next phases? What refinements are suggested? 16. What has been done to address awareness of gender disparities and imbalances? 17. What is the per participant cost of various forms of training and cant his be sustained by the MOE at the completion of the project? 18. Are there funds for the development of training materials and the capacity to use the new IEMS sys￾tem to document all the participants? 19. Is there a fully functioning professional Development Center at the MOE and what training will be provided to the new director? 20. What deficiencies are there in the overall training program? 21. How was donor coordination facilitated to assure the efforts were not duplicated and high training standard were provided to all participants? LIBERIA TEACHER TRAINING PROGRAM PHASE II 105 MOE SENIOR OFFICIALS—INITIAL MEETING MEETING/INTERVIEW GUIDE Meeting Setup Self-introduce team members. State purpose of the evaluation: This assessment is meant to serve a dual purpose: (1) to learn to what extent the project’s objectives and goals are on track to be achieved by project close out in 2015; and (2) to inform the design of future USAID basic education investments. Give brief overview of multiple data sources, field visits, central-level interviews, and document review. There will be a need for follow-up meetings with individual MOE deputies and department managers, but you want to use this meeting to get the MOE’s perspective on basic education provision in Liberia and to orient the Ministry about this evaluation. Agree on an amount of time for the meeting and the team’s need to speak with individual officials afterward to set up follow-up meetings. List name, title of MOE official interviewed. Meeting Questions 1. Would His Excellency (Minister of Education) provide us with his perspective on both the LTTP and on the larger concerns of the Ministry in providing basic education to the nation’s children? 2. To what extent is the Ministry of Education involved with the LTTP? 3. What is your understanding as to what the program would do in the area of basic education? 4. What indications do you have that the program is engaged in those activities to the expected extent? 5. Does the program’s planned activities seem appropriate to the needs of Liberia’s education system? Why or why not? 6. To what extent have the project staff collaborated with Ministry staff and kept you informed of their activi￾ties and concerns? 7. What kind of concerns or obstacles, if any, has the LTTP experienced in trying to meet their own goals and objectives? To what extent have these concerns affected the MOE? 8. Does the Ministry have any suggestions on how to address these concerns and obstacles? 9. Moving on to the larger context of education, what limitations does the Ministry see in providing adequate education to all Liberian children? 10. What, in particular, do you think this team should look at during our site visits to a sampling of RTTIs, schools, and county and district education offices in Liberia? LIBERIA TEACHER TRAINING PROGRAM PHASE II 106 PRIMARY SCHOOL PRINCIPALS FOCUS GROUP INTERVIEW GUIDE Purpose: Determine if the principals are aware of the LTTP’s objectives and activities and outline how gradu￾ates from the program are providing quality instruction to the children. Interview Questions 1. Are you aware of goals and activities of LTTP and which are relevant to your school? 2. Is there a difference in the quality of teaching between a LTTP graduate working at your school and other teacher? What are they? 3. Have you participated in the project and have you provided input in the design of the program? 4. What are you teachers’ major professional development needs? 5. What are the strengths and weakness of the program? 6. What feedback have your received from the teachers who completed this programs in terms of what they are able to apply? 7. What training is needed for principals so that can support effective instruction? 8. What are your greatest challenges as a principal and how has the LTTP helped? 9. What follow-up support do your teachers need? 10. Has the teaching coach been useful in helping teachers apply what they have learned? 11. What school improvement strategy would you like to plan, if you had additional resources? LIBERIA TEACHER TRAINING PROGRAM PHASE II 107 12. Have you integrated a quality reading and mathematics curriculum at your school? 13. How are you dealing with gender disparities in you school? What are they? 14. How is the community supporting your school and instructional program? LIBERIA TEACHER TRAINING PROGRAM PHASE II 108 INTERVIEW QUESTIONS LTTP Managers and Staff 1. In looking over the last two years of operation, what has gone particularly well with the LTTP? 2. Your initial assessment of the primary teacher training system uncovered a number of serious con￾straints that the LTTP would face in starting operations. Can you describe how these constraints, ranging from a huge number of untrained teachers to non-functioning teacher training institutions, affected your planning and implementation of project activities? 3. What has been the nature of the LTTP’s relationship with the Liberia Ministry of Education in un￾dertaking LTTP activities? Has the MOE been an active and effective partner in working toward the project’s goals? 4. The LTTP chose to simultaneously launch both a resurrected pre-service training program and a ge￾ographically limited in-service training system. What has been your experience in using both these modes to tackle Liberia’s large teacher training needs? 5. Your goals also included helping to improve the capacity of the MOE to manage an adequate mod￾ern teacher training system? What has been your experience in achieving this goal? 6. What kind and severity of constraints have you faced in trying to help Liberia overcome the effects of its past conflict on education? 7. After two years, how far has the LTTP been able to go in improving the quality of teacher training in Liberia? 8. What is the nature of your activities with regard to strengthening the College of Education at the University of Liberia? 9. Do you have any indication that the early trainee graduates of the revised training programs are ap￾plying these more modern, participatory teaching skills in actual classrooms? 10. What has been your experience to date that indicates about the direction technical assistance for basic education should take in Liberia? LIBERIA TEACHER TRAINING PROGRAM PHASE II 109 RTTI Principals’ and Lecturers’ Questionnaire Purpose: Document the quality of preserve teacher training and the RTTI’s capacity to deliver an effective program. 1. How was the teacher training curriculum been revised to more appropriately reflect the education needs of Liberia and the basis for developing teacher training curriculum? 2. How has early grade reading and math been incorporated in the primary curriculum and offered in all teacher training institutions as a full course? 3. Are RTTIs fully-functional self-governing, as granted by the MoE, with the teaching staff trained in curriculum design, subject pedagogy, recruitment strategies and student assessment and the adminis￾trative staff trained in planning management? 4. Have RTTIs been strengthened with computer labs and technology to help them manage property, finance and personnel, including educational technology training to enhance lessons through the in￾ternet facilities? 5. Have RTTI’s attached demonstration schools been strengthened to serve as model schools and clini￾cal avenue along with catchment schools for applying pedagogical skills? 6. Have RTTI’s teachers acquired basic computer skills to enable them to produce teaching and learn￾ing materials? To use PowerPoint presentations to support video episodes of teaching practice expe￾rience, some of which are taken at the demonstration schools and others taken at the catchment schools? 7. How is your college a better resourced and more fully functional training institution given LTTP in￾puts? 8. What are the training needs of lectures and management staff? 9. What are the greatest challenges in delivering an outstanding training program? LIBERIA TEACHER TRAINING PROGRAM PHASE II 110 10. Do you have suggestions for redesigning the LTTP? LIBERIA TEACHER TRAINING PROGRAM PHASE II 111 RTTI INSTRUCTORS FOCUS GROUP INTERVIEW GUIDE Interview Preparation Collaborate with RTTI principal to select a group of about 10-12 representative instructors to interview in a group. As discussed in the research methodology, the sequencing of questions begins with positive aspects and accomplishments that respondents are proud of and moves later to probe aspects that are problematic. Introductory Statement. Inform respondents of the purpose of study: “We are looking at the role of the RTTI in training teachers, the work of the LTTP in assisting the RTTI in its work, and your own personal roles both as instructors and with the LTTP staff. I will ask you some questions and will take notes. I need for you to feel free to tell me honestly what you think, and I promise to be careful with your opinions because they are very important. I will not quote you by name and will integrate what you tell me with the statements of many other RTTI instructors around the country.” Write down names, sex, subject(s) taught, years in teaching, and previous training and education, including certificates and diplomas. Note length of time working with LTTP staff. Interview Questions: 1 What are your greatest accomplishments and what have you enjoyed most about teaching? 2. Are you familiar with the primary school curriculum? 3. Tell us a little bit about what your goals and objectives are in instructing the trainees? 4. What have you discovered about the students who have enrolled as teacher trainees? Are they ready for learning the content that you are teaching them? Why or why not? Are they motivated to become good teachers? 5. (If applicable) What are you doing to make up for the trainees’ lack of adequate education and training be￾fore entering the RTTI? 6. What is your opinion of the LTTP-facilitated training of trainers that you were given? Did it prepare you well for the actual tasks of training young pre-service trainees? 7. Are you able to apply the practices of active-learner, child-centered pedagogy in your classes at the RTTI? If yes, how do you do this? If no, why are you not able to do this? 8. How much support do you and your RTTI colleagues receive from the LTTP in being able to successfully train pre-service teachers? 9. What other kinds of constraints, if any, impinge negatively on the environment at your RTTI? How is this being addressed? LIBERIA TEACHER TRAINING PROGRAM PHASE II 112 Interview Guide for Counties Target: County Education Officer AND/OR Chairperson of County Ed Board Purpose: To obtain information about the relationship between the MOE and the principal education service delivery management unit. The Education Reform Act has promoted decentralization to the county level for the major responsibilities for service delivery. The delegation of authority and responsibilities are not included in the legislation. The objective of the questions is to see how policy, systems and institutional capacity are being developed under the new structure. Specifically the interview will address these evaluation questions: 1. How effectively is the LTTP addressing underlying institutional, policy and systems weaknesses that impact the capacity to deliver a professional development program for Liberia’s educators? 2. What are the likely and sustainable impacts of the EMIS on the MOE’s planning and manage￾ment, as well as on issues of MOE accountability and ownership of the education process and outcomes? 3. To what degree is there MOE ownership, leadership and accountability for teacher training and early grade reading? 4. How effectively have the different LTTP components (pre-service, in-service, EGRA/EGMA, Higher Education, MOE policy and systems) engaged the MOE at the county level? 5. What are the CEO’s views on the LTTP’s activities in the areas of policy and institutional strengthening in areas such as:  EMIS  biometrics identity cards  payroll reform Approach – The team will meet with the County Education Officer for a face to face interview. Since R-1 is aimed at enabling the MOE to effectively support decentralized operations at the county level, this will be the basis for the interview. This questionnaire is focused on the MOE –COE administrative relationship. 1. Please discuss the services provided by each of the three main Departments/Bureaus of the MOE LIBERIA TEACHER TRAINING PROGRAM PHASE II 113 MOE Department Services Provided Administration Instruction Planning research and development 2. The LTTP is strengthening the MOE to provide services to the counties AND is providing ser￾vices directly to counties in a number of key areas of planning and administration LTTP Services Aware Importance 1 =(tops); 2= (average) 3= (not so much) Impact 1= effective 2= partial 3 = ineffective Education Management Information System and capacity building to operate/comply Training in strategic planning, management, ca￾pacity building etc. Training and Systems Development for Instruc￾tional Leadership LIBERIA TEACHER TRAINING PROGRAM PHASE II 114 Improved Communication (from MOE to CEO) of policies, priorities, changes in procedures, new developments Training and Systems for Education Quality Mon￾itoring and Instructional Supervision Training for School Boards (Other Services) Early Grade Reading and Math Programs Teacher Training Payroll Reform/Biometric Cards 3. What are the key obstacles that you face? Has the LTTP helped the county respond to the chal￾lenges in the education system in Liberia? What more is needed? 4. Budgeting and funding for educational service delivery is an essential characteristic of both na￾tional commitment to educational reform and the MOE’s commitment to support decentraliza￾tion. Has the level of funding from the MOE increased, decreased or remained about the same for this county? 5. Implementing an effective and sustainable teacher training and professional development pro￾gram is a key priority for the MOE. What are the institutional or policy constraints that need to be addressed so that this goal can be achieved? 6. The MOE, with support from the LTTP, is introducing biometric cards for teacher identifica￾tion and ease of administration. Has this service been successfully introduced in the county? What is its impact to date? What more needs to be done? 7. Coordination between the county and the districts (DEO) is important. What are the difficulties that you encounter in this relationship? What are the administrative challenges in this relation￾ship:  Planning  Budgeting  Information Flow/Reporting LIBERIA TEACHER TRAINING PROGRAM PHASE II 115  Human Resources (Assignments, Appointments, Transfers, Attendance)  Supervision 8. What have been the most significant accomplishments over the past two years that can be at￾tributed to the LTTP? LIBERIA TEACHER TRAINING PROGRAM PHASE II 116 Interview Guide Target: Ministry of Education  Deputy Minister for Administration  Assistant Minister/Bureau of Fiscal Affairs & Human Resources Development  Assistant Minister/Bureau of General Administration Purpose: To meet with key administrators for the purpose of collecting information aimed at addressing the following evaluation questions: 1. How effective were both the LTTP design and its implementation in responding to challenges in the Liberian education context, especially in light of the adoption of the “Education Reform Act” of 2011? 2. How effectively is the LTTP addressing underlying institutional, policy and systems weaknesses that impact capacity to deliver a professional development program for Liberia’s educators? 3. What are the likely and sustainable impacts of the EMIS on the MOE’s planning and manage￾ment, as well as on issues of MOE accountability and ownership of education process and out￾comes? 4. What are the constraints, challenges and lessons learned from the program’s implementation thus far? 5. How receptive has the MOE been to the LTTP II’s intervention design, objectives and (planned) outcomes? 6. What is the effectiveness of the LTTP’s approach to increasing women’s access to, and participa￾tion in MOE more generally? 7. What implementation successes and challenges have been documented in implementing the LTTP components? 8. What are the preliminary outcomes of the policy efforts? What are the preliminary outcomes of the decentralization efforts? LIBERIA TEACHER TRAINING PROGRAM PHASE II 117 Approach: The evaluation team will meet with senior MOE officials and administer an open-ended ques￾tionnaire about changes in policy, management or financial support for professional development. Key in￾quiries about decentralization and the means and methods of funding decentralized activities at the county level will also be made. (Relationships between counties and districts will be assessed through questionnaires at that level). In addition a future meeting will be held to either collect baseline financial data on a specific form that will attempt to look at the education budgets over the past six years as a way of determining com￾mitment and assessing any contribution by LTTP. Interview Questions 1. Are you aware of USAID’s teacher training project - the LTTP II project - that has been assisting the Ministry in the areas of policy and management, institutional strengthening, early reading, teacher training and performance management through a computerized Education Management Information System (EMIS)? 2. What have been the most important and valuable contributions of the program in your Department? 3. Have there been some activities that have not yet met your expectations? How would you suggest that these shortcomings be addressed? 4. The LTTP provides assistance to each of the Departments in the new MOE structure. What mecha￾nisms internal to the MOE are in place to coordinate, promote synergies, and avoid duplication of these separate components? 5. The Education Reform Act (Section 3.2.6 (b) and (c) establishes the mandate of the Bureau of Fiscal Affairs and Human Resources Development t… “train staff” …and … “develop and strengthen the human structural and other capacity of the Ministry”. How is the Department/Bureau fulfilling this mandate as it relates to strengthening counties? Is there an annual grant for each county? Is it based on a formula? 6. Since the Department of Administration is generally responsible for staffing issues, what specifically is being done regarding policies for gender equity in administrative and management positions at the MOE? What tools (policies, guidelines, directives, sensitivity training) does the MOE use to assure that the counties are also aware of and following gender equity mandates of the Education Reform Act (Section 1.5 (g))? 7. Decentralization of the education function may be more advanced than decentralization policies gen￾erally? How will this affect the funding and management of teacher training, capacity building, etc.? LIBERIA TEACHER TRAINING PROGRAM PHASE II 118 Interview Guide Target: FHI/360 Institutional Development Team  Institutional Capacity Development Advisor  EMIS Advisor  M&E Advisor  Policy and Systems Strengthening Advisor Purpose: To identify the pre-existing policy and management conditions/issues to which the LTTP project was directed (R-1); to assess progress and identify constraints; to identify work to be completed by 2105; and, to identify priorities for future interventions. Approach: The evaluation team will use an interview/questionnaire to capture information necessary to draw conclusions that will benefit the contractor, the Government and USAID. The structured portion of the in￾terview is closely tied to the SOW questions. Additional follow-up meetings and data collection may be re￾quired to complete the process. 1. What were the specific policies, leadership management, human resources, and systems is￾sues/constraints related to the delivery of a professional development program that existed at the inception of the LTTP II. Please provide specific examples at the central MOE level and at the County level. Areas Specific Weaknesses Central MOE % Specific Weaknesses Counties % Policy Making Capacity Instructional Leadership Management Systems o Strategic LIBERIA TEACHER TRAINING PROGRAM PHASE II 119 Planning o Budgeting o Monitoring o Perfor￾mance Manage￾ment o Monitoring o Reporting 2. How effectively has the program been able to address these issues to date? Please assign a per￾cent complete to the five major factors, i.e., policy, leadership, management, systems, human resources. 3. How was the project structured to address these issues and to address the three primary result areas? What are the strengths of the LTTP organization structure? Has it been effective? After some experience on the ground what would you change, if anything? 4. Concurrently building the centralized capacity of the MOE to plan and manage a decentralized program, while building the capacity of counties to plan, manage AND deliver service seems challenging in a post-conflict environment, both from a capacity/readiness perspective and in terms of competition for resources. How has the program managed to sequence these tasks? Is building MOE capacity a “condition precedent” to other project delivery systems or is it a par￾allel activity equal in importance to other interventions? 5. Two specific examples may be instructive.  The project provided strategic planning training. How were they similar and how were they different between the content provided to central MOE administrators and the content provided to county/district administrators? o Why? o What outputs from either of the trained personnel would best reflect their under￾standing of the principles of strategic planning? LIBERIA TEACHER TRAINING PROGRAM PHASE II 120  The project undertook a very significant initiative to introduce an EMIS. (First of all: de￾scribe and provide data concerning the physical investments and the costs of each component) The EMIS seems to be intended to be one of most important means initiated by the program to promote vertical linkages and common ownership between the levels of the educational hierarchy (MOE. CEO, DEO). o What is the current status of that initiative? o Training in both the purpose of an EMIS and its actual operation was provided. Has the latter been absorbed and where will the team “see” a successful, operating EMIS? o EMIS is being implemented nation-wide? o What are the challenges associated with that decision? o R1.3 seems to be the foundation of the R.1 results framework, i.e., county monitor￾ing of performance; ability to communicate results to decision-makers, donors, parents, etc.; and most critically vis a vis project goals “equitable access to educa￾tion”) Can R.1 be achieved if R 1.3 is not operating at full capacity by 2015? 6. The transition to increased ownership by MOE is commendable and is a key to longer term ownership and sustainability. The most recent quarterly report indicates that three Deputy Min￾isters have been removed from their senior management positions. Is this a tradeoff between “sovereignty” and “sustainability”? How can the project address these issues? 7. Since the counties are the target for much of the capacity building:  Is it possible to concentrate on a sample of the counties for the purpose of introducing a national standard?  What are the contractor’s observations about this decision?  Will replication occur?  Can this be done without external support?  Are there other resources (donor assistance) that the non-participating counties can use to build the same sort of teacher training models?  Are national MOE funds allocated in a way that seeks perverse equity, i.e., non-LTTP counties get MORE funding because it is assumed that USAID/LTTP funds are being used to support the others?  What are the most successful county level capacity building initiatives, e.g., training, technical assistance, informing sharing, partnering, observation and how are those manifest? LIBERIA TEACHER TRAINING PROGRAM PHASE II 121  What have been the most challenging areas?  Is it essential to achieving the project’s results and education goals to continue with these efforts?  If so what is being contemplated as an approach that will improve prospects for suc￾cess? 8. Decentralization in education management may be dependent on the greater decentralization policies of the country.  What forces and factors in fiscal decentralization will impact education management and service delivery?  Has the LTTP project factored this into the design of project activities going for￾ward?  Significant personnel movement seems to be a characteristic of Liberia’s decentrali￾zation efforts. How has the program coped with this? 9. Coordination among donors working in the same sector is often lacking.  Is that the case with initiatives in teacher training?  Early reading+math?  School-based management? Others? LIBERIA TEACHER TRAINING PROGRAM PHASE II 122 ANNEX 7: OBSERVATIONAL TOOLS Checklists and Observation Instruments The team expects systematically to observe teachers and instructors who have participated in the LTTP-II pre-service and in-service teacher training program, as well as TOT training, to determine if the participants are able to apply the new skills and approaches. A checklist is also provided to examine the relevancy and quality of the curriculum. Three instruments are attached which focus on teacher and trainer effectiveness as well interactive teaching. 1. Pre-Service Teacher Training Curriculum Checklist RTTI’s TEACHER TRAINIG COURSE MAPPING QUESTIONS and INSTRUCTIONAL MATERIALS Purpose: To document the quality of the LTPP teacher training curriculum design and to determine the ef￾fect on improved classroom instruction. This includes suggestions for refinements in the program. LTPP Manager and other Staff AIM/PURPOSE 1. Is the Aim/Purpose statement clearly stated and aligned to teacher standards and the primary school curriculum especially in math and reading? 2. Do the Aim/Purpose of each course link logically and developmentally to each other? 3. How does the Aim/Purpose of each course contribute to the development of future teachers for Li￾berian primary schools? 4. What additional learning experience are now required? COURSE OBJECTIVES 5. How do course objectives achieve the Aim/Purpose of the course? 6. Do each of the course objectives related to quality instruction? 7. Is each course objective relevant to producing future teachers for Liberian primary schools? LIBERIA TEACHER TRAINING PROGRAM PHASE II 123 8. How will each course objective help graduates to implement the primary curriculum framework? 9. Do the objectives of each course link logically and developmentally to each other? 10. Are there critical objectives/outcomes missing? If so, what are they? 11. Are any of the listed objectives now redundant? If so, which ones? Why? 12. How will the program of lead lectures/ tutorials/ independent study/ practicum and assessment go together to help students achieve each of these objectives? ASSESSMENT 13. Which planned assessment tools assess each of the above course objectives? 14. Are the planned assessment tools valid and reliable indicators of quality learning outcomes? 15. What weighting is given to each course objective – is this weighting reflected in the assessment tools used? 16. Are assessment tasks linked to learning activities? How is this done? 17. Are students taught all of the knowledge, skills, understandings, etc. that they need to successfully complete each assessment task? Where and when does this take place in each course? 18. Are there detailed rubrics for each assessment tool? 19. What standards are used to compare learning outcomes – standards/ benchmarks, etc.? 20. Are model answers provided as standards against which assessment outcomes are measured? STUDENT TEACHING 21. Where in each course are the student teaching competencies, linked to that course modeled, prac￾ticed and applied? LIBERIA TEACHER TRAINING PROGRAM PHASE II 124 Trainer Workshop Observation Instrument Purpose: This can help document the effectiveness of the in-service trainers and provide feedback on addi￾tional training needs for the trainers. 1. The trainer’s explanation of workshop objectives 2. The trainer’s explanation of the objectives for each workshop session and learning act 3. The trainer’s ability to arise interest when introducing and instructional activity 4. The trainer’s explanation of the work expected from each participant 5. The trainer’s ability to maintain a clear relationship between the workshop content and the workshop objectives 6. The trainer’s skill in clarifying the relationships among the various topics treated in the workshop 7. The trainer’s skill in making clear the distinction between major and minor topics 8. The trainer’s skill in adjusting the rates at which new ideas are covered so that the material can be followed and understood 9. The trainer’s ability to clarify material which needs elaboration 10. The trainer’s speaking skills 11. The trainer’s ability to ask easily understood questions 12. The trainer’s ability to ask thought provoking questions 13. The trainer’s ability to answer questions clearly and concisely 14. The trainer’s overall effectiveness as a discussion leader 15. The trainer’s ability to get students to participate in class discussions 16. The trainer’s skill in facilitating discussions among students as opposed to discussions only between the teacher and students 17. The trainer’s ability to wrap things up before moving on to a new topic 18. The trainer’s ability to tie things together at the end of a class 19. The trainer’s explanation of precisely how performance is to be evaluated. 20. The trainer’s ability to design evaluation procedures which are consistent with workshop objectives 21. The trainer’s performance in periodically informing participants on their progress 22. The trainer’s selection of materials and activities which are thought-provoking 23. The trainer’s ability to select materials and activities which are not too difficult 24. The trainer’s provision of a variety of materials and activities LIBERIA TEACHER TRAINING PROGRAM PHASE II 125 25. The trainer’s ability to use a variety of teaching techniques 26. The trainer’s demonstration of creativity in teaching methods 27. The trainer’s management of day-to-day administrative details 28. The trainer’s flexibility in offering options for individual students 29. The trainer’s ability to take appropriate action when students appear to be bored 30. The trainer’s availability to for personal consultation 31. The trainer’s ability to relate to people in ways which promote mutual respect 32. The trainer’s maintenance of an atmosphere which actively encourages learning 33. The trainer’s ability to inspire excitement or interest in the content of the workshop 34. The trainer’s ability to relate the subject matter to other academic disciplines and real school situa￾tion 35. The trainer’s willingness to explore a variety of points of view 36. The trainer’s ability to get participants to challenge points of view raised in the workshop 37. The trainer’s performance in helping participants to explore the relationship between personal values and school improvement. 38. The trainer’s performance in making participants aware of the value issues within the school im￾provement process. LIBERIA TEACHER TRAINING PROGRAM PHASE II 126 3. Teacher Effectiveness Classroom Observation Instrument Purpose: To document how the LTPP’s training skills and competencies are applied by teachers in the classroom and suggest refinements in the pro￾gram. Indicators Evaluation Measures Inefficient (1) Satisfactory (2) Efficient (3) Evaluation 1 Planning 1.1 Identifying Teacher Effectiveness Observation Tool: LTTP Mid-Term Evaluation Children’s Needs and Interests 1.1.1 Using various tools and sources to identify the children’s educational needs and interests. Planning does not reflect the fact that the teacher resorts to any information sources to identify the children’s needs. Planning reflects the chil￾dren’s needs through teachers resorting to ex￾perts: educational supervi￾sor, parents, or evaluation results for children (or Planning reflects the children’s needs through teachers resorting to experts: educational supervi￾sor, parents, or evaluation results for children (or both). Planning reflects the children’s needs through teachers resorting to experts: educa￾tional supervisor, parents, and evalu￾ation results for children LIBERIA TEACHER TRAINING PROGRAM PHASE II 127 both). 1.1.2 Preparing enriching or remedial activities for children with special needs. There are no enriching or remedial activities for chil￾dren with special needs. There are enriching or remedial activi￾ties, but they do not serve the children’s actual needs. There are enriching or remedial activi￾ties and they serve the children’s actu￾al needs. 1.2 Familiarization With About Planning 1.2.1 The teacher analyzes the educational content The content analysis is not available. The content analysis is available, but the elements are in￾complete. The content analy￾sis is available and the elements are complete based on the approved form. 1.2.2 The teacher prepares the semester plans that suit the implementation of the curriculum. The semester plans are not available. The semester plans are not prepared according to the designated model. The semester plans are prepared ac￾cording to the des￾ignated model. 1.2.3 The teacher prepares the suitable daily plans. The daily plans are not available. The daily plans are not prepared ac￾cording to the des￾ignated model. The daily plans are not prepared ac￾cording to the des￾ignated model. 1.2.4 A suitable period of time is specified for the implementation of ac￾tivities/units. The period of time for the implementation of activi￾ties/units is not specified. A suitable period of time is specified for the implementation of activities/units. The specified peri￾od of time takes the semester and daily plans into account. 1.2.5 The teacher takes con￾tinuity of daily planning in to regard. The teacher does not take continuity of daily plan￾ning in to regard. Continuity and doc￾umentation of daily planning. Continuity and documentation of daily planning, and LIBERIA TEACHER TRAINING PROGRAM PHASE II 128 organizing it ac￾cording to the specified period of time. 1.2.6 The teacher takes inte￾gration in preparation of activities into ac￾count. Not taking integration with between curriculum units and activities into account. Integration within curriculum units. Taking integration within curriculum units and activities into account, and keeping them in order. Total of First Rubric .2 Implementation 2.1 Good knowledge about effective classroom management 2.1.1 Organizing educational corners. Distributing the corners inappropriately. Taking the following into account, while distributing the corners: 1. corners are distributed appropriately 2. The classroom is clean. Taking the follow￾ing into account, while distributing the corners: 1. cor￾ners are distributed appropriately 2. The classroom is clean. Taking the follow￾ing into account, while distributing the corners: 1. corners are dis￾tributed appropri￾ately 2. The classroom is clean 3. Spaces allocated for cor￾ners are suitable 2.1.2 The teacher distributes the corners to help her implement the curricu￾Distributing the corners does not help facilitate movement or communica￾Distributing the corners while taking the height of furni￾ture and shelves into Taking the follow￾ing into account when distributing LIBERIA TEACHER TRAINING PROGRAM PHASE II 129 lum. tion with children. account so as not to block the vision. the corners: 1.height of furni￾ture and shelves does not block the vision 2. Corners are apart 3. the morn￾ing activities space allows communica￾tion between chil￾dren and the teach￾er 2.1.3 The teacher puts toys and teaching aids in order. Toys and teaching aids are not distributed in the cor￾ners. Toys and teaching aids are distributed appropriately in the corners. Toys and teaching aids are distribut￾ed appropriately in the corners and are tagged correctly. 2.1.4 General safety rules are taken into account when using the furni￾ture, toys, and teaching aids. General safety rules are not taken into account when using the furniture, toys, and teaching aids. The safety of furni￾ture, teaching aids, and toys is ensured through regular maintenance. The safety of furni￾ture, teaching aids, and toys is ensured by: 1. Doing mainte￾nance to furniture, toys, and teaching aids 2. Appropriate use of furniture and teaching aids LIBERIA TEACHER TRAINING PROGRAM PHASE II 130 2.1.5 Necessary materials and aids are prepared for appropriate use. Necessary materials and aids related to activities are prepared in advance. Necessary materials and aids related to activities are pre￾pared in advance and by arrangement. Following a partic￾ular sequence in using the prepared necessary materials and aids, which should be inexpen￾sive (made from raw materials) 2.1.6 The teacher employs communication skills. The teacher does not show communication skills and physical and lin￾guistic communication with children. The teacher em￾ploys communica￾tion skills The teacher shows one of the following practices: 1. Active listening to children 2. Answering their questions 3. Using positive language when deal￾ing with the child The teacher shows the following prac￾tices: 1. Active listening to children 2. Answering their questions 3. Using positive language when dealing with the child 4. Using positive physical communi￾cation skills 2.1.7 The teacher is keen that the morning activities provide an educational The morning activities do not provide an educational learning experience. The morning activi￾ties achieve the edu￾cational learning The morning activ￾ities achieve the educational learn- LIBERIA TEACHER TRAINING PROGRAM PHASE II 131 learning experience. experience when one of the following conditions is met: 1. Activation of the fixed boards smoothly 2. Effective imple￾mentation of the activity in question 3. The children’s interaction with the teacher activity ing experience when all of the following condi￾tions are met: 1. Activation of the fixed boards smoothly 2. Effec￾tive implementa￾tion of the activity in question 3. The children’s interaction with the teacher activity 2.1.8 The teacher reflects positive personal char￾acteristics in dealing with children. The teacher’s treatment of children does not reflect positive personal charac￾teristics. When dealing with children, the teacher shows the following characteristics: 1. Her treatment reflects love, affec￾tion and patience toward children The teacher is characterized by at least three person￾ality traits as fol￾lows: 1. Her treatment reflects love, affec￾tion and patience toward children 2. She is keen on being just and fair in dealing with the children LIBERIA TEACHER TRAINING PROGRAM PHASE II 132 3. She accepts both the positive and negative feelings and emotions of children 4. She accepts con￾structive criticism 2.1.9 She respects the feel￾ings of children and encourages them to empathize with each other. There is nothing that shows her respect for the feelings and emotions of children. The teacher’s re￾spect for children is reflected by achiev￾ing one of the fol￾lowing: 1. Encouraging children to respect the feelings of each other 2. The teacher ac￾cepts the feelings and emotions of children, both posi￾tive and negative 3. The teacher trains children to know their own feelings and the feelings of others and to sym￾pathize with others The teacher’s re￾spect for children is reflected by achieving one of the following: 1. Encouraging children to respect the feelings of each other 2. The teacher ac￾cepts the feelings and emotions of children, both positive and nega￾tive 3. The teacher trains children to know their own feelings and the feelings of others and to sympathize LIBERIA TEACHER TRAINING PROGRAM PHASE II 133 with others 2.1.10 The teacher provides feedback. The teacher provides feedback inappropriately. Providing feedback in a variety of ways. Appropriate feed￾back is achieved through the follow￾ing: 1. Diversity in the methods of provid￾ing feedback 2. Providing feed￾back in a timely manner 2.1.11 Direct and indirect guidance techniques are used appropriately. Direct and indirect guid￾ance techniques are not activated. The teacher’s use of direct and indirect guidance techniques is reflected in one of the following: 1. Diversity of strat￾egies for guiding direct behavior 2. Diversity of strat￾egies for guiding indirect behavior The teacher’s use of direct and indi￾rect guidance tech￾niques is reflected in the following: 1. Diversity of strategies for guid￾ing direct behavior 2. Diversity of strategies for guid￾ing indirect behav￾ior 3. Using behavior￾guiding drawings and expressions LIBERIA TEACHER TRAINING PROGRAM PHASE II 134 2.1.12 The teacher joins the children in the break￾fast event. The teacher does not join the children the breakfast event. The teacher joins the children in the breakfast event by doing one of the following: 1. Preparing the right place for eating 2. Urging children to abide by time and duration of the meal 3. Having breakfast with them 3. Urging children to follow the eating etiquette. 4. Encouraging children to eat healthy meals The teacher joins the children in the breakfast event by doing three of the following: 1. Preparing the right place for eat￾ing 2. Urging children to abide by time and duration of the meal 3. Urging children to follow the eating etiquette 4. Encouraging children to eat healthy meals 2.1.13 The teacher makes sure to give a good example for children. The teacher does not fol￾low the appropriate behav￾ior modeling. The teacher’s be￾havior indicates one of the following: 1. The teacher gives a role model for appropriate behav￾ior The teacher’s be￾havior indicates the following: 1. The teacher gives a role model for appropriate behavior LIBERIA TEACHER TRAINING PROGRAM PHASE II 135 2. The teacher gives children a role mod￾el of positive behav￾ior 2. The teacher gives children a role model of posi￾tive behavior 3. The teacher gives intended be￾havioral role mod￾els 2.1.14 The teacher respects children. The teacher does not show respect for children. The teacher’s be￾havior indicates one of the following: 1. Sitting at the level of children during communication 2. Preparing activi￾ties that help chil￾dren accept each other's behaviors The teacher’s be￾havior indicates the following: 1. Sitting at the level of children during communica￾tion 2. Preparing activi￾ties that help chil￾dren accept each other's behaviors 3. Activities indi￾cate that the teach￾er respects them and understands the differences among. children 2.1.15 The teacher takes into account individual dif￾Nothing indicates that the teacher takes individual The teacher takes individual differ￾The teacher takes individual differ- LIBERIA TEACHER TRAINING PROGRAM PHASE II 136 ferences among chil￾dren. differences among chil￾dren into account. ences among chil￾dren into account in the following as￾pects: 1. Diversity of activ￾ities. 2. Carrying out ac￾tivities to suit the capacity of children. ences among chil￾dren into account in the following aspects: 1. Diversity of ac￾tivities 2. Carrying out activities to suit the capacity of chil￾dren. 3. Availability of work sheets with different degrees of difficulty 2.1.16 The teacher takes intra￾differences in children into account. Nothing indicates that the teacher takes intra￾differences in children into account. The teacher’s be￾haviors indicate the following: 1. She takes into account the devel￾opmental progress in the child 2. She constantly follows up the child’s completion of activities. The teacher’s be￾haviors indicate two of the follow￾ing: 1. She takes into account the devel￾opmental progress in the child 2. She constantly follows up the child’s completion LIBERIA TEACHER TRAINING PROGRAM PHASE II 137 of activities. 3. She records de￾velopmental pro￾gress in the child’s behavior and com￾pletion of activities. 2.1.17 The teacher ensures a smooth transition from one activity to another Nothing indicates that the teacher takes into account the smooth transition from one activity to an￾other. The teacher’s be￾haviors indicate one of the following: 1. The teacher pro￾vides a summary of what has been ac￾complished. 2. The teacher uses mechanisms and interesting materials for a smooth transi￾tion between cor￾ners All the following requirements should be taken into account in the case of a smooth transition: 1. The teacher pro￾vides a summary of what has been ac￾complished. 2. The teacher uses mechanisms and interesting materi￾als for a smooth transition between corners 3. She gives enough time to rearrange the class￾room environment 2.1.18 The teacher carries out activities related to in￾Nothing indicates that the teacher takes into account The teacher’s be￾havior indicates one The teacher’s be￾havior indicates the LIBERIA TEACHER TRAINING PROGRAM PHASE II 138 tegration within the curriculum units. integration within the cur￾riculum units. of the following: 1. She employs ac￾tivities in the curric￾ulum units in an integrated way 2. She carries out activities in the cor￾ners in an integrated manner following: 1. She employs activities in the curriculum units in an integrated way 2. She carries out activities in the corners in an inte￾grated manner. 3. She carries out activities in the outer yard in an integrated manner. 2.1.19 The teacher takes into account smooth transi￾tion to the first grade. Nothing indicates that the teacher takes into account smooth transition to the first grade. The teacher takes into account smooth transition to the first grade as follows: 1. Learning educa￾tional experiences as a required outcome 2.Children master￾ing educational ex￾periences as a re￾quired outcome The teacher takes into account the smooth transition to the first grade as follows: 1. Learning educa￾tional experiences as a required out￾come 2.Children master￾ing educational experiences as a LIBERIA TEACHER TRAINING PROGRAM PHASE II 139 required outcome 3. Engaging chil￾dren in morning school activities 2.1.20 Activating the parental involvement program. Nothing indicates that the teacher activates the pa￾rental involvement pro￾gram. Activating the pa￾rental involvement program through: 1. A board on pa￾rental involvement 2.A program plan, supportive hands form, attendance form, and activities report Activating the pa￾rental involvement program through: 1. A board on pa￾rental involvement 2.A program plan, supportive hands form, attendance form, and activities report 3.Effective partici￾pation by volun￾teers in the kinder￾garten, and docu￾menting this partic￾ipation 2.1.21 The teacher manages and invests times in the implementation of tasks. Nothing indicates that the teacher manages and in￾vests times in the imple￾mentation of tasks. The teacher manag￾es and invests times in the implementa￾tion of one of the following tasks: The teacher man￾ages and invests times in the im￾plementation of one of the follow￾ing tasks: LIBERIA TEACHER TRAINING PROGRAM PHASE II 140 1. The teacher bal￾ances between the time allocated for her role and the role allocated for the children’s role 2. The teacher matches between the planned time and the real time for implementation. 1. The teacher bal￾ances between the time allocated for her role and the role allocated for the children’s role 2. The teacher matches between the planned time and the real time for implementation 3. The teacher achieves real-time implementation of the intended out￾comes 4. The teacher adapts time to emergencies 2.2 Using teaching strategies to meet children’s needs and interests. 2.2.1 The teacher prepares a classroom environment that stimulates think￾ing. Nothing indicates that the teacher prepares a class￾room environment that stimulates thinking. The teacher pre￾pares a classroom environment that stimulates thinking through one of the following: 1. The teacher uses learning situations The teacher pre￾pares a classroom environment that stimulates thinking through one of the following: 1. The teacher uses learning situations LIBERIA TEACHER TRAINING PROGRAM PHASE II 141 to stimulate thinking 2. The teacher im￾plements various activities that stimu￾late thinking. to stimulate think￾ing 2. The teacher im￾plements various activities that stim￾ulate thinking. 3. The teacher pro￾vides equipment and materials nec￾essary for stimulat￾ing thinking 4. The teacher rais￾es a variety of questions to stimu￾late thinking 2.2.2 The teacher encourages children to experiment and explore things. Nothing indicates that the teacher encourages chil￾dren to experiment and explore things. The teacher pro￾vides safe materials to help children in experimentation and exploration. The teacher en￾courages experi￾mentation and ex￾ploration among children through the following: 1. The teacher pro￾vides safe materials to help children in experimentation and exploration 2. The teacher car￾ries out activities in LIBERIA TEACHER TRAINING PROGRAM PHASE II 142 which the child engages in experi￾mentation and ex￾ploration. 2.2.3 The teacher follows up on children and joins them in performing activities. Nothing indicates that the teacher follows up on children and joins them in performing activities. The teacher’s be￾havior indicates one of the following: 1. The teacher walks in between the chil￾dren to check on their performance. 2. The teacher joins children in perform￾ing activities and urges them to make achievements. The teacher’s be￾havior indicates the following: 1. The teacher walks in between the children to check on their per￾formance 2. The teacher joins children in performing activi￾ties and urges them to make achieve￾ments 3. The teacher watches for cases in which some children misunder￾stand activities 2.2.4 The teacher employs teaching aids effective￾ly. The teacher does not em￾ploy teaching aids. The teacher uses teaching aids in a sequence that achieves the objec￾The teacher em￾ploys teaching aids by activating the following: LIBERIA TEACHER TRAINING PROGRAM PHASE II 143 tive in question. 1. Using teaching aids in a sequence that achieves the objective in ques￾tion 2. Choosing the right time for dis￾play 3. Involving chil￾dren in using the aids effectively 2.2.5 The teacher employs information and com￾munication technology in children’s learning. Nothing indicates that the teacher employs infor￾mation and communica￾tion technology in chil￾dren’s learning. The teacher em￾ploys computer in children’s learning. The teacher em￾ploys information and communica￾tion technology in children’s learning by activating the following: 1. The teacher em￾ploys computers in children's learning 2. The teacher uses a variety of com￾puter software 3. The teacher em￾ploys television and radio in the LIBERIA TEACHER TRAINING PROGRAM PHASE II 144 educational process 2.2.6 The learning and teach￾ing activities are linked with the children’s life experience. 1. The teacher pre￾sents the life expe￾riences in the form of intended activi￾ties. 2. The teacher links between the learn￾ing and teaching activities and the children’s life expe￾rience. Nothing indicates that the teacher takes into account the link between the learn￾ing and teaching activities and the children’s life ex￾perience. The teacher pre￾sents the life experi￾ences in the form of intended activities. The learning and teaching activities are linked with the children’s life expe￾rience as follows: 1. The teacher pre￾sents the life expe￾riences in the form of intended activi￾ties 2. The teacher links between the learn￾ing and teaching activities and the children’s life expe￾rience Total of Second Rubric 3 Evaluation and Correction 3.1 Identifying the comprehensive and continuous evaluation of the child’s performance 3.1.1 The teacher uses dif￾ferent evaluation strat￾egies and tools. Nothing indicates that the teacher uses different evaluation strategies and tools. The teacher uses the right evaluation tools. The teacher does the evaluation by: 1.Using the right evaluation tools LIBERIA TEACHER TRAINING PROGRAM PHASE II 145 2. Matching evalua￾tion strategies with special outcomes 3.1.2 The teacher uses dif￾ferent sources to evalu￾ate the children’s per￾formance. Nothing indicates that the teacher uses different sources to evaluate the children’s performance. The teacher uses a variety of child per￾formance evaluation sources (teacher, child, councilor, parent, principal, or specialist). The teacher evalu￾ates the children’s performance by using the following sources: 1. A variety of child performance evaluation sources (teacher, child, councilor, parent, principal, or spe￾cialist) 2. Preparing evalu￾ation tools that are compatible with the outcomes 3.1.3 The teacher interprets the results of children performance evalua￾tion. Nothing indicates that the teacher interprets the re￾sults of children perfor￾mance evaluation. The teacher analyzes the data of evalua￾tion tools. The teacher inter￾prets the results of the children’s per￾formance evalua￾tion through the following: 1.Analyzing the data of evaluation tools 2.Providing expla- LIBERIA TEACHER TRAINING PROGRAM PHASE II 146 nations that are compatible with the analysis results 3.1.4 Interpreting the results is followed by appro￾priate post-procedures. (Daily, semester, enrich￾ment, remedial) plans are not set up or adjusted based on the evaluation results. Different (daily, semester, enrich￾ment, or remedial) plans are set up. The results are in￾terpreted through post-procedures as follows: 1. Setting up dif￾ferent (daily, se￾mester, enrich￾ment, or remedial) plans 2. Adjusting differ￾ent (daily, semester, enrichment, or re￾medial) plans in light of the results 3.2 Upgrading the teacher’s performance through self-evaluation 3.2.1 The teacher encourages children to evaluate themselves. Nothing indicates that the teacher encourages chil￾dren to evaluate them￾selves. The teacher encour￾ages children to dis￾cuss their activities. The teacher en￾courages children to evaluate them￾selves through: 1. Encouraging children to discuss their activities 2.Directing chil￾dren to improve LIBERIA TEACHER TRAINING PROGRAM PHASE II 147 their performance based on the dis￾cussions Total of Third Rubric 4 Professional Aspect 4.1 Keenness on sustainable professional development and commitment to codes of ethics. 4.1.1 The teacher cooperates with her colleagues at school. Nothing indicates that the teacher cooperates with her colleagues at school. The teacher cooper￾ates with her col￾leagues in preparing educational aids and activities. The teacher coop￾erates with her col￾leagues as follows: 1. Cooperating with her colleagues in preparing educa￾tional aids and ac￾tivities. 2.Contributing with her colleagues to develop and improve the school’s perfor￾mance 4.1.2 The teacher is keen on maintaining profes￾sional development, which reflects in her performance. The teacher does not en￾hance her knowledge through training courses, research, or enrolment in university. The teacher en￾hances her knowledge through training courses, research, or enrol￾ment in a university, but she does not apply what she The teacher en￾hances her knowledge through training courses, research, or enrol￾ment in a universi￾ty, and she applies what she learns in LIBERIA TEACHER TRAINING PROGRAM PHASE II 148 learns in the educa￾tional process. the educational process. LIBERIA TEACHER TRAINING PROGRAM PHASE II 149 ANNEX 8: SURVEYS AND SCORECARDS The team will use a variety of evaluation methodologies, including scorecards and surveys, as appropriate. These are intended to provide a quantitative dimension to the evaluation and enable retrospective compari￾sons using time series data where possible to track the impact of various LTTP activities. The team aimed to employ scorecards for the following: 1. Comparison of budget data over three two-year intervals, using 2008/2009 as the baseline 2. Collection of data from the MOE 3. Collection of data from the MOE Department of Administration 4. Collection of data from CEOs In addition, it is anticipated that a qualitative survey instrument will be developed and used after the team has had the opportunity to meet with USAID and the contractor to determine what data may already exist Baseline Financial Data to Evaluate MOE/Liberia Ownership and Commitment Purpose: Collect and analyze time series budget data to provide a quantitative perspective for the following assessment questions:  How effectively is the LTTP addressing underlying institutional, policy and systems weaknesses that impact capacity to deliver a profes￾sional development program?  To what degree is there MOE ownership, leadership and accountability for teacher training and early grade reading?  How sustainable are the different components of the project?  What are the preliminary outcomes of the policy efforts?  What are the preliminary outcomes of the decentralization efforts? Candidate Indicators of MOE Ownership and Commitment Indicator Formula Budget Year 2008/2009 2010/2011 2012/2013 1. Absolute Change in Education Budget Baseline Education Budget (2008/2009) =100% (x) Percent Compared to Baseline 2010/2011 / x Percent Compared to Baseline 2012/2013 / x 100% 2. Relative Financial Commitment to Education Budget Appropriated Total Budget Appropriated % % % Education 3. Education Budget Allocated to Teacher Training Total Budget for Teacher Training Education Budget Appropriated % % % 4. Classroom Teach￾ers : Increase per period Baseline Classroom Teachers (2008/2009) = x Increase Compared to Baseline (x+1) x # # % # % 5. Budgets Allocated to Counties MOE Budget Allocated to Counties MOE Budget Increase Compared to Baseline % 100% % % % % 6. Budgets Allocated to RTTIs MOE Budget Allocated to RTTIs MOE Budget Increase Compared to Baseline % % % % % % Interview Guide/Scorecard Portion Target: FHI/360 Institutional Development Team  Institutional Capacity Development Advisor  EMIS Advisor  M&E Advisor  Policy and Systems Strengthening Advisor Purpose: To identify the pre-existing policy and management conditions/issues to which the LTTP program was directed (R-1); to assess progress and identify constraints; to identify work to be completed by 2105; and, to identify priorities for future interventions. Approach: The Evaluation Team will use an interview/questionnaire to capture information necessary to draw conclusions that will benefit the con￾tractor, the Government and USAID. The structured portion of the interview is closely tied to the SOW questions. Additional follow-up meetings and data collection may be required to complete the process. 10. What were the specific policies, leadership management, human resources, and systems issues/constraints related to the delivery of a pro￾fessional development program that existed at the inception of the LTTP II. Please provide specific examples at the central MOE level and at the County level. Areas Specific Weak￾nesses Central MOE % Specific Weaknesses Counties % Policy Making Capacity Instructional Leadership Management Systems o Strategic Planning o Budgeting o Monitor￾ing o Perfor￾mance Manage￾ment o Monitor￾ing o Reporting Scorecard for CEO’s 9. Please discuss the services provided by each of the three main Departments/Bureaus of the MOE MOE Department Services Provided Administration Instruction Planning Research and development 10. The LTTP is strengthening the MOE to provide services to the counties and is providing services directly to counties in a number of key areas of planning and administration. LTTP Services Aware Importance 1 =(tops); 2= (average) 3= (not so much) Impact 1= effective 2= partial 3 = ineffective Education Management Information System and capacity building to operate/comply Training in strategic planning, management, ca￾pacity building, etc. Training and systems development for instruc￾tional leadership Improved communication (from MOE to CEO) of policies, priorities, changes in procedures, new developments Training and systems for education quality moni￾toring and instructional supervision Training for school boards (other services) Early grade reading and math programs Teacher Training Payroll reform/biometric cards 11. What are the key obstacles that you face? Has the LTTP helped the county respond to the challenges in the education system in Liberia? What more is needed? 1 U.S. Agency for International Development 1300 Pennsylvania Avenue, NW Washington, DC 20523 Tel: (202) 712-0000 Fax: (202) 216-3524 www.usaid.gov