June 2014 This publication was produced at the request of the United States Agency for International Development. It was prepared independently by EMC (Emerging Markets Consulting), Phnom Penh. End of Project Performance Evaluation of The Improved Basic Education in Cambodia Project END OF PROJECT PERFORMANCE EVALUATION OF THE IMPROVED BASIC EDUCATION IN CAMBODIA PROJECT PROMOTING BETTER EDUCATED YOUTH IN CAMBODIA WITH INCREASED ACCESS TO A QUALITY AND RELEVANT BASIC EDUCATION June 2014 DISCLAIMER The author’s views expressed in this publication do not necessarily reflect the views of the United States Agency for International Development or the United States Government. Performance Evaluation on Improved Basic Education in Cambodia (IBEC) p. 3 CONTENTS END OF PROJECT PERFORMANCE EVALUATION OF THE IMPROVED BASIC EDUCATION IN CAMBODIA PROJECT .................................................................................................................................................... 2 Contents .......................................................................................................................................................................... 3 Table of Figures ............................................................................................................................................................. 4 Acronyms ........................................................................................................................................................................ 5 Executive Summary ....................................................................................................................................................... 6 Evaluation Purpose and Evaluation Questions ................................................................................................... 6 Project Background .................................................................................................................................................. 6 Evaluation Questions, Design, Methods and Limitations ................................................................................ 7 Findings And Conclusions ....................................................................................................................................... 8 Evaluation Purpose & Evaluation Questions ......................................................................................................... 13 Evaluation Purpose ................................................................................................................................................. 13 Evaluation Questions ............................................................................................................................................. 14 Project Background .................................................................................................................................................... 15 Evaluation Methods & Limitations ........................................................................................................................... 17 Findings, Conclusions & Recommendations ......................................................................................................... 20 Findings ...................................................................................................................................................................... 20 Innovations, Best Practices and Lessons Learned ........................................................................................... 33 Conclusions .............................................................................................................................................................. 38 Recommendations .................................................................................................................................................. 40 Annexes ......................................................................................................................................................................... 41 Annex I: Evaluation Statement of Work ........................................................................................................... 42 Annex II: Evaluation Methods and Limitations................................................................................................. 46 Annex III: Data Collection Instruments ............................................................................................................ 47 Annex IV: Sources of Information ...................................................................................................................... 63 Annex V: Disclosure of any Conflicts of Interest ........................................................................................... 69 Annex VI: Enrollment Tables and Charts ......................................................................................................... 73 Annex VII: Dropout and Repetition Charts ..................................................................................................... 75 Annex VIII: Gender Parity Index Rates ............................................................................................................. 79 Annex IX: IBEC Life Skills Activities .................................................................................................................. 82 Annex X: IBEC Primary School Results ............................................................................................................ 84 Annex XI: ADDENDUM: Additional Observations on Lessons Learned & Successes to Add to the IBEC Evaluation Record ........................................................................................................................................ 99 Performance Evaluation on Improved Basic Education in Cambodia (IBEC) p. 4 TABLE OF FIGURES Figure 1 Verification of Data on IBEC Project Indicators ..................................................................................... 18 Figure 2 Secondary Enrollment in Target Provinces .............................................................................................. 20 Figure 3 Secondary Enrollment over Four Years ................................................................................................... 21 Figure 4 IBEC Dropout Rate ....................................................................................................................................... 22 Figure 5 IBEC Repetition Rate .................................................................................................................................... 22 Figure 6 Kampong Cham Dropout ............................................................................................................................ 23 Figure 7 Kampong Cham Repetition.......................................................................................................................... 23 Figure 8 Kratie Dropout ............................................................................................................................................... 24 Figure 9 Kratie Repetition ............................................................................................................................................ 24 Figure 10 Siem Reap Dropout ..................................................................................................................................... 25 Figure 11 Siem Reap Repetition .................................................................................................................................. 25 Figure 12 IBEC Gender Parity Index for Enrollment ............................................................................................. 26 Figure 13 Kampong Cham Pupil-Teacher Ratio ...................................................................................................... 28 Figure 14 Kratie Pupil-Teacher Ratio ........................................................................................................................ 28 Figure 15 Siem Reap Pupil-Teacher Ratio ................................................................................................................ 29 Figure 16 Life Skills Activities ...................................................................................................................................... 30 Figure 17 Tier Change at Primary Schools ............................................................................................................... 35 Figure 18 Tier Changes at Secondary Schools ........................................................................................................ 36 Figure 19 Percent Secondary Schools Reaching Tier 1 ......................................................................................... 37 Performance Evaluation on Improved Basic Education in Cambodia (IBEC) p. 5 ACRONYMS BSDA CCC CEFAC CFS CWCC DEO EDA EMIS GPI Buddhist Social Development Association Committee for Cooperation in Cambodia Commune Education For All Committee Child Friendly Schools Commune Women and Children Committee District Education Officer Economic Development Association Education Management Information System Gender Parity Index IBEC KAPE M&E MoEYS MOI NEP NGO PB PEO PS PT Ratio SDP SIDA SIG SS TOR Improved Basic Education in Cambodia Kampuchean Action for Primary Education Monitoring and Evaluation Ministry of Education, Youth and Sport Ministry of Interior NGO Education Partnership Non-Government Organization Program Budget (from the Ministry) Province Education Officer Primary School Pupil-Teacher Ratio School Development Plan Swedish International Development Agency School Improvement Grant Secondary School Terms of Reference USAID WCRD WEI U.S. Agency for International Development Women and Children's Rights Development World Education International Performance Evaluation on Improved Basic Education in Cambodia (IBEC) p. 6 EXECUTIVE SUMMARY EVALUATION PURPOSE AND EVALUATION QUESTIONS The purpose of the evaluation is to assess the project’s accountability and extent to which it was able to meet its intended objectives –at all result levels. The evaluation also documents lessons learned and best practices, as well as providing recommendations to inform evidence-based future programming. The Evaluation Questions guiding this Performance Evaluation were: 1) To what extent did the project achieve its objectives? 2) Among all the interventions implemented by the partner, identify the interventions that have been the most effective in achieving the project objectives. 3) What are key lessons learned? The audience of the evaluation report will be the USAID/Cambodia Mission, the Asia Bureau, and USAID implementing partners. USAID may consider the findings, particularly the evidence-based findings, in its strategic approach to education. An Executive Summary will be provided to the Ministry of Education, Youth and Sport (MoEYS). It is expected that the host country partners and donors will also be able to use the report to better assist them in their future goals. PROJECT BACKGROUND The strategic objective of this five-year project, which began in 2009 and will end in 2014, is to improve access, quality, and relevance of basic education in Cambodia. More specifically, the IBEC project is to increase lower secondary school enrollments, retention, and completion rates, providing Cambodia’s adolescent youth population with an opportunity to be better educated and lead productive lives. The IBEC project targets selected schools in three provinces, Kampong Cham, Kratie and Siem Reap. There are four inter-related, mutually reinforcing, and complementary components to achieving this overarching goal. These four main components include: 1. Local NGO and Government Capacity Building & Advocacy for Sustainability: IBEC accomplishes this by training, coaching, and monitoring its sub-grantee, Kampuchea Action for Primary Education (KAPE). The core themes covered in the trainings are: a) Organizational Management, b) Program Management, c) Project Performance Management, d) Governance, e) Administration, f) Human Resource Management, and g) Financial Management. 2. More Equitable School Access: IBEC provides scholarships for youth from underserved populations who are poor, belong to ethnic and religious minorities, or are handicapped. The scholarships provided are in the form of bicycles, clothing, school supplies, and monetary support. IBEC is also improving access to school by building temporary classrooms to accommodate more students. Other school improvements may range from building wells for drinking water, latrines, or other facilities for sanitation purposes. Access to education is also improving by recruiting teachers from the local community. 3. Improved School Management and Community involvement: IBEC provides training to school principals on leadership and management skills and supports the Parent Teacher Associations in the Performance Evaluation on Improved Basic Education in Cambodia (IBEC) p. 7 project’s target schools. IBEC programming is aimed towards training school managers to identify priorities, develop proposals and budgets, develop and implement maintenance plans, and to effectively advocate for grants from the Cambodian government and other potential donors. 4. Improved Educational Relevance: IBEC developed and piloted a Life Skills curriculum composed of 30 modules on different relevant topics. This curriculum has been adopted by the Ministry of Education, Youth, and Sports (MoEYS). The project works closely with community, including commune councils, to improve their engagement and ownership. The development hypothesis underlying the IBEC project is that a more relevant curriculum, combined with better trained teachers and school managers would encourage more students to attend lower secondary school and to stay long enough to complete the lower secondary cycle. Improved physical facilities (wells for drinking water, sanitary latrines, life skills laboratories, etc.) were also believed to play a positive role in inducing students to attend and remain in school. In addition, a limited number of scholarships would directly help vulnerable youth obtain a quality education. In total, these improvements were to create a better educated youth population who will have a knowledge base more relevant to the economic needs of Cambodia, with skills more compatible with the demands of the local job market. EVALUATION QUESTIONS, DESIGN, METHODS AND LIMITATIONS The data sought for this evaluation included: 1. Quantitative data sent by target schools to the project; 2. Quantitative and qualitative data collected by project staff from target schools; and 3. Evaluation team interviews with stakeholders (Ministry of Education Youth and Sport; IBEC project management and staff; IBEC project implementing partners; School directors, staff, teachers and students at target schools; and Commune officials, village officials and parents in target school areas.) Analysis of the quantitative data related to accountability included comparison of current rates to baseline rates for the indicators examined. Where feasible, the EMIS rates (especially from the target provinces) were also included in the comparisons. Interviews with key stakeholders in MoEYS, key IBEC project staff, and key persons in its implementing partner organizations were essential to understand the current Cambodia context for education projects. Interviews at target schools and communes and villages were limited by time and budget and by the distances between target locations. The impressions we gathered at this level provided some local perceptions of sustainability to supplement the understanding we had of the project from key Ministry and IBEC stakeholders. Quantitative data that IBEC had received from schools was available for the four years of project implementation, although it had not been systematically compiled and aggregated in a single database covering consecutive years. We created this multi-year database ourselves by merging the data for individual years, which allowed us to see trends against the IBEC baseline and national EMIS data, which we also included in the database. Because such a comprehensive multi-year data was not maintained during the project, an opportunity was lost during implementation to identify trends from analysis of this M&E data over time and to consider fine-tuning the implementation accordingly. Summary statements about the findings of this data had been made in Annual and Quarterly Reports, but for many performance indicators we could not find the actual data on which the statements of findings were based. Accordingly, we could not evaluate the accuracy, validity or representativeness of those data and the related Performance Indicators. Performance Evaluation on Improved Basic Education in Cambodia (IBEC) p. 8 A baseline for many of the project quantitative indicators was provided, against which to compare current rates. However, for a majority of the quantitative indicators, the baseline changed from year to year. It was evidently adjusted to account for schools that had been added or removed from the project. Neither the data on which the baseline had originally been constructed nor data supporting any baseline modifications during project implementation were available to the evaluation. FINDINGS AND CONCLUSIONS FINDINGS Enrollment1 Our analysis of IBEC school data shows that enrollment at target schools in one province, Kratie, had increased over the four years of implementation examined. The other two provinces, Kampong Cham and Siem Reap registered declines in enrollment at target schools. This reflects a larger national trend of declining school enrollments, which can in part be attributed to smaller family sizes and increased mobility. On this measure, the project had mixed results in increasing access in the targeted schools. A full explanation for these observed trends would call for a detailed examination of project implementation factors, provincial education data, and the socio-economic and ecological conditions of the provinces where the target schools are located, which was beyond the scope of this evaluation. Dropout, Repetition Dropout rate and repetition rate reductions over the four years of IBEC project implementation compared to project baselines generally confirm increased access to education for children in target schools. At the primary level, target schools show dropout and repetition rates that are below the project baseline rate for all years of project implementation. At the secondary level, in Year 4 (2012-13) the total project secondary dropout rate exceeded the baseline rate, but the repetition rate for the last two years of implementation is below the baseline rate. For all four years of implementation, the female dropout rate at target secondary schools exceeded the baseline rate. The dropout rate for secondary school females is probably associated with increasing opportunities for employment in the expanding factory sector in the country, where female workers are preferred, to accelerated rates of migration, especially among young people. In all three target provinces, for both primary and secondary schools, the IBEC school dropout rate is below province EMIS rates for all four years of project implementation. However, in all three provinces, for both primary and secondary schools, the IBEC school repetition rate is above EMIS rates for all four years of implementation for both all students and female students. Either EMIS consistently understates the actual repetition rates, or IBEC and EMIS calculate repetition differently, or repetition in IBEC target schools is a real and persistent problem. Gender Parity on Enrollment Gender parity rates for IBEC target primary schools are below the range for parity (0.97-1.03), indicating a general preponderance of boys for all years of project implementation at the primary level. In IBEC target secondary schools the GPI rises over the four years of implementation. In Year 3 and Year 4, the 1 School enrollments viewed over several years may reveal important trends. The evaluation did not consider technical indicators like Gross Enrollment Rate (GER) or Net Enrollment Rate (NER), which are not relevant to the present evaluation questions. Performance Evaluation on Improved Basic Education in Cambodia (IBEC) p. 9 GPI is well above the parity range, indicating a preponderance of females enrolled. This is probably due to project emphasis on access for females, especially at the secondary level, and helps explain the elevated dropout rate for girls in these secondary schools. Quality One of the key strategic decisions the IBEC project made to improve the quality of education at both the primary and secondary levels was to address the serious problem of teacher shortages for the country's schools. The evident reduction of the pupil teacher ratio in IBEC schools supports the empirical perception that increased teacher density in classrooms leads to increased quality of education.. In all three target provinces, at both the primary and secondary levels, IBEC pupil teacher ratios for all years of IBEC implementation were generally below EMIS rates. The relatively low pupil-teacher ratio for urban target secondary schools highlights the persistent problem of persuading teachers to work in rural and remote schools. Relevance Within the IBEC framework, relevance is purposefully reflected in the Life Skills curriculum developed by the project. The project was successful in gaining Ministry approval for most of their new Life Skills curriculum modules. The associated IBEC Life Skills texts were also imprinted with the official MoEYS logo. Project advocacy contributed to new Ministry policy which now officially allocates time in school schedules for Life Skills instruction and activities. These project accomplishments attest to IBEC success in increasing the relevance of education, especially related to the curriculum. Progress in extending relevance is demonstrated in the increasing frequency of Life Skills activity implementation each year at both target primary and secondary schools. Civil Society Strengthening The success of IBEC in building the capacity of two of its key partners, KAPE (Kampuchean Action for Primary Education) and BSDA (Buddhist Social Development Association), is attested by the fact that these local civil society organizations were both able to achieve Certification of Compliance with the Code of Ethical Principles and Minimum Standards for NGOs from the Committee for Cooperation in Cambodia (CCC). CCC has a well-regarded accreditation process of great rigor. A three-step process for certification is used involving (i) document review and investigation; (ii) verification process; and (iii) external review and certification and covers six aspects of organizational excellence, 1) human resource management; 2) financial resource management; 3) service delivery; 4) external relations; 5) organizational learning; and 6) strategic management. Innovations, Best Practices, Lessons Learned A significant innovation of the IBEC project is the classification of its target schools into "development readiness" tiers. Tier1 are advanced schools with excellent school governance. Tier 2 schools are satisfactory in regard to management. Tier 3 schools are at a poor stage of development. Higher tier schools receive greater funding through their School Improvement Development Plan, as they are considered able to absorb more external aid more effectively. Lower tier schools receive greater capacity building assistance to enable them to move up in tier level. IBEC concludes that the innovation of adjusting aid to readiness and performance is an intervention that helps promote good governance practices in target areas. MoEYS officials also agree that this "results based" budgeting and grants management device based on tier distinctions correctly places the responsibility for effectiveness on the School Director. Other donors considering school development Performance Evaluation on Improved Basic Education in Cambodia (IBEC) p. 10 grant assistance may also appreciate the tier structure as it shifts accountability for performance to the Provincial Education Office, which manages local Ministry staff, including school directors. The tier classification may have the unintended consequence of exacerbating the gap between strong and weak schools. The tier concept shifts resources to the strongest and most successful of the attractive schools, and away from the relatively weaker schools. The legitimacy of tier assignment and justification for tier mobility also depends on the perception that development readiness will be assessed objectively and transparently. An analysis of tier change in IBEC target schools reveals significant differences between the primary and secondary levels. Between start up and Year 4, at the primary level, at least 25% of project schools in the three target provinces were downgraded in tier and 40% to 50% were upgraded in tier. By contrast, between start up and Year 4 at the secondary level, only between 2% and 5% of project schools were downgraded in tier and over 75% of target schools were upgraded in tier. These differences may reflect different management and governance challenges at the primary and secondary levels, or may reflect different methods of tier assessment at the two levels, or significantly different project implementation efforts at the two levels. No data was found in IBEC materials to help account for these differences found. CONCLUSIONS The end-of-project performance evaluation provides evidence to confirm that IBEC largely achieved its objectives to improve access, quality and relevance of basic education in Cambodia. Dropout rates have generally been reduced in target schools, although dropout of secondary school females remains a problem. Repetition rates are trending below the project baseline, but above the national EMIS rates, which may be understated. As repetition is a school decision, it may be informal school practice to understate these numbers to accord with Ministry policy to reduce repetition. This would give rise to an artificially low EMIS rate. Gender parity rates show a slight preponderance of boys in enrollment at IBEC primary schools and a preponderance of girls in IBEC secondary school enrollment. This reflects the many efforts directed at improving school access for girls especially in secondary schools (scholarships, separate latrines, girls' counselors and many others.) As the 2013 IBEC Annual Report (p.2) notes proudly in regard to these access objectives, "these very positive bottom line changes in school efficiency are among the project's most important achievements." Improvements in quality of education are reflected in reduced pupil-teacher ratios in IBEC schools. This is probably due to the many efforts of the project to cope with the perennial teacher shortage in Cambodian schools. IBEC has supported contract or temporary teachers, especially in remote schools and provides scholarships for students at Teacher Training Colleges. Province data shows that the values IBEC promotes, such as improving quality for poorer remote schools, affects local policy that is then reflected in the trends of the pupil-teacher ratio. IBEC's great efforts to advance a relevant education in Cambodia have been crowned with success as the Ministry of Education, Youth and Sport endorsed the IBEC Life Skills curriculum and made life skills an obligatory subject at schools. Much of the training of teachers and capacity building of school directors to advance relevance and to support the Life Skills program in IBEC has also improved the general quality of teaching and management at target schools as measured by IBEC performance standards, according to interviewed stakeholders’ perceptions. Performance Evaluation on Improved Basic Education in Cambodia (IBEC) p. 11 Intensive project support helped two implementing partners gain accreditation from the Committee for Cooperation in Cambodia. This achievement reflects a USAID commitment to strengthening civil society as a way to guarantee sustainability of development interventions. A key intervention to improve access and quality in Cambodian schools has been the distribution of discretionary school grants, based on school improvement development plans. IBEC has introduced a significant innovation of "development readiness" tiers, as a way to direct resources to schools that prove to be most effective in utilizing the support properly. The development readiness tier concept holds great promise because it combines two central development concerns, aid effectiveness and good governance. An unintended drawback of the tier scheme may be to widen the separation between good schools, where most resources will go, and poor schools that may be neglected or avoided. Careful attention to the most vulnerable and least advantaged schools should balance the favor shown to excellent schools. IBEC managers undoubtedly have insights into how Cambodian schools really work and what the challenges are to improving effectiveness and governance at schools. It remains for them to distill lessons learned from their experience with Cambodian schools in order to clarify what "readiness" means in a development context, so future projects may then be designed to foster and engage this elusive quality. The value of a concept like "development readiness" in education is that it challenges us to return to first principles and open a discussion about what fundamental goals education should have in Cambodian society. There is a new generation of leaders and educators in the Ministry of Education Youth and Sport. Many of these younger officials do not remember the Khmer Rouge and did not grow up in a civil war milieu. These new young leaders may use a conversation about development in education to stimulate fresh and productive thinking about what kind of citizen Cambodian education is supposed to create. RECOMMENDATIONS  Project monitoring and evaluation data should be collected maintained and analyzed with the same care that project financial data is treated. Analysis of data during the project would highlight emerging trends and provide an opportunity for adjustments as needed.  Suddenly rising dropout and repetition rates in Year 4 of implementation in some schools as the project ends and as support is reduced may threaten the gains made. The IBEC project exit strategy should include sensitive encouragement of these schools to continue dropout and repetition reductions.  Trends in access, quality, relevance and tier mobility reflect great differences between levels and among target provinces, which are likely associated with the different demographic, social and economic conditions. Effectiveness would be enhanced if project implementation were fine-tuned to the peculiar needs and requirements of each level and province, based on adequate monitoring and evaluation reports.  An analysis of what qualities increase a school's development readiness, or make a school too weak to remain in the project would reveal important characteristics of management and governance in Cambodian schools. This is the kind of "lesson learned" that would be useful in designing future interventions.  The encouragement of "risk-taking" by school directors to get results often asks them to use personal funds for project activities until project funds are sent as reimbursement. The commingling of personal and project funds is not appropriate for a civil servant. A line of credit at a local bank should be provided to project school directors, and interest expenses should be borne by the project.  There is high-level agreement in the Government that Ministry of Interior funds could be used at the local level for education, but school directors are often reluctant to question local authority Performance Evaluation on Improved Basic Education in Cambodia (IBEC) p. 12 in the commune about the availability of funding. Management and citizenship training for school directors should include strengthening political advocacy skills in dealing with local elected officials.  Tier classification appears to be an attractive method for allocating scarce resources in a rational way. The concept of "development readiness" combines two key concerns of donors, aid effectiveness and good governance, and may be adopted in other major projects in education in connection with school grants. A critical analysis of the concept would be concerned with unequal school readiness and would consider how decisions about a just allocation of assistance in the education sector might be made. This analysis would provide a useful basis on which future interventions could be planned. Performance Evaluation on Improved Basic Education in Cambodia (IBEC) p. 13 EVALUATION PURPOSE & EVALUATION QUESTIONS EVALUATION PURPOSE The evaluation policy of the U.S. Agency for International Development (USAID) articulates the crucial role that evaluations play in the application and management of development resources. With this in mind, USAID/Cambodia’s Office of Public Health and Education (OPHE) sought an expert team to conduct an end-of-project performance evaluation of the Improved Basic Education project in Cambodia (IBEC). The purpose of the evaluation is to assess the project’s accountability and extent to which it was able to meet its intended objectives –at all result levels. The evaluation also documents lessons learned and best practices, as well as providing recommendations to inform evidence-based future programming. The Improved Basic Education in Cambodia project was designed in direct response to USAID’s goal to promote a better educated youth. The strategic objective of this five-year project, which began in 2009 and will end in 2014, is to improve access, quality, and relevance of basic education in Cambodia. More specifically, the IBEC project is to increase lower secondary school enrollments, retention, and completion rates, providing Cambodia’s adolescent youth population with an opportunity to be better educated and lead productive lives. There are four inter-related, mutually reinforcing, and complementary components to achieving USAID’s overarching goal to promote a better educated youth. These four main components of the IBEC project include: 1. Local NGO and Government Capacity Building & Advocacy for Sustainability: IBEC accomplishes this by training, coaching, and monitoring its sub-grantee, Kampuchea Action for Primary Education (KAPE). The ore themes covered in the trainings are: a) Organizational Management, b) Program Management, c) Project Performance Management, d) Governance, e) Administration, f) Human Resource Management, and g) Financial Management. 2. More Equitable School Access: IBEC provides scholarships for youth from underserved populations who are poor, belong to ethnic and religious minorities, or are handicapped. The scholarships provided are in the form of bicycles, clothing, school supplies, and monetary support. IBEC is also improving access to school by building temporary classrooms to accommodate more students. Other school improvements may range from building wells for drinking water, latrines, or other facilities for sanitation purposes. Access to education is also improving by recruiting teachers from the local community. 3. Improved School Management and Community involvement: IBEC provides training to school principals on leadership and management skills and supports the Parent Teacher Associations in the project’s target schools. IBEC programming is aimed towards training school managers to identify priorities, develop proposals and budgets, develop and implement maintenance plans, and to effectively advocate for grants from the Cambodian government and other potential donors. Performance Evaluation on Improved Basic Education in Cambodia (IBEC) p. 14 4. Improved Educational Relevance: IBEC developed and piloted a Life Skills curriculum composed of 30 modules on different relevant topics. This curriculum has been adopted by the Ministry of Education, Youth, and Sports (MoEYS). The project works closely with community, including commune councils, to improve their engagement and ownership. The audience of the evaluation report will be the USAID/Cambodia Mission, the Asia Bureau, and USAID implementing partners. USAID may consider the findings, particularly the evidence-based findings, in its strategic approach to education. An Executive Summary will be provided to the Ministry of Education, Youth and Sport (MoEYS). It is expected that the host country partners and donors will also be able to use the report to better assist them in their future goals. EVALUATION QUESTIONS The Evaluation Questions guiding this End of Project Performance Evaluation were: 1) To what extent did the project achieve its objectives? 1.1. Did the project increase access to lower secondary education with respect to enrollment, retention, promotion, and completion rates of youth from underserved populations (including marginalized girls and boys, poor, ethnic and religious minorities, and handicapped children) in targeted schools? 1.2. Did the project improve the quality of lower secondary education in targeted schools? 1.3. Did the project increase the relevance of education, particularly related to curriculum, and how did the implementer measure progress? 1.4. How effective has the local NGO capacity building component been and what are the strengths/weaknesses of the local partners? 2) Among all the interventions implemented by the partner, identify the interventions that have been the most effective in achieving the project objectives. 2.1. What activities/interventions introduced by IBEC are likely to continue after the project ends and why? 2.2. Were there any unintended consequences or results of the project interventions? 2.3. How effectively has the project addressed gender differences in education throughout all its interventions? 3) What are key lessons learned? 3.1. Which interventions, based on evidence, should be continued or expanded to improve access (enrollment, retention, promotion, and completion rates) to and the quality of lower secondary education or improve relevance of education? 3.2. How might future investments be refocused? The evaluation specifically states a focus on secondary school results, although the IBEC program had a very large primary school component. In the report that follows, both primary and secondary results are provided. Given the limitations on report length, most of the primary school results have been placed in an Annex. Performance Evaluation on Improved Basic Education in Cambodia (IBEC) p. 15 PROJECT BACKGROUND When the IBEC project was developed, Cambodia’s education system was affected by a weak public￾sector service-delivery system, nominal teacher capacity, lack of adequate school facilities, poor governance, and lack of quality teaching and learning resources. Of all primary and lower secondary schools, almost 50 percent of them did not have clean water, and approximately 20 percent did not have toilets. These factors resulted in low enrollment levels and high drop-out rates, especially at the lower secondary level and higher. Repetition and absenteeism were also a major concern and vulnerable children, particularly girls, the disabled, and minorities, were at risk of being deprived of an education. Lower secondary enrollment rates were near 35%, which put Cambodia globally in the bottom 20 countries worldwide. In community meetings, parents and community members often identified the lack of a relevant curriculum as an obstacle to lower secondary school enrollment. Additionally, limited access, such as distance to and from school, lack of classrooms, and school-associated costs, as well as the lack of qualified teachers are factors that affected enrollment and retention. Due in large part to inadequate access to a quality and relevant basic education, at best 25 percent of the 250,000 individuals who were eligible to enter the job market each year didn’t have the necessary and sufficient skills to find employment in the formal sector1. Furthermore, workers with little formal basic education experience were more likely to suffer from low pay and benefits, poor working conditions, and job insecurity. Therefore, the IBEC project was created to focus on designing, developing, pilot testing, and evaluating a more relevant lower secondary school life skills curriculum, using lessons learned from previous USAID education projects. Teachers were to be trained in the use of the new curriculum and school administrators and Parent Teacher Associations (PTAs) were to be trained in such areas as planning, how to develop small grant applications, school management, and facilities maintenance. Access issues were to be addressed by providing scholarships to vulnerable students and school improvement grants to schools. Additionally, at the end of the project, one or more local Education NGOs should be capable of providing quality assistance in primary and lower secondary education, creating in-country sustainability and building the capacity of local civil society. In the past decade, the Cambodian government has made significant improvements in the education sector. Net enrolment has increased from 84 percent in 2000 to 95 percent in 2010 for primary schools and from 14 percent in 2000 to 32 percent in 2010 for lower secondary schools. To ensure access to education for all children, the Cambodian government has steadily increased the number of schools in the country. In 2000, there were only 5,468 primary and 367 lower secondary schools, but in 2010 there were 6,565 primary and 1,122 lower secondary schools. Despite these striking improvements and achievements in Cambodia's educational system, significant challenges persist related to access and to quality, as well as institutional capacity development. This is particularly the case for those residing in remote and rural areas, and those marginalized by poverty, ethnicity, and/or gender. Among other challenges, a shortage of teachers is a chronic issue that Cambodia faces. This shortage has resulted in inadequate access to education, especially for children who live in remote areas of the country. The host government has attempted to address this issue by sending 95 percent of newly graduated teachers to teach in remote areas. Despite these efforts, teachers often do not stay in these remote areas to teach, because living conditions are typically much harsher than in urban areas. While the Ministry Performance Evaluation on Improved Basic Education in Cambodia (IBEC) p. 16 recruits around 5,000 new teachers annually, the teacher shortage continues to be a big gap and newly graduated teachers are only enough to replace those who have retired and those who have left their jobs. Another major challenge is the lack of classrooms which needs to be addressed in order to accommodate an increasing number of student enrolments. In addition to the lack of classrooms, many existing schools are also poorly equipped. Unfortunately, schools are not receiving adequate resources from the government and are thereby, unable to adequately address these challenges. In 2008, 18.10 percent of the national budget was allocated to education. This figure decreased to 15.92 percent in 2012, approximately 75 percent of which is allocated for teacher salaries. Despite the low budget input from the host government, the Ministry is supported by a number of donors namely: Asian Development Bank (ADB), World Bank (WB), UNESCO, UNICEF, European Union, Swedish International Development Cooperation Agency, Japanese International Cooperation Agency, Korea International Cooperation Agency, and others. These development partners support the Ministry through loans and grants. More specifically, ADB and WB provide loans to the Ministry; where as other donors primarily provide grants. In addition, Cambodia was qualified in April 2008 to receive $57.4 million Fast Track Initiative (FTI) Funding (2008-2012) this level of support has allowed Cambodia to work towards achieving its Education for All (EFA) goal by 2015. The country is currently applying for funding through the Global Partnership for Education (GPE), and will soon receive another $38.5 million to continue its efforts toward achieving its EFA goal. The development hypothesis underlying the IBEC project is that a more relevant curriculum, combined with better trained teachers and school managers would encourage more students to attend lower secondary school and to stay long enough to complete the lower secondary cycle. Improved physical facilities (wells for drinking water, sanitary latrines, life skills laboratories, etc.) were also believed to play a positive role in inducing students to attend and remain in school. In addition, a limited number of scholarships would directly help vulnerable youth obtain a quality education. In total, these improvements were to create a better educated youth population who will have a knowledge base more relevant to the economic needs of Cambodia, with skills more compatible with the demands of the local job market. Performance Evaluation on Improved Basic Education in Cambodia (IBEC) p. 17 EVALUATION METHODS & LIMITATIONS The data sought for this evaluation included: 1. Quantitative data sent by target schools to the IBEC project, mainly in the form of statistics on enrollment, dropout, number of teachers and so on. Schools are accustomed to collecting this sort of statistical information annually for the national EMIS publications of the Ministry. 2. Quantitative and qualitative data collected by IBEC project staff from target schools on their monitoring and assessments over the four years of implementation. 3. Evaluation team interviews with stakeholders (Ministry of Education Youth and Sport; IBEC project management and staff; IBEC project implementing partners; School directors, staff, teachers and students at target schools; and Commune officials, village officials and parents in target school areas.) Analysis of the quantitative data related to accountability included comparison of current rates to baseline rates for the indicators examined. Where feasible, the EMIS rates (especially from the target provinces) were also included in the comparisons. Interviews with key stakeholders in MoEYS and with key IBEC project staff and with key persons in its implementing partner organizations were useful in gaining a context for the project as it began to wind down to a conclusion, and invariably included concerns about the future. Interviews at target schools and communes and villages were limited by time and budget and by the distances between target locations. The impressions we gathered at this level often included tours of the project accomplishments, especially computer rooms, new school buildings and water and sanitation systems. (See Annex III for interview guides) Quantitative data that IBEC had received from schools was available covering the four years of project implementation. It had never been compiled into a single cumulative database, nor analyzed at a meta level before this evaluation. Summary statements about the findings of this data had been made in Annual and Quarterly Reports, but for many performance indicators we could not find the actual data on which the statements of findings were based. Accordingly, we could not evaluate the accuracy, validity or representativeness of those findings A baseline for many of the project quantitative indicators was provided, against which to compare current rates. However, for a majority of the quantitative indicators, the baseline changed from year to year, and had evidently been adjusted to account for schools that had been added or removed from the project. Neither the data on which the baseline had originally been constructed nor data supporting any baseline modifications during project implementation were available to the evaluation. An apparently outstanding model for accurate and complete record keeping can be found in the IBEC project in the Finance Department. In the finance section of the Project Implementation Center in Kampong Cham, there is an entire wall of bookcases filled with carefully labeled file folders holding a comprehensive account of the detailed financial transactions over the first four years of project Performance Evaluation on Improved Basic Education in Cambodia (IBEC) p. 18 implementation. This is what we would expect, as finances are subject to audit, and any shortcomings in record keeping would result in failing the audit. The IBEC project monitoring and evaluation data were scattered among a variety of printed and electronic sources, and held by various individuals with component responsibilities. We found that annual data from schools were available but had not been consolidated, reviewed or summarized on a cumulative basis from year to year. We were able to compile that data into a database covering the four years of project implementation. With data in that form, we were able to conduct an analysis of access, quality and relevance, as provided below. We found that data on IBEC monitoring and evaluation assessments of many of the project performance indicators were either lacking, or only partially substantiated the assertions of progress on the indicators found in annual reports. In many cases the assessment tool was not available. Often there were no details on the sampling method, nor a list of schools actually participating in a monitoring exercise, nor the original data generated by the monitoring. Moreover, the design of the Monitoring and Evaluation Framework were in parts inconsistent and with several shortcomings. The Project Indicators were of varying quality and relevance. The Project Indicators were activities and outputs and gave no actual indications of goal achievements. In addition several of the indicators were almost identical. The Result framework should follow a consistent and transparent design. Figure 1 Verification of Data on IBEC Project Indicators Inputs •What resources are needed/used? Activities • What is/should be done? Outputs •What is produced or delivered? Outcomes •What do the project wish to achieve? Impact •What are desired the long-term changes/effects ? Indicators •Verifies goal achievement at outcome/impact level 80.00% 50.00% 37.50% 35.00% 0.00% 10.00% 20.00% 30.00% 40.00% 50.00% 60.00% 70.00% 80.00% 90.00% COMPONENT 1: Local NGO and Government Capacity Building & Advocacy COMPONENT 2: More Equitable School Access COMPONENT 3: Improved School Management and Community Engagement COMPONENT 4: Improved Educational Relevance Verification of Data on IBEC Project Indicators (Year 4) % Verified % Unclear/ Partial % Not Verified Performance Evaluation on Improved Basic Education in Cambodia (IBEC) p. 19 The chart above shows an analysis of the availability and our verification of valid project generated data on performance indicators from the most recent full year of IBEC project implementation, 2012-13. Data for earlier years of project implementation was so difficult to access or incomplete that no analysis was attempted. For Component 1, most data seems to be available (on 80% of the indicators under the component) and only a few questions were raised by partial or unclear data files. For Component 2, only 50% of the indicators had satisfactory data available that verified the achievement of the indicator. In Components 3 and 4, over 60% of indicators the data was absent, or could not be located, or there was partial or incomplete data that prevented verification that the data substantiated the claim of progress on the indicators. To maximize validity and accuracy of this valuation, the data from schools used in the evaluation was spot checked for accuracy against the original documentation from schools. The data found in IBEC was also checked against the data kept at the schools we visited. Data from schools was compared with baseline figures and with EMIS, and displayed in charts to identify trends. For indicators on which data were unavailable, like completion rates, we were unable to include that indicator in our evaluation. We based the evaluation on the data that was available and reasonably complete, and did not attempt to generate independently additional data from IBEC target schools. To do that would have required large scale survey methods that were far beyond the resources or time allocated to this performance evaluation. However, such an independent impact assessment to generate data that is not available in IBEC might be worthwhile if adequate resources can be mobilized. Performance Evaluation on Improved Basic Education in Cambodia (IBEC) p. 20 FINDINGS, CONCLUSIONS & RECOMMENDATIONS FINDINGS ACCESS2 Enrollment In Year 1 of project implementation, 2009-10, Siem Reap primary and secondary schools had not yet joined IBEC. Accordingly, changes in total target province enrollments at IBEC are meaningful only from Year 2 to Year 4 when all three provinces participated in the project. Figure 2 Secondary Enrollment in Target Provinces From 2010-11 to 2012-13, EMIS enrollment for all secondary schools in the three target provinces declined 3.2% from 169,777 to 164,308 students. By comparison, in that same period, IBEC target secondary school enrollments declined 8% from 46,344 to 42,608 students. It is instructive to consider enrollment in each of the IBEC target provinces separately. In this case we can observe changes in enrollment from when the province joined IBEC to the present. In the tables below, data for both the IBEC target schools in each province and the EMIS rates for total enrollments in the province are given. 2 Data from IBEC target schools were available on enrollment, dropout and repetition. Data on completion rates were not available. 42,222 46,344 41,391 42,608 168,032 169,777 165,296 164,308 0 50,000 100,000 150,000 200,000 2009-10 2010-11 2011-12 2012-13 Target Province Secondary School Enrollment (Total and IBEC) IBEC SS Student Enrollment EMIS SS Enrollment in Target Provinces The data shows that enrollment at IBEC target schools, in one province, Kratie, increased over the four years of implementation examined. The other two provinces, Kampong Cham and Siem Reap registered declines in enrollment at target schools. On this measure, the project had mixed results in increasing access in the targeted schools. Performance Evaluation on Improved Basic Education in Cambodia (IBEC) p. 21 IBEC Target schools in Kratie showed enrollment increases at both primary and secondary levels. Target schools in the other two provinces registered declines in both primary and secondary enrollments. (Annex X contains all the tables and charts relating to primary school findings). IBEC Secondary Level Enrollment and Total Secondary Level Enrollment in the Three Target Provinces Kampong Cham Siem Reap Kratie IBEC Enrollment EMIS Total Enrollment IBEC Enrollment EMIS Total Enrollment IBEC Enrollment EMIS Total Enrollment 2009-10 25,859 96,360 54,019 5,760 17,653 2010-11 24,542 97,457 15,038 55,297 6,764 17,023 2011-12 22,097 94,752 12,948 53,843 6,346 16,701 2012-13 23,099 94,144 13,291 54,257 6,218 15,907 % Change -10.67% -2.30% -11.62% 0.44% 7.37% -9.89% Figure 3 Secondary Enrollment over Four Years At IBEC target secondary schools in Kratie, enrollment increased 7.4%, from 5760 in Year 1 to 6218 in Year 4. By comparison, EMIS total enrollment in secondary schools in Kratie declined almost 10%. IBEC in Kratie showed much greater success in providing increased access to students at both primary and secondary levels than either Kampong Cham or Siem Reap. This finding could be related to the different demographic, social and economic conditions that exist in each province. For example development of infrastructure, expansion of local industry and agriculture, changing employment opportunities, flooding and other climatic patterns are all factors that may give rise to new migration and demographic consequences. A performance evaluation like this may be able to identify trends, but in￾depth analysis of potential causes would require detailed surveys of such environmental factors in each province, which was beyond the scope of this evaluation. Several School Directors interviewed at Provincial level did point to the fact that many of their students at secondary level come from remote areas and a significant proportion consists of ethnic minority students, who have to travel great distances to reach the school. This distance affects enrollment and establishment of student dormitories near the school could have a positive effect on student enrollment. A School Director in Kratie explained that some Pnong and Kroal ethnic minority students from a remote area were able to study at his secondary school because IBEC support enabled them to rent a house together near the school. They cook and eat together and help one another with their studies. If there were a dormitory, he said, probably many more ethnic minority students would come to study here. Flow Rates Project Level. The charts below show the dropout and repetition rates over time at IBEC secondary schools. The dashed line represents the project baseline, against which the current project dropout each year is compared. The assemblage of IBEC project schools will no longer be a grouping once the project ends. Accordingly, comparisons of flow rates with the official EMIS statistics will be made below, for each province. Any lessons learned from the IBEC trends may be useful and relevant to these enduring province structures of the Ministry of Education, Youth and Sport. Dropout rate and repetition rate reductions over the four years of IBEC project implementation generally confirm increased access to education for children in target schools for all pupils and especially for girls. Performance Evaluation on Improved Basic Education in Cambodia (IBEC) p. 22 Figure 4 IBEC Dropout Rate Secondary dropout hovered below the baseline in 2010-11 but then rose in 2012-13 above the baseline. Female dropout has stayed persistently above the baseline for four years (see Annex VII figure 7.1). The steep rise in dropout in 2012-13 is worrying, and may be related to the wind-down of scholarships and other project interventions that helped reduce dropout rates in earlier years of implementation. Figure 5 IBEC Repetition Rate Repetition rates in IBEC secondary schools have shown declines in the last two years. Repetition rates at the project level for all students and for females is now well below the baseline rate, which attests to the success of the IBEC project in improving access and orderly progress through the grades at the secondary level. (For female repetition rate see Annex VII figure 7.2). Province Level. For each target province, the total rates (from EMIS), and IBEC province baseline and current IBEC rates for students at the target schools are given in the charts below. (The charts on dropout and repetition rates at the secondary level for female students at target schools are placed in Annex VII.) Kampong Cham It is instructive to consider dropout and repetition in each of the IBEC target provinces separately. Kampong Cham illustrates the trends which are mirrored in the data from the other two provinces 12.50% 13.00% 13.50% 14.00% 14.50% 15.00% 15.50% 16.00% 16.50% 2009-10 2010-11 2011-12 2012-13 IBEC Secondary Schools Total Project Dropout Rate Baseline Dropout Current Dropout 0.00% 2.00% 4.00% 6.00% 8.00% 2009-10 2010-11 2011-12 2012-13 IBEC Secondary Schools Total Repetition Rate Baseline Repetition Current Repetition Performance Evaluation on Improved Basic Education in Cambodia (IBEC) p. 23 Figure 6 Kampong Cham Dropout The rates of dropout in Kampong Cham secondary schools are very stable. They hover about the IBEC baseline and fall consistently below the EMIS rates for the province. A school director said that IBEC support for a guidance counsellor for female students was very significant in reducing their dropout. But, he said that with the end of IBEC support, that service would end because the school did not have funds to pay the counsellor to make the visits to girls' homes in the villages after her teaching duties. Figure 7 Kampong Cham Repetition Repetition rates for IBEC secondary schools in Kampong Cham are falling, and for the past two years have been below the project baseline. This is significant because repetition is mainly a school decision (while dropout is largely a family decision). Reducing repetition means that schools are reducing absences (the main criterion for repetition decisions). Reduced absences and reduced repetition reflects greater access. These IBEC repetition rates are, however, significantly higher than the EMIS repetition rates. This is the same profile that we see in all three IBEC provinces. There may be systematic differences in the way EMIS and IBEC has calculated repetition rates. Kratie 0.00% 5.00% 10.00% 15.00% 20.00% 25.00% 2009-10 2010-11 2011-12 2012-13 Kampong Cham IBEC Secondary Total Dropout Baseline Dropout Current Dropout EMIS Dropout 0.00% 1.00% 2.00% 3.00% 4.00% 5.00% 6.00% 7.00% 2009-10 2010-11 2011-12 2012-13 Kampong Cham IBEC Secondary Total Repetition Baseline Repetition Current Repetition EMIS Repetition Performance Evaluation on Improved Basic Education in Cambodia (IBEC) p. 24 Figure 8 Kratie Dropout The secondary schools in Kratie show a consistent pattern of dropout rates below the baseline for all years of project implementation. These IBEC rates are also somewhat below the EMIS rates for secondary schools in the province. Figure 9 Kratie Repetition Repetition rates in Kratie at the secondary level have dropped steeply from the IBEC baseline rates. The current rates for repetition are above the EMIS rates for repetition in secondary schools in the province, but those EMIS rates are declining rather slowly. The steep decline in IBEC schools suggests an intentional effort at IBEC schools in Kratie to enable students to complete the year's school work and promote to the next grade. Siem Reap 0.00% 5.00% 10.00% 15.00% 20.00% 25.00% 2009-10 2010-11 2011-12 2012-13 Kratie IBEC Secondary Total Dropout Baseline Dropout Current Dropout EMIS Dropout 0.00% 2.00% 4.00% 6.00% 8.00% 2009-10 2010-11 2011-12 2012-13 Kratie IBEC Secondary Total Repetition Baseline Repetition Current Repetition EMIS Repetition Performance Evaluation on Improved Basic Education in Cambodia (IBEC) p. 25 Figure 10 Siem Reap Dropout Dropout in IBEC Siem Reap secondary schools are high, and above the IBEC baseline for both total and female students. These IBEC rates are very close to the EMIS rates for the province. Figure 11 Siem Reap Repetition Secondary level repetition rates in Siem Reap IBEC schools are consistently higher than the IBEC baseline rates, and much higher than the EMIS rates for the province. The repetition rates have declined slightly since Siem Reap joined the IBEC project, but the province stands out in the project for this high rate of secondary repetition. Since repetition of a grade is usually due to large number of absences during the year, it would be worthwhile asking if secondary students are absent at certain times during the year, (and why these students return to school in the following year to repeat the grade). Perhaps adjustment in the school calendar for some province schools to account for seasonal work demands or regular family migration (transhumance) would assure fewer student absences and lower repetition rates. A number of School Directors and one PEO also noted that seasonal labor and student migration were common causes of absenteeism. According to them, some smaller schools are able to actively track vulnerable students and migration to reduce repetition. However, most secondary schools don’t have access to such detailed information since their coverage areas are often too large. 0.00% 5.00% 10.00% 15.00% 20.00% 25.00% 2009-10 2010-11 2011-12 2012-13 Siem Reap IBEC Secondary Total Dropout Baseline Dropout Current Dropout EMIS Dropout 0.00% 2.00% 4.00% 6.00% 8.00% 10.00% 2009-10 2010-11 2011-12 2012-13 Siem Reap IBEC Secondary Total Repetition Baseline Repetition Current Repetition EMIS Repetition Performance Evaluation on Improved Basic Education in Cambodia (IBEC) p. 26 School Director in Kratie: “Migration is a primary concern. There are two types of migration: those who move temporary and those who move permanently. Also we are seeing students dropping out because they get married typically around 16-17 years old and some even 21-22 years old. When the school first started in 2003 – 2004, some students where only a couple of years younger than the school director”. Gender Parity Index The IBEC project supports the Education For All goal progressively to eliminate gender disparities in primary and secondary education and achieve gender parity in education. The Gender Parity Index is obtained by dividing the rate for an indicator (like gross enrollment) for females by the rate for the indicator for males. A GPI value equal to 1.0 shows equality for males and females for that indicator. A GPI value less than 1 indicates difference in favor of males. A GPI value of greater than 1.0 indicates a difference in favor of females. Educators consider that there is no gender disparity if the GPI ranges between 0.97 and 1.03. Figure 12 IBEC Gender Parity Index for Enrollment In IBEC secondary schools in the three target provinces, the current GPI for enrollment is above the baseline for all years of project implementation, and is actually rising. The rate now shows a rate outside the range for parity and decidedly in favor of female enrollment over male enrollment, as the GPI is above 1.10. This is contrast to the GPI at the primary level, which reveals a preponderance of boys at that level. (See Annex X, figure 10.19). The increased enrollment of females at the secondary contributes to the finding of increased female dropout rate for females noted above. IBEC only monitored GPI at schools where scholarships were provided. The scholarship awards at the secondary level were in the proportion of 80% for females and 20% for males. The elevated GPI suggests the success of the scholarships in improving female access to secondary education. QUALITY 0.95 1.00 1.05 1.10 1.15 2009-10 2010-11 2011-12 2012-13 IBEC Secondary Schools Gender Parity Index GPI at Baseline Current GPI One of the key strategic decisions the IBEC project made to improve the quality of education at both the primary and secondary levels was to address the serious problem of insufficient teachers for the country's schools. The evident reduction of the pupil teacher ratio in IBEC schools attests to the increased quality of education in target schools Performance Evaluation on Improved Basic Education in Cambodia (IBEC) p. 27 IBEC aimed to increase the capacity of schools to deliver quality education. One of the ways IBEC did this was by providing improved teaching materials and resources for schools and by providing a variety of management and capacity building trainings. Nearly every school director noted the importance of these materials and opportunities for in-service training and workshops and school visits as the key improvements in quality brought by the project. Another important contribution to increased capacity for improving quality was the support IBEC provided for School Support Committees. The participation of these committees was essential to the formulation of an adequate school plan, which in turn was the key element in the release of IBEC grant funds for the school projects that were proposed at the school. Nearly every stakeholder we interviewed said that IBEC support of their school plans had made a large contribution to improved quality of education at their schools. IBEC also approached the capacity of schools to deliver quality education by addressing the vexing problem of teacher shortages. IBEC provided community teachers, who are qualified, but not Ministry employees and are paid by the project. Another IBEC strategy has been to provide support for students from particularly remote or impoverished areas, who are attending Teacher Training Colleges. The project may then arrange with the Ministry that these teachers will eventually be assigned to work in their home areas, where the need is great. One source for empirical data on quality is, accordingly, the Pupil-Teacher Ratio over the years of project implementation. In developing the data on IBEC schools to produce the following charts for each province we observed a distinction between urban, rural and remote schools (which is included in the school identification data in EMIS). The reason to include this distinction is the perennial problem that teachers prefer to work in Interviewed stakeholders’ perception on quality An important question for this evaluation was whether the quality of basic education in the target provinces was improved through a combination of school-level and provincial level IBEC activities. Understandably, quality is a complex and multifaceted concept in any context and can be assessed from several angles. In addition to analyzing quantitative data from schools, the evaluation team asked several questions during stakeholder interviews in order to gain additional anecdotal information on the quality of education and any perceived changes in the target area. We received a broad range of answers covering virtually all aspects of quality of basic education: from access to library books and computer labs to applying student-centered teaching methods to new school management practices. Many individual school directors and teachers tended to focus first and foremost on the material aspects of education quality first, as they understood it: latrines, libraries, computers, study and teaching materials, water stations, etc. In some cases, school directors and government officials on both provincial and district levels praised leadership training provided by IBEC staff that helped improve the quality of managing school activities, including preparing and monitoring school budgets, developing and implementing school plans and increasing team work effectiveness. One PEO pointed out that one of the major quality aspects of IBEC was a well-rounded approach to education, as defined by the MoEYS, emphasizing student￾centered learning (particularly Student Councils) and capacity building activities for school teachers and management. School directors generally emphasized the aspect of training that had to do with developing school plans which were key to obtaining increased funding for the teachers and activities of their school. These school plans were supposed to be based on an inclusive process, so there was frequent mention of community or school support committee participation. Regular monitoring of project activities by IBEC staff was encouraged and in some cases even requested by school directors and seen as an important factor in ensuring higher quality of implementing project activities and providing education in general. Performance Evaluation on Improved Basic Education in Cambodia (IBEC) p. 28 urban areas and they tend to avoid assignment to remote areas, where life is much more difficult. Accordingly, the pupil-teacher ratio in remote areas is generally higher than in urban areas. IBEC has successfully supported special efforts to address the teacher shortage problem in the underserved rural and remote areas in their target provinces and accordingly target schools show reduced pupil-teacher ratios and enhanced quality. Kampong Cham Figure 13 Kampong Cham Pupil-Teacher Ratio At the secondary level in Kampong Cham, IBEC schools are well below the EMIS rates for the province, but there is also a distinct difference growing among remote, rural and urban secondary schools in the province. As expected, remote schools have higher pupil-teacher ratio than urban schools, with rural schools in between. Kratie The secondary level in Kratie shows that adjustments were made in IBEC schools in urban areas in 2011- 12 to deal with a sudden rise in pupil-teacher ratio. Figure 14 Kratie Pupil-Teacher Ratio Evidently teachers from urban schools were assigned to remote schools at the end of 2010-11which resulted in a spike in urban pupil-teacher ratio and a decline in remote pupil-teacher ratio in 2011-12. An 0 10 20 30 40 50 2009-2010 2010-2011 2011-2012 2012-2013 Pupil-Teacher Ratio Kampong Cham IBEC Secondary Remote Rural Urban EMIS 0 10 20 30 40 50 2009-2010 2010-2011 2011-2012 2012-2013 Pupil-Teacher Ratio Kratie IBEC Secondary Remote Rural Urban EMIS Performance Evaluation on Improved Basic Education in Cambodia (IBEC) p. 29 infusion of new teachers may have helped restore the IBEC urban schools to a rate below EMIS and close to rural and remote rates in 2012-13. Kratie is receiving large numbers of migrants and new settlers from other provinces, and IBEC has helped build many temporary school structures in the province to cope with the demographic shifts. The pupil-teacher ratio is likely to fluctuate as provincial policy adjusts to the increasing need for quality schools and more teachers. Student Council members in Kampong Cham: “There are not enough teachers. There are only 21 teachers in the school. Some of teachers are from Kampong Cham, but some of them come from different provinces”. School Director in Kratie: “IBEC has played an important role in improving the teaching quality. I think if we compare some IBEC schools to non IBEC schools there will be differences in teaching quality, study quality, environment, teaching material and human resources. Also we the Directors get trainings in management and school development planning”. Siem Reap In Siem Reap schools, the distinction between urban and other schools is most evident. At the primary level, IBEC schools are all below the EMIS pupil-teacher ratio. There is also a convergence of the urban, rural and remote rates, showing the success of the project helping to balance the distribution of teachers at the target primary schools (see Annex X, figure 10.25). At the secondary level the reduced pupil-teacher ratio for the IBEC urban schools is pronounced and persistent and reflects a decided preference teachers have for working in town. The pupil-teacher ratio for IBEC rural and remote secondary schools hovers at or slightly below the general province rate represented by the EMIS line. Remote schools in Siem Reap seem to have lost teachers (or gained students) in Year 4, as the pupil-teacher Ratio exceeded the EMIS rate in 2012-13. This is a trend that bears watching. Figure 15 Siem Reap Pupil-Teacher Ratio 0 10 20 30 40 50 2009-2010 2010-2011 2011-2012 2012-2013 Pupil-Teacher Ratio Siem Reap IBEC Secondary Remote Rural Urban EMIS Performance Evaluation on Improved Basic Education in Cambodia (IBEC) p. 30 MoEYS Director: “The IBEC contribution to quality has been to link theory and practice in LS classes for Lower Secondary school students. Students in LS develop their own projects, learn to solve problems in their community and improve their life”. RELEVANCE The sense of relevance adopted here is an imperative to provide a basic education to students that will enable them to enter the contemporary workforce in Cambodia. The IBEC records were analyzed to establish which Life Skills activities were selected at target schools over four years of project implementation and to ascertain progress made. The menu or panel of Life Skills options available to schools changed over the project life, as the Life Skills curriculum was finalized. The names of the activities also changed, as the modules were piloted and refined. Figure 16 Life Skills Activities At the secondary level the progress of Life Skills implementation is particularly striking. The total number of activities implemented at IBEC primary schools has grown from 88 in 2009-10 to a remarkable 836 in 2012-13. The chart above also shows the distribution of Life Skills activities over the three provinces, which is related to the number of target schools in each province. Performance Evaluation on Improved Basic Education in Cambodia (IBEC) p. 31 Sustainability of IBEC Life Skills programs Sustainability of a curriculum (that is the likelihood that it will persist after project support has ended), may be positively affected by advantages perceived of the curriculum and negatively affected by challenges noted for the curriculum. The evaluation has summarized some anecdotes about sustainability of the Life Skills curriculum according to whether the opinions reflect the perception of an advantage or the sense of a challenge by interviewed informants. Primary Level Informants noted the following advantages of the Life Skills curriculum that are likely to promote the sustainability of the interventions, and improve the economic productivity of students in the future:  School choice of activities assured that the modules selected were locally appropriate.  Gardening continues the tradition of having pupils care for the school grounds. This tradition goes back to the pre-French era when primary schools were located on pagoda grounds and children were responsible for helping maintain the grounds.  Life Skills products can be consumed by students or sold. Informants suggested that the sustainability of the Life Skills intervention faced the following challenges:  There is a shortage of skilled or trained Life Skills teachers.  Additional funds are needed for materials used in Life Skills activities.  Extra pay is needed for teachers who extend their workday supervising LS activities.  Schools have difficulty finding the required two hours for LS in an already crowded school / schedule.  The harvests of gardening activities may only be available after the school year ends.  LS activities tend to be gendered--boys doing construction, girls doing cooking. Secondary Level A highly appreciated Life Skills innovation that IBEC brought to some target secondary schools was computer education and computer labs. Informants noted the following key challenges to the sustainability of computers in schools:  The computer lab takes up a whole dedicated classroom in an already crowded school. Interviewed stakeholders’ perception on relevance Overall, relevance seemed to be a difficult concept for our respondents to evaluate; often, if was inseparable from the quality of education, and the evaluation team received very similar answers, especially in a wider context of improvements to the infrastructure of their schools. Nonetheless, many informants, particularly those who had received IBEC training, understood the special use of the term "relevance" in the project context to refer to the role of the Life Skills (LS) introduced in the curriculum by IBEC. Frequently they would tell the evaluation team how students learned skills in various LS classes that they consider useful in real life and could even be used to generate additional income for their families, such as mushroom and vegetable growing. Some students reportedly expressed interest in taking more LS classes related to agriculture in order to be able to better help their families. More socially oriented classes, such as Safe Migration and Drug and Alcohol Abuse Prevention, were also generally regarded as highly relevant knowledge for the students. Higher level education officials, especially PEOs, stipulated that IBEC stood out among similar projects due to its focus on various practical skills for everyday life, from vegetable growing techniques to problem solving. The PEO in Siem Reap pointed out that, based on his observations during school visits, IBEC had been especially successful in enhancing confidence and teamwork skills among his students. Performance Evaluation on Improved Basic Education in Cambodia (IBEC) p. 32  There is a shortage of skilled computer teachers and technicians to troubleshoot the network connections between thin client terminals and the server computer.  The solar electricity systems provide free electricity but the batteries are beginning to fail.  The server computers are aging and need maintenance.  Once computers are available, students desire internet access which is expensive and slow (by smart phone modem).  Computers tend to be used more by males than females. Informants noted the following advantages of the IBEC computer intervention that are likely to promote the sustainability of the advances in information technology at these schools and to increase the employability of graduating students who have participated in these life skills activities:  Some student computer clubs raise their own money for electricity.  Some student computer clubs are able to maintain the computers at their school.  Teachers also use the computers for developing teaching materials.  LSS students use the Khmer keyboard, while USS students use the English keyboard. Computers can assist language proficiency. It was highlighted by a school director of a Kampong Cham secondary school that the computer club students have opened a small business in town repairing computers. These enterprising students also help the younger students learn computer skills. He remarked that students-helping-other-students was often the best form of education in a new field like computer technology. CIVIL SOCIETY STRENGTHENING The World Education International project, IBEC, working in the Cambodian Education Sector, was designed to replicate the success of USAID programming in the Health Sector. The design included building the capacity of local IBEC partners involved in education. The IBEC project capacity building efforts succeeded in helping KAPE, IBEC’s biggest local partner, and BSDA, to achieve certification of compliance with the Code of Ethical Principles and Minimum Standards for NGOs from the Committee for Cooperation in Cambodia (CCC). CCC has a well-regarded accreditation process of great rigor. A three-step process for certification is used involving (i) document review and investigation; (ii) verification process; and (iii) external review and certification. IBEC Local Partners The implementing partners of IBEC identified for this evaluation were WEI and KAPE and several other smaller NGOs outlined below. Kampuchean Action for Primary Education (KAPE) is the largest local NGO in the education sector in Cambodia. The organization implements projects and research at all levels of the education sector across eleven Cambodian provinces. KAPE's strength is that has long experience implementing projects in the Cambodian education sector, but its weakness is in documenting systematically the lessons learned and best practices developed in its projects. Buddhist Social Development Association (BSDA) is led by monks and former monks and lay￾people. Its mission, vision and values are inspired by Buddhist philosophy. BSDA started out in 2005 focusing on English and Computer classes but its activities now include education and vocational training for Orphans and Vulnerable Children (OVC), community and democracy development, as well as work with drug users and HIV/AIDS infected people. BSDA's strength is to include an ethical dimension in all its projects, but its weakness is attempting too broad a range of activities leading to a lack of depth and focus in any sector. Performance Evaluation on Improved Basic Education in Cambodia (IBEC) p. 33 Women and Children’s Rights Development (WCRD) was founded in 1998 by a program officer at Save the Children UK. WCRD is a small organization of six persons and works only in Kratie. Its role as a partner for IBEC included administering and monitoring of scholarship assistance distribution to IBEC schools. The organization has faced management challenges during its work with IBEC, including the loss of a year's documentation. Its strength is a distinct focus on education in Kratie, where it has extensive experience. Economic Development Association (EDA) is a "micro" local NGO in Kampong Cham consisting of three persons. It has assisted KAPE by following up scholarship students at IBEC schools and helping them at home with Life Skills instruction, especially in the area of sewing and weaving. EDA's strength is providing Life Skills instruction in agriculture, emphasizing gardening and chicken raising. Its weakness is lack of core funding and uncertain prospects for staff and project development. NGO Education Partnership (NEP) is an umbrella organization established in 2002. NEP now has 131 NGO members from all parts of Cambodia working in the education sector. NEP's strengths are promoting dialogue and cooperation among key stakeholders to improve the quality and accessibility of education in Cambodia and representing the views of its civil society members to Government and Development Partners. NEP's weakness is lack of analytical skills needed to challenge the education paradigm and to consider critically the place of education in Cambodian society. Several informants including KAPE consider "civic awareness and empowerment" may be a controversial or sensitive topic in the current Cambodian political context. Focusing on a “re-branding” to “Education Support Networking" would capture the approach of fostering links between schools, NGOs and government and fostering links between students, teachers and the school director to improve the quality of the schools. INNOVATIONS, BEST PRACTICES AND LESSONS LEARNED Field observations on IBEC partners WEI apparently assumed management responsibilities in IBEC and KAPE assumed local implementation responsibilities. However, the evaluation team could not distinguish WEI and KAPE personnel at the IBEC office in Kampong Cham, and our overall impression was that even for many of the IBEC staff, the boundary between WEI and KAPE in the project was often not distinct. It is also interesting to point out that several of the Commune Council members we interviewed claimed to be unfamiliar with IBEC, but knew KAPE and sometimes other implementing partners well and were nevertheless aware of the project activities in their communes. Overall, the DEOs gave very positive experience with the project coordination and cooperation of different stakeholders, good feedback on the programs. For example in Siem Reap, KAPE project staff cooperated well with local authorities, according to one of the DEOs who reported they visited the community and held meetings twice a year, including the parents to help them understand the objectives of the project. Some informants however, voiced criticisms of IBEC that may hint at some weaknesses of the implementing partners. It should be noted that these remarks are wholly anecdotal, with no claim to representativeness, and should be treated as such. PEOs and DEOs from both Kampong Cham and Siem Reap noted difficulties in communication between IBEC and the local education authorities, mainly DEOs. Some IBEC project officers were also deemed to be young, resulting in occasional perceived communication issues with much older teachers and school directors. School directors in both Kampong Cham and Siem Reap mentioned delays in receiving funds from the project for school plans, as well as some miscommunications in monitoring project activities. Without further research, it is impossible to assess whether these issues indicate systemic problems or simply reflect incidental matters of concern to the informants we happened to encounter. Performance Evaluation on Improved Basic Education in Cambodia (IBEC) p. 34 DEVELOPMENT READINESS TIERS Probably the most significant innovation of the IBEC project is the classification of its target schools into "development readiness" tiers. Tier 1 are well managed advanced schools with excellent school governance. Tier 2 schools are satisfactory in regard to management. Tier 3 schools are at a poor stage of development. Higher tier schools receive greater funding through their School Development Plan, as they are able to absorb more external aid more effectively. Lower tier schools receive greater capacity building assistance to enable them to move up in tier level. IBEC concludes that the innovation of adjusting aid to readiness and performance is an intervention that helps promote improved management at target schools. MoEYS officials agree that this "results based" budgeting and grants management device correctly places the responsibility for effectiveness on the School Director. In order to classify schools into appropriate developmental readiness tiers, project personnel compile performance and management data from a wide range of sources, including (i) a spot checks which examine Financial Accountability and Governance, as it concerns the administration of school grants and Quality of Activity Implementation, which relates to IBEC activity completion. (ii) anecdotal assessments conducted in collaboration with Provincial Working Group members, (iii) internal rankings of schools conducted by Provincial Offices of Education, (iv) classroom observation results, and (v) hands-on surveys of schools that were borderline in their initial assessments. Mobility within the tier classification enabled IBEC to make important statements about school management or governance. Some schools were upgraded in tier as a reward for their performance. Schools with high levels of developmental readiness receive more resources given their ability to use such resources effectively Some schools were downgraded in tier. The most common reason that schools were downgraded appeared to be linked with a change in leadership at the school, usually in the form of a new director who had less of a professional commitment to improving the school than the previous school director. IBEC provided Tier 3 schools with further support to help them learn how to improve their financial accountability as well as their general governance. The project also held provincial teams accountable for the performance of their schools. In this regard IBEC asked Provincial Office of Education members to play a critical role in mentoring their Ministry staff (the school directors) in order to be compliant with IBEC requirements, and again addressing management and governance issues at the provincial level. IBEC also asked some schools to leave the project. Schools phased out were judged to have made little progress, or actually regressed due to lack of motivation and interest in development among the school directors and teachers. Some schools were eliminated due to their failure to attend to agreed responsibilities, failure to expend funds for school projects, or for "egregious mismanagement." Performance Evaluation on Improved Basic Education in Cambodia (IBEC) p. 35 This practice enabled the project to retain schools that complied with IBEC requirements, and reject schools that did not. This selection process would seem to promote more homogeneity among target schools, and eliminate the worst problem cases that could not fulfill IBEC requirements. The tier mobility mechanism provides a way to analyze this important IBEC innovation that promises more effective project implementation (the distribution of grants) and promises improvements in school governance (the management of funds and activities). Tier Change at Primary and Secondary The following chart shows that for IBEC target schools in all three target provinces at the primary level there were relatively high rates of both upgrading and downgrading of tier level over the four years of project implementation. The schools that were "phased out" or eliminated from the project altogether are not included in these numbers, as only schools that changed from one tier to another within IBEC are counted. Figure 17 Tier Change at Primary Schools The high rates of tier level movement were evidently the result of careful assessments of the schools that resulted in decisions to adjust schools upward or downward in tier level over four years of the project. The consequence would have been adjustments in the project resources provided, depending on observed changes in "development readiness." Several school Directors note that this development could be caused 41.82% 40.00% 50.00% 26.36% 36.67% 25.00% 31.82% 23.33% 25.00% 0.00% 10.00% 20.00% 30.00% 40.00% 50.00% 60.00% 70.00% KCM KRA SRP Percent of Target Schools Tier Change from Start-up to Y-4 IBEC Primary Schools Upgrade Downgrade Same One critical opinion on the tier system was given by a District Office of Education officer in Kampong Cham. This official asked "What about schools at tier 10 or 11?" (referring to hypothetical tiers far below the existing system). His point was that there are extremely weak and marginalized schools that he believes rarely come into the view of donors. He estimated that in his district about 25% of schools were "tier 10," and not anywhere close to being able to participate in a project like IBEC. Many of these schools serve families that move seasonally. Children leave school as the rivers begin to rise and their families move to farms on higher ground. The students miss the last several weeks of school. A wider implication of this may be that the tier system, should it be adopted by other donor projects in the future, could shift donor resources away from such extremely weak and marginalized schools in the Cambodian countryside. These schools would need additional targeted support and capacity building, before they can be brought into a tier performance system project such as IBEC. Performance Evaluation on Improved Basic Education in Cambodia (IBEC) p. 36 by “teacher retirement and cronic teacher shortages” in many schools, increasing the workload and leading to a drop in quality. Figure 18 Tier Changes at Secondary Schools In contrast to the situation at the primary level, the IBEC target secondary schools in all target provinces show very high rates of upgrade in tier and very low rates of downgrade over the four years of project implementation. These observations of tier mobility over time raise questions about the real situation at primary and secondary schools and the meaning of "development readiness" at these two levels. In order to learn lessons from this pilot innovation, IBEC should explain what these patterns in upgrading and downgrading of tier at the primary and secondary levels might mean. The following questions might be considered:  Are the differences in tier mobility decisions due to the design of the assessment tool, which may have provided different kinds of information about primary schools than about secondary schools?  Are primary schools inherently more prone to management or governance variability or challenges than secondary schools, giving rise to more tier movement? Can these challenges be specified so that training could be better designed for the actual needs at each level?  Was the project capable of providing the additional capacity building so clearly needed at the primary level for schools that declined in tier, or were phased out?  The evidence suggests that target secondary schools have inherently greater potential for "development readiness" than target primary schools. Is this due to a difference in quality of management and governance in the two levels of schools, or some other reason?  Did secondary schools receive significantly different kind of implementation effort than primary schools, which might account for the trend of upgrading of tier over time at the secondary level, and the greater mix of upgrading and downgrading of tier at the primary level? Another perspective on tier mobility can be obtained by comparing the percent of IBEC secondary schools in each target province that achieved the highest, Tier 1, level of "development readiness" over the four years of IBEC project implementation. 75.51% 85.71% 80.00% 0.00% 20.00% 40.00% 60.00% 80.00% 100.00% KCM KRA SRP Percent of Target Schools Tier Change from Start-up to Y-4 IBEC Secondary Schools Upgrade Downgrade Same Performance Evaluation on Improved Basic Education in Cambodia (IBEC) p. 37 Figure 19 Percent Secondary Schools Reaching Tier 1 In Kampong Cham, the proportion of Tier 1 secondary schools increased over five-fold from 13.64% of schools at start up to 73.47% at Year 4 of IBEC project implementation. In Kratie, the Tier 1 secondary schools increased nine-fold from 9.52% at start-up to 85.71% of schools at Year 4. In Siem Reap, the Tier 1 secondary schools in this province increased over eleven-fold from 6.67% at start-up to 76.67% of schools at Year 4. The achievement of the highest level of "development readiness" in target secondary schools revealed by this analysis would seem to suggest that significant improvements were made in the area of management and governance at these schools and resulted in increased access to resources. But, in order to obtain lessons learned on how this success was accomplished, IBEC would have to explain what evidence or results they used to support their decisions on assigning Tier 1 status to schools and how their interventions had led to those results. There are insights about dealing effectively with Cambodian schools that were utilized by IBEC management in their decision-making, but the insights remain as yet unvoiced. It was noted by the evaluation team that several school directors at the strongest schools receiving IBEC support aspired to ASEAN standards of English language and Computer skills for their students. They were confident that their students were capable of great achievements at University, if they obtained the help needed to access higher education. Although we can discern the patterns in project decision making about tier assignment, the monitoring and evaluation data were not available in IBEC to explain why or how the decisions on tier advancement were taken. Careful consideration of these tier mobility decisions could reveal significant insights about the nature of school governance and its challenges at primary and secondary levels in Cambodian schools and in disparate Cambodian province settings. Tier classification appears to be an attractive method for allocating scarce resources in a rational way. The concept of "development readiness" combines two key concerns of donors, aid effectiveness and good governance, and may be adopted in other major projects in education. A critical analysis of the concept would be concerned with unequal school readiness and consider how decisions about just allocation of assistance in the education sector might be made. This analysis would provide a useful basis on which future interventions could be planned. Another concern was raised by school directors in Siem Reap, which voiced the concern of fund allocation, and the fact that IBEC support came to the cluster not the school. This led to uncertainty about how the money would be divided among the three schools in the cluster, so he was not really able to plan well. He thought that a grant directly to the school would be better, as was done in other provinces. The Evaluation team notes the SIDA are implementing the SIG programme with direct cash transfers to all Primary and secondary schools in Cambodia starting from 2014. Performance Evaluation on Improved Basic Education in Cambodia (IBEC) p. 38 OTHER INNOVATIVE INTERVENTIONS Engaging Community Support for Education Informants gave many examples of traditional forms of community support that have the promise of sustainability. These traditional forms were utilized by their School Support Committee, especially if this committee included village elders and monks. These are the kinds of culturally appropriate fund-raising activities that could be supported and encouraged to benefit local schools. These traditional community contributions might complement the more formal mechanisms of funding allocations by local government.  Bon Pkha (flower offering) is a method of raising cash contributions from villagers. Householders tie cash offerings to a small silver or gold foil-wrapped ritual tree.  The monks may place a collection box in the pagoda for contributions to the school.  The Wat Committee may sponsor a Kathen ceremony (usually in support of the monks) that could include raising funds for the school. (This is a memory of the pre-French tradition in which all primary schools were in pagodas)  Wealthy private donors may make substantial contributions. Villagers received remittances from wealthy villagers now living in Australia or the USA. Villagers sought funds from "Excellencies," rich government officials, or Oknya, rich business men. Informants also shared views on the role of Commune Councils in helping to fund education.  IBEC attempted to model transparent funding to education through the Commune Council by passing project funding for schools through the Commune Council, which was to use the funds to respond to the School Development Plans (that had formerly been submitted to the IBEC project).  Some Commune Councils added small amounts of matching funds to the IBEC funding for schools.  Commune Councils were transparent on IBEC funds received, and often called a meeting of all the IBEC school directors in the commune in order to decide on a distribution of the funding.  By contrast, Commune Councils are not transparent on funds they received from the Ministry of Interior. School Directors are generally afraid to question the local authority about the availability of funds for education, although there is high-level agreement in the Government that MOI funds can be used for education at the local level.  Within many Commune Councils there is some confusion and overlap between the education roles of CWCC (Commune Women and Children Committee) and the CEFAC (Commune Education For All Committee). Some projects have used the CWCC to support pre-schools and early childhood care and development activities. Other projects (like IBEC) have used the CEFAC to support basic education interventions in primary and lower secondary school.  Many informants noted that party politics prevailed at the Commune Council. The mekhum (Commune Head) might be supportive of education if he felt that by doing so he would get a stronger name, or improve the likelihood of his re-election. CONCLUSIONS Performance Evaluation on Improved Basic Education in Cambodia (IBEC) p. 39 The end-of-project performance evaluation provides evidence to confirm that IBEC largely achieved its objectives to improve access, quality and relevance of basic education in Cambodia. Dropout rates have generally been reduced in target schools, although dropout of secondary school females remains a problem. Repetition rates are trending below the project baseline, but above the national EMIS rates, which may be understated. (As repetition is a school decision, it may be informal school practice to understate these numbers to accord with Ministry policy to reduce repetition. This would give rise to an artificially low EMIS rate.) Gender parity rates show a slight preponderance of boys in enrollment at IBEC primary schools and a preponderance of girls in IBEC secondary school enrollment. This reflects the many efforts directed at improving school access for girls especially in secondary schools (scholarships, separate latrines, girls' counselors and many others.) As the 2013 IBEC Annual Report (p.2) notes proudly in regard to these access objectives, "these very positive bottom line changes in school efficiency are among the project's most important achievements." Improvements in quality of education are reflected in reduced pupil-teacher ratios in IBEC schools. This is probably due to the many efforts of the project to cope with the perennial teacher shortage in Cambodian schools. IBEC has supported contract or temporary teachers, especially in remote schools and provides scholarships for students at Teacher Training Colleges. Province data shows that the values IBEC promotes, such as improving quality for poorer remote schools, affects local policy that is then reflected in the trends of the pupil-teacher ratio. IBEC's great efforts to advance a relevant education in Cambodia have met with success as the Ministry of Education, Youth and Sport endorsed the IBEC Life Skills curriculum and made life skills an obligatory subject at schools. Much of the training of teachers and capacity building of school directors to advance relevance and to support the Life Skills program in IBEC has also improved the general quality of teaching and management at target schools. Intensive project support helped two implementing partners gain accreditation from the Committee for Cooperation in Cambodia. This achievement reflects a USAID commitment to strengthening civil society as a way to guarantee sustainability of development interventions. A key intervention to improve access and quality in Cambodian schools has been the distribution of discretionary school grants, based on school improvement development plans. IBEC has introduced a significant innovation of "development readiness" tiers, as a way to direct resources to schools that prove to be most effective in utilizing the support properly. The development readiness tier concept holds great promise because it combines two central development concerns, aid effectiveness and good governance. An unintended drawback of the tier scheme may be to widen the separation between good schools, where most resources will go, and poor schools that may be neglected or avoided. Careful attention to the most vulnerable and least advantaged schools should balance the favor shown to excellent schools. IBEC managers undoubtedly have insights into how Cambodian schools really work and what the challenges are to improving effectiveness and governance at schools. It remains for them to distill lessons learned from their experience with Cambodian schools in order to clarify what "readiness" means in a development context, so future projects may then be designed to foster and engage this elusive quality. The value of a concept like "development readiness" in education is that it challenges us to return to first principles and open a discussion about what fundamental goals education should have in Cambodian society. There is a new generation of leaders and educators in the Ministry of Education Youth and Sport. Many of these younger officials do not remember the Khmer Rouge and did not grow up in a civil war Performance Evaluation on Improved Basic Education in Cambodia (IBEC) p. 40 milieu. These new young leaders may use a conversation about development in education to stimulate fresh and productive thinking about what kind of citizen Cambodian education is supposed to create. RECOMMENDATIONS  Project monitoring and evaluation data should be collected maintained and analyzed with the same care that project financial data are treated. Analysis of data during the project would highlight emerging trends and provide an opportunity for adjustments as needed.  Suddenly rising dropout and repetition rates in Year 4 of implementation in some schools as support from the project is reduced may threaten to forfeit gains made. The IBEC project exit strategy should include sensitive encouragement of these schools to continue dropout and repetition reductions.  Trends in access, quality, relevance, and tier mobility reflect great differences between levels and among target provinces, which are likely associated with the different demographic, social and economic conditions. Effectiveness would be enhanced if project implementation were fine-tuned to the peculiar needs and requirements of each level and province, based on adequate monitoring and evaluation reports.  An analysis of what qualities increase a school's development readiness, or make a school too weak to remain in the project would reveal important characteristics of management and governance in Cambodian schools. This is the kind of "lesson learned" that would be useful in designing future interventions.  The encouragement of "risk-taking" by school directors to get results often asks them to use personal funds for project activities until project funds are sent as reimbursement. The commingling of personal and project funds is not appropriate for a civil servant. A line of credit at a local bank should be provided to project school directors, and interest expenses should be borne by the project.  There is high-level agreement in the Government that Ministry of Interior funds could be used at the local level for education, but school directors are often reluctant to question local authority in the commune about the availability of funding. Management training for school directors should include strengthening advocacy skills in dealing with local political officials.  Tier classification appears to be an attractive method for allocating scarce resources in a rational way. The concept of "development readiness" combines two key concerns of donors, aid effectiveness and good governance, and may be adopted in other major projects in education in connection with school grants. A critical analysis of the concept would be concerned with unequal school readiness and would consider how decisions about a just allocation of assistance in the education sector might be made. This analysis would provide a useful basis on which future interventions could be planned.  Student absences due to seasonal work, or due to regular family migration (due to seasonal flooding for example) might be reduced by adjustment in the school calendar for affected province schools. Reduced student absences would lower repetition rates. Performance Evaluation on Improved Basic Education in Cambodia (IBEC) p. 41 ANNEXES Performance Evaluation on Improved Basic Education in Cambodia (IBEC) p. 42 ANNEX I: EVALUATION STATEMENT OF WORK DESCRIPTION/SPECIFICATIONS/STATEMENT OF WORK TITLE: End-of-Project Evaluation, Improved Basic Education in Cambodia (IBEC) Project I. Purpose The evaluation policy of the U.S. Agency for International Development (USAID) articulates the crucial role that evaluations play in the application and management of development resources With this in mind, USAID/Cambodia’s Office of Public Health and Education (OPHE) is seeking an expert team to conduct an end-of-project performance evaluation of the Improved Basic Education project in Cambodia (IBEC). The purpose of the evaluation is to assess the project’s accountability and extent to which it was able to meet its intended objectives –at all result levels The evaluation will also document lessons learned and best practices, as well as provide recommendations to inform evidence-based future programming. See IBEC results framework at Annex 1 II. Description of the Project Project Title Improved Basic Education in Cambodia Implementing Partner World Education Inc Implementing Period October 1, 2009 – September 30, 2014 Project Total Cost $10,000,000.00 Cooperative Agreement Number AID-442-A-09-00003 Contract/Agreement Officer’s Representative Sieng Heng The Improved Basic Education in Cambodia project was designed in direct response to USAID’s goal to promote a better educated youth. The strategic objective of this five-year project, which began in 2009 and will end in 2014, is to improve access, quality, and relevance of basic education SOL-442-14-000009 3 in Cambodia. More specifically, the IBEC project is to increase lower secondary school enrollments, retention, and completion rates, providing Cambodia’s adolescent youth population with an opportunity to be better educated and lead productive lives There are four inter-related, mutually reinforcing, and complementary components to achieving this overarching goal. These four main components include: 1. Local NGO and Government Capacity Building & Advocacy for Sustainability: IBEC accomplishes this by training, coaching, and monitoring its sub-grantee, Kampuchea Action for Primary Education (KAPE). The ore themes covered in the trainings are: a) Organizational Management, b) Program Management, c) Project Performance Management, d) Governance, e) Administration, f) Human Resource Management, and g) Financial Management 2. More Equitable School Access: IBEC provides scholarships for youth from underserved populations who are poor, belong to ethnic and religious minorities, or are handicapped. The scholarships provided are in the form of bicycles, clothing, school supplies, and monetary support. IBEC is also improving access to school by building temporary classrooms to accommodate more students. Other school improvements may range from building wells for drinking water, latrines, or other facilities for sanitation purposes. Access to education is also improving by recruiting teachers from the local community 3. Improved School Management and Community involvement: IBEC provides training to school principals on leadership and management skills and supports the Parent Teacher Associations in the project’s target schools. IBEC programming is aimed towards training school managers to identify priorities, develop proposals and budgets, develop and implement maintenance plans, and to effectively advocate for grants from the Cambodian government and other potential donors Performance Evaluation on Improved Basic Education in Cambodia (IBEC) p. 43 4. Improved Educational Relevance: IBEC developed and piloted a Life Skills curriculum composed of 30 modules on different relevant topics. This curriculum has been adopted by the Ministry of Education, Youth, and Sports (MoEYS). The project works closely with community, including commune councils, to improve their engagement and ownership III. Background The development hypothesis underlying the IBEC project is that a more relevant curriculum, combined with better trained teachers and school managers would encourage more students to attend lower secondary school and to stay long enough to complete the lower secondary cycle Improved physical facilities (wells for drinking water, sanitary latrines, life skills laboratories, etc.) were also believed to play a positive role in inducing students to attend and remain in school. In addition, a limited number of scholarships would directly help vulnerable youth obtain a quality education. In total, these improvements were to create a better educated youth population who will have a knowledge base more relevant to the economic needs of Cambodia, with skills more compatible with the demands of the local job market When the IBEC project was developed, Cambodia’s education system was affected by a weak public￾sector service-delivery system, nominal teacher capacity, lack of adequate school facilities, poor governance, and lack of quality teaching and learning resources. Of all primary and lower secondary schools, almost 50 percent of them did not have clean water, and approximately 20 percent did not have toilets. These factors resulted in low enrollment levels and high drop-out rates, especially at the lower secondary level and higher. Repetition and absenteeism were also a major concern and vulnerable children, particularly girls, the disabled, and minorities, were at risk of being deprived of an education. Lower secondary enrollment rates were near 35%, which put Cambodia globally in the bottom 20 countries worldwide In community meetings, parents and community members often identified the lack of a relevant curriculum as an obstacle to lower secondary school enrollment. Additionally, limited access, such as distance to and from school, lack of classrooms, and school-associated costs, as well as the lack of qualified teachers are factors that affected enrollment and retention. Due in large part to inadequate access to a quality and relevant basic education, at best 25 percent of the 250,000 individuals who were eligible to enter the job market each year didn’t have the necessary and sufficient skills to find employment in the formal sector1. Furthermore, workers with little formal basic education experience were more likely to suffer from low pay and benefits, poor working conditions, and job insecurity Therefore, the IBEC project was created to focus on designing, developing, pilot testing, and evaluating a more relevant lower secondary school life skills curriculum, using lessons learned from previous USAID education projects. Teachers were to be trained in the use of the new curriculum and school administrators and Parent Teacher Associations (PTAs) were to be trained in such areas as planning, how to develop small grant applications, school management, and facilities maintenance. Access issues were to be addressed by providing scholarships to vulnerable students and school improvement grants to schools. Additionally, at the end of the project, one or more local Education NGOs should be capable of providing quality assistance in primary and lower secondary education, creating in-country sustainability and building the capacity of local civil society In the past decade, the Cambodian government has made significant improvements in the education sector. Net enrolment has increased from 84 percent in 2000 to 95 percent in 2010 for primary schools and from 14 percent in 2000 to 32 percent in 2010 for lower secondary schools. To ensure access to education for all children, the Cambodian government has steadily increased the number of schools in the country. In Performance Evaluation on Improved Basic Education in Cambodia (IBEC) p. 44 2000, there were only 5,468 primary and 367 lower secondary schools, but in 2010 there were 6,565 primary and 1,122 lower secondary schools Despite these striking improvements and achievements in Cambodia's educational system, significant challenges persist related to access and to quality, as well as institutional capacity development. This is particularly the case for those residing in remote and rural areas, and those marginalized by poverty, ethnicity, and/or gender Among other challenges, a shortage of teachers is a chronic issue that Cambodia faces. This shortage has resulted in inadequate access to education, especially for children who live in remote areas of the country. The host government has attempted to address this issue by sending 95 percent of newly graduated teachers to teach in remote areas. Despite these efforts, teachers often 1 Evaluation of USAID/Cambodia’s Labor and Industrial Productivity Activities, February 2009 do not stay in these remote areas to teach, because living conditions are typically much harsher than in urban areas. While the Ministry recruits around 5,000 new teachers annually, the teacher shortage continues to be a big gap and newly graduated teachers are only enough to replace those who have retired and those who have left their jobs Another major challenge is the lack of classrooms which needs to be addressed in order to accommodate an increasing number of student enrolments. In addition to the lack of classrooms, many existing schools are also poorly equipped Unfortunately, schools are not receiving adequate resources from the government and are thereby, unable to adequately address these challenges. In 2008, 18.10 percent of the national budget was allocated to education. This figure decreased to 15.92 percent in 2012, approximately 75 percent of which is allocated for teacher salaries Despite the low budget input from the host government, the Ministry is supported by a number of donors namely: Asian Development Bank (ADB), World Bank (WB), UNESCO, UNICEF, European Union, Swedish International Development Cooperation Agency, Japanese International Cooperation Agency, Korea International Cooperation Agency, and others. These development partners support the Ministry through loans and grants. More specifically, ADB and WB provide loans to the Ministry; where as other donors primarily provide grants In addition, Cambodia was qualified in April 2008 to receive $57.4 million Fast Track Initiative (FTI) Funding (2008-2012.) This level of support has allowed Cambodia to work towards achieving its Education for All (EFA) goal by 2015. The country is currently applying for funding through the Global Partnership for Education (GPE), and will soon receive another $38.5 million to continue its efforts toward achieving its EFA goal IV. Evaluation Questions 1) To what extent did the project achieve its objectives? 1.1. Did the project increase access to lower secondary education with respect to enrollment, retention, promotion, and completion rates of youth from underserved populations (including marginalized girls and boys, poor, ethnic and religious minorities, and handicapped children) in targeted schools? 1.2. Did the project improve the quality of lower secondary education in targeted schools? 1.3. Did the project increase the relevance of education, particularly related to curriculum, and how did the implementer measure progress? 1.4. How effective has the local NGO capacity building component been and what are the strengths/weaknesses of the local partners? Performance Evaluation on Improved Basic Education in Cambodia (IBEC) p. 45 2) Among all the interventions implemented by the partner, identify the interventions that have been the most effective in achieving the project objectives 2.1. What activities/interventions introduced by IBEC are likely to continue after the project ends and why? 2.2. Were there any unintended consequences or results of the project interventions? 2.3. How effectively has the project addressed gender differences in education throughout all its interventions? 3) What are key lessons learned? 3.1. Which interventions, based on evidence, should be continued or expanded to improve access (enrollment, retention, promotion, and completion rates) to and the quality of lower secondary education or improve relevance of education? 3.2. How might future investments be refocused? V. Audience and Intended Use The audience of the evaluation report will be the USAID/Cambodia Mission, the Asia Bureau, and USAID implementing partners. USAID will consider the findings, particularly the evidence-based findings, in its strategic approach to education. An Executive Summary will be provided to the Ministry of Education, Youth and Sport (MoEYS). It is expected that the host country partners and donors will also be able to use the report to better assist them in their future goals Performance Evaluation on Improved Basic Education in Cambodia (IBEC) p. 46 ANNEX II: EVALUATION METHODS AND LIMITATIONS The data sought for this evaluation included: 1. Quantitative data sent by target schools to the project; 2. Quantitative and qualitative data collected by project staff from target schools; and 3. Evaluation team interviews with stakeholders (Ministry of Education Youth and Sport; IBEC project management and staff; IBEC project implementing partners; School directors, staff, teachers and students at target schools; and Commune officials, village officials and parents in target school areas.) Analysis of the quantitative data related to accountability included comparison of current rates to baseline rates for the indicators examined. Where feasible, the EMIS rates (especially from the target provinces) were also included in the comparisons. Interviews with key stakeholders in MoEYS and with key IBEC project staff and with key persons in its implementing partner organizations were essential to understand the current Cambodia context for education projects. Interviews at target schools and communes and villages were limited by time and budget and by the distances between target locations. The impressions we gathered at this level provided some local perceptions of sustainability to supplement the understanding we had of the project from key Ministry and IBEC stakeholders. Quantitative data that IBEC had received from schools was available for the four years of project implementation, although it had never been compiled into a single database, nor analyzed before this evaluation. Quantitative and qualitative data generated by IBEC field and project staff were largely not available to the evaluation. Summary statements about the findings of this data had been made in Annual and Quarterly Reports, but for many performance indicators we could not find the actual data on which the statements of findings were based. Accordingly, we could not evaluate the accuracy, validity or representativeness of those data. A baseline for many of the project quantitative indicators was provided, against which to compare current rates. However, the baseline changed from year to year. It was evidently adjusted to account for schools that had been added or removed from the project. Neither the data on which the baseline had originally been constructed nor data supporting any baseline modifications during project implementation were available to the evaluation. Performance Evaluation on Improved Basic Education in Cambodia (IBEC) p. 47 ANNEX III: DATA COLLECTION INSTRUMENTS Interview guides 1. Interview guide IBEC Partners Evaluation Questions Sub Questions Specific questions Comments To what extent did the project achieve its objectives? Did the project improve the quality of lower secondary education in targeted schools? 1. How was/is quality defined?  What assessments were made to ascertain quality?  Were the assessments, in your opinion, valid, accurate and representative?  Your level of involvement and influence 2. What baseline studies were undertaken? 3. What kind of assessments was undertaken to ensure and measure quality of the multiple quality oriented interventions in the schools? 4. Was the “improvement/decline of quality” measured against a control group/s? 5. If improved quality has been achieved - what have “improved” quality led to in terms of definable, tangible and measurable results at an impact level over the entire IBEC project period? Did the project increase the relevance of education, particularly related to curriculum, and how did the implementer measure progress? 1. How was relevance defined and what goals were targeted? 2. Through what participatory processes was “relevance of education” integrated into the curriculum, what results was achieved? 3. What kind of assessments was undertaken to ensure and Performance Evaluation on Improved Basic Education in Cambodia (IBEC) p. 48 measure quality of the multiple quality oriented interventions? 4. What baseline studies were undertaken? How effective has the local NGO capacity building component been and what are the strengths/weaknesses of the local partners? 1. Which preliminary assessments were conducted to assess capacity – and what definitions of “capacity” was used? 2. What assessments were made to study training effectiveness? 3. Were the assessments valid, accurate and representative? 4. What kind of assessments was undertaken to ensure and possible realign/improve on interventions in the course of the project period? Among all the interventions implemented by the partner, identify the interventions that have been the most effective in achieving the project objectives. What activities/interventions introduced by IBEC are likely to continue after the project ends and why? 1. What were the strengths, weaknesses, opportunities and challenges in working with the IBEC project? 2. If you were going to begin a new project like IBEC, what would you do differently, in design, implementation? 3. What were missed opportunities that you feel had promise, but were not pursued in the IBEC project? 4. What were the key innovations IBEC brought to the education sector? 5. What IBEC activities have the promise to be sustainable after funding ends? Performance Evaluation on Improved Basic Education in Cambodia (IBEC) p. 49 Were there any unintended consequences or results of the project interventions? 1. Did any unintended consequences, positive and negative, occur as a consequence of the planning and implementation of any of the IBEC interventions? 2. If yes, what and how was these consequences addressed? 3. Was there any overarching strategy/process for addressing unintentional and negative consequences? How effectively has the project addressed gender differences in education throughout all its interventions? 1. What gender assessments were conducted? 2. Were the assessments, in your opinion, valid, accurate and representative? 3. How was gender focus integrated into the IBEC planning and interventions? What are key lessons learned? Which interventions, based on evidence, should be continued or expanded to improve access (enrollment, retention, promotion, and completion rates) to and the quality of lower secondary education or improve relevance of education? 1. What were the strengths, weaknesses, opportunities and challenges in working with the IBEC project? 2. If you were going to begin a new project like IBEC, what would you do differently, in design, implementation and assessment? 3. What were missed opportunities that you feel had promise, but were not pursued in the IBEC project? 4. What were the key innovations IBEC brought to the education sector? 5. What IBEC activities have the promise to be sustainable after funding ends? Performance Evaluation on Improved Basic Education in Cambodia (IBEC) p. 50 2. Interview guide MoEYS Officials Evaluation Questions Sub Questions Specific questions Comments To what extent did the project achieve its objectives? Did the project improve the quality of lower secondary education in targeted schools? 1. Please elaborate on the role of the CG in the planning, implementation and monitoring of IBEC? 2. What national efforts are currently being effected in order to improve the quality of lower secondary education? How does this coincide with IBEC interventions? Did the project increase the relevance of education, particularly related to curriculum, and how did the implementer measure progress? 1. Please elaborate on the consultative process of curriculum development and, if any, the involvement of IBEC key partners and implementers in this process? Among all the interventions implemented by the partner, identify the interventions that have been the most effective in achieving the project objectives. What activities/interventions introduced by IBEC are likely to continue after the project ends and why? 1. To your knowledge, what were the strengths, weaknesses, opportunities and challenges in the IBEC project? 2. In the view of CC interviewees, what are the strengths, weaknesses of the IBEC program to date? 3. What is the likelihood that the CC will fund education in the future with its MoI budget? What are the obstacles to this funding? 4. Does CC have experience coordinating any other donor funding for education/schools than IBEC? (eg local Wat, School Support Committee, Performance Evaluation on Improved Basic Education in Cambodia (IBEC) p. 51 LNGO, overseas remittances, etc) 5. How does CC interact, cooperate with MoEYS: local School Directors, DOE, POE? Were there any unintended consequences or results of the project interventions? 1. Was any unintentional consequences (negative and/or positive) of the IBEC interventions brought to the attention of the CG?  If yes, how was these issues addressed? How effectively has the project addressed gender differences in education throughout all its interventions? What are key lessons learned? Which interventions, based on evidence, should be continued or expanded to improve access (enrollment, retention, promotion, and completion rates) to and the quality of lower secondary education or improve relevance of education? 1. Was there any collaboration between IBEC and other partners or government initiatives/interventions? 2. Sustainability Issues  Are there any sustainability plan/s to continue the activities after the end of the project period?  Are there any willingness and availability to continue the activities after the project ends?  Future education programming and areas that stakeholders would like to see improved? Performance Evaluation on Improved Basic Education in Cambodia (IBEC) p. 52 3. Interview guide Provincial Office of Education POE/ District Office of Education DOE Evaluation Questions Sub Questions Specific questions Comments To what extent did the project achieve its objectives? Did the project improve the quality of lower secondary education in targeted schools? 1. What is the POE/DOE`s notion of improved quality? 2. Have the POE/DOE ascertained improvement in quality amongst the targeted secondary schools, if so in what way? 3. In what way, if any, have the POE/DOE interacted and communicated with the implementing parties in order to ensure increased quality? 4. Have there been an overall improvement in the general quality of lower secondary education in your Province/district? Please elaborate? 5. What in your opinion have been the primary contribution of the multiple IBEC interventions, to achieve improved quality of lower secondary education in the targeted schools? And how does this differ from the specific work of the Province/District? Did the project increase the relevance of education, particularly related to curriculum, and how did the implementer measure progress? 1. Please elaborate on the role of the CG in the planning, implementation and monitoring of IBEC? 2. What experience of/with interaction/cooperation/coordination in the IBEC projects? 3. What were the strengths, weaknesses, opportunities and challenges in working with the IBEC project and achieving “increased relevance of education”? Among all the interventions What activities/interventions introduced by IBEC 1. What were the strengths, weaknesses, opportunities and Performance Evaluation on Improved Basic Education in Cambodia (IBEC) p. 53 implemented by the partner, identify the interventions that have been the most effective in achieving the project objectives. are likely to continue after the project ends and why? challenges in working with the IBEC project? 2. If you were going to begin a new project like IBEC, what would you do differently, in design, implementation and assessment? 3. What were missed opportunities that you feel had promise, but were not pursued in the IBEC project? 4. What were the key innovations IBEC brought to the education sector? 5. What IBEC activities have the promise to be sustainable after funding ends? Were there any unintended consequences or results of the project interventions? 1. Did any unintended consequences, positive and negative, occur as a consequence of the planning and implementation of any of the IBEC interventions? 2. If yes, what and how was these consequences addressed? 3. Was there any overarching strategy/process for addressing unintentional and negative consequences? 4. How and if where the POE and DOE involved/included in the IBEC project? Performance Evaluation on Improved Basic Education in Cambodia (IBEC) p. 54 How effectively has the project addressed gender differences in education throughout all its interventions? 1. How has the POE/DOE underlined the gender perspectives in the IBEC planning and interventions? 2. Has there also been a national gender focus during the project implementation? 3. What are the views of key stakeholders on the question of gender issues in primary and secondary education? 4. How was attention to gender appropriately integrated into the IBEC planning and interventions? 5. How did the POE/DOE ensure a proper and effective gender focus in the IBEC project? What are key lessons learned? Which interventions, based on evidence, should be continued or expanded to improve access (enrollment, retention, promotion, and completion rates) to and the quality of lower secondary education or improve relevance of education? 6. Collaboration with other partners or government initiatives/interventions 7. What were the strengths, weaknesses, opportunities and challenges in working with the IBEC project? 8. If you were going to begin a new project like IBEC, what would you do differently, in design, implementation and assessment? 9. What were missed opportunities that you feel had promise, but were not pursued in the IBEC project? 10. What were the key innovations IBEC brought to the education sector? 11. What IBEC innovations are worth adapting/adopting/refining for the future? 12. What IBEC activities have the promise to be sustainable after funding ends? 4. Interview guide School Directors Performance Evaluation on Improved Basic Education in Cambodia (IBEC) p. 55 Evaluation Questions Sub Questions Specific questions Comments To what extent did the project achieve its objectives? Did the project improve the quality of lower secondary education in targeted schools? 1. What is the Director`s notion of improved quality? 2. Have the Director ascertained improvements in quality amongst the targeted secondary schools, if so in what way? 3. In what way, if any, have the Director interacted and communicated with the implementing parties in order to ensure increased quality in the school? 4. What, in your opinion, have been the primary contribution of the multiple IBEC interventions, to achieve improved quality of lower secondary education in the targeted schools? Did the project increase the relevance of education, particularly related to curriculum, and how did the implementer measure progress? 1. Number of teachers in the school? 2. Total school enrollment? 3. Dropout/Repetition/Promotion rates as reported to EMIS? 4. Scholarship student statistics if available? 5. Total funding received from IBEC? 6. Number of IBEC supported teachers from PTTC? 7. What types of projects were supported by IBEC at the school? 8. How effective has the funding transfer from IBEC been? What problems? 9. What problems/challenges were faced in IBEC program implementation at your school? Among all the interventions What activities/interventions introduced by IBEC 6. What were the strengths, weaknesses, opportunities and Performance Evaluation on Improved Basic Education in Cambodia (IBEC) p. 56 implemented by the partner, identify the interventions that have been the most effective in achieving the project objectives. are likely to continue after the project ends and why? challenges in working with the IBEC project? 7. What the school join a similar project again?  Why/Why not?  Anything that you or the school would do differently? 8. What were missed opportunities that you feel had promise, but were not pursued in the IBEC project? Were there any unintended consequences or results of the project interventions? 1. Did any unintended consequences, positive or negative, occur as a consequence of the planning and implementation of any of the IBEC interventions in your school?  If yes, what and how was these consequences addressed? 2. How was the communication with the IBEC implementers and decision makers? 3. Was there any overarching strategy/process for addressing unintentional and negative consequences in your school?  How and if where the school Directors involved? How effectively has the project addressed gender differences in education throughout all its interventions? 1. What assessments were made to focus on gender? 2. Were the assessments valid, accurate and representative? 3. How was gender focus appropriately integrated into the IBEC planning and interventions? 4. How is gender considerations part of the education planning in this school? What are key lessons learned? Which interventions, based on evidence, should be continued or expanded to 1. What suggestions would you make for a potential new project based on the lessons you have learned - in project design, implementation, Performance Evaluation on Improved Basic Education in Cambodia (IBEC) p. 57 improve access (enrollment, retention, promotion, and completion rates) to and the quality of lower secondary education or improve relevance of education? monitoring, reporting, funding, training, technical support. 2. Sustainability Issues  Are there any sustainability plan/s to continue the activities after the end of the project period?  Stakeholders’ willingness and availability to continue the activities after the project ends?  Future education programming and areas that stakeholders would like to see improved?  What have been the benefits of being an USAID implementing partner or beneficiary of IBEC (besides purely financial gains)? 5. Interview guide Teachers Evaluation Questions Sub Questions Specific questions Comments To what extent did the project achieve its objectives? Did the project improve the quality of lower secondary education in targeted schools? 1. What is your notion of improved quality in the classroom and in regards to teaching/activities? 2. What specific strategies have been put in place to improve on quality in the classroom – stemming from the IBEC intervention?  Any weaknesses and short￾comings? Did the project increase the relevance of education, particularly related to curriculum, and how did the implementer measure progress? 1. Please elaborate on the relevance of education, particularly related to the curriculum? 2. What are the evidence/signs of progress and noticeable achievements? Performance Evaluation on Improved Basic Education in Cambodia (IBEC) p. 58 3. How did the implementer measure progress and quality achievements (in the classroom)? 4. What problems/challenges were faced in IBEC program implementation at your school/classroom? Among all the interventions implemented by the partner, identify the interventions that have been the most effective in achieving the project objectives. What activities/interventions introduced by IBEC are likely to continue after the project ends and why? 1. What were the strengths, weaknesses, opportunities and challenges in working with the IBEC project? 2. If you were going to begin a new project like IBEC, what could/should be done differently? 3. What were missed opportunities that you feel had promise, but were not pursued in the IBEC project? Were there any unintended consequences or results of the project interventions? 1. Did any unintended consequences, positive and/or negative, occur as a consequence of the planning and implementation of any of the IBEC interventions in your school?  If yes, what and how was these consequences addressed? 2. How was the communication with the IBEC implementers, decision makers and you as a teacher? 3. How specifically where the teachers involved? Performance Evaluation on Improved Basic Education in Cambodia (IBEC) p. 59 How effectively has the project addressed gender differences in education throughout all its interventions? 1. How are gender issues and concerns addressed at your school and in your classroom? 2. How is gender issues and concerns integrated in to the teaching and educational planning? What are key lessons learned? Which interventions, based on evidence, should be continued or expanded to improve access (enrollment, retention, promotion, and completion rates) to and the quality of lower secondary education or improve relevance of education? 1. What suggestions would you make for a potential new project based on the lessons you have learned - in project design, implementation, monitoring, reporting, funding, training, technical support. 6. Interview guide PTA representatives Evaluation Questions Sub Questions Specific questions Comments To what extent did the project achieve its objectives? Did the project improve the quality of lower secondary education in targeted schools? 1. What is your notion of improved quality in lower secondary education? 2. How has the PTAs been involved in the IBEC project? 3. Level of Parental (PTA) influence in the IBEC project? Did the project increase the relevance of education, particularly related to curriculum, and how did the implementer measure progress? Among all the interventions What activities/interventions Performance Evaluation on Improved Basic Education in Cambodia (IBEC) p. 60 implemented by the partner, identify the interventions that have been the most effective in achieving the project objectives. introduced by IBEC are likely to continue after the project ends and why? Were there any unintended consequences or results of the project interventions? 1. Was any unintended consequences, positive and negative, occurring as a consequence of the planning and implementation of any of the IBEC interventions brought to the PTA? during the project implementation? 2. If yes, was these consequences addressed in conjunction with the PTA? 3. How was the communication with the IBEC implementers and decision makers/school directors and the PTA? How effectively has the project addressed gender differences in education throughout all its interventions? What are key lessons learned? Which interventions, based on evidence, should be continued or expanded to improve access (enrollment, retention, promotion, and completion rates) to and the quality of lower secondary education or improve 1. From a PTA perspective, which interventions have been successful and could be continued after the end of IBEC funding? Performance Evaluation on Improved Basic Education in Cambodia (IBEC) p. 61 relevance of education? 7. Interview guide Community Councils Evaluation Questions Sub Questions Specific questions Comments To what extent did the project achieve its objectives? Did the project improve the quality of lower secondary education in targeted schools? Did the project increase the relevance of education, particularly related to curriculum, and how did the implementer measure progress? 4. What is the likelihood that local schools can obtain funds from Commune Councils? What would be needed to improve the likelihood? Among all the interventions implemented by the partner, identify the interventions that have been the most effective in achieving the project objectives. What activities/interventions introduced by IBEC are likely to continue after the project ends and why? Were there any unintended consequences or results of the project interventions? How effectively has the project addressed gender differences in education throughout all its interventions? Performance Evaluation on Improved Basic Education in Cambodia (IBEC) p. 62 What are key lessons learned? Which interventions, based on evidence, should be continued or expanded to improve access (enrollment, retention, promotion, and completion rates) to and the quality of lower secondary education or improve relevance of education? 1. Commune Council perspective on contributing of commune funds to support school activities Performance Evaluation on Improved Basic Education in Cambodia (IBEC) p. 63 ANNEX IV: SOURCES OF INFORMATION Number of respondents The quantitative information and databanks are provided accordingly to USAID. No. Person to Name Title Institution Date Start time End time Status Address and contact information 1 Mr. Eng Kimly Director Department of Curriculum Development, MoEYS May 19, 2014 10:00 11:00 Done Chao Ponheahok Secondary School, Phnom Penh 2 Mr. Chan Narin, Mr. Kurt Bredenburg, Mr. Eng Sok, and Ms. Jacole Douplas Chief of party Senior advisor Deputy chief of party Program officer World Education Cambodia May 19, 2014 12:30 13:30 Done #20 Street 222, Boeung Raing, Khan Daun Penh, Phnom Penh, Cambodia 3 Ms. Song Kunthea Mr. Keo Sokha, and Mr. Phan Chanhoeurn Education and members service coordinator Program officer Finance and admin officer NGO Education Partnership May 19, 2014 15:00 16:00 Done No. 41, St. 464, 12311 Phnom Penh 4 Ms. Chum Rathneary Director Preah En Kaosa Secondary School May 19, 2014 15:00 16:00 Done Siem Reap city, Siem Reap province 5 Ms. Phon Tara Deputy director Department of Planning, MoEYS May 20, 2014 08:30 09:30 Done #169, Norodom Boulevard, Phnom Penh 6 Mr. Oung Sereydy Director Siem Reap'sfa Provincial Office of Education May 20, 2014 09:00 10:00 Done Siem Reap city, Siem Reap province 7 Mr. Heom Hann Mr. Heout Chanthorn School director Teacher Phum Komrou Primary School May 20, 2014 09:30 11:30 Done Phum Komrou Primary School, Kralanh district, Siem Reap province PP Kampong Cham Kratie Siem Reap Kampong Thom MoEYS 3 0 0 0 0 3 School director 0 5 4 12 7 28 POE 0 1 1 1 1 4 DOE 0 2 2 3 0 7 Implementing partner 2 3 1 0 0 6 Student council 0 2 0 1 5 8 Teacher 0 8 8 14 10 40 Commune chief 0 1 1 2 2 6 Total 5 22 17 102 Respondent Location # of respondents Performance Evaluation on Improved Basic Education in Cambodia (IBEC) p. 64 8 Mr. Ung Ngohok Director Department of Secondary Education, MoEYS May 20, 2014 10:00 11:00 Done #169, Norodom Boulevard, Phnom Penh 9 Mr. Pot Samnang Head of office Siem Reap's District Office of Education May 20, 2014 10:30 11:30 Done Siem Reap city, Siem Reap province 10 Ms. Sum Sambath Ms. Hoy La Mr. Phatt Theoung Mr. Ouy Sereyvon School director Teacher Teacher Teacher Samdech Euv High School May 20, 2014 13:30 15:30 Done Samdech Euv High School, Sranger commune, Siem Reap district, Siem Reap province 11 Mr. Phong Peng Head of office Tboung Khmum's District Office of Education May 20, 2014 16:00 17:00 Done Tboung Khmum district, Kampong Cham province 12 Mr. Tep Sarin Mr. Bun Yeoun Head of office Deputy head of office Kralanh's District Office of Education May 21, 2014 08:00 09:00 Done Kralanh district, Siem Reap province 13 Mr. Sao Vanna Director Kampuchean Action for Primary Education May 21, 2014 09:00 10:00 Done Provincial Teachers Training College (PTTC), National Road 7, Kampong Cham city, Kampong Cham province 14 Ms. Phin Chanrey Mr. Seom Seung Director Deputy director Roung Kou Primary School May 21, 2014 09:30 10:30 Done Roung Kou Primary School, Kralanh district, Siem Reap Province 15 Ms. Sum Kimsreang, Ms. Oum Ry, and Mr. Heang Lay Director Deputy Director Advisor Economic Development Association May 21, 2014 10:15 11:15 Done Krola village, Krola commune, Kompong Siem district, Kampong Cham province 16 Mr. Keo Sopheak Mr. Dok Chhat Ms. Thorn Chantha School director Teacher Teacher Kampong Thkaw Primary School May 21, 2014 11:00 12:30 Done Kampong Thkaw Primary School, Kralanh district, Siem Reap province 17 Mr. Tiv Kiri Mr. Sorn Chin Ms. Earb Sokheng Ms. Seng Tisen School director Teacher Teacher Teacher Komrou Srok Primary School May 21, 2014 13:00 15:00 Done Komrou Srok Primary School, Sout Nikum district, Siem Reap province Performance Evaluation on Improved Basic Education in Cambodia (IBEC) p. 65 18 Mr. Tourn Sokhum Mr. Plang Vannpat Mr. Chea Sokchamreoun Deputy Director Head of secondary office Head of primary office Kampong Cham's Provincial Office of Education May 21, 2014 15:00 16:30 Done Kampong Cham City 19 Mr. Sok Sang Ms. Cheng Chanty Teacher Teacher Phum Komrou Secondary School May 21, 2014 15:30 17:30 Done Phum Komrou Secondary School, Sranal commune, Kralanh district, Siem Reap province 20 Mr. Chey Nara Head of office Krouch Chhmar's District Office of Education May 22, 2014 09:00 10:00 Done Krouch Chmar district, Kampong Cham province 21 Mr. Tann Laen Head of office Sout Nikom's District Office of Education May 22, 2014 09:00 10:00 Done Sout Nikum district, Siem Reap province 22 Mr. Chay Tipa, Ms. Slaimann Hasanas 5 Students School director Teacher Students council Beush Pi High School May 22, 2014 10:15 11:45 Done Beush Pi High School, Krouch Chmar district, Kampong Cham province 23 Mr. Som Se Director Samraong Secondary School May 22, 2014 10:30 11:30 Done Samraong Secondary School, Samraong commune, Sout Nikum district, Siem Reap province 24 Mr. Heng Latt Commune chief Domdaek Commune Hall May 22, 2014 13:00 13:30 Done Domdaek Commune, Sout Nikum district, Siem Reap province 25 Mr. Kly Kriya Director Beush Mouy Primary School May 22, 2014 13:30 14:30 Done Beush Mouy Primary School, Krouch Chmar district, Kampong Cham province 26 Mr. Thaing Mono Director Mukh Pen Primary School May 22, 2014 14:00 15:00 Done Mukh Pen Primary School, Puok district, Siem Reap province 27 Mr. Pich Sophy Mr. Sou Se 10 Students School Director Teacher Students council Domdaek High School May 22, 2014 15:00 18:00 Done Domdaek High School, Domdaek commune, Sout Nikum district, Siem Reap province Performance Evaluation on Improved Basic Education in Cambodia (IBEC) p. 66 28 Mr. Thorn Vandong Director Buddhist Social Development Association May 22, 2014 16:30 17:30 Done Near the capital city of Kampong Cham province 29 Mr. Ly Puthea Deputy director Tboung Khum High School May 23, 2014 08:00 08:15 Done Tboung Khmum High School, Tboung Khum district, Kampong Cham province 30 Mr. Nhok Sophy Ms. Teok Ramun Director Teacher Tuol Rovieng Primary School May 23, 2014 09:00 11:30 Done Tuol Rovieng Primary School, Puok district, Siem Reap province 31 Mr. Om Kea Commune chief Roka Por Pram Commune Hall May 23, 2014 09:40 10:20 Done RoKa Por Pram commune, Tboung Khmum district, Kampong Cham province 32 Mr. Siek Phanet Director Chanimith Primary School May 23, 2014 10:30 11:30 Done Chanimith Primary School, RoKa Por Pram commune, Tboung Khmum district, Kampong Cham province 33 Mr. Tha Sieng Lai Member of student council Toul Kandoal Secondary School May 23, 2014 13:15 13:30 Done Toul Kandoal Secondary School, Chikor commune, Tboung Khmum district, Kampong Cham province 34 Mr. Nhok Chhour Commune chief Tuol Rovieng Commune Hall May 23, 2014 13:30 14:00 Done Tuol Rovieng commune, Puok district, Siem Reap province 35 Mr. Mao Soman Director Toul Kandoal Secondary School May 23, 2014 13:40 14:40 Done Toul Kandoal Secondary School, Chikor commune, Tboung Khmum district, Kampong Cham province 36 Mr. Rith Salin Mr. Eng Borey Mr. Chan Chin Director Deputy director Secreatary Athipadey Primary School May 23, 2014 15:00 16:00 Done Athipadey Primary School, Puok district, Siem Reap province 37 Mr. Lay Bora Director Kratie's Provincial Office of Education May 26, 2014 09:00 10:00 Done Kratie City Performance Evaluation on Improved Basic Education in Cambodia (IBEC) p. 67 38 Mr. Chou Vonthy Director Kampong Thom's Provincial Office of Education May 26, 2014 09:00 10:00 Done Kampong Thom city, Kampong Thom province 39 Mr. Phon Siputhea Mr. Kae Kimhong Mr. Tann Honglim Mr. Som Soksour 10 Students Director Deputy director Teacher Teacher Students Hun Sen Tbaeng High School May 26, 2014 10:30 12:00 Done Hun Sen Tbaeng High School, Kampong Svay district, Kampong Thom province 40 Mr. Chheng Sophal Mr.Sreng Menghong Ms. Sroy Dane Ms. Un Sorsdey Director Teacher Teacher Teacher Svay Chraeh Secondary School May 26, 2014 13:00 14:00 Done Svay Chraeh Secondary School, Snoul district, Kratie province 41 Mr. Chhorn Chhoung Mr. Leng Naly Mr. Seom Pisith Mr. Sun Sarin 10 students Director Teacher Teacher PTA Students council Moreak Secondary School May 26, 2014 13:30 17:30 Done Moreak Secondary School, Prasat Balang district, Kampong Thom province 42 Mr. Ton Ngaeth Head of office Snoul's District Office of Edcuation May 26, 2014 15:00 16:00 Done Snoul district, Kratie province 43 Mr. Kith Dimang Mr. Chhun Nonn Commune chief Commune council Hanchey Commune Hall May 27, 2014 08:30 09:10 Done Hanchey commune, Chlong district, Kratie province 44 Mr. Orn Sorn Mr. Ourn Seang Im Ms. Yeom Sophana 10 students Director Teacher Teacher Students council Hun Sen Prasat Secondary School May 27, 2014 09:00 12:00 Done Hun Sen Prasat Secondary School, Santuk district, Kampong Thom province 45 Mr. Ouk Liphan Director Hanchey Leu Primary School May 27, 2014 09:30 10:30 Done Hanchey Leu Primary School, Hanchey commune, Chlong district, Kratie province 46 Ms. Mey Thidapov Mr. Norng Sarom Mr. Hor Teangseng Director Teacher Teacher Preah Monykesor Secondary School May 27, 2014 13:30 14:30 Done Preah Monykesor Secondary School, Chlong district, Kratie province 47 Mr. Oem On Commune chief Sala Visey Commune Hall May 27, 2014 13:30 14:00 Done Sala Visey Commune, Prasat Balang district, Kampong Thom Province Performance Evaluation on Improved Basic Education in Cambodia (IBEC) p. 68 48 Mr. Sun Limseng Mr. Maen Thy Ms. Chhor Sarin Mr. Ich Vuth 10 students Director Deputy director Teacher Teacher Students council Prasat Balang High School May 27, 2014 14:30 17:30 Done Prasat Balang High School, Prasat Balang district, Kampong Thom province 49 Mr. Mean Sarik Head of office Chlong's District Office of Education May 27, 2014 15:00 16:00 Done Chlong district, Kratie province 50 Mr. Nheok Kimvy Director Sandann High School May 28, 2014 08:40 09:40 Done Sandann High School, Sambo district, Kratie province 51 Mr. Lakk Chheon Commune chief Prasat Commune Hall May 28, 2014 10:00 11:00 Done Prasat commue, Santuk district, Kampong Thom province 52 Mr. Tok Thyrith Ms. Hun Taing Ngorn Mr. Heom Vuth Ms. Kum Sineom 10 students Director Deputy director Teacher Teacher Students council Kampong Thmar High School May 28, 2014 13:30 17:30 Done Kampong Thmor High School, Santuk district, Kampong Thom province 53 Mr. Prum Von Director Women and Children Rights Development May 28, 2014 16:00 17:00 Done Malob Doung Restaurant, Kratie city 54 H.E. Sam Sereyrath Director General Ministry of Education Youth and Sport May 30, 2014 14:00 15:00 Done #80, Norodom Blvd, Phnom Penh Performance Evaluation on Improved Basic Education in Cambodia (IBEC) p. 69 ANNEX V: DISCLOSURE OF ANY CONFLICTS OF INTEREST Performance Evaluation on Improved Basic Education in Cambodia (IBEC) p. 70 Performance Evaluation on Improved Basic Education in Cambodia (IBEC) p. 71 Performance Evaluation on Improved Basic Education in Cambodia (IBEC) p. 72 Performance Evaluation on Improved Basic Education in Cambodia (IBEC) p. 73 ANNEX VI: ENROLLMENT TABLES AND CHARTS ENROLLMENT Figure 6.1 Total Enrollments in three target provinces at primary and secondary levels. (Only data from 2010-11 to 2012-13 is comparable, because for those three years all three target provinces participated in IBEC. Siem Reap had not yet joined IBEC in 2009-10) IBEC Primary School Total Student Enrollment EMIS PS Enrollment in Target Provinces EMIS PS Enrollment total Cambodia IBEC Secondary School Student Enrollment EMIS SS Enrollment in Target Provinces EMIS SS Enrollment Total Cambodia 2009-10 56,700 525,928 2,240,651 42,222 168,032 908,698 2010-11 90,960 519,967 2,191,192 46,344 169,777 895,602 2011-12 85,589 510,562 2,142,464 41,391 165,296 859,312 2012-13 82,729 523,979 2,173,384 42,608 164,308 823,499 Figure 6.2 Comparison of IBEC primary enrollments in three target provinces. 48878 53392 48883 47713 26328 26328 24714 7822 11240 10378 10302 0 10000 20000 30000 40000 50000 60000 2009-10 2010-11 2011-12 2012-13 IBEC Primary Enrollment in Three Target Provinces Kampong Cham Siem Reap Kratie Performance Evaluation on Improved Basic Education in Cambodia (IBEC) p. 74 Figure 6.3 Comparison of IBEC Secondary enrollment in three target provinces 25859 24542 22097 23099 15038 12948 13291 5760 0 6764 6346 6218 10000 20000 30000 40000 50000 60000 2009-10 2010-11 2011-12 2012-13 IBEC Secondary Enrollment in Three Target Provinces Kampong Cham Siem Reap Kratie Performance Evaluation on Improved Basic Education in Cambodia (IBEC) p. 75 ANNEX VII: DROPOUT AND REPETITION CHARTS (Secondary Level Female Rates) IBEC Total Female Dropout and Repetition Figure 7.1 Figure 7.2 12.50% 13.00% 13.50% 14.00% 14.50% 15.00% 15.50% 16.00% 16.50% 2009-10 2010-11 2011-12 2012-13 IBEC Secondary Schools Total Female Dropout Rate Baseline Female Dropout Current Female Dropout 0.00% 2.00% 4.00% 6.00% 8.00% 2009-10 2010-11 2011-12 2012-13 IBEC Secondary Schools Total Female Repetition Rate Baseline Female Repetition Current Female Repetition Performance Evaluation on Improved Basic Education in Cambodia (IBEC) p. 76 Kampong Cham Female Dropout and Repetition Figure 7.3 Figure 7.4 0.00% 5.00% 10.00% 15.00% 20.00% 25.00% 2009-10 2010-11 2011-12 2012-13 Kampong Cham IBEC Secondary Female Dropout Baseline Female Dropout Current Female Dropout EMIS Female Dropout 0.00% 1.00% 2.00% 3.00% 4.00% 5.00% 6.00% 2009-10 2010-11 2011-12 2012-13 Kampong Cham IBEC Secondary Female Repetition Baseline Female Repetition Current Female Repetition EMIS Female Repetition Performance Evaluation on Improved Basic Education in Cambodia (IBEC) p. 77 Kratie Female Dropout and Repetition Figure 7.5 Figure 7.6 0.00% 5.00% 10.00% 15.00% 20.00% 25.00% 2009-10 2010-11 2011-12 2012-13 Kratie IBEC Secondary Female Dropout Baseline Female Dropout Current Female Dropout EMIS Female Dropout 0.00% 2.00% 4.00% 6.00% 8.00% 2009-10 2010-11 2011-12 2012-13 Kratie IBEC Secondary Female Repetition Baseline Female Repetition Current Female Repetition EMIS Female Repetition Performance Evaluation on Improved Basic Education in Cambodia (IBEC) p. 78 Siem Reap Female Dropout and Repetition Figure 7.7 Figure 7.8 0.00% 5.00% 10.00% 15.00% 20.00% 25.00% 2009-10 2010-11 2011-12 2012-13 Siem Reap IBEC Secondary Female Dropout Baseline Female Dropout Current Female Dropout EMIS Female Dropout 0.00% 2.00% 4.00% 6.00% 8.00% 2009-10 2010-11 2011-12 2012-13 Siem Reap IBEC Secondary Female Repetition Baseline Female Repetition Current Female Repetition EMIS Female Repetition Performance Evaluation on Improved Basic Education in Cambodia (IBEC) p. 79 ANNEX VIII: GENDER PARITY INDEX RATES Kampong Cham Figure 8.1 In Kampong Cham IBEC secondary schools, the current rate for GPI again follows the EMIS rate closely, at least over the last three years, and is consistently within the parity range. Female students appear to have equal access to secondary education in the province, both in IBEC schools and, according to the EMIS data, more generally in non-project schools in Kampong Cham. 0.80 0.85 0.90 0.95 1.00 1.05 1.10 2009-10 2010-11 2011-12 2012-13 Kampong Cham IBEC Secondary GPI Rates GPI at Baseline Current GPI EMIS GPI Performance Evaluation on Improved Basic Education in Cambodia (IBEC) p. 80 Kratie Figure 8.2 The IBEC current GPI for secondary schools follows the EMIS trend closely, but again stays above the EMIS rate. This again shows that at the secondary level both IBEC schools and non-project schools are strongly favoring female access--to the point where both IBEC and EMIS rates now exceed the parity range. Steps to restore gender balance are needed here. 0.90 0.95 1.00 1.05 1.10 1.15 2009-10 2010-11 2011-12 2012-13 Kratie IBEC Secondary GPI Rates GPI at Baseline Current GPI EMIS GPI Performance Evaluation on Improved Basic Education in Cambodia (IBEC) p. 81 Siem Reap Figure 8.3 By contrast, the secondary rate for IBEC schools is far above the baseline, and above EMIS rates. The GPI at the secondary level in Siem Reap, for both IBEC schools and non-project schools is now well above the parity range, distinctly favoring female enrollment. Some steps to restore balance between male and female enrollment are needed here. 0.80 0.90 1.00 1.10 1.20 1.30 2009-10 2010-11 2011-12 2012-13 Siem Reap IBEC Secondary GPI Rates GPI at Baseline Current GPI EMIS GPI Performance Evaluation on Improved Basic Education in Cambodia (IBEC) p. 82 ANNEX IX: IBEC LIFE SKILLS ACTIVITIES For indicative purposes, the most comprehensive list of activities is shown in the following table. IBEC Life Skills Activities Life Skill related to social issues Life Skill (Civic Awareness) Life Skill (Personal Hygiene) Life Skill (Cultural Awareness) Life Skill (Environment Awareness) Life Skill on Trading, Economic, Career Life Skill on (Saving and Expenditure) Life Skill on (Social and Personal Understanding) Life Skill on (Money Management) Life Skill on (Place of Work) Life Skill on (My Marketing) Life Skill on (My Future) Life Skill on (Market Simulation) Life Skill on (Safe Migration) Life Skill on (Technology Awareness) Life Skill on (Drug Prevention) Life Skill on (Alcohol Prevention) Life Skill on (Understanding on Gender) Life Skill on (Music and Arts) Life Skill on (ICT) Life Skill on (Social and Child Enterprise) Practical Livelihood Skill Practical Livelihood Skill (Vegetable) Practical Livelihood Skill (Fish) Practical Livelihood Skill (Rice) Practical Livelihood Skill (Frog) Practical Livelihood Skill (Chicken) Practical Livelihood Skill (Mushroom Production) Practical Livelihood Skill (Bio-Garden) Practical Livelihood Skill (House Painter) Practical Livelihood Skill (Hair-Cut) Practical Livelihood Skill (Bicycle Repairer) Practical Livelihood Skill (Sewing) Practical Livelihood Skill (Fruit Sugar) Curriculum Enhancement (For Cooking) Performance Evaluation on Improved Basic Education in Cambodia (IBEC) p. 83 As a measure of progress, the increase in number of activities each year can be summarized. Primary Level. At the primary level the total number of activities implemented at IBEC primary schools has grown from 169 in 2009-10 to 182 in 2012-13, which demonstrates the progress made in Life Skills implementation. The chart below shows the distribution of Life Skills activities over the three provinces, which is varies with the number of target schools in each province. IBEC Primary Schools Number of Activities 2009-10 10 2010-11 11 2011-12 12 2012-13 11 Total LS Activities Implemented at IBEC Primary Level TOTAL Kampong Cham Kratie Siem Reap 2009-10 169 129 40 0 2010-11 190 123 35 32 2011-12 186 111 27 48 2012-13 182 107 27 48 Secondary Level At the secondary level the progress of Life Skills implementation is particularly striking. The total number of activities implemented at IBEC primary schools has grown from 88 in 2009-10 to a remarkable 836 in 2012-13. IBEC Secondary Schools Number of Activities 2009-10 8 2010-11 11 2011-12 32 2012-13 31 Total LS Activities Implemented at IBEC Secondary Level TOTAL Kampong Cham Kratie Siem Reap 2009-10 88 68 20 0 2010-11 331 163 54 114 2011-12 866 384 223 259 2012-13 836 379 213 244 Performance Evaluation on Improved Basic Education in Cambodia (IBEC) p. 84 ANNEX X: IBEC PRIMARY SCHOOL RESULTS The Evaluation specifically required focus on project secondary school results. However, considerable IBEC effort was also directed at the primary feeder schools from which the secondary schools drew their pupils. Accordingly, some data on primary level target schools were collected by IBEC and are summarized in the charts in this Annex. Primary Project Enrollment Figure 10.1 From 2010-11 to 2012-13, EMIS enrollment for all primary schools in the three target provinces increased 0.77% from 519,967 to 523,979 pupils. By comparison, in that same period, IBEC target primary school enrollments declined 9% from 90,960 to 82,729 pupils. Primary Province Enrollment Figure 10.2 IBEC Primary Level Enrollment and Total Primary Level Enrollment in the Three Target Provinces Kampong Cham Siem Reap Kratie IBEC Enrollment EMIS Total Enrollment IBEC Enrollment EMIS Total Enrollment IBEC Enrollment EMIS Total Enrollment 2009-10 48,878 298,318 172,008 7,822 55,602 2010-11 53,392 292,247 26,328 171,485 11,240 56,235 2011-12 48,883 283,270 26,328 170,509 10,378 56,783 2012-13 47,713 284,295 24,714 180,059 10,302 59,625 % Change -2.38% -4.70% -6.13% 4.47% 24.07% 6.75% At IBEC target primary schools in Kratie, enrollment increased 24%, from 7822 in Year 1 to 10302 in Year 4. By comparison, EMIS enrollment in the province primary schools showed an increase of only 6.75%. 56,700 90,960 85,589 82,729 525,928 519,967 510,562 523,979 0 100,000 200,000 300,000 400,000 500,000 600,000 2009-10 2010-11 2011-12 2012-13 Target Province Primary School Enrollment (Total and IBEC) IBEC PS Total Student Enrollment EMIS PS Enrollment in Target Provinces Performance Evaluation on Improved Basic Education in Cambodia (IBEC) p. 85 Project Dropout Figure 10.3 The current dropout rate in IBEC primary schools is clearly below the baseline rate. The change in baseline value in 2010-11 is due to adjustments made when Siem Reap joined the project. Project Female Dropout Figure 10.4 0.00% 1.00% 2.00% 3.00% 4.00% 5.00% 6.00% 7.00% 8.00% 2009-10 2010-11 2011-12 2012-13 IBEC Primary Schools Total Dropout Rate Baseline Dropout Current Dropout Performance Evaluation on Improved Basic Education in Cambodia (IBEC) p. 86 Project Repetition Figure 10.5 There is an unmistakable upward trend in primary repetition in 2012-13, which is troubling. Although the rate is still below the baseline, the rate should be watched carefully in the final implementation year. Project Female Repetition Figure 10.6 0.00% 5.00% 10.00% 15.00% 20.00% 25.00% 2009-10 2010-11 2011-12 2012-13 IBEC Primary Schools Total Repetition Rate Baseline Repetition Current Repetition Performance Evaluation on Improved Basic Education in Cambodia (IBEC) p. 87 Kampong Cham Dropout Figure 10.7 In Kampong Cham, the dropout rate at IBEC target primary schools stays below the baseline for all years. The IBEC dropout rate is also far below the EMIS published dropout rates for the province. This is the kind of profile that confirms the project success in improving access for primary grade pupils Kampong Cham Female Dropout Figure 10.8 0.00% 2.00% 4.00% 6.00% 8.00% 10.00% 12.00% 2009-10 2010-11 2011-12 2012-13 Kampong Cham IBEC Primary Total Dropout Baseline Dropout Current Dropout EMIS Dropout 0.00% 2.00% 4.00% 6.00% 8.00% 10.00% 12.00% 2009-10 2010-11 2011-12 2012-13 Kampong Cham IBEC Primary Female Dropout Baseline Female Dropout Current Female Dropout EMIS Female Dropout Performance Evaluation on Improved Basic Education in Cambodia (IBEC) p. 88 Kampong Cham Repetition Figure 10.9 The repetition rates in Kampong Cham IBEC primary schools show a dramatic rise in 2012-13--rising above the baseline and also above EMIS rates. The decision to have a student repeat a grade depends on the judgment of the school and is generally based on exceeding the allowed days of absence. But it could be that other factors are also at play in this last year of program implementation, as project activities are curtailed, or resources withdrawn. Kampong Cham Female Repetition Figure 10.10 0.00% 5.00% 10.00% 15.00% 2009-10 2010-11 2011-12 2012-13 Kampong Cham IBEC Primary Total Repetition Baseline Repetition Current Repetition EMIS Repetition 0.00% 5.00% 10.00% 15.00% 2009-10 2010-11 2011-12 2012-13 Kampong Cham IBEC Primary Female Repetition Baseline Female Repetition Current Female Repetition EMIS Female Repetition Performance Evaluation on Improved Basic Education in Cambodia (IBEC) p. 89 Kratie Dropout Figure 10.11 The dropout rates for IBEC primary schools in Kratie after the initial year are consistently far below the adjusted IBEC baseline rate. Dropout in IBEC schools is also consistently below the EMIS rates for the province. Kratie Female Dropout Figure 10.12 0.00% 5.00% 10.00% 15.00% 20.00% 25.00% 30.00% 2009-10 2010-11 2011-12 2012-13 Kratie IBEC Primary Total Dropout Baseline Dropout Current Dropout EMIS Dropout 0.00% 5.00% 10.00% 15.00% 20.00% 25.00% 30.00% 2009-10 2010-11 2011-12 2012-13 Kratie IBEC Primary Female Dropout Baseline Female Dropout Current Female Dropout EMIS Female Dropout Performance Evaluation on Improved Basic Education in Cambodia (IBEC) p. 90 Kratie Repetition Figure 10.13 Repetition in IBEC primary schools in Kratie follows the same pattern as dropout. The current repetition each year after Year 1 in Kratie is far below the baseline rate, and generally very close to the EMIS rates for repetition in primary schools in this province. Kratie Female Repetition Figure 10.14 0.00% 20.00% 40.00% 60.00% 80.00% 100.00% 2009-10 2010-11 2011-12 2012-13 Kratie IBEC Primary Total Repetition Baseline Repetition Current Repetition EMIS Repetition 0.00% 20.00% 40.00% 60.00% 80.00% 2009-10 2010-11 2011-12 2012-13 Kratie IBEC Primary Female Repetition Baseline Female Repetition Current Female Repetition EMIS Female Repetition Performance Evaluation on Improved Basic Education in Cambodia (IBEC) p. 91 Siem Reap Dropout Figure 10.15 Siem Reap primary schools entered the IBEC project in Year 2, 2010-11. The dropout rates at the primary level have stayed just below the IBEC baseline rate. These current rates and baseline rates in IBEC schools is far below the EMIS dropout rates for primary schools in the province Siem Reap Female Dropout Figure 10.16 The current repetition rates in Siem Reap primary schools falls below the baseline in all years. However the IBEC repetition rates are above the EMIS rates published for the primary schools in Siem Reap. 0.00% 2.00% 4.00% 6.00% 8.00% 10.00% 12.00% 2009-10 2010-11 2011-12 2012-13 Siem Reap IBEC Primary Total Dropout Baseline Dropout Current Dropout EMIS Dropout 0.00% 2.00% 4.00% 6.00% 8.00% 10.00% 12.00% 2009-10 2010-11 2011-12 2012-13 Siem Reap IBEC Primary Female Dropout Baseline Female Dropout Current Female Dropout EMIS Female Dropout Performance Evaluation on Improved Basic Education in Cambodia (IBEC) p. 92 Siem Reap Repetition Figure 10.17 Siem Reap Female Repetition Figure 10.18 0.00% 5.00% 10.00% 15.00% 2009-10 2010-11 2011-12 2012-13 Siem Reap IBEC Primary Total Repetition Baseline Repetition Current Repetition EMIS Repetition 0.00% 5.00% 10.00% 15.00% 2009-10 2010-11 2011-12 2012-13 Siem Reap IBEC Primary Female Repetition Baseline Female Repetition Current Female Repetition EMIS Female Repetition Performance Evaluation on Improved Basic Education in Cambodia (IBEC) p. 93 Primary School Gender Parity Index Figure 10.19 The GPI for all IBEC target primary schools in the three target provinces is stable, but below the baseline and outside the range for gender parity, showing a predominance of males. Kampong Cham GPI Figure 10.20 The GPI at the IBEC primary schools in Kampong Cham tracks just below the project baseline for the last three years. The GPI for project schools follows the EMIS published rate for GPI in the province closely, but remains below the range for parity. 0.85 0.90 0.95 1.00 1.05 1.10 2009-10 2010-11 2011-12 2012-13 IBEC Primary Schools Gender Parity Index GPI at Baseline Current GPI 0.80 0.85 0.90 0.95 1.00 1.05 1.10 2009-10 2010-11 2011-12 2012-13 Kampong Cham IBEC Primary GPI Rates GPI at Baseline Current GPI EMIS GPI Performance Evaluation on Improved Basic Education in Cambodia (IBEC) p. 94 Kratie GPI Figure 10.21 Kratie is unlike the other two IBEC provinces in showing improved gender parity at the primary school level. The GPI for IBEC schools is rising above the EMIS rate for the province, showing that IBEC schools are exceeding the trend for the province as a whole in providing access to primary school for female students. Siem Reap GPI Figure 10.22 At the primary level, Siem Reap shows disappointing results, with the current rate generally falling below the EMIS rate and below the IBEC baseline rate. The EMIS rate for gender parity is also falling, showing a reduced access for females in the province as a whole, while the IBEC seems to show slightly improved access, at least in 2012-13, when the IBEC GPI exceeds the EMIS rate. 0.90 0.95 1.00 1.05 1.10 1.15 2009-10 2010-11 2011-12 2012-13 Kratie IBEC Primary GPI Rates GPI at Baseline Current GPI EMIS GPI 0.80 0.90 1.00 1.10 1.20 1.30 2009-10 2010-11 2011-12 2012-13 Siem Reap IBEC Primary GPI Rates GPI at Baseline Current GPI EMIS GPI Performance Evaluation on Improved Basic Education in Cambodia (IBEC) p. 95 Pupil-Teacher Ratio Kampong Cham PTR Figure 10.23 Kratie PTR Figure 10.24 At the primary level, the pupil teacher ratio in Kratie is generally below the EMIS rate. The EMIS rate for primary schools in this province is also well below that in Kampong Cham and Siem Reap. The urban primary schools tend to have the lowest pupil-teacher ratio, but the rates for urban, rural and remote are very close. 0 20 40 60 80 2009-2010 2010-2011 2011-2012 2012-2013 Pupil-Teacher Ratio Kampong Cham IBEC Primary Remote Rural Urban EMIS 0 20 40 60 80 2009-2010 2010-2011 2011-2012 2012-2013 Pupil-Teacher Ratio Kratie IBEC Primary Remote Rural Urban EMIS Performance Evaluation on Improved Basic Education in Cambodia (IBEC) p. 96 Siem Reap PTR Figure 10.25 Life Skills Activities Figure 10.26 At the primary level the total number of activities implemented at IBEC primary schools has grown from 169 in 2009-10 to 182 in 2012-13, which demonstrates the progress made in Life Skills implementation. 0 20 40 60 80 2009-2010 2010-2011 2011-2012 2012-2013 Pupil-Teacher Ratio Siem Reap IBEC Primary Remote Rural Urban EMIS Performance Evaluation on Improved Basic Education in Cambodia (IBEC) p. 97 Province Level Tier Changes at Primary Schools Kampong Cham Figure 10.27 In Kampong Cham, the rate of Tier 1 primary schools doubles between start up and Year 4 of implementation from 28.18% of schools at start-up to 59.09% of schools in Year 4. Kratie Figure 10.28 In Kratie, the rate of Tier 1 primary schools barely increases from start up to Year 4 of implementation, from 43.48% of schools at start-up, to 46.67% of schools in Year 4. 28.18% 32.73% 59.09% 0.00% 20.00% 40.00% 60.00% 80.00% 100.00% Start-up Year 3 Year 4 Percent of Target Schools Tier Change at IBEC Primary Schools, Kampong Cham Tier 1 Tier 2 Tier 3 43.48% 33.33% 46.67% 0.00% 20.00% 40.00% 60.00% 80.00% 100.00% Start-up Year 3 Year 4 Percent of Target Schools Tier Change at IBEC Primary Schools, Kratie Tier 1 Tier 2 Tier 3 Performance Evaluation on Improved Basic Education in Cambodia (IBEC) p. 98 Siem Reap Figure 10.29 In Siem Reap the rate of Tier 1 primary schools increases nearly five-fold from 11.54% at start-up to 53.85% in Year 4. 11.54% 32.69% 53.85% 0.00% 20.00% 40.00% 60.00% 80.00% 100.00% Start-up Year 3 Year 4 Percent of Target Schools Tier Change at IBEC Primary Schools, Siem Reap Tier 1 Tier 2 Tier 3 Performance Evaluation on Improved Basic Education in Cambodia (IBEC) p. 99 ANNEX XI: ADDENDUM: ADDITIONAL OBSERVATIONS ON LESSONS LEARNED & SUCCESSES TO ADD TO THE IBEC EVALUATION RECORD Drafted by World Education, Inc & Kampuchean Action for Primary Education The following observations are added to the Independent Evaluation of the Improved Basic Education in Cambodia Project (IBEC) to provide additional clarity on important Lessons Learned during the project’s implementation. These comments are not intended to refute or contradict the findings of the Evaluation Team but rather to both contextualize and add to the Assessment, given that significant time constraints resulted in a very limited window for the evaluators to meet all relevant stakeholders. Monitoring Framework and Project Complexity: The current evaluation has noted the need to improve project accountability in terms of data keeping and analysis. However, the assessment of the project’s accountability should best be understood in terms of the project’s overall context and the design of its M&E framework, since the project feels that one of the biggest lessons learned from the IBEC experience (not mentioned in the Evaluation Report) relates to the dangers of complex programming vis a vis the development of evaluation frameworks and indicator identification. In this regard, it should be noted that IBEC’s holistic approach to educational development ensured that the project would be highly complex in its scope, encompassing many activities across multiple components (e.g., access, educational quality, management capacity, community engagement, etc). Because USAID’s M&E template, known as the PMP, does not limit the number of project indicators or prioritize them in terms of their importance, this limitation, combined with the project’s high level of complexity, resulted in a profusion of indicators that in many cases did not adequately capture the overall impact of the project and created many unnecessary burdens for data collection. For example, reductions in repetition and dropout were accorded the same status as indicators with far less important measures of success such as the execution of tracer studies. As a result of the situation described above, the project identified 70 indicators each with their own methodologies for sampling and data collection. The burden of data collection stemming from this number of indicators somewhat overwhelmed the project’s manpower assigned to undertake it. Thus, the project feels that it is important to contextualize the assessment of the project’s evaluation framework and emphasize the tendency for complex projects with holistic designs to promote overly complex assessment frameworks with a profusion of unnecessary indicators. This is an important Lesson Learned that the project would like to add to the record of that stated in the project evaluation document. These observations were also echoed in DQA statements issued by USAID that were not mentioned in the Evaluation Report. The above observations should not be interpreted as an indictment of holistic programming; indeed, IBEC was able to largely achieve most of its key objectives, as the evaluators point out, ‘because’ of its holistic scope. But holistic designs have many inherent dangers for the development of assessment frameworks. Innovations, Best Practices, and Lessons Learned (p.20) The following observations are intended to complement the findings of the Evaluation Team noted in the Section on Innovations, Best Practices, and Lessons Learned (p.20). Once again, these observations are intended to add to and contextualize the current findings of the Evaluation Report. Ministry Buy-in: Another important observation that should be added to IBEC’s evaluation record relates to the high level of ‘buy-in’ by MoEYS stakeholders at the highest levels, especially by the Director General of the Directorate of General Education who chaired IBEC’s oversight committee (known as the Consultative Group). Due to time constraints and schedule conflicts, this key individual was not interviewed during the project evaluation but his support and advocacy for the project have been key to Performance Evaluation on Improved Basic Education in Cambodia (IBEC) p. 100 both its success and ability to leverage impacts. For example, the DG has been a key advocate for the project’s ability to clarify the Ministry’s policy regarding the use of overtime payments for secondary school life skills teachers, ensuring one of the most important policy changes that will help to sustain the new life skills curriculum and implementation framework developed. Since the completion of the evaluation, the DG has also been able to advocate successfully for a recent decree issued by the Minister of Education, Youth, and Sport himself instructing all schools to use the new life skills curriculum manuals developed by IBEC from the next school year onwards. This formalized endorsement of the curriculum at the highest level is one of the most important aspects of Ministry buy-in that the project would like to add to the record. This achievement is especially remarkable when one considers that the project did not provide any form of salary supplement to its Ministry counterparts, as is the practice of most other development partners.3 Innovations in Public Private Partnership: One of the cross-cutting themes in IBEC programming mandated by USAID relates to the need for the project to promote PPP in educational development. Once again, time limitations did not permit for any discussion of contributions made by the project in this area. In addition to successfully soliciting support from several corporations such as Apple and Microsoft for specific project activities, IBEC successfully established a social enterprise affiliated with the lead local partner in the project, i.e., KAPE. This social enterprise, known as Thuntean Seksa or TTS, develops and markets teaching and learning aids for schools. TTS became independent of the project in October 2012 and is now a self-sustained operation with revenues of over $200,000. This success addresses the frequent dependency that schools often develop for teaching aid access that hinge entirely on project-mediated supply chains. In most projects, this access ends when the project does, even in cases where schools have money to buy teaching aids. The establishment of TTS has sustainably solved this problem and promises to be a lasting contribution to the education system in Cambodia with continuing access to innovative teaching aids that uses market forces for development and distribution. School Selection Approaches: The Evaluation Report discusses the interesting innovation of classifying schools into development tiers and experimentation in concepts relating to ‘developmental readiness.’ One aspect of the project’s approach to modulating project assistance based on schools’ developmental readiness relates to the process through which schools were selected. In this respect, IBEC departed from the usual practice in development projects of basing school selection solely on criteria of ‘need.’ The project also considered motivational factors and habits of ‘risk-taking’ as additional key criteria in school selection. The project reasoned that schools that are averse to risk-taking behavior or who have no interest in participating in a development project focusing on innovation would mute the effectiveness of development aid. During project start-up, all schools in the province were invited to an informational workshop on the project with explanations of the expectations for participating schools. All schools were invited to ‘apply’ for participation completing a standardized form that assessed motivational factors and risk-taking behavior. Of great interest and concern, only about 70% of schools in some provinces that attended the workshop bothered to apply, reflecting the very low levels of motivation and interest in innovation among many state schools. As a result of this process, target schools were able to self-select for project participation resulting in a pool of target schools that reflected not only need factors but also motivational ones as well. This is an interesting innovation that the project feels should be noted as an important innovation and lesson learned. Innovations in ICT Access: The Evaluation Report discusses the ability of the project to provide access to ICT facilities; however, the report did not adequately highlight the revolutionary innovations in 3 USG regulations prohibit the payment of salary supplements to the civil servants of foreign governments to do tasks for which they are already paid by their own civil service. Performance Evaluation on Improved Basic Education in Cambodia (IBEC) p. 101 technology using thin client devices that have successfully addressed many of the challenges undermining other donor investments in ICT labs. In this respect, equipment costs, energy usage, and maintenance needs have all ensured that the number of labs built in Cambodian schools is small and characterized by a short shelf life. That is, labs usually fall out of use within a period of one to two years. While IBEC did not solve all of these problems as the report points out, it did successfully reduce equipment costs by 56%, energy usage by 81%, and maintenance needs by over 90%. These successes have been empirically validated in a research report carried out by the project that should be mentioned in the project’s evaluation record. Career Counseling: With a youth population comprising 65% of Cambodia’s total population, career counseling has become a critically important need for the nation’s school system. Unfortunately, the state school system has historically had no formal provisions for such counseling on school premises. Project surveys have found that 75% of students rely primarily on their parents, siblings, and friends for career advice with schools playing a very small role. IBEC is the first project to develop a hands-on manual on career counseling that was successfully piloted in collaboration with the Vocational Orientation Department. The lessons learned from the pilot including the need for teaching staff dedicated to career counseling functions should also be mentioned in the Evaluation Report. Other Innovations in Educational Quality: The project would also like to add observations relating to other improvements in educational quality not mentioned in the Evaluation Report. These observations include the introduction of (i) subject classrooms, (ii) provisions to maximize science lab utilization, and (iii) the use of Student Councils and Subject Clubs to amplify student learning. In this respect, the project has pioneered an effective response to the problem of ‘talk and chalk’ methods in secondary school instruction by using the concept of Subject Classrooms. Because classes remain stationery while subject teachers must move from room to room, there is a disincentive for teachers to carry around bulky supplies such as maps, science equipment (even when it is available), and math tools for their teaching. As a result, teachers primarily rely on talk and chalk for their instruction. By establishing fully equipped classrooms dedicated to a particular subject (e.g., Geography, Science, Math, etc) that require students to move from classroom to classroom, the project has made it much easier for teachers to easily access teaching and learning aids for their instruction. This institutional change in how schools work is spreading to other provinces and projects and deserves mention in the project’s evaluation record. In addition, the project has also made significant headway in improving the low utilization of science labs that has plagued other projects. The approach used in this regard has relied a combination of strategies including specialized curricular materials referenced to the textbook that make science experiments much clearer, modeling over 60 experiments that teachers actually do themselves at teacher training workshops, and school director training that introduces accountability for lab usage. Record books that help track the use of science labs have been very effective in this regard. Again, these lessons learned should be added to the project’s evaluation record. Finally, the IBEC Project increased student ownership of the school through improved capacity-building of Student Councils. In addition to building student confidence, these efforts contributed to the success of life skills classes, library maintenance, and computer/science lab utilization. The project has also been the first to introduce the idea of Subject Clubs to provide opportunities to students with particular interests in specific topics to increase their knowledge and experience learning in an enjoyable context. Capacity-building manuals in these areas have all been documented and successfully implemented. U.S. Agency for International Development 1300 Pennsylvania Avenue, NW Washington, DC 20523