DO ESA
GJee v )
inal Ev
Ma ag g as u a
Submitted To:
USAlD IIndonesia
Submitted By:
The Mitchell Group, Inc.
1816 11 th Street .W.
Washington, D.C. 2000 I
Telephone: (202) 745-1919
Facsimile: (202) 234-1697
Internet: the-mitchellgroup.com
.-~-{iE -
~--=eROUP.INC.
:t~itchell
March 2007
ACRONYMS
ALPS
AusAID
Bappeda
CI
CLCC
CP
CRC
DBE
DEO
Dewan
Pendidikan
DINAS
Pendidikan
DPRD
D4
GOI
IAPBE
KKG
Madrasah
MBE
MGMP
MONE
M'ORA
MSS
MTT
PAKEM
PKG
PMEP
PMPTK
RAPBS
RENSTRA
RIPS
SBM
SC
SI
UNESCO
UNICEF
USAID
Active Learning through Professional Support
Australian Agency for International Development
District Planning Agency
Community Involvement
Creating Learning Communities for Children
Community Participation
Cluster Resource Centers
Decentralized Basic Education (DBEI = Package 1)
District Education Office
District Education Board
Education Office ofLocal Government (District & Provincial Levels) .
District Legislature.
Four-year Diploma
Government of Indonesia
Indonesia Australia Partnership for Basic Education
Teachers' Working Group (Primary Schools)
Islamic school
Managing Basic Education
Subject Teacher Working Group
Ministry ofNational Education
Ministry ofReligious Affairs
Minimum service standard
Master Teacher Trainers
Active, Creative, Effective and Joyful Learning (an active learning approach
used in schools)
Strengthening the work ofteachers
Performance Monitoring and Evaluation Plan
Directorate General for Quality Improvement of Teachers and Education
Staff
School Budget
Strategic Plan
School Plan
School-Based Management
School Committee
Bachelor's degree
United Nations Education, Scientific and Cultural Organization
United Nations Children's Fund
United States Agency for International Development
Final Evaluation ofthe Managing Basic Education (MBE) Project ABE-BE IQC (Contract # EDH-I-02-05-00035-00)
I
ACKNOWLEDGEMENTS
This final evaluation of the MBE project and the report of those findings were carried out by a
four-person team consisting of: Rasdi Ekosiswoyo, former Rector, Semarang State University;
David P. Evans (Team Leader), Senior Education Consultant; Michael Thair, International
Development Specialist; and M. Basri Wello, Vice Rector, State University of Makassar. The
four-person team was hired under a contract administered by The Mitchell Group, Inc. (TMG)
and assisted by the South-East Consortium for International Development (SECID).
The team gratefully acknowledges the assistance provided by:
• SECID, especially Adjeng Pudji Astrini, Tina Hendarto, and Danker Schaareman;
• USAID/Jakarta, especially Loretta Garden, Jill Gulliksen, and Jim Hope;
• Research Triangle Institute (RTI), especially Robert Cannon, Lynne Hill, Prima
Setiawan, Stuart Weston and the rest ofthe MBE staff;
• The staffs ofthe Decentralized Basic Education projects (DBE) 1,2, and 3;
• Representatives ofthe Ministry ofNational Education (MaNE);
• Representatives ofthe Ministry ofReligious Affairs (MORA);
• Representatives ofthe Coordinating Ministry for People's Welfare;
• Representatives ofthe international donor community;
• Local government officials in each ofthe districts visited;
• Community leaders and other key staff ofthe districts visited;
• Members ofthe school committees in the schools visited;
• Students and parents in all the schools visited; and particularly to
• Supervisors, principals and teachers in all the schools visited.
To all of those people and to any others who contributed to the provision of information to the
team, thank you. Without their valuable assistance the evaluation could not have been completed.
Annex IV lists key people interviewed. Any errors are regrettable but, of course, the sole
responsibility ofthe team.
Final Evaluation oftheManaging Basic Education (MBE) Project
II
ABE-BE IQC (Contract # EDH-I-02-05-00035-00)
TABLE OF CONTENTS
ACRONYMS
ACKNOWLEDGEMENTS
TABLE OF CONTENTS
EXECUTIVE SUMMARY
BACKGROUND
FINDINGS
LESSONS LEARNED
CONCLUSIONS
SECTION I. INTRODUCTION
I.1 EVALUATION OBJECTIVES AND SCOPE
1.2 METHODOLOGY
1.3 CONSTRAINTS IN UNDERTAKING THE EVALUATION
1.4 STRUCTURE OF THE REpORT
SECTION II. BACKGROUND
11.1 OVERVIEW OF USAID EDUCATION STRATEGY IN INDONESIA
11.2 DESCRIPTION OF USAID INTERVENTIONS IN INDONESIA
11.3 OVERVIEW OF GOVERNMENT OF INDONESIA EDUCATION POLICY
11.4 PROJECT DESIGN AND RELEVANCE
SECTION III. FINDINGS
111.1 DISTRICT AND SCHOOL-BASED MANAGEMENT
m.2 COMMUNITY INVOLVEMENT
111.3 TEACHING AND LEARNING
IlIA PROJECT EFFICIENCY
SECTION IV. IMPLICATIONS: LESSONS LEARNED
IV.l LESSON LEARNED 1: CLEAR AND FOCUSED CONCEPTUAL FRAMEWORK
I
II
III
VI
VI
VII
IX
XI
1
1
1
2
2
3
3
3
4
6
8
8
13
16
19
23
23
Final Evaluation ofthe Managing Basic Education (MBE) Project ABE-BE IQC (Contract # EDH-I-02-05-00035-00)
III
IV.2 LESSON LEARNED 2: WELL SEQUENCED AND COORDINATED WHOLE SCHOOL
APPROACH 23
IV.3 LESSON LEARNED 3: START SMALL AND BUILD ON SUCCESS 24
IVA LESSON LEARNED 4: Do WHAT YOU DO WELL 24
IV.5 LESSON LEARNED 5: PAKEM PLATEAU 25
IV.6 LESSON LEARNED 6: SELECTION PROCESSES 25
IV.7 LESSON LEARNED 7: COORDINATION 26
IV.8 LESSON LEARNED 8: POLICY DIALOGUE (OR LACK THEREOF) 26
SECTION V. SUSTAINABILITY
V.1
V.2
V.3
COMPONENTS OF IN-BUILT MBE PROJECT SUSTAINABILITY
EVIDENCE OF SUSTAINABILITY AT THE DISTRICT LEVEL
IMPLICATIONS FOR THE GOI
27
27
27
27
VI.1 OPPORTUNITY 1
VI.2 OPPORTUNITY 2
VI.3 OPPORTUNITY 3
VIA OPPORTUNITY 4
SECTION VI. STRATEGIES AND APPROACHES TO SUPPORT MBE
AFTER PROJECT ENDS 30
30
30
31
31
ANNEXES 33
ANNEX I SUMMARY OF SCHOOL VISITS 34
ANNEX II SUMMARY OF THE CONTENTS OF THE MBE TRAINING
PACKAGES 40
ANNEX III LIST OF SCHOOLS VISITED 42
ANNEX IV LIST OF PEOPLE CONTACTED 43
ANNEX V SELECTED LIST OF DOCUMENTS CONSULTED 46
ANNEX VI AREAS FOR FURTHER RESEARCH 48
Final Evaluation ofthe Managing Basic Education (MBE) Project
IV
ABE-BE IQC (Contract # EDH-I-02-05-00035-00)
ANNEX VII S"LTMMARY OF MEETING WITH Gor AND DONORS TO
REVIEW FINDINGS 49
Final Evaluation ofthe Managing Basic Education (MBE) Project ABE-BE IQC (Contract # EDH-I-02-05-00035-00)
V
EXECUTIVE SUMMARY
BACKGROUND
Purpose
The objective ofthis evaluation is two-fold: first, the evaluation examines the overall results and
impact of the MBE program, including the project's strengths and weaknesses, implementation
mechanisms and lessons learned; and second the evaluation explores approaches and strategies
for continued support of successful elements after its completion by linking it with the
Government ofIndonesia (GOI), USAID's on-going DBE program, and/or other implementers.
Methodology
The evaluation team used four basic methods to obtain its findings, including: reviewing of
project documents; observation or MBE-assisted, non-MBE-assisted, and DBE-assisted schools,
principals, teachers and other educators; targeted questions and interviews with individuals and
officials knowledgeable about the MBE and DBE projects; and targeted questions in focus group
discussions with single and mixed groups ofstakeholders.
Constraints
There were several constraints to the evaluation ofthe MBE:
• No independent data could be found to measure the program's impact on the quality of
education from formal tests and examinations conducted by local and national
authorities;
• It was not possible to undertake an in-depth analysis of classroom behavior and the time
spent in each school was sufficient to form impressions only;
• The sample ofschools selected was small and could not be random;
• Schools were selected by national and provincial MBE project coordinators and the final
sample may be skewed towards better performing schools and most schools were
expecting the evaluation team and were very prepared for our visit.
These constraints resulted in the team being somewhat cautious in drawing conclusions.
Project Design
The Managing Basic Education (MBE) project works with 23 local district governments in three
provinces (East and Central Java and Nanggroe Aceh Darussalam - NAD), as well as in Jakarta,
in three project components: district and school-based management, community participation
and teacher training. The district and school-based management component seeks to strengthen
the capacity of local government and school principals to effectively plan, manage and deliver
quality basic education services. The community participation component works to strengthen
the position and role of local stakeholders - parents, teachers, school committees, community
organizations, local parliaments - in planning and managing basic education. The teacher
training component trains teachers in active learning methodologies (known in Indonesia as
PAKEM) and learning material development. The three components are intended to be mutually
reinforcing and the program is delivered at the school level in a whole school approach.
Final Evaluation ofthe Managing Basic Education (MBE) Project
VI
ABE-BE IQC (Contract # EDH-I-02-05-00035-00)
FINDINGS
A. General Findings
Well positioned
The MBE is well aligned with the Gal decentralization policy and MONEIMORA strategic
plans, especially in the areas of improving the quality of teaching and learning, increasing the
involvement of local communities in schools, improved governance and accountability through
the strengthening of school committee capacity, and the increased availability of instructional
materials.
Dramatic Visible Change
For the most part, MBE-assisted schools undergo a dramatic visible change in the physical
characteristics of the school. Although the schools are not always in tip-top shape, old physical
infrastructure has been spruced up with minor and sometimes major repairs frequently made by
parents. Classrooms are often newly painted in bright, light colors and many classrooms have
become a showcase for student work and learning materials. The improved classroom learning
environment is one ofthe most striking impacts ofthe MBE program.
Energy and Enthusiasm
Although most teachers in MBE-assisted schools are not well-trained in the formal system and
are not certified at the new Gal required level (S1), MBE-assisted schools show a marked
increase in the amount of enthusiasm and energy in the classroom. Most ofthe classrooms have
been reorganized away from traditional seating into small work groups of 4-6 students and
teachers were often observed presiding over student-centered exercises. Some teachers lead
classes using excellent, well-thought out active-learning exercises and techniques that might rival
the best but, most appear to be learning the methods recently introduced. Teachers in MBE￾assisted schools have made great strides toward improved and diversified teaching methods but,
for the most part, the progress in improving teaching methods is incomplete.
Improved School Management
With few exceptions, district officials and school principals demonstrated good management
practice and a commitment to transparency and accountability. Generally, principals had
developed school strategic plans in concert with a wide group of·stakeholders and were
reasonably articulate in the short- to long-term needs and priorities ofthe school. School budgets
were nearly always displayed prominently. Some principals exhibited fairly sophisticated
knowledge of teacher evaluation processes and incentives, teacher and school development
needs, and how to access resources for school improvement. Nearly all ofthe principals seemed
to be comfortable with displaying the new-found transparency and enjoyed the new attention the
community appeared to be focusing on the school. At the same time, some principals appeared to
accept the status quo as a given, did not see obvious problems, and did not seem to provide
dynamic leadership. In addition, it appeared that some sub-district and district managers offered
lack-luster support to the schools and could profit from a better definition of their roles and
responsibilities as well as training in management and leadership.
Final Evaluation ofthe Managing Basic Education (MBE) Project ABE-BE IQC (Contract # EDH-I-02-05-00035-00)
VII
Very Active Parental Participation
Most schools appeared to have parents and parent organizations that were very active in the
schools. The level of involvement varied widely with some parents working as volunteer
teachers-aids, repairing or building classrooms, or providing nutritional supplements and
teaching materials to the schools, while others simply cleaned the school rooms periodically. The
parents appeared to be energized by the new attitudes ofthe teachers and principals and the focus
on improving the quality of education within the school. At the same time, a minority of the
parents at the school appeared to be little more than window dressing and were hard-pressed to
describe any kind ofrole in the school.
Engaged School Committees and Community Leaders
In all but a very few schools, school committee members and community leaders were present
and appeared to be actively involved in the management and governance of the school. School
committee members arid community leaders provided input into school development plans,
reviewed school budgets, developed performance standards, and assisted in developing
additional revenue streams or in-kind assistance in the implementation of the school
development plan. Most maintained that they met a minimum of four times a year. Some ofthe
school committee members appeared to see their role as perfunctory and there were some cases
where the school committee chairperson had an obvious conflict of interest with the school
principal and were less than fully independent ofthe principal. By and large, however, the school
committee and the community leaders appeared to playa prominent and useful role in the school.
The willingness of community leaders and school committee members to participate in meeting
with the evaluation team, often on the basis of a last minute invitation, was impressive and a
measure ofwhat high regard theMBE project is held in the communities.
B. Findings by Type of School
MONE vs. MORA
Generally speaking there appeared to be little differences in the impact of the MBE model being
applied to a public primary school as opposed to a public or private madrasah (an Islamic
school). The madrasah represent a wider range in the quality of primary education than those of
the public primary system and the madrasah represented both the best and the worst school we
saw. Some ofthe madrasah suffered badly from being under-resourced but others, which showed
the same signs of limited funding, were some of the best encountered. In any case, the impact of
the MBE program on school management, community participation and active learning methods
appeared to be equally strong in both types ofschools.
Junior Secondary Schools
In general, the teacher training component of the MBE project was less successful in junior
secondary schools. The reasons for this are largely because in primary schools the MBE project
trained all the teaching staff and the small number of teachers in primary schools form a
"community of interest" that is generally a coherent, mutually supporting group, while junior
secondary schools have larger staff, not as large numbers of teachers were trained, and teachers
are split by content areas and across all of these content areas do not form one cohesive school
Final Evaluation ofthe Managing Basic Education (MBE) Project
VIII
ABE-BE IQC (Contract # EDH-I-02-05-00035-00)
·grouping of teachers. In all other respects - school-based management and community
participation - the junior secondary schools appeared to be the same as the primary schools.
Province
The widely-held perception before the field visits was that the MBE had achieved the best results
in East Java followed by the results achieved in Central Java. NAD was seen as a special case.
This review challenges some of those perceptions. Firstly, the evaluation team found excellent
schools and good examples ofthe MBE approach in all three provinces. Secondly, the review of
MBE-assisted schools in East Java and Central Java suggested that the impact of the project was
about the same in the two provinces; East Java was not measurably better than Central Java.
Thirdly, it appears that there is a stronger correlation between the quality of teaching in the
school and the length of time a school has been in the MBE program than in the province where
the school is located. There are exceptions, but schools that had been in the MBE program for a
little over a year were generally not as accomplished as schools that had been in the MBE
program for three or more years. In all likelihood, teaching methods change slowly from didactic
to active learning methods and teachers need some time to master the new methodologies. The
fourth finding comes from NAD province where we found the program to be generally less
impressive than the programs in Central and East Java. The implementation ofthe MBE program
in NAD was different and not as well coordinated illustrating, perhaps, that a well coordinated
approach is very important to program success.
LESSONS LEARNED
Clear and Focused Conceptual Framework
The MBE has a very clear and focused conceptual framework that the evaluation team found was
well understood by all stakeholders at all levels. Further, this conceptualization is very clearly
framed in contemporary understandings of schools, classrooms and teachers, and is well
informed by lessons learned and best practice in the Indonesian context. This clear and focused
conceptual framework appears to be an important factor in the success ofthe program.
Well Sequenced and Coordinated Whole School Approach
It was very clear to the evaluation team that the well sequenced and coordinated whole school
approach adopted by MBE where teachers are actively supported by the school principal, other
teachers, and the wider school community is instrumental in the effective implementation of
student focused learning. While there may have been some differences between schools or, even
within individual NIBE schools in terms of the extent of changes and their effectiveness, it was
generally observed that the entire school had made some progress in adopting PAKEM.
Start Small and Build on Success
The MBE approach demonstrates very clearly the advantages of starting small and then building
on success, both in the origins of CLCC and in MBE implementation itself. The CLCC program
developed pilot approaches initially when it started in 1999, and by 2002 was adopted by MONE
as the official approach to school-based management in primary schools. Similarly, the NIBE
began as a small intervention focusing on strengthening local government capacity to manage
basic education, and was scaled up with an added emphasis and funding for the improvement of
teaching and learning and the coverage extended in Central and East Java. As a result of the
Final Evaluation ofthe Managing Basic Education (MBE) Project ABE-BE IQC (Contract # EDH-I-02-05"00035-00)
IX
small start and the creed of building on success, the MBE implementation engaged schools in an
integrated and meaningful series of interventions which resulted in very high levels of school
ownership.
Do what you do well
There has been a tendency in the past for donors and the Gal to maximize the number ofschools
that receive project interventions. The lesson learned across a number of donors and projects is
that this approach often has the effect of spreading the quality and duration of inputs and
therefore limiting the impact and long-term sustainability of the intervention. A more effective
approach, observed within the MBE experience and approach, is to limit the number of schools
to ensure that the investment in individual schools is sufficient to facilitate a whole school
development and involve enough personnel to provide sufficient critical mass for a sustainable:
change.
PAKEM Plateau
The MBE program's most significant outcomes have been in improving the physical classroom
learning environments, with less spectacular progress being made in improving teaching
methodologies. In most classes visited the extent of changes in the classroom varied widely. In
some classes observed the lessons were still largely didactic, while in others teachers were able
to implement the very best student-centered teaching practices. Most teachers were somewhere
between these extremes. In any education system, changing teaching practices will always be a
significant challenge as these changes are difficult to achieve and it will take longer to produce
measurable impacts than the other types of MBE interventions. This situation may have been
hampered by the relatively soft indicators used by MBE (and CLCC) in classroom monitoring. A
more focused classroom monitoring approach coupled with targeted training materials
responding specifically to deficiencies in teaching methodologies may produce better results and
might avoid the possible plateau that we may be seeing in teaching methodologies.
Selection Process may be too Prescriptive for Widespread Use
The selection processes used by MBE is orientated towards schools that demonstrate a capacity
and willingness to change and, as a result, many of those selected are quite good schools within
the general Indonesian context. The MBE model has an approach and implementation process
that may not apply equally well to poorly developed schools. The MBE project support provided
is prescriptive in that it tends to direct schools along pre-determined expectations and outputs. A
problem may be that a particular project intervention within an implementation timetable may
not be relevant to a particular school at that particular time. This may be particularly damaging
for a model that proposes to introduce the MBE intervention in a few schools in a district and
then encourage the adoption of the MBE model throughout the district. A more ideal model
might be a school selection and engagement process involving a detailed assessment of
individual schools with interventions planned and tailored as a result ofthis assessment.
Coordination with GOI
While the MBE project is obviously pitched at the district level, which is appropriate within the
context of decentralization, a number of MaNE and MORA officials made the point that they.
were not very well informed about the project. The points raised included that while they may
Final Evaluation ofthe Managing Basic Education (MBE) Project
X
ABE-BE IQC (Contract # EDH-I-02-05-00035-00)
have received project newsletters and reports periodically, these generally did not provide
sufficient insights into the challenges and lessons learned that could inform GOI policy and
strategic planning. The MONE officials wished to become better informed about USAID
education project activities, and while efforts are being made by the projects, this area of
communication and dialogue needs to be explored and improvements made.
Policy Dialogue
The MBE project has provided some·good gains in individual schools and districts, and these
gains are in support of GOI broad education policy. However, the MBE project is not designed
.or implemented in such a way that would encourage central government ownership with
subsequent impact on broader policy formation and integration. There has been some integration
at the district level but, the results vary considerably. Consequently, this is an area where there is
an opportunity to inform or engage in a policy dialogue at the central level in what is a critical
policy area.
CONCLUSIONS
On all accounts, the MBE project has had impressive impacts on district and school
management, community involvement, and teaching and learning. When compared to the
average Indonesian school, MBE-assisted schools are better managed institutions, with active
community participation, richer learning environments, and teachers with a wider variety·of
teaching approaches and materials.
Despite the very obvious impressive progress made by the MBE project, there are a number of
improvements in the MBE model that can be made. These include:
• Further improving teaching methodologies and learning resources, especially advanced
student questioning techniques, classroom management, and student assessment;
• Further strengthening teacher and principal professional development that is school-based
and performance-led;
• Further strengthening and reinforcing school management practices by encouraging
better management practices at the sub-district, district and provincial levels;
• Further encouraging community participation by providing incentives to continued
community involvement in schools; and
• Further encouraging system-wide policy review and dialogue.
Final Evaluation ofthe Managing Basic Education (MBE) Project ABE-BE IQC (Contract # EDH-I-02-05-00035-00)
XI
SECTION I. INTRODUCTION
1.1 EVALUATION OBJECTIVES AND SCOPE
This report is the Final Evaluation of USAIDlIndonesia's Managing Basic Education (MBE)
project. The MBE project has served as the pilot for USAID/lndonesia's Decentralized Basic
Education (DBE) program which is now the cornerstone of the U. S. Government's assistance to
Indonesia's education sector.
The objective of this evaluation is two-fold: first the evaluation examines the overall results and
impact of the MBE program, including the project's strengths and weaknesses, implementation
mechanisms and lessons learned; and second the evaluation explores approaches and strategies
for continued support of successful elements after its completion by linking it with the
Government of Indonesia (Gal), the DBE, and/or other implementers. It this regard, the MBE
evaluation is both backward-looking (at the program accomplishments) and forward-looking
(linkage with DBE). More specifically, the evaluation seeks to:
• Determine to what extent the project has accomplished the tasks laid out in the contract
and work plans;
• Report how the project stakeholders perceive theMBE intervention and what elements
are seen to be most useful or ineffective;
• Develop actionable, prioritized recommendations for the GOIand the DBE implementers
for enhancing their performance;
• Identify key lessons learned from the experience and relate them to the on-going DBE
program; and
• Assist USAIDlIndonesia to determine whether and how to continue support to the MBE
target districts in East and Central Java.
1.2 METHODOLOGY
The evaluation was conducted between January and March 2007 by a team of national and
international cqnsultants. Field work was led by the independent consultants but guided by the
very able staff ofthe MBE and DBE projects.
The evaluation team used four basic methods to obtain its findings, including:
• Reviewing of project documents, other evaluation findings from similar programs, and
research and background studies in education;
• Observation of MBE-assisted, non-MBE-assisted, and DBE-assisted schools, principals,
teachers and other educators;
• Targeted questions and interviews with individuals and officials knowledgeable about the
MBE and DBE projects; and
• Targeted questions in focus group discussions with single and mixed groups of
stakeholders.
Stakeholders interviewed included Ministry of Education (MaNE) and Ministry of Religious
Affairs (MORA) officials at the national, provincial, and district levels; school supervisors;
Final Evaluation ofthe Managing Basic Education (MBE) Project ABE-BE IQC (Contract # EDH-I-02-05-00035-00)
1
school principals; teachers; school committee members; parents and parents' groups
(paguyaban); MBE project facilitators and administrators; and students. Although checklists for
classroom observation were developed, no effort was made to collect data using these formats.
Moreover, USAID/Indonesia provided the evaluation team with an illustrative list of 38 key
questions to guide the investigation and the evaluation team used these questions to develop a
work plan.
A total of 40 schools were visited in three provinces, with 19 in Central Java, 8 in NAD, and 13
in East Java. The 40 schools represent about nine percent of the total schools assisted. The
sample ofschools was in 7 school districts. We intentionally reduced the number of school visits
mid-evaluation and saw fewer schools in NAD and East Java so that we could increase our time
with stakeholder interviews. Of the 40 schools we visited, 22 were public (MaNE) primary
schools, 10 were madrasah (MORA) primary schools, three were junior secondary schools, and
five were non-MBE public primary schools. We interviewed hundreds of stakeholders in these
schools, many in focus groups, making it impractical to record all the names of the persons
interviewed.
1.3 CONSTRAINTS IN UNDERTAKING THE EVALUATION
There were several constraints to the evaluation of the MBE. Little baseline data was collected
prior to commencement of the MBE program interventions on key variables like student
achievement and, although the MBE program developed project tests to measure student
achievement based on project interventions, there was no standardized data available from
formal tests and examinations conducted by local and national authorities to demonstrate the
program's impact on the quality of education.
Given the scale and geographic spread of the JYlBE project, there was insufficient time and
human resources for the evaluation team to undertake an in-depth analysis of classroom behavior
and the time spent in each school was sufficient to form impressions only. The sample ofschools
selected was small and could not be random owing to the logistics of visiting school within the
available time. Although a wide variety of schools were visited, some types of schools such as
junior secondary, were underrepresented. Moreover, schools were selected by national and
provincial MBE project coordinators and the final sample of schools may be skewed towards
better performing schools. In addition, schools were obviously expecting the evaluation team and
they were very prepared for our visit. These constraints were taken into account by the
evaluation team in evaluating the data and forming conclusions.
1.4 STRUCTURE OF THE REPORT
The report is organized in six main sections. Section I is the introduction. Section II provides
background information about the USAID/lndonesia education strategy and interventions, a brief
description of the MBE and DBE programs, as well as a review of the Gal education policy.
Section III outlines the evaluation findings (accomplishments, strengths and weaknesses, and
lessons learned by each of the three components: district and school-based management,
community involvement and teaching and learning. Section IV summarizes the lessons learned
from the findings, examines how well the MBE project has accomplished its tasks, and measures
the cost-effectiveness ofthe program. Section V examines the implications ofthe lessons learned
and reviews the issues surrounding the sustainability of the MBE intervention. The report
Final Evaluation ofthe Managing Basic Education (MBE) Project
2
ABE-BE IQC (Contract # EDH-I-02-05-00035-00)
finishes with Section VI which outlines strategies for supporting the MBE intervention once the
program is concluded.
SECTION II. BACKGROUND
ILl OVERVIEW OF USAID EDUCATION STRATEGY IN INDONESIA
Indonesia's school system is immense and the population of school-age children is one of the
largest in the world. Despite rapid progress achieving almost universal enrollment in primary
education under a centralized management system over the last 30 years, significant problems in
the education system continue to exist, including low public funding, poor completion rates, low
teacher qualifications and poor classroom methodologies. Improving educational quality with
limited central government funds is a key educational challenge and, given the magnitude ofthe
problem, the Government ofIndonesia (GOI) has initiated a decentralization process in the hopes
of spreading the governance, financial, and managerial responsibility for improving education
across different stakeholders. Under decentralization, school committees and district
governments are expected to hold schools and teachers accountable for educational quality and,
the GOI is developing school and teacher accreditation criteria and minimum service standards
to measure their performance.
Within the framework of school management and teacher accountability measures that are
currently being put in place by the GOI, USAIDlIndonesia's Strategic Objective is to Improve
the Quality of Basic Education in Indonesia by achieving three intermediate results: 1) More
effective Decentralized Management and Governance of Schools; 2) Improved Quality of
Teaching and Learning; and 3) Increased Education Relevance and Workforce Skills for Youth.
To achieve these results, USAID intends to invest $157 million over the period 2005-2010 in
four programs. The portfolio ofprograms USAIDlIndonesia is supporting includes: 1) Managing
Basic Education (MBE), the subject of this evaluation described more fully below, 2)
Decentralized Basic Education (DBE), 3) Sesame Street Indonesia, the renowned television
show aimed at better equipping children to start school; and 4) Opportunities for Vulnerable
Children, a program that promotes inclusive education for children with special needs. Some of
the major results expected from these investments are:
• Increased capacity oflocal governments to plan for and manage education services;
• Increased community participation in the provision of education;
• Better teaching performance as a result ofin-service teacher training;
• Better student and school performance;
• Livelihood skills for in-school and out-of-school youth increased; and
• Replication of best practices.
. 11.2 DESCRIPTION OF USAID INTERVENTIONS IN INDONESIA
The main component of the USAIDlIndonesia's education program and the cornerstone of
President Bush's Indonesia Education Initiative is the Decentralized Basic Education program
(DBE). The DBE focuses on improving the quality of basic education in primary and junior
secondary schools, both public and private. The program has three main goals: strengthening the
capacity of local governments and communities to manage educational services (DBE1);
Final Evaluation ofthe Managing Basic Education (MBE) Project ABE-BE IQC (Contract # EDH-I-02-05-00035-00)
3
enhancing teaching and learning to improve student performance in key subjects such as science,
math, and reading at primary level·(DBE2); and assisting Indonesian youth to gain more relevant
life and work skills to better compete in a world economy (DBE3). These programs are expected
to reach 9,000 public and private schools, 2.5 million students, 90,000 educators, and one million
youth by 20 IO.
Serving as the pilot program for the DBE program, the Managing Basic Education (MBE)
project works with 23 local district governments in three provinces (East and Central Java and
NAD), as well as in Jakarta, in three project components: district and school-based management,
community participation and teacher training. The district and school-based management
component seeks to strengthen the' capacity of local government and school principals to
effectively plan, manage and deliver quality basic education services. The community
participation component works to strengthen the position and role of local stakeholders - parents,
teachers, school committees, community organizations, local parliaments - in planning and
managing basic education. The teacher training component trains teachers in active learning
methodologies (known in Indonesia as PAKEM) and learning material development. The three
components are intended to be'mutually reinforcing and the program is delivered at the school
level in a whole school approach.
II.3 OVERVIEW OF GOVERNMENT OF INDONESIA EDUCAnON POLICY
In January 2001 the Government of Indonesia (Gal) enacted new laws on decentralization and
revenue sharingI that put in place a series of swift and comprehensive changes transferring
power, management authority and funds for the delivery of basic services from the central
government to districts and municipalities.
The subsequent promulgation in July 2003 of the new Education Law 20/20032 saw the transfer
of responsibility for the delivery of education to lower levels of government. Some decision
making power was given to schools with school-based management encompassed in Articles 51
and 56 whereby communities through school committees can become involved in planning,
monitoring and school improvement. An important exception is that the responsibility for
religion was not handed over to the districts. Subsequently the Ministry of Religious Affairs
(MORA) has opted to make decentralization voluntary, and has not yet decentralized
responsibility for madrasah (an Islamic school), an important reason being that the large majority
of madrasah (approximately 90%) are private, while most other schools are publicly owned and
supported.
The Gal policy objectives and sector reform priorities are encompassed in the Ministry of
National Education (MaNE) Strategic Plan 2005-2009 (RENSTRA) and the MORA Strategic
Plan 2006-2015. Other important legislation includes a National Plan of Action for achieving
Education for All, and a new curriculum for all levels of schooling. priorities include (i) an
expansion of access to junior secondary education, (ii) improved education quality and standards
through the establishment of minimum service standards (MSS), the enhanced availability of
instructional materials and improved learning environments, and (iii) the strengthening of
1Law Number 22 ofYear 1999 concerning Regional Autonomy; Law Number 25 ofYear 1999 concerning the Fiscal Balance between the
Center and the Regions .
2Law Number 20 ofYear 2003 concerning the National Education System, signed by President Megawati Sukarnoputri on July 8, 2003
Final Evaluation ofthe Managing Basic Education (MBE) Project
4
ABE-BE IQC (Contract # EDH-I-02-05-00035-00)
governance and accountability mechanisms, including both financial management and control
and the wider accountability ofschools to their communities through school committees.
The GOI recognizes the need to upgrade the qualifications of teachers in order to address these
and other education quality issues. In response, a new Directorate General for Quality
Improvement of Teachers and Education Staff - Direktorat Jenderal Peningkatan Mutu Pendidik
dan Tenaga Kependidikan (PMPTK) has been established. The task ofthis Directorate General is
to ensure that teachers and other education personnel meet minimum academic and competency
standards linked to appropriate remunerations.
The new Law on Teachers passed in December of 20053 provides a framework ofteacher rights,
roles and responsibilities, and defines the responsibilities of various levels of government in
relation to teachers. The wording has been kept fairly broad in some areas, with the intention to
specify more detail through subsequent regulations. The Law includes, among others, sections on
(i) professional principles; (ii) qualifications and competencies; (iii) rights and obligations; (iv)
compulsory work and post-graduation service contracts; (v) appointment, placement, transfer and
discharge; (vi) guidance and development; and (vii) rewards.
While national policy provides for a staffing formula for primary and secondary schools, there
are many inequities in teacher deployment across both schools and districts. The districts have
authority to address these inequities, however, there is a general reluctance to do so. Further,
stafftransfers between districts and provinces appear to be very difficult.
In the area of qualifications and competencies, standards have been increased to require teachers
to have at least a bachelor's degree (Sl) or four year diploma (D4), in addition to demonstrated
competencies in four areas covering pedagogy, personal, social and professional qualities. These
competencies will require training credits to be earned through accredited universities. The GOI
goal is that 40% of teachers will have Sl level qualifications by 2009. This is an ambitious task
given that there are 1.36 million primary teachers and 490,000 junior secondary teachers, and
that only 17 percent of primary and 29 percent of the junior secondary teachers hold S1
qualifications. Obviously the current system could not cope will a full-time upgrading approach
which would take teachers out of the classroom, therefore there will be the need to provide
teacher in-service training.
In this regard the Law provides that both the national and regional governments have a duty to
provide in-service training budgets to facilitate the increase in academic qualifications of
teachers. Professional certificates will be issued to teachers meeting both the academic
qualifications and competencies. This will have significant salary implications as a teachers
awarded these professional certificates will be eligible to receive a professional allowance equal
to their base salary, which in effect will double take-home salary. An important element of the
Law is that these professional allowances will- be funded by the national government, regardless
of whether the teachers are employed by national or regional governments, or communities.
3 Law Number 14 ofYear 2005 concerning Teachers and Lecturers
Final Evaluation ofthe Managing Basic Education (MBE) Project ABE-BE 1QC (Contract # EDH-I-02-05-00035-00)
5
11.4 PROJECT DESIGN AND RELEVANCE
Project Design
The MBE commenced in February 2003 as a 19-month, $3 million, USAID Office of
Decentralized Local Government capacity building program focusing on strengthening local
governments in their capacity to manage basic education. This original design was a relatively
small intervention targeting 10 local governments, and the focus was on community participation
in school management and the development of clearly defined roles for local institutions
involved in managing education that included the Dinas Pendidikan (District Education Office /
DEO), DPRD (District Legislature), Bappeda (District Planning Agency), and Dewan
Pendidikan (District Education Board). The overall aim was to improve district-level
management of the education sector and to improve transparency, effectiveness, and
accountability in education funding.
In October 2003, President George W. Bush announced the $157 million Indonesia Education
Initiative. Subsequently the USAID education strategy was further developed, and USAID
created a new office to manage education in Indonesia. Given the changed strategic direction of
USAID, the MBE was modified anq extended in August 2004 through to March 31, 2007 with
funding increased to $10,099,564. Consequently the project extension and modification saw an
added emphasis and funding for the improvement of teaching and learning in grades 1 to 9, and
extended coverage for Central and East Java in 20 districts.
At the district level, interventions focused on data collection, planning, and the development of
formula funding to support school-based management. At the school and community level the
focus was on (i) developing school-based management, (ii) increasing community participation,
and (iii) improving the quality of teaching and learning through the adoption of PAKEM.
Additional expansions have occurred in December 2005 where the MBE commenced work in
two districts of post-tsunami NAD province in support ofthe DBE program, and in January 2006
.with an additional seven schools inCentral Jakarta.
The MBE design allows for the selection within a district a lO-school cluster in two target sub￾districts. This selection is done in consultation with district education authorities, and the schools
selected may include a mix of primary and junior secondary schools from the public, private,
secular and religious sectors. Approximately 450 schools in 23 districts have been targeted for
assistance, of which approximately 75 percent are primary schools, and the remainder are junior
secondary. Ofthat total, approximately 20 percent are madrasah.
In addition to the targeted MBE assistance, monitoring reports have indicated relatively high
dissemination by district governments into schools not participating in the MBE project. For
example project monitoring reports from 2006 indicate that district governments using their own
funding and resources had disseminated elements of the MBE program to more than 6,000
schools. This has generally been achieved with trainer support from the MBE project
The approaches adopted by the MBE are relatively widespread in Indonesia,·and have evolved
from activities that commenced in the 1980s such as the Permantapan Kerja Guru (PKG) -
strengthening the work of teacher - and the Active Learning through Professional Support
(ALPS) program that focused on changing teaching behavior and encouraging students to
become mo~e active learners. In 1999 UNICEF and UNESCO, along with other bilateral donors,
joined with MONE to implement reform in what became known as the Creating Learning
Final Evaluation ofthe Managing Basic Education (MBE) Project
6
ABE-BE IQC (Contract # EDH-I-02-05-00035-00)
•
Communities for Children (CLCC) program. This program continues today, supported by a
range of donors and is made up of three components: School-Based Management (SBM),
Community Participation (CP) and Active, Creative, Effective and Joyful Learning which is a.
student-centered teaching and learning approach commonly referred to by its Bahasa Indonesian
acronym as PAKEM. The MBE project replicates the UNICEFIUNESCO's models for SBM,
Community Participation and the PAKEM approaches of CLCC with the addition of district￾level management assistance. The CLCC approach has now been adopted by the Gal as the
official approach to SBM.
As described on the DBE project website (www.dbe-usaid.org) the DBE project aims to improve
the quality of education through interventions over three integrated components: 0) decentralized
education management and governance (DBEl), (ii) teaching and learning (DBE2), and (iii)
relevant life skills for youth (DBE3).
The prime focus of the DBEI interventions is capacity building of local government in their
management of basic education, improving governance in schools, and introducing the use of
information resources to enhance management. At the primary school level, DBE2 includes
cluster-based in-service activities focusing on active learning and improved student assessment
approaches. The DBE3 interventions focus on developing relevant life-skills for junior secondary
school-aged students in preparation for lifelong learning and entry into the workforce.
The MBE is often described as a pilot project, and in this sense the DBE in broad terms can be
seen as a follow on activity in that DBEI and DBE2 expand on the original MBE interventions.
The major differences are that DBEI has a more focused approach to improving the capacity of
local government whereas the MBE has focused on both districts and school level. With DBE2
the major differences are that a formalized cluster approach has been adopted, local universities
are involved in improving teaching and learning for teacher accreditation, Cluster Resources
Centers (CRCs) are to be developed along with the provision of learning resources, pilot
kindergartens using audio instruction will be introduced, and a range of information and
communication technologies such as digital libraries and "Education Hotspots" through wi-fi
will be introduced. The DBE3 component was never part of the MBE project, although DBE3
has taken on responsibility for working in junior secondary schools.
Project Relevance
Both the MBE and DBE are well aligned with the Gal decentralization policy and
MONEIMORA strategic plans, especially in the areas of increasing the involvement of local
communities in schools, improved governance and accountability through the strengthening of
school committee capacity, and the increased availability of instructional materials and improved
learning environments.
The MBE was implemented prior to the new Law on Teachers and establishment of the
Directorate General for Quality Improvement of Teacher and Education Personnel, and
consequently does not provide the now necessary focus on improving the academic
qualifications of teachers. The DBE2, with the links to local universities, is in a good position to
provide accredited in-service teacher training activities that can be linked to the required
standards in teacher qualifications and competencies.
The evaluation team found very high levels of ownership among all levels of MBE stakeholders.
This ownership was supported by good levels of understanding ofthe purposes ofthe MBE, how
Final Evaluation ofthe Managing Basic Education (MBE) Project ABE-BE IQC (Contract # EDH-I-02-05-00035-00)
.7
all the various components fitted together, and stakeholders generally had very clear visions for
the future.
SECTION III. FINDINGS
lILt DISTRICT AND SCHOOL-BASED MANAGEMENT
As has been the case in most of Indonesia's public sector, the Indonesian education system has
been traditionally quite centralized. In 2003, the Education Law transferred principal
responsibilities, authorities, and resources for the delivery of education to lower levels of
government and transfers some decision-making power to schools. School-based management is
embodied in Articles·51 and 56, which engage the wider community through school committees
in planning, monitoring and improving school quality. Responsibility for education now lies at
the district level, except for madrasah which continue to be managed centrally by the Ministry of
Religious Affairs (MORA).One ofIndonesia's major challenges is to confront pre-existing goals
like improving educational quality in the context of decentralization.
As noted in the National Plan of Action for Achieving Education for All, the 2004 Indonesia
Education Sector Review, and the 2005-09 Education Renstra (strategic plan), Indonesia has
many challenges to overcome to achieve its goal of quality education. Among others some ofthe
issues related to management are ineffective management at the school level, inefficient
deployment of staffwithin schools, large numbers of primary school teachers who do not meet
the minimum qualifications or have not mastered the curriculum, and high teacher absenteeism.
The MBE project supports a variety of activities at the school, sub-district, and district levels to
improve school management as well as school committees, education boards, and local
government officials to improve their capacity to better manage educational resources
(personnel, infrastructure, supplies, and finances).
The project.seeks to spread governance responsibility across different stakeholders, train
principals in management and leadership, strengthen accountability and transparency
mechanisms, involve parents and community stakeholders in the schools, and train educators in
the system. It appears that the MBE project has made significant strides toward improved school￾based management but that there remain many structural problems to overcome. Although it is
somewhat outside the responsibilities and scope of this report to analyze, some of the problems
noted are: 1) it appears that functions, structures and financing up and down the education
system are unclear and possibly misaligned; and 2) there is mixed management performance
across districts.
Findings: Accomplishments and Weaknesses
When schools perform well, it is the result of a variety of factors but perhaps the two most
important factors are: the quality and performance ofthe teaching force and strong leadership by
the school principal. Under decentralization, school committees and district governments are
expected to hold schools and teachers accountable and school and teacher accreditation criteria
and standards are being developed to help measure performance but, since the law has yet to be
enacted, it is not yet clear how the accreditation will be used or how schools can improve their
performance.
Final Evaluation ofthe Managing Basic Education (MBE) Project
8
ABE-BE IQC (Contract # EDH-I-02-05-00035-00)
Role of School Principal
Under the centralized model, school principals received directions and instructions from various
levels of government which were then relayed on to school staff. In a decentralized system,
school principals are expected to discard the authoritarian model, be more independent of
authority, and more open and cooperative with the community. In addition, the school principal
should show 1) open, flexible, and adaptive school management leadership; and 2) provide
instructional leadership. By and large, the majority of school principals interviewed in for this
report demonstrated those qualities.
As now required by law, school principals develop, in concert with the school committee,
teachers, and community leaders, a school development or strategic plan that outlines the goals,
objectives, priorities and methods of achieving those outputs for the school. When most school
principals were asked to describe the process by which the strategic plan had been developed,
most described a very open process and they demonstrated a good understanding of their new
more democratic and consensus-building role. Although almost always in a proscribed format,
many of the strategic plans were extremely well done, well thought-out and articulate
documents. Others were somewhat perfunctory with confused and somewhat contradictory
objectives. Most school principals could relatively easily articulate the key objective and
priorities in the strategic plan but, some could not or badly fumbled questions related to the
strategic plan, perhaps indicating an incomplete involvement in the development of the plan or
lack ofinterest in the content ofthe plan. Only a few ofthe principals had considered updating or
reexamining the strategic plan, although in some cases as many as the first five priorities had
been achieved and the plan appeared to be out-of-date. Strategic plans were almost always
prominently displayed in the school but it was often as if the principal had come to see the
display ofthe plan as proof of his or her new openness rather than as a useful tool.
School budgets were also always prominently displayed somewhere in the school and, even more
than strategic plans, principals were inclined to point to them as evidence of a new era of
transparency and accountability. The display of school budgets clearly has ushered in a new era
of openness but, on close inspection the budgets were nearly identical from school to school and
the information in them was generally nothing different from that already widely known, like a
listing of the teachers' annual salary. Interestingly, budget information on school fees or other
.funds collected by school committees were not always available and the several times new
classrooms were being constructed, budgets for the project were not displayed.
Most principals felt that they were better leaders as a result of the MBE training but, they were
hard-pressed to cite examples of their skills, perhaps because of modesty, other than the school
development plan, collaboration with parents and the community and the budget process. Some
principals cited increased confidence in their role as principal and claimed they were better able
to manage a positive learning environment. Others maintained they had provided leadership in
the classroom by initiating change and innovation but, the influence on the principals was
difficult to substantiate and, although teachers generally gave credit to principals for maintaining
a good vision, few noted much classroom involvement or leadership from principals.
Despite the fact that most MBE-assisted schools visited were obviously markedly different from
what they had been and were now dynamic and happy places where real learning was taking
place, there were some issues not easily or openly discussed and one got the impression that
principals were unwilling to change the status quo. For example, some principals managed badly
Final Evaluation ofthe Managing Basic Education (MBE) Project ABE-BE IQC (Contract # EDH-I-02-05-00035-00)
9
overstaffed schools - some with student/teacher ratios below 10 - but either saw nothing amiss or
were unwilling to discuss the issue. In more than a few cases, school committees were not very
active and provided limited support and involvement in the school and, for whatever reasons, the
principal was content not to address the problem. It may be that these issues are outside the
purview ofthe principal or that repeated attempts to address the issue had lead to frustration and
apathy but, in some areas principals were not always change agents or for that matter good
stewards ofpublic resources.
for the most part, school principals in MBE-assisted schools have embraced their new role in
both instructional and school management leadership but they seem to be more comfortable as
school managers rather than as instructional leaders. The educational reforms of school-based
management introduced by decentralization and reinforced under the tutelage ofthe MBE project
has resulted in dramatically changed behaviors in school principals, including for example
encouraging transparent, accountable, and inclusive school plans and finances, and creating a
more open, innovative and positive learning atmosphere in schools. However, if the
decentralization effort and the enabling climate for change and innovation at the school level are
to be successful, the program will need greater support from the education office (DINAS) at the
district and sub-district level.
District Education Office (DEO)
Within the decentralized system of education, the district education offices occupy a pivotal
position between policy makers and service deliverers. Any intervention that involves
development at the school level should also involve interaction and provision of training and
other capacity building at the district level. Similarly, any intervention aimed at supporting
national policy initiatives should also engage at the district level to facilitate implementation of
the required policies. It appears that the MBE project did an excellent job of the former; it
provided training in capacity building at the sub-district and district levels that went a long way
toward creating the space for teachers and principals to continue to grow and innovate. Perhaps,
additional attention needs to be devoted to improve district and provincial policy making
(provincial capacity building was beyond the scope ofwork ofthe MBE).
One ofthe key objectives ofthe MBE project was to increase the capacity oflocal government to
plan for and to manage education services. The MBE targeted three particular institutions, the
District Parliament (DPRD), the Education Council (Dewan Pendidikan), and School
Committees (Komite Sekolah), and included them in training and other workshop activities. In
particular, these groups were trained at the district and sub-district in problem analysis, school
mapping and the interpretation of data, planning, budgeting, and the implementation of plans.
Virtually all the district and sub-district officials interviewed extolled the merits of the training
and explained that the program provided them with the skills needed to participate in the
planning, budgeting, monitoring, and implementation of school reforms. Most everyone
suggested that they were much better informed as a result of the MBE intervention and they
believed that the school system had become much more transparent and accountable. When
pressed to outline examples where the MBE training had made a difference in their behavior,
most DEO officials pointed to the new found skills in the analysis of data and systematic
planning that had begun to improve the allocation of funds within the district. Despite these
generally very good findings, there were some major inefficiencies observed and a number of
"mixed" signals that might indicate more needs to be done in this area.
Final Evaluation ofthe Managing Basic Education (MBE) Project
10
ABE-BE IQC (Contract # EDH-I-02-05-00035-00)
One important indicator of the "mixed" impact of the MBE project may be in the area of Gal
ownership of the MBE project. The evaluation found that the ownership of the MBE-assisted
reforms varied from district to district. Some districts have embraced the MBE concept
completely and, using their own funds, have extended and replicated the MBE intervention to
every school in the district. Other districts have budgeted and allocated funds for substantial
follow-on and replication activities using MBE facilitators. Some districts have made plans to
replicate the MBE training program in some portion of their schools but have not allocated
funding to expand the MBE program in other sub-districts, while other districts have no plans in
the short-run to continue the MBE program.
It is not clear why there is considerable variation in the ownership of the MBE or even if a lack
of commitment to replicate the MBE program indicates lack of ownership. And the variety of
responses from district governments on the issue ofreplication is a two-edged sword; it clearly
indicates that some enlightened district officials have seen the importance of delivering the
program to all schools. But, the wide variety of responses concerning the replication of the
program also seems to indicate that the efficacy of the MBE intervention in support of MaNE
objectives has not been sufficiently made. Instead, the MBE project appears to be seen by some
in the Gal as an interesting and worthwhile program that parallels MaNE's objectives but not
enough has been done to show the importance of the school-based intervention on district and
national policy making.
A second "mixed" signal that is difficult to interpret is the fact that a substantial amount of
inefficiency was observed in the school system. If the goal of the school-based management
intervention is to assist in the creation of a transparent and accountable school system and
everyone is quick to point to the symbolic importance of school budgets and strategic plans on
the wall, then it may be equally important to see that the system is only partially accountable. For
example, some schools were over-staffed and had very low student/teacher ratios, while others
had student/teacher ratios so high (45-50 pupils per class) as to make the application of active
learning methods difficult. Another example of inefficiency might be seen in the often heard
stories of principals being trained in school-based management techniques only to be transferred
to another school, leaving one school without a SBM-trained leader and a MBE-trained principal
in a school without a responsive faculty or community. Other examples could be shown in repair
and maintenance ofschools, in the failure to introduce money-saving approaches like, multigrade
classrooms, and the apparent failure to apply formula funding in some districts. Surely, the
rational deployment of staff and the maximization of scarce funds are hallmarks of a well￾managed, accountable school system and evidence to the contrary suggests that, while the issue
may be outside the purview of a district and school-based intervention, perhaps the MBE project
should have worked more closely with national, provincial and district level officials to include
these issues in a policy dialogue.
On the other hand, the MBE project was at the mercy of local governments to make difficult
decisions. The MBE introduced district level officials to data-based management techniques and
formula funding in an effort to promote equitable and transparent funding in schools. The MBE
helped district government to collect and analyze data, make strategic plans, and base district
funding of schools on needs. But, without decisive action to address these inefficiencies and the
"political will" to make difficult decisions, these issues remained unsolved and perhaps outside
the purview ofthe MBE project.
Final Evaluation ofthe Managing Basic Education (MBE) Project ABE-BE IQC (Contract # EDH-I-02-05-00035-00)
11
Implications and Lessons Learned: Whole school approach works well
The school-based management program is based on practices started by the Directorate General
of Primary and Secondary Education and refined in UNICEF's CLCC. The SBM approach is by
no means new but, what may be unique is the approach of training multiple stakeholders in an
integrated training package. As all stakeholders receive training in school-based management,
community involvement and active learning techniques each stakeholder understands the role
and responsibility of the others and what they are trying to accomplish and the whole becomes
more than the sum ofthe parts.
SBM training makes a significant contribution to school management practices
Although schools are pre-selected for their interest in change and leadership and the schools at
the start may be some ofthe best managed in Indonesia, the MBE SBM program clearly makes a
significant difference in transforming schools. The Gal requires the development of budgets and
school plans but the MBE SBM training has produced some excellent examples of school
strategic plans and is an important first step in the community banding together behind the
school. Moreover, the leadership training is clearly a foundation and confidence-builder for
many school principals who appear to use the training to great advantage.
School principals are more comfortable as managers
Most of the principals appeared to be more comfortable as school managers than as instructional
leaders. Since most principals have come from the ranks of teachers and should, therefore, be
comfortable in the classroom, the finding is difficult to understand. On the one hand, it may be
that principals are more comfortable with traditional teaching methods or, that they, as former
teachers, resented the role ofother instructional leaders and would prefer not to interfere, unless
asked into the classroom. On the other hand, it may be that school principals see themselves as
too busy or too important to worry about what happens in the classroom. In any case, it appears
that some adjustments may be needed in the training program to emphasize the role of the
principal as an instructional leader.
District education offices playa pivotal role in the success of the SBM approach
Nearly all DEOs praised the efforts of the MBE program but, at this point near the end of the
program, only about half of them had committed funds or had definite plans to extend or sustain
the MBE program in their district. In addition, some DEOs had inadvertently sabotaged the MBE
effort by transferring principals who had been recently trained by MBE to non-MBE schools.
Finally, as evidenced by the incidence of over- and understaffing and despite the best efforts of
MBE and SBM training, there is considerable more work ahead. before the education system is
seen as accountable and transparent. Given these and other factors, it would appear that MBE, its
successor (DBE), or USAID could and should do more to engage the Gal in policy dialogue
and, as necessary, finance key studies leading to policy change (Please see Annex VI: Areas for
further Research).
Final Evaluation ofthe Managing Basic Education (MBE) Project
. 12
ABE-BE IQC (Contract # EDH-I-02-05-00035-00)
Strategies and approaches to sustain SBM
Financial sustainability may be demonstrated when there is "take-up" of the program's
methodology, approach, or modules in any replication by local authorities. As we have seen,
about halfthe DEOs have committed to continue the MBE approach andthe SBM once the MBE
program ends, a good record by any account. In addition, the ideas promoted by MBE and its
predecessors promoted by UNICEF and AusAID have been codified into the education law and
have changed the way educators in Indonesia view school-based reform. Given the success ofthe
MBE approach, it is unlikely that anyone looking to improve school performance should use
anything but an integrated whole school approach with SBM, community involvement and active
learning elements in the training package.
III.2 COMMUNITY INVOLVEMENT
Findings: Accomplishments and Weaknesses
Community involvement in school (one ofthe main objectives ofthe MBE program) is meant to
empower the school stakeholders to participate in various ways to increase the quality of
education at schools. The evaluation team found that community involvement at school has
substantially occurred with both school committee (SC) level and at other levels of the
community including parents, businesses, and other school stakeholders.
The SC is involved in schools in four managerial aspects: advising, controlling, supporting, and
mediating. In advising, the SC provided inputs and suggestions on developing school RIPS
(strategic plans) and RAPBS (budget plans) while in controlling, the SC works in supervising
and monitoring the implementation of the RIPS and RAPBS. In the supporting aspects, theSC
provides efforts to reach out for financial resources as well as to provide various kinds of in-kind·
.supports such as intellectual and free labor contributions. In the role as mediating, the SC
channels other school stakeholders' wishes, suggestions, complaints, and expectations about the
school programs to the proper officials. Examples of SCs involved in all four aspects of school
management were seen by the evaluation team.
In order to make those stakeholders more knowledgeable about how to participate in SCs or what
kind of participation they could provide to schools, MBE provided training for SC, teachers,
school principals, parents, and community members. They were trained in how to develop RIPS
and RAPBS. The information gathered from different sources shows that there has been
extensive and widespread participation in the training from schools and communities in MBE￾assisted districts. Many of the participants even came from non-target MBE schools. The MBE
training has Shown great impacts on the role of community in schools. The evaluation team
found that community participation in schools has increased: (1) parental and community
assistance to schools in financial and in-kind terms; and (2) community support of teaching and
learning in schools. .
Financial and in-kind assistance occurs in all of the kinds of schools surveyed, including junior
secondary schools, primary schools and madrasah. Some schools require student daily and or
monthly donations to improve classroom conditions·and learning resources, while others raise
financial support regularly through public alliances for constructing new school facilities -
classrooms, laboratories, and so forth. In fact, community participation is seen by many parents
as a way of sharing the load in supporting children's education. They indicated that it enables
Final Evaluation ofthe Managing Basic Education (MBE) Project ABE-BE lQC (Contract # EDH-I-02-05-00035-00)
13
those on low incomes to contribute in-kind rather than in cash. Instead of cash, they can provide
labor for school maintenance and participate in other school activities. The parents also
underlined the principle of shared responsibility through school committees. Other school
stakeholders were very positive about the effectiveness of SC in increasing community
participation in school activities.
Another form of community participation that has been fostered by MBE is to involve parents
and other community members in teaching and learning process. MBE has encouraged the idea
of parents and members of community working regularly in the classroom to assist in teaching
the students. As result, across the MBE target schools, parents have been providing in-kind
participation in the education process. At the primary level, some parents at lower grades (1 and
2) involved themselves in the classroom to help students learn to hold pencils, write letters, read
alphabets, or on how to work on tasks assigned by teachers. Outside the classroom, some parents
prepare learning resources, filed students' work, while others teach craft skills, English,
computer and other skills after school hours. At the lower junior secondary level, parents also
provided in-kind supports for school activities but they were not as involved in the classroom as
they are at primary schools. One reason given for this was that parents may feel intimidated by
the higher academic levels in junior secondary and therefore they are less confident in becoming
involved in supporting the teaching and learning process.
All of the community involvement and contributions were intended for improving education
process and facilities in order to increase the students' achievement. Therefore, in most schools
any contribution, in whatsoever form, that is going into the schools is under a close watch by the
SC and is managed by the principal based on the foundation of transparency and accountability.
Most ofthe school principals and SCs across the MBE-targeted schools have applied the practice
oftransparency and accountability ranging from fairly good to a very good level.
Across the MBE target schools, SC and parents' classroom-based forums have been established
as a key mechanism to increase community participation in schools. School principals reported
that the role of parents, local communities, local companies, and SCs in funding and providing
support to schools is growing. The support comes in the form of direct cash payment and in the
form ofin-kind contributions to improve the classroom and school physical environment. Parents
and community members are more willing to contribute to school levies when they know how
and for what the money will be spent. This increasing community participation is an indication
oftheir high credibility and commitment to their schools.
The growth of community participation at schools has also provided greater opportunity for
schools to improve their learning facilities in every classroom, which makes it easier for the
PAKEM approach to be implemented. The school community has provided not only learning
resources and some help in classroom but also some necessary classroom furniture needed by
teachers. Additionally, c,ommunity involvement has created more effective management of the
school. because the SC might confront the school principal if there is any wrong doing. The SCs
also assists the principal in ways where both parties can learn and develop their skills in dealing
with tasks in a more transparent and accountable way.
While the community participation is generally increased, the practice is not yet equal across the
districts. The concept of transparency and accountability is relatively new for school
stakeholders and they need to go through a kind of transitional period in order to be more
familiar with the system. An unfavorable condition for some schools is that not all schools can
Final Evaluation ofthe Managing Basic.Education (MBE) Project
14
ABE-BE IQC (Contract # EDH-I-02-05-00035-00)
rely as much on their community for additional funding sources; schools with more open
management and with well-informed stakeholders will receive more support from the
community. On the other hand, schools with closed management and with a less educated and
less fortunate community will also be less exposed to the possibility of having higher
participation and contribution from the stakeholders. Accordingly, some schools will always find
it difficult to raise the level of community participation through parents and community. Instead,
they will either go for more public alliances with companies or depend mostly on government
funding.
Schools are routinely producing the RIPS and RAPBS, and these are prominently displayed.
However, the evaluation team concluded that these plans are not consistently updated, and the
level of actual participation in the development and monitoring ofthese plans by members ofthe
school community was questionable in many schools. In support of these conclusions, it was
found that in many schools the SC met only three or four times a year, and that any community
involvement outside of these formal SC meetings was more consultative than collaborative.
Additionally, in a number of schools the level of community membership of the SC was fairly
limited, often involving only two or three school committee members. Limiting the community
participation in SCs has the potential to reduce transparency. Therefore, there is a need to ensure
that the composition of SCs represents all of the key stakeholder groups within a particular
school community.
Implications & Lessons Learned: Community involvement is ~n important part of the
MBEModeI.
The MBE model of intervention has suggested parents and other community members share the
responsibilities with the government to manage schools. This idea is based on the fact that
schools can be assisted to provide better quality education by community participation. In all
MBE target schools, community involvement is obvious and it has surely brought about a lot of
changes not only to physical appearances but also to many aspects of school management
practices particularly in terms of education process.
There is evidence that the good practice of school-based management increases the willingness
of some community members to provide financial contributions and in others cases to provide
free labor in the classroom. It is also evident that high level of transparency and accountability
will result in high level of community participation at schools. Parents want to participate
because they know the school program and they feel that they own the school.
Involved parents influence educational quality
Perhaps, one of the lessons learned here is that regular parent participation in the classroom
indicates an important change in the educational process and in opening up schools to the
community. This new but essential change has to go through a transitional period at some
schools. Teachers need to change their attitude from being closed to being open to the presence
of other people in the classroom while teaching. For many teachers, however, they find it very
helpfui and motivating to improve their teaching performance to ensure the better learning
outcome and to possibly attract more support from the community.
Final Evaluation ofthe Managing Basic Education (MBE) Project ABE-BE IQC (Contract # EDH-I-02-05-00035-00)
15
Ownership
Another significant lesson learned is that community involvement in most school activities
increased the degree of ownership among the school stakeholders. school committees, parents
and other members ofthe community find themselves to be respected when they get involved in
the school activities. When these people feel that they are actually part ofthe school community
and understand that the school is not only managed by the government but also by all
stakeholders, the sense of ownership is established. In most MBE target schools visited by the
evaluation team, there were always many school stakeholders attending the meeting and
interviews. This clearly shows that the community attention and commitment to the school is at
high level and it is imperative that such momentum of enthusiasm be maintained.
DEOs need more oversight
While the evaluation team observed some good examples of community participation, the same
enthusiasm or the level of understanding was not always evident in District Education Offices
(DEOs). As the DEOs are ultimately responsible for school funding and staffing, and more
importantly are an essential link in the implementation of national policy, a lesson learned here is
that these officers need to become better informed and involved in the dynamics of community
participation. It could be expected that community-driven school initiatives will become more
and more mature and innovative in the future as SCs engage more community members, become
more knowledgeable about schools, and increasingly confident in their role. However without
full support and understanding of DEOs, new initiatives coming from SCs and the school
community are unlikely to be supported.
Targeted support
There is some variation in the extent to which community participation has been adopted
between different schools. A lesson learned here, therefore, is that there is a need for more
targeted support to maximize school participation. However, this does not imply a "one-size-fits￾all" approach, as it was evident to the evaluation team that there was considerable variation in
the capacity of communities to become involved in schools and also differences between
principals in their willingness to "let-go" ofsome management and decision making as they have
traditionally used a very authoritarian and top down management model. Therefore, there is a
need to allow schools to develop at their own pace, depending on their particular circumstances
and the personalities involved. This would be essential to allow sustainable change to occur and
to avoid the encouragement of principals to set up community participation facades where they
really want to maintain a strong grip on school management
III.3 TEACHING AND LEARNING
Findings: Accomplishments and Weaknesses
On all accounts, the MBE project has had impressive impacts on classrooms. When compared to
the average Indonesian classroom, the MBE classrooms provide significantly richer learning
environments and teachers have adopted a wider variety of teaching approaches and materials to
make their teaching more interesting. Another impressive and significant achievement is that in
MBE schools there is unanimous support amongst all stakeholders for active learning generally,
and the PAKEM approach in particular.
Final Evaluation ofthe Managing Basic Education (MBE) Project
16
ABE-BE IQC (Contract # EDH-I-02-05-00035-00)
There was anecdotal evidence to suggest that student achievement had improved as a result of
MBE interventions. For example, many schools reported that student' results had improved as
evidenced by improved school ratings and the achievements in various inter-school competitions.
Data provided by the MBE project on the ranking of schools in two districts, based on Class 6
examination results, demonstrated significant ranking improvements in the MBE target schools.
Additionally, the MBE project has conducted its own testing in 54 schools using special tests
developed in conjunction with the CLCC program. These results have shown increased student
scores between 2004 and 2006 in reading (Class 1), reading comprehension (Class 4),
mathematics (Class 4), and science (Class 5).
While these results and other anecdotal evidence do not demonstrate conclusively that student
performance is improved as a result of MBE interventions, the evaluation team concluded that
there. were probably some gains in student performance. However, this is an area that requires
closer attention, and there would be considerable advantages in USAlD and the GOl cooperating
in establishing appropriate monitoring to demonstrate the impact of project interventions on
student performance
A number of teachers commented that with the recent introduction of competency-based
curriculum by the government there appeared to be an endorsement of PAKEM approaches.
Even though PAKEM invariably increases teacher workloads, there was a genuine and strong
commitment among teachers to its implementation and an appreciation of the benefits and
results that PAKEM had achieved in classrooms and schools as a whole. This change in teacher
attitude and behavior is an impressive and significant outcome of the MBE project, and
demonstrates quite clearly that one of the major obstacles to change in classrooms has been
overcome, that is teacher resistance to change.
The most frequently observed indicators of PAKEM implementation were the student group
seating arrangements and visual displays of learning materials on walls that included examples
of student work. The most significant accomplishments therefore were the fairly rapid and
impressive improvements in the physical classroom learning environments, the very genuine
attempts by teachers to move away from traditional didactic methodologies, and the strong
support among all stakeholders for PAKEM.
Implications and Lessons Learned:
PAKEM may have reached a plateau
While impressive improvements have been made in the physical classroom learning
environments along with some progress in improving teaching methodologies in MBE schools,
these gains appear to have reached a "plateau" in schools that have received all of the PAKEM
modules. There is the risk therefore that further improvements may not occur, or the.gains made
to date may not be sustainable in the long term ifsome corrective measures are not adopted.
The MBE use of facilitators overall appears to have been very effective, largely attributed to the
fact that these people are regular classroom teachers, are generally highly committed, and
therefore can generally relate very well within their own schools and neighboring schools to the
challenges of teachers implementing PAKEM. However, as the MBE project comes to an end
they have probably taken their fellow teachers as far as their own experience and training with
PAKEM will allow, thus resulting in plateauing. Teacher professional development is a long￾term and continuous process requiring on-going guidance, monitoring, and support to bring
Final Evaluation ofthe Managing Basic Education (MBE) Project ABE-BE IQC (Contract # EDH-I·02-05-00035·00)
17
about change in classrooms. This does not imply that teachers require simply more formal
training programs, as it is well known in Indonesia that many teachers have been trained over
and over again and there has been no significant change in classroom practices.
The MBE interventions have established a good environment for further teacher
development
What is important is that the MBE interventions have resulted in teachers clearly recognizing
that within schools other teachers are an important source of knowledge about teaching.
Consequently, the most impressive outcomes ofMBE interventions are that teacher resistance to
change has been overcome, classroom learning environments have improved significantly, and
teachers have made some progress in improving classroom methodologies. However, of even
more significance is that the MBE has developed a school environment, which if properly
utilized, will allow further improvements in teaching and learning.
As a consequence this provides a good opportunity in which to introduce school-based
interventions for the improvement of classroom teaching practices without having to rely on
expensive inputs from formal training.
The MBE teachers are well positioned to further improve their teaching methodologies
As outlined above, the evaluation team concluded that the MBE has made significant and
impressive achievements in improving classroom teaching and learning. At the same time the
evaluation team concluded that PAKEM classrooms could become more effective and teaching
skills greatly improved, ifteachers could confidently adopt more advanced methodologies in, the
following areas:
• Explaining skills,
• Demonstrations,
• Different strategies for beginning qnd concluding lessons depending on content and
expected learning outcomes,
• More effective and varied use of blackboards,
• More variety in the use ofsmall-group cooperative learning,
• Selecting and using appropriate instructional resources,
• Distinguishing between the different assessments ofstudent learning to meet the different
needs ofteachers, students, parents, education authorities, and for student motivation, and
• More sophisticated approaches to formative and summative student assessment.
A further very important consideration is that a number ofteachers admitted that they were in the
unenviable position of realizing the benefits of PAKEM, but at the same time having to use a lot
of rote learning in order to meet the needs of local examinations that were largely based on
factual recall.
Importantly the MBE inputs have provided an ideal whole school environment and classroom
methodology skill-base that potentially can support teachers to further develop and build on the
initial gains in teaching practice introduced by PAKEM. Therefore, this places a significant onus
on USAID and the GOI to work together to capitalize on the MBE achievements to date made in
classrooms. Approaches to support these achievements are outlined in more detail elsewhere in
Final Evaluation ofthe Managing Basic Education (MBE) Project
18
ABE-BE IQC (Contract # EDH-I-02-05-00035-00)
this report, and should include elements such as the strengthening of KKG/MGMP at the cluster
level.
IlIA PROJECT EFFICIENCY
The evaluation of project efficiency included an examination of annual progress monitoring
reports coupled with the field observations by the evaluation team.
While theMBE team and stakeholders are able to present a project conceptual framework that on
the surface appears simple and straightforward, in effect it is a complex mix of interventions
covering decentralized district management and governance of education, school- and
community-based management, teaching and learning in schools, and the replication and
dissemination of good practice. The MBE Team have managed to implement this complex set of
interventions in an impressive manner that has achieved a very high level of stakeholder
ownership.
Of considerable note is MBE team's effective response to USAID demands for expansion ofthe
program from 10 to 20 districts, in addition to the direct and indirect support for NAD, Jakarta
and the DBE project. This demonstrates two important qualities. Firstly, the high caliber of the
MBE Team, and secondly the robustness ofthe MBE model and approach.
Analysis ofImplementation and Achievements
The 8 December 2006 annual progress monitoring report indicates that 17 of the 24 outputs as
defined in the Performance Monitoring and Evaluation Plan (PMEP) have been achieved, and of
these, 12 outputs (50 percent of the total outputs) have been exceeded. The six outputs not
achieved in the 2006 Report include; a small shortfalls in 4 schools out of the targeted 280
displaying RAPBS, 23 schools from a targeted 240 for functioning school committees, 28
schools from a targeted 200 where community assistance increased in financial and in-kind
terms, and a shortfall of 28 schools from a targeted 200 adopting an active community strategy
for maintaining and improving school facilities. Additionally the survey of stakeholder
satisfaction had not been conducted, and there were no targets set for MBE dissemination.
Considering the program expansion demands placed on MBE, the evaluation team does not
consider these shortfalls to be serious, especially when considering that they have exceeded
targets in other outputs. It is also important to note that good progress appears to be being made
on these shortfalls and none have been a total failure, and that some targets may have been on the
high side and influenced by factors outside ofthe project.
Strengths and Weaknesses ofMBE's Technical Approach
As outlined elsewhere in this report, the impact of MBE on participating schools has been
impressive in a number of aspects, and these schools certainly appear to be a lot different to non￾MBE schools. However the evaluation team detected a possible implementation plateau, where
improvements may have stagnated, and where schools are relying on physical changes in the
classroom as evidence of progress. To a large extent this situation may have been perpetuated by
a number of weaknesses in the MBE monitoring approaches. Below is a description of some of
these weaknesses.
Final Evaluation ofthe Managing Basic Education (MBE) Project ABE-BE IQC (Contract # EDH-I-02-05-00035-00)
19
Teacher Performance
The MBE uses a number of classroom indicators to provide evidence of changes in teacher
performance to support student-centered learning. The indicators used include; use of pair or
. group work, asking non-recall questions, making and using own teaching aids, helping students
individually with tasks, adopting formative assessment methods, and giving feedback to students.
The monitoring targets for these behaviors provided in the annual progress monitoring reports
for October 2004 - September 2005(1 February 2006) and September 2006 (8 December 2006)
were 70 percent and 80 percent respectively of teachers trained that demonstrated these
behaviors. The results reported in the 8 December 2006 report were:
"The target has been achieved. In all phases, in excess of90% ofteachers demonstrate at
least two new behaviors... "
While on the surface this appears to be an impressive achievement, in effect these are weak
indicators of both success and changes in classroom teaching behavior. Firstly, the approach is
soft in that only two new behaviors need to be demonstrated by the teachers monitored. For
example this may be the "use of pair/group work" and "making and using own teaching aids."
Interestingly, both the February 2006 and December 2006 monitoring reports say that "Overall,
the most common changes observed in teacher performance were the adoption of group work in
classes..." This is hardly surprising as the majority of MBE classes have student seating
arranged in groups, and clear criteria for exactly what constitutes pair or group work is not
defined in the monitoring reports, therefore achievement ofthis behavior may be relatively easy
for teachers. The same applies to the use ofteaching aids; there is no reporting in the monitoring
reports of the quality of these aids, lesson relevance, and effectiveness in achieving the desired
learning outcomes for which.the aids were designed. Therefore, these indicators are very broad
and open to wide interpretation.
Considering the inherent weaknesses of this monitoring, and as a consequence the lack of
diagnostic feedback to the project as a whole, and teachers in particular, this could be one
contributing factor to the plateau reached in improvements in teaching methodologies described
elsewhere in this report. In fairness to the MBE, however, the indicators used by MBE were
taken from the CLCC program and the "soft" indicators were initially used to provide some.
easily-made targets when schools were just coming on-stream. Some of the indicators were
"hardened" as the program matured, for example in the area of lesson planning, however they
did· not focus on the more critical areas of teacher-student interaction promoting improved
learning.
School- and Community-Based Management
The annual progress monitoring reports provide a number of indicators related to the project
outcome of: MBE schools meet criteria of having active, functioning school committee and
increased community support. In support of this outcome a number of related project outcomes
and indicators are reported covering areas including school- and community-based management,
role ofthe school committee, and role ofthe community
However, a number of the monitoring indicators are, once again, fairly soft. For example, in the
area of school development plans there is a pre-occupation with their public display, which may
have some bearing on transparency and accountability, but is not a good indicator of more
Final Evaluation ofthe Managing Basic Education (MBE) Project
20
ABE-BE IQC (Contract # EDH-I-02-05-00035-00)
important aspects such as the quality and level of stakeholder involvement in school
management and planning, which are major determinants of good management practices.
While the MBE monitoring reports make statements such as "the school community had been
involved in their preparation", there is no analysis of exactly what this involvement is. For
example, was the involvement purely consultative or was it more collaborative, and in what
ways? More concrete indicators of the level of community involvement would include, for
example, an analysis of the composition of school committees in individual schools, an
indication of the quality of meeting minutes, committee member attendance rates correlated to
who they actually were (for example are meetings conducted while community members are not
attending), and the voting patterns of individuals and the types of decisions made. If meeting
minutes are kept, this type of analysis is very straightforward.
Another example is the indicator used in the 8 December 2006 Report, where for school
committees the indicator is "actively involved in monitoring plans." While the Report indicates
that monitoring of these school plans is important and that the 54 percent of schools met the
criteria, there is no discussion or analysis of what this monitoring by school committees should
be, and how it was actually carried out.
The lack of this kind of monitoring data points to a need for more rigorous monitoring and
feedback in this area.
Cost Effectiveness
When economists evaluate the efficiency of projects they usually conduct benefit/cost or cost
effectiveness analyses. Benefit/cost analysis calculates a rate of return to the investment from the
ratio of the discounted benefits and costs of a project over some time period. Cost effectiveness
analysis measures the unit costs of the project and compares those costs with similar project
alternatives.
Although the benefits of the MBE project can be clearly enumerated, they are not easily
measured. For example, the MBE project has clearly transformed schools and made them
interesting places to learn, teachers are using varied and more interesting teaching·methods,
school management is clearly more transparent, accountable and efficient, but how does one
quantify those benefits? The standard method for measuring the benefit of the intervention like
this one might be to identify the learning gains through some measure like student achievement
scores and then propose some monetary stream that might result from those gains, e.g., greater
productivity; higher earnings, improved education system savings because of reduced dropouts.
Since many learning gains are achieved in primary school, attributing many of the gains to the
intervention would be heroic because a fair amount of time and other intervening variables
would pass between the learning gain and the measurable impact. Since there are no reliable,
independent data (student achievement,enrollment gains, or reduced dropouts) that might be
attributed to the impact of the intervention, it is probably impossible to carry out a benefit/cost
analysis ofthis project.
In regard to the cost effectiveness of the MBE project, it is difficult to draw any firm
conclusions. The problem appears to be that it is very difficult to make comparisons of similar
items across different donors.
It might be said that the MBE program is relatively low-cost. For example, a simple calculation
made by dividing the total estimated project expenses ($9.5 million spent to date) by the number
Final Evaluation ofthe Managing Basic Education (MBE) Project ABE-BE IQC (Contract # EDH-I-02-05-00035-00)
21
ofschools assisted under the MBE project (449) yields a figure of an average school intervention
of $21,158. More sophisticated calculations that estimate annual average costs (in an effort to
avoid double counting oftrained persons) yield the following:
• Average cost per school per year of $6,139;
• Average cost per school to receive the full training package of four training packages of
$23,212;
• Annual average cost per educator trained in partner schools of $455;
• Annual average cost per person trained in partner schools of $358;
• Annual average cost per student for school development of $19.25.
When these estimates are compared to other similar projects such as UNICEF's CLCC, however,
the MBE project looks to be relatively expensive. For example, a simple calculation of the total
CLCC project costs divided by the number of schools yields an estimated cost per school per
intervention of $2,205, which is only a fraction ofthe average costs under the MBE project. The
figures for average student cost per student are higher under the MBE project when compared to
the CLCC project ($19.25 compared to $12.50), while the average cost of a person trained under
CLCC at $385 is higher than the MBE project's cost of $358.
On the other hand, when the MBE program is compared to AusAID's IAPBE program, the MBE
program appears to be relatively inexpensive. For example, the $US 7,691,307 applied to the 180
schools in the IAPBE program yield an average school cost of $43,729 but, in fairness, the
AusAID program provides much more assistance to district and provincial government. In
another case, the IAPBE average cost oftraining per participant trained is $2,084, which is more
than four times the average cost under MBE. Clearly, one must be very cautious using this
analysis because it is not clear that these comparisons are comparing like units.
Final Evaluation ofthe Managing Basic Education (MBE) Project
22
ABE-BE IQC (Contract # EDH-I-02-05-00035-00)
SECTION IV. IMPLICATIONS: LESSONS LEARNED
IV.1 LESSON LEARNED 1: CLEAR AND FOCUSED CONCEPTUAL FRAMEWORK
The MBE has a very clear and focused conceptual framework that the evaluation team found was
well understood by all stakeholders at all levels. A major strength ofthe MBE approach was that
everyone understood very clearly the objectives and importance of a whole school approach, and
as a result all stakeholders were working towards a common goal.
Further, this conceptualization of the MBE program is very clearly framed in contemporary
understandings of schools, classrooms and teachers, and is well informed by lessons learned and
best practice in the Indonesian context. For example, contemporary understanding of teacher
development tells us the following:
• Changes in teacher behavior are best achieved when the interrelated complexities of
community, school, and classroom are taken into account.
• Effective programs need to be well planned and understood by all stakeholders.
• Apart from close consultation about their needs, teachers, principals and other local
participants along with administrative officials need to be involved in planning the
program.
• The most effective and relevant in-service programs are those that allow high levels of
local participation in both the design and implementation. There is also a need to involve
local parents and community members. Here, teachers, principals, and teacher educators
work as a team in the school to educate the community about its schools, the curriculum,
and at the same time learn about the characteristics ofthe local children and families.
• Programs that focus on continuous development to guide, monitor, and support necessary
skills, knowledge, and new ideas tend to be more successful in bringing about change at
the classroom level than those that seek quick fixes to fill up deficiencies or those that
simply provide a qualification.
• Ongoing support is essential for effective transfer of skills· and is easiest developed
through the school-based model. The main support comes from within the school
environment (e.g., principal, other teachers).
Quite clearly, the MBE program conforms to all of the above components of contemporary
knowledge, and the evaluation team concluded that the MBE has demonstrated exemplary
performance in all ofthese elements.
IV.2 LESSON LEARNED 2: WELL SEQUENCED AND COORDINATED WHOLE
SCHOOL ApPROACH
It was very clear to the evaluation team that the well sequenced and coordinated whole school
approach adopted by MBE where teachers are actively supported by the school principal, other
teachers, and the wider school community is instrumental in the effective implementation of
student-focused learning. While there may have been some differences between schools or even
within individual MBE schools in terms of the extent of changes and their effectiveness, it was
generally observed that the entire school had made some progress in adopting PAKEM.
Final Evaluation ofthe Managing Basic Education (MBE) Project ABE-BE IQC (Contract # EDH-I-02~05-00035-00)
23
The implications of this lesson were particularly evident in NAD as well as in Central Java. In
NAD, DBEI had been responsible for school management and community participation
activities, whereas MBE has taken responsibility for the teaching and learning components with
the introduction of PAKEM. Putting aside the facts that at the time of the evaluation schools had
only received Modules I and 2 of PAKEM, and also the post-trauma effects of the tsunami, it
was generally felt by the evaluation team that while PAKEM adoption by schools was generally
good, overall this was generally less impressive than in the other provinces visited. The
evaluation team largely attributed this to the lack of ~omparable progress made by the DBEI
interventions when compared to the other MBE provinces in the areas of SBM and community
involvement in schools, especially in terms of implementing a well-coordinated and integrated
whole school approach. This, therefore, demonstrated very clearly to the evaluation team the
need to engage individual schools in a well-planned and integrated approach encompassing
effective sequencing ofthese elements to support changes in the classroom. Also important is the
need for schools and teachers to have a clear "roadmap" of interventions and targets, something
that appeared to be lacking in NAD, but was clearly evident in the other provinces visited.
In Central Java, the schools in Kudus had received components of the DBEI and DBE 2
programs, but the impact of these interventions was muted. Moreover, the schools could not
explain the package of interventions they were to receive, when or what the next set of
interventions might be, and the overall purpose of the program. Again, the lack of a well￾coordinated and integrated approach with a clear roadmap was highlighted in Kudus.
IV.3 LESSON LEARNED 3: START SMALL AND BUILD ON SUCCESS
The MBE approach demonstrates very clearly the advantages of starting small and then building
on success, both in the origins of CLCC and in MBE implementation itself. The CLCC program
developed pilot approaches initially when it started in 1999,and by 2002 was adopted by MaNE
as the official approach to school-based management in primary schools. Similarly, the MBE
began as a small intervention focusing on strengthening local government capacity to manage
basic education, and was scaled up with an added emphasis and funding for the improvement of
teaching and learning and the coverage extended in Central and East Java. As a result of the
small start and the creed of building on success, the MBE implementation engaged schools in an
integrated and meaningful series of interventions which resulted in very high levels of school
ownership.
IV.4 LESSON LEARNED 4: Do WHAT YOU DO WELL
There has been a tendency in the past for donors and the Gal to maximize the number ofschools
that receive project interventions. The lesson learned across a number of donors and projects
reviewed by the evaluation team is that this approach often has the effect ofstretching the quality
and duration of inputs and therefore limiting the impact and long-term sustainability of the
intervention. A more effective approach, observed within the MBE experience and approach, is
to limit the number of schools to ensure that the investment in individual schools is sufficient to
facilitate. a whole school development and involve enough personnel to provide sufficient critical
mass fora sustainable change.
Final Evaluation ofthe Managing Basic Education (MBE) Project
24
ABE-BE IQC (Contract # EDH-I-02-05-00035-00)
IV.5 LESSON LEARNED 5: PAKEM PLATEAU
PAKEM methods may have reached an implementation plateau, with potential for future
backsliding.
One of the MBE program's most significant outcomes has been in improving the physical
classroom learning environments, with less spectacular progress being made in improving
teaching methodologies. In most classes visited the extent of changes in the classroom varied
widely~ In a few classes lessons were still largely didactic, while in a few other schools teachers
were able to implement more student-centered teaching practices. Most teachers were
somewhere between these extremes. In any education system, changing teaching practices will
always be a significant challenge as these changes are difficult to achieve and it will take longer
to produce measurable impacts than the other types of MBE interventions. This situation may
have been hampered by the relatively soft indicators used by MBE in classroom monitoring. A
more focused classroom monitoring approach coupled with targeted training materials
responding specifically to deficiencies in teaching methodologies may produce better results and
might avoided the possible plateau that we may be seeing in teaching methodologies.
IV.6 LESSON LEARNED 6: SELECTION PROCESSES
Selection processes may be too prescriptive if schools selected probably would do well
anyway.
.The selection processes used by MBE is orientated towards schools that demonstrate a capacity
and willingness to change and, as a result, many of those selected are quite good schools within
the general Indonesian context. T~e MBE model has an approach and implementation process
that may not apply equally well to poorly developed schools. The MBE project support provided
is prescriptive in that it tends to direct schools along pre-determined expectations and outputs. A
problem may be that a particular project intervention within an implementation timetable may
not be relevant to a particular school at that particular time.. This may be particularly damaging
for a model that proposes to introduce the MBE intervention in a few schools in a district and
then encourage the adoption of the MBE model throughout the district. A more ideal model
might be a school selection and engagement process involving a detailed assessment of
individual schools with interventions planned and tailored as a result ofthis assessment.
It is interesting to note here that in the evaluation team's discussions with UNICEF, it was
revealed that they are considering for the next phase of CLCC a model that may be more
applicable in disadvantaged provinces, and considering options such as longer or more intensive
interventions of certain components oftheir model.
This approach being considered by UNICEF may also have implications in othe~ aspects of the
CLCC, as they also acknowledge the plateau effect in their schools that has been highlighted by
the evaluation team in MBE schools. If a more flexible model is adopted that is less prescriptive
and more responsive to individual school needs, and with more intensive interventions based on
these needs, this may tend to counter the potential for the CLCCIMBE approach reaching a
plateau. Of course this may result in a slightly reduced number of schools being targeted;
however, it has the advantage ofmaximizing the effect in individual schools.
Final Evaluation ofthe Managing Basic Education (MBE) Project ABE-BE IQC (Contract # EDH-I-02-05-00035-00)
25
IV.7 LESSON LEARNED 7: COORDINATION
There has been a lack of meaningful coordination with key directorates at the centralleveI.
While the MBE and DBE projects are obviously pitched at the district level, which is appropriate
within the context of decentralization, a number of MONE officials made the point that they
were not very well informed about these projects. The points raised included that while they may
have received project newsletters and reports periodically, these generally did not provide
sufficient insights into the challenges and lessons learned that could inform GOl policy and
strategic planning. The MONE officials wished to become better informed of USAID education
project activities, and while efforts are being made by the projects, this area of communication
and dialogue needs to be explored and improvements made.
IV.8 LESSON LEARNED 8: POLICY DIALOGUE (OR LACK THEREOF)
MBE and DBE are well positioned to assist the GOI but there has been little policy
dialogue and integration.
As outlined elsewhere in this report the MBE has provided some good gains in individual
schools and districts, and these gains are in support of GOl broad education policy. However the
MBE and DBE projects are not designed or implemented in such a way that would encourage
central government ownership with subsequent impact on broader policy formation and
integration.
There has been some integration at the district level for example with the MBE activities in the
areas of district-level planning, increasing efficiency in the use of resources through school
mergers, teacher deployment that is more closely related to student numbers, and more equitable
school funding. However the results vary considerably. For example the MBE annual progress
monitoring report, September 2006, indicates that a large percentage of MBE districts had made
plans for teacher redeployment based on local needs and student numbers. However it was
acknowledged that in most cases this planning did not produce many outcomes. Consequently
this in an area where in fact there has been limited integration into local policy, and as a result an
opportunity to inform or engage in a policy dialogue at the central level in what is a critical
policy area has been missed.
Final Evaluation ofthe Managing Basic Education (MBE) Project
26
ABE-BE IQC (Contract # EDH-I-02-05-00035-00)
SECTION V. SUSTAINABILITY
V.l COMPONENTS OF IN-BuILTMBE PROJECT SUSTAINABILITY
The well coordinated whole school approach adopted by the MBE project is a major element of
sustainability. It was very clear to the evaluation team that the components of SBM and
PAKEM, supported by community involvement, had been well established in the schools visited.
This was evidenced by the general opinion ofstakeholders at various levels that the program was
successful, and it was clear that MBE enjoyed high levels of local ownership, especially at the
school level. .
The MBE project has established an impressive team of facilitators. These classroom teachers
who go through a rigorous selection process and then take on the additional role of facilitator
generally maintained their teaching duties, and apart from having their expenses covered,
received no additional remuneration for these additional duties. The groups of facilitators met by
the evaluation team were all very highly committed to the MBE, and willing to continue their
role after the MBE project is finished. This is an impressive achievement of the MBE project to
establish this cadre offacilitators, and is a significant factor to ensure sustainability.
The evaluation team also observed that planning at the school level showed strong commitment
and sustainability, where in a number of instances the MBE program was made the highest
priority in school RlPS and RAPBS. Coupled with this, in some DEOs it was indicated that
considerable amounts ofmoney were to be allocated to the replication ofMBE.
Consequently it was concluded by the evaluation team that the· MBE project design and
implementation had successfully built in a number of elements that potentially, given the
appropriate political environment and support, could ensure sustainability.
V.2 EVIDENCE OF SUSTAINABILITY AT THE DISTRICT LEVEL
The MBE annual progress monitoring reports indicated high levels ofreplication, for example in
September 2006 it is reported that dissemination by all MBE districts between 2004 and 2006
had reached 6,075 schools and 51,630 participants. The report indicated, "typically,.
dissemination takes the form of locally funded training activities, teacher exchanges and study
visits." However details are not provided in the MBE monitoring ofthe impact ofthese activities
on the schools involved, therefore this raises issues regarding the technical quality ofthis rollout.
Coupled with this, while the MBE design and monitoring approaches included elements of
replication and dissemination, these were largely quantitative measures (number of non-target
schools trained, number of participants trained, number of study visits, etc.) and did not include
indicators ofthe actual capacity oflocal authorities to manage and expand the program.
The evaluation team generally had mixed responses from DEOs regarding replication. In some
areas there was obviously a high commitment, in others they were considering it, while in some
there were obviously no plans.
V.3 IMPLICAnONS FOR THE GOI
The rollout of MBE by DEOs raises a number of implications for the GOI, USAID, and other
donors with similar programs. These are discussed below.
Final Evaluation ofthe Managing Basic Education (MBE) Project ABE-BE IQC (Contract # EDH-I-02-05-00035-00)
27
Lack of Appropriate Monitoring and Technical Support to Ensure Quality
The lack of appropriate monitoring coupled with the appropriate technical support may
ultimately have a negative impact on sustainability For example, teacher exchanges and study
visits by district officials to successful MBE schools no doubt provide powerful impressions and
strong stimulus for change, and at the same time obviously raise expectations. However if local
attempts to replicate are unsuccessful and expectations are not met, this will send a powerful
message to local decision makers and ultimately impact on the reputation and sustainability of
MBE replication. This will apply particularly with the PAKEM component, as it was apparent to
the evaluation team that this was the most difficult component to implement due mainly to the
time required to instill changes in classroom teaching practice. Coupled with this, the formal
student assessment approaches currently in use may not clearly demonstrate gains in student
performance as a result ofintroducing PAKEM.
Therefore for future rollouts of MBE there will be a critical need to have in place an effective
monitoring system linked with the appropriate technical support that can respond effectively to
specific weaknesses. The monitoring should not be entirely quantitative, and should be designed
using a diagnostic approach to give early warnings of problems. Imperative to this approach will
be easy access by schools and DEOs to appropriate support mechanisms and technical assistance
. to correct any failings or shortcomings in implementation.
Staffing
The evaluation team found quite a few "champions" of MBE at all levels, including teachers,
~chool principals, and officials at the sub-district and district levels. It was clear that these
"champions" had produced very positive impacts on the success ofMBE at the local level.
However a local reality is that many of these key people are moved around the system, and the
evaluation team was aware of examples where these moves had slowed the pace of reform in
individual schools and within DEOs. This therefore demonstrates that the reliance on
"champions" can be precarious in terms ofsustainability, and that ultimately a critical imperative
for sustainability is government ownership at all levels. Therefore what will be required to
ensure sustainability are two things. Firstly, commitments from the GOI at all levels to ensure
that these "champions" or key agents of change remain in place for sufficient time to ensure
sustainability. Secondly, ownership needs to be established concurrently at all levels, so that
schools, sub-district, district and provincial offices are all working towards the same goal and
they all appreciate the ramifications ofmoving key staff during critical phases of MBE rollout.
System-Wide Policy
There are a number of other elements within the broader education system itselfthat will impact
on the degree of MBE sustainability. For example the lack of commitment by local government
to address the issues of student-staff ratios and teacher deployment will ultimately impact on
MBE sustainability. Schools that are grossly overstaffed, where many teachers have not much to
do and are seldom in the classroom, cannot implement effective changes in teacher behavior. On
the other hand, understaffed schools where teachers have higher than normal teaching loads
cannot be expected to take on the additional work loads imposed by PAKEM.
Any misalignment of formal student assessment approaches implemented by districts or schools
with the active learning promoted by PAKEM will ultimately be major a determinant of MBE
Final Evaluation ofthe Managing Basic Education (MBE) Project
28
ABE-BE IQC (Contract # EDH-I-02-05-00035-00)
sustainability. If parents need to pay for the "extra classes" needed to coach students in rote
learning for assessment based largely on recall type questions, this may become a major
disincentive.
There are obviously a number offunding issues that need to be addressed at the policy level. For
example, districts which elect to rollout initiatives such as the MBE will obviously be required to
inject more funding into schools, and the issue here is how are these initiatives to be encouraged
through funding mechanisms.
As mentioned elsewhere in this report, the role of school facilitators has been critical to the
success of MBE. A policy issue here is how to maintain these facilitators as classroom teachers,
and at the same time formalize arid support their role as facilitators. There is no implication of
additional salaries here, but simply a formal recognition of their status to allow them to operate
more effectively in schools and to have their role formally recognized.
Final Evaluation ofthe Managing Basic Education (MBE) Project ABE-BE IQC (Contract # EDH-I-02-05-00035-00)
29
SECTION VI. STRATEGIES AND APPROACHES TO SUPPORT
MBE AFTER PROJECT ENDS
Given the lessons learned, there are various opportunities post-MBE. The section below
identifies four such opportunities and describes the related project design considerations that
need to be considered.
VIol OPPORTUNITY 1
A number of districts have allocated funds for the expansion of MBE activities into schools
not previously targeted
A number of districts demonstrated to the evaluation team a high level of commitment, local
ownership, and understanding of the MBE model. However a number of concerns were raised
regarding the quality of these rollouts. Strategically, with the confidence and good will
developed through MBE, USAID and the GOI are in a good position to facilitate quality rollouts
through the provision oftechnical assistance, and at the same time sustain and improve the gains
made in exiting MBE schools.
What is critical here, as demonstrated in the lessons learned, is that PAKEM is unlikely to reach
its full potential unless the elements of SBM and community support are fully addressed and
integrated. The success of PAKEM is highly dependant on the MBE approach that provides the
teachers with appropriate support as developed by training of district officers and school
principals, and encourages principals by developing community support. Separating school
interventions into separate components is a convenient but artificial process since, in reality, a
school works as a whole in a complex web of relationships. Secondly, it should be understood
that the districts are very familiar and comfortable with the MBE "package," and therefore any
technical assistance should strictly adhere to the MBE model and should not be a watered-down
version or consist ofmodified interventions.
VIo2 OPPORTUNITY 2
The evaluation has highlighted the need for a systematic and sustained approach to teacher
development if real improvements in teaching and learning are to occur in the classroom
As outlined above, with the good progress being made to date in the improved classroom
physical environments there remains the need to make further advances in teaching
methodologies. Two approaches are recommended here.
Firstly, sustainability and the capacity to introduce more advanced methodologies will require an
improvement in the skills of MBE facilitators. There was some limited criticism among teachers .
offacilitators, and perhaps the key here is to address the issue oftheir skills as effective mentors.
Secondly, another requirement is the need to strengthen the capacity ofKKGs and MGMPs, not
as an opportunity to provide top-down training inputs, but to improve the capacity and
knowledge of individual key members and 1IillE facilitators to respond to local teacher
challenges in implementing PAKEM, and further improving teaching methodologies.
government grants are now available to support these activities, therefore the first step is to
Final Evaluation ofthe Managing Basic Education (MBE) Project
30
ABE-BE IQC (Contract # EDH-I-02-05-00035-00)
facilitate the formation of KKGslMGMPs where they do not already exist, and to improve their
capacity in applying for and effectively using the grant money to meet their particular needs.
As it has been demonstrated in the past that significant development ofteachers can take place at
the KKGIMGMP cluster level, USAID and the GOI should take the opportunity to facilitate and
extend these structures in order to fully capitalize on the investments made to date in MBE. The
KKGIMGMPs have the advantage of being low cost and conveniently located at the grassroots
level.
Another important consideration is that with the introduction ofteacher certification the demands
for teacher in-service training have changed significantly since the implementation ofMBE. As a
result KKGslMGMPs will become an important focus for teachers seeking new opportunities to
develop their knowledge and skills for the examinations and tests leading to certification.
VI.3 OPPORTUNITY 3
The new qualifications and competencies being introduced require teachers to attain a
bachelor's degree (S1) or four-year diploma (D4), in addition to demonstrated
competencies in four areas, which obviously would be done through teacher in-service
training/professional development
While riot finalized yet, certain universities will be accredited to provide teacher certification,
and there is the capacity to award credits for prior learning and skills. There is therefore some
potential for teachers, given the appropriate support, to gain some credits towards certification as
a result oftheir PAKEM achievements.·
The DBE2 project's links to universities provide an ideal opportunity to start teachers in the
MBE target schools on the pathway to certification and accreditation, and this provides an ideal
incentive for continued teacher involvement in teaching skill development. Therefore concerted
efforts should be made to link MBE teachers with accredited universities through the DBE2
project.
VI.4 OPPORTUNITY 4
MONE is making considerable progress on preparing for the implementation of teacher
certification requirements and developing the appropriate Regulations to accompany the
Teacher Law
This work is being supported with technical assistance from the World Bank and financial
support from the Netherlands Government. In discussions with Fasli lalal (Director General,
Quality Improvement of Teachers & Education Staff) it was made clear to the evaluation team
that MONE would appreciate additional inputs into these processes. USAID is therefore
strategically well positioned with the experiences of MBE and the current engagement with
universities through the DBE2 to engage in dialogue with MONE to elaborate the type of inputs
required, and to provide the appropriate technical assistance.
Given the MBE and DBE2 experience to date, valuable input and support to MONE in preparing
. for the implementation ofteacher certification could be provided in a number of areas, including
for example:
Final Evaluation ofthe Managing Basic Education (MBE) Project ABE-BE IQC (Contract # EDH-I-02-05-00035-00)
31
• Investigating and providing further guidelines on the effectiveness of KKGIMGMP in
supporting improved teacher performance;
• Supporting the work of (and establishment of, where they do not exist) KKGIMGMP
cluster groups and supporting their workshop training programs with resources and
advice from those personnel directly involved in DBE2IMBE;
• Identifying supervisors, principals, expert teachers, facilitatorslMTT and senior teachers
for training as assessors to undertake classroom observation and other activities at local
schools for teacher certification procedures; .
• Assisting in the development of a district monitoring system to standardize the
implementation ofthe teacher competency test;
• Assisting in the development of a Teacher Performance Assessment Instrument; and
• Providing advice on approaches to the delivery of on-the-job training by universities, and
the impacts Ofsuch training.
Final Evaluation ofthe Managing Basic Education (MBE) Project
32
ABE-BE IQC (Contract # EDH-I-02-05-00035-00)
ANNEXES
Final Evaluation ofthe Managing Basic Education (MBE) Project ABE-BE IQC (Contract # EDH-I-02~05-00035-00)
33
ANNEX I SUMMARY OF SCHOOL VISITS
Background
The overall purpose of the school visits was to review the Managing Basic Education (MBE)
project to inform USAID/Indonesia of how well the contractrequirements were met and what
results were achieved. It was also expected that the evaluation ofthe MBE project, which was a
pilot program, would inform the Mission regarding program elements that could be further
developed under the Mission's on-going flagship education program, Decentralized Basic
Education (DBE). More specifically, the purpose ofthe study ofschools in the three provinces in
which MBE worked was to identify the projects' accomplishments and impact; strengths and
weaknesses and why elements of the program worked and why others did not; lessons learned in
implementation; and recommend options for sustaining the interventions started by MBE by
linking them with the efforts ofthe government ofIndonesia and/or the DBE program.
Although the methodological approach is outlined elsewhere, it is important to·note that the
evaluation team used a variety of methods to obtain its findings, including reviewing project
documents; observation of the schools, teachers and classrooms; targeted questions and
interviews with individuals and focus group discussions with single and mixed groups of various
stakeholders. The stakeholders interviewed included, national, provincial, and district Ministry of
Education (MaNE) and Ministry of Religious Affairs (MORA) officials; school supervisors;
school principals; teachers; school committee members; parents and parents' groups
(paguyaban); MBE project facilitators and administrators; and students.
A total of 40 schools were visited, with 19 in Central Java, 8 in NAD, and 13 in East Java. We
intentionally reduced the number of school visits mid-evaluation and saw fewer schools in NAD
and East Java so that we could increase our time with stakeholder interviews. Ofthe 40 schools
we visited, 22 were public (MaNE) primary schools, 10 were madrasah (MORA) primary
schools, 3 were junior secondary schools, and 5 were non-MBE public primary schools.
Findings
(a) Typical Indonesian primary school
To better understand the impact ofthe MBE project, the project team visited five "virgin"
public primary schools that had not received MBE or MBE-type assistance. The idea was
to establish a "baseline" or typical Indonesian school that had not received much
assistance from any source. In fact, Indonesia's school system is immense and diverse
and there are no typical primary schools.
Indonesian public primary schools are remarkably homogeneous and many appear to
have been constructed at about the same time. The school is most often constructed in a
"U~ shape and generally has 6 classrooms and one or two other rooms that act as the
principal's office and teacher's room. In addition, a characterization of a typical
Indonesian school might be described as a school with:
Well-worn physical infrastructure in need of some repairs with dull, poorly lighted
classrooms displaying little student work or learning materials on the walls;
A somewhat unenthusiastic, mostly not qualified (80 percent of indonesian teachers
do not meet the qualification standards) and sometimes absent faculty teaching in a
Final Evaluation ofthe Managing Basic Education (MBE) Project
34
ABE-BE IQC (Contract # EDH-I-02-05-00035-00)
traditional "chalk-n-talk" and unimaginative lecture method with little student-teacher
or student-student interaction;
A harried school principal with few clear goals, objectives and priorities, whose
operating style is authoritarian, paternalistic and go-it-alone;
Parents who are resigned to their children receiving, at best, a mediocre quality
education and having no tradition of participating in their child's education; and
An inactive school committee and/or little evidence of community involvement in the
school and little community pride.
Although the above may be overblown and there are many schools in Indonesia that
provide quality education, the "virgin" schools we visited exhibited all or most of these
characteristics.
(b) Overall findings
1. Dramatic Visible Change
For the most part, MBE-assisted schools undergo a dramatic visible change in the
physical characteristics of the school. Although the schools are not always in tip-top
shape, the often old physical infrastructure has been spruced up with minor and
sometimes major repairs frequently made by parents. Classrooms are often newly painted
in bright, light colors and many classrooms have become a showcase for student work
and/or learning materials. The improved classroom environment is one of the most
striking impacts ofthe MBE program.
2. Energy and Enthusiasm
Although most teachers in MBE-assisted schools are not well-trained in the formal
system and are not certified at the new government of Indonesia required level (S 1),
MBE-assisted schools show a marked increase in the amount of enthusiasm and energy in
the classroom. Most of the classrooms have been reorganized away from traditional
theater seating into small work groups of 4-6 students and teachers were often observed
presiding over student-centered exercises. Some teachers lead classes using excellent,
well thought-out active learning exercises and techniques that might rival the best but,
most appear to be learning the methods and they are somewhat uncomfortable leading the
active learning approaches in front of a group of strangers. Teachers in MBE-assisted
schools have made great strides toward improved and diversified teaching methods but,
for the most part, the changes in teaching methods are incomplete.
3. Improved School Management
With few exceptions, school principals demonstrated generally good management
practice and a commitment to transparency and accountability. Generally, principals had
developed school strategic plans in concert with a wide group of stakeholders and were
reasonably articulate in the short- to long-term needs and priorities of the school. School
budgets were nearly always displayed prominently. Some principals exhibited fairly
sophisticated knowledge of teacher evaluation processes and incentives, teacher and
school development needs, and how to access resources for school improvement. Nearly
all of the principals seemed to be comfortable with displaying the new-found
transparency and enjoyed the new attention the community appeared to be focusing on
the school. At the same time, many of the principals appeared to accept the status quo as
Final Evaluation ofthe Managing Basic Education (MBE) Project ABE-BE IQC (Contract # EDH-I-02-05-00035-00)
35
a given, did not see obvious problems, and did not seem to provide dynamic leadership.
In addition, it appeared that the sub-district and district managers offered lack-luster
support to the schools and could profit from a better definition of their roles and
responsibilities as well as training in management and leadership.
4. Very Active Parental Participation
Most schools appeared to have parents and parent organiz~tions that were very active in
the schools. The level of involvement varied widely with some parents working as
volunteer teacher aids, repairing or building classrooms, or providing nutritional
supplements and teaching materials to the schools to others that simply cleaned the
school rooms periodically. The parents appeared to be energized by the new attitudes of
the teachers and principals and the focus on improving the quality of education within the
school. At the same time, a minority of the parents at the school appeared to be little
more than window dressing and were hard-pressed to describe any kind of role in the
school. One measure of the success of the MBE program was that we met large numbers
of parents at almost every school and most were eager and proud to show off their
community and newly transformed school.
5. Engaged school committees and Community Leaders
In all but a very few schools, school committee members and community leaders were
present and actively involved in the management and governance of the school. School
committee members and community leaders provided input into school development
plans, reviewed school budgets, developed performance standards, and assisted in
developing additional revenue streams. or in-kind assistance to assist in the
implementation of the school development plan. Most maintained that they met a
minimum offour tiIJ;les a year. Some of the school committee members appeared to see
their role as perfunctory and there were some cases where the school committee
chairperson had an obvious conflict of interest with the school principal and was less than
fully independent of the principal. By and large, however, the school committee and the
community leaders appeared to playa prominent and useful role in the school, especially
in generating and managing alternative revenue streams for the school. The willingness
of community leaders and school committee members to participate in meetings with the
evaluation team, often on the basis of a last minute invitation, was impressive and a
measure ofwhat high regard the MBE project is held in the communities.
6. Time Matters
Although we surveyed a small sample of schools (only about 10 percent of the targeted
schools), there appears to be a rough correlation between the length of time a school has
been in the MBE program and, therefore, the number of teacher training (PAKEM)
packets to which the school has been exposed, and the quality of teaching in the school.
There are exceptions but schools that had been in the MBE program for a little over a
year were generally not as accomplished as schools that had been in the MBE program
for three or more years. Moreover, schools that had received the full complement of four
PAKEM packets, appeared to have more confident and accomplished teachers. In all
likelihood, teaching methods change slowly from stand-up lecturing to active learning
methods and teachers need some time to get comfortable and master the new concepts.
7. Areas for Further Investment
Final Evaluation ofthe Managing Basic Education (MBE) Project
36
ABE-BE IQC (Contract # EDH-I-02-05-00035-00)
.Despite the very obvious impressive progress made by the MBE project, once one probes
beneath the surface there are a number of issues that suggest that the glass may be only
three-quarters full and that some improvements in the MBE model can be made. It is
likely that many of the issues below will need to be addressed to prevent backsliding in
the schools and/or to ensure that the MBE investment can be fully realized. Some ofthe
issues (discussed elsewhere in the report in more depth) that may need to be addressed
include:
further improving teaching methodologies and learning resources, especially
advanced student questioning techniques, classroom management, and student
assessment;
further strengthening teacher and principal professional development that is school￾based and performance-led;
further strengthening and reinforcing school management practices by encouraging
better management practices at the sub-district, district and provincial levels; and
further encouraging community participation by providing incentives to continued
community.involvement in schools.
(c) Findings by Type of School
Generally speaking there appeared to be little differences in the impact of the MBE
model being applied to a public primary school as opposed to a madrasah primary school.
The madrasah schools represent a Wider range in the quality of primary education than
those of the public primary system and the madrasah schools represented both the best
school and the worst school we saw. Some of the madrasah schools suffered badly from
being underresourced but others, which showed the same signs of limited funding, were
some ofthe best we encountered. In any case, the impact of the MBE program on school
management, community participation and active learning methods appeared to be
equally strong in both types ofschools, public and madrasah.
(d) Findings in junior secondary schools.
Although our sample ofjunior secondary schools was small (we were able to see three of
the four we were scheduled to review), the conclusions from those three schools were
very similar. In general, the teacher training component of the MBE project was less
successful in junior secondary schools. The reasons for this are two fold. First, although
the numbers of teachers trained in both schools were roughly equivalent, the impact of
the training on the two types of schools was different. In primary schools, the MBE
project trained every teacher in the school as part of its whole school approach. In junior
secondary schools, which are generally two- to-three times larger schools with a
.correspondingly larger faculty, the MBE project was able, because of budgetary reasons,
to train only ten teachers - two from every content area. Training only 10 teachers in a
faculty of say, 40 teachers meant that the tra.ined teachers had to train their colleagues in
a kind of cascade, which tpok more time and slowed the impact of the training on the
school. In addition, primary schools are generally small with only nine teachers and
easily form a "community of interest" that is generally a coherent, mutually supporting
group with similar concerns. By contrast, junior secondary schools, with a larger number
of teachers split by content areas, do not generally form such communities. Second,
Final Evaluation ofthe Managing Basic Education (MBE) Project ABE-BE IQC (Contract # EDH-I-02-05-00035-00)
37
junior secondary schools are different from primary schools insofar as they are subject
matter focused and the children circulate among teachers rather than spend their whole' .
day with one teacher. Ifsome ofthe teachers in a subject matter area have been trained in
active learning methods and some have not been trained in the methodology, the students
are likely to get a mix of both traditional stand-up lecturing and active learning methods
and the impact ofthe new methods on the school and students is muted. A good example
of this difference was apparent in a junior secondary school where two teachers in the
same subject area, geography, were teaching side-by-side in different classrooms the
same lessons with very different methods and very different results...one group of
students was engaged and challenged, while the other group was uninterested. In all other
respects - school-based management and community participation - the junior secondary
schools appeared to be the same as the primary schools.
(e) Findings by Province
The widely-held perception we heard before the field visits was that the MBE had
achieved the best results East Java followed by the results achieved in Central Java. NAD
was seen as a special case and much different than the project implementation in Java.
This review challenges some ofthose perceptions.
.First, the evaluation team found excellent schools and terrific examples of the MBE
approach in all three provinces. Second, the review ofMBE-assisted schools in East Java
and Central Java suggested that the impact of the project was about the same in the two
provinces; East Java was not measurably better than Central Java. There surely were
differences within the provinces but when one compared schools that had been in the
MBE program for about the same length of time (three years: Pati in Central Java with
Probolinggo in East Java; two years: Banyumas in Central Java with Batu in East Java;
one Year: Semarang in Central Java with Malang in East Java) the program and its results
appeared to be generally very good and about the same success and/or problems in each
province. It appears that the program can be implemented with the same level of success
in different locations.
The third finding about possible regional differences in the application of the MBE
project comes from a review of NAD province where we found the program to be
generally less impressive than the programs in Central and East Java.
The implementation of the MBE program in NAD was different than those in the other
provinces. Apparently, although DBE 1 was ready, DBE 2 was not ready to begin its
program in NAD in the post-tsunami period and USAID asked MBE to step in and work
with DBEI to begin the program. In other words, NAD is different because DBE 1 was
responsible for the school management and community participation activities, while
MBE was responsible for the teaching and learning components, including the
introduction of PAKEM. Setting aside the horrible trauma resulting from the tsunami and
the loss of 10 percent of the population and the host of other factors that have effected
NAD in the last 25 years, the MBE program in NAD was clearly less impressive,
illustrating, perhaps, that implementing a program that was not as well coordinated as the
integrated MBE program, even by the same contractor (both DBEI and MBE are
managed by RTI) may not be as successful.
Final Evaluation ofthe Managing Basic Education (MBE) Project
38
ABE-BE IQC (Contract # EDH-I-02-05-00035~OO)
There are a number of possible reasons for the less than impressive results. First, MBE
was able to only implement two of the four packets of PAKEM training in the year and a
half that· it had worked in NAD. Most teachers in NAD seem less confident and
accomplished applying active learning methods and a constant refrain from teachers,
principals and supervisors was that they needed more training in PAKEM. Second, it
appeared that the DBE lIMBE combined program was not as well-coordinated and
integrated in the whole school approach normally supplied by the MBE staff. Since the
several components of MBE - School-based management, community participation, and
teacher training - are mutually reinforcing and the school seems to gain strength from the
application ofthe components together (the whole is greater than the parts), it may be that
sequencing of the program elements was not as efficient in NAD. Interestingly, the same
problem was found in Kudus, Central Java in the DBE schools. The Kudus schools had
received a dose of school-based management and community development training
followed, after a long hiatus, by one teacher training program in teaching mathematics. In
both NAD and Kudus, school principals, supervisors and teachers had little notion of how
the program fit together or what was the next planned intervention and both programs
seemed to suffer from the confusion that the a clear "roadmap" could provide.
Final Evaluation ofthe Managing Basic Education (MBE) Project ABE-BE IQC (Contract # EDH-I-02-05-00035-00)
39
ANNEX II SUMMARY OF THE CONTENTS OF THE MBE
TRAINING PACKAGES
Package I (10 llnits) Package 2 (7 l'nits) Package 3 (8 l'nits) Package -t (9 units)
SBM What is SBM? Review ofthe SBM Implementing the
Making School Program in each School Development
Budgets and School school Plan
plans Developing the Role The School Budget,
The Role ofthe ofthe School Operational Funding
Principal and Committee and Community
Supervisor in leading Participation
Professional
Development
Community Community Developing the Role
Participation Participation in ofthe Community in
Education Supporting Learning
Creativity in
Gathering Resources
Public Accountability
PAKEM What is PAKEM? Designing PAKEM Keeping a Learning Organizing and Using
Developing PAKEM lessons: Journal the School Library
lessons • Modeling good The Competency Assessment and
Creating a Good activities Based Curriculum Evaluation
Learning Environment Questioning Skills (CBC) Training Teachers • Mapping the through the Teachers' Implementing the
Teacher Working • Class Curriculum Working Groups
Groups Organization Competencies (KKG and MGMP)
• Cooperative Subject Based On-the-job Training
Learning Approaches (Mentoring) Learning to the Curriculum Developing the role of
Practical Teaching Planning the School the School Principals'
On-the-Job Training Curriculum (KTSP) and School
Assessment and Supervisors' Working
Evaluation Groups (KKKS and
Implementing the KKPS)
Planned Curriculum
Reviewing the
Implementation ofthe
Curriculum
Other Developing Developing Student
Monitoring Indicators Potential: Gender
Issues
Monitoring the Impact
ofthe Training
Notes about the training packages:
• The third package is especially focused on PAKEM and developing the school
curriculum and has technical sections covering specific approaches to learning the core
subjects.
Final Evaluation ofthe Managing Basic Education (MBE) Project
40
ABE-BE IQC (Contract # EDH-I-02-05-00035-00)
• The packages have been designed to be used in a flexible manner - either in a period of
extended training or in a series ofshort one day training periods.
• The training packages are supported by a number of books of lesson plans I ideas for
lessons.
Final Evaluation ofthe Managing Basic Education (MBE) Project ABE-BE IQC (Contract # EDH-I-02-05-00035-00)
41
ANNEX III LIST OF SCHOOLS VISITED
Central.Java NAD East .Java
Banyumas District
Kebasen:
SD 3 Kalisalak
MIN Bentul
SD 1 Gambus
Ajibarang:
SMP 3 Ajibarang
SD3 Pancasan
SD Kalibendo
Semarang District
SDN Siswa
Pringapus:
SD 2 Wonorejo
SD Wonorejo
Ambarawa:
SD Mlilir
MIN Amberawa
SMP 1 Ambarawa
Pati District
Jakenan:
SD Sonorejo
MTsN Winong
Pati:
SD 1,2,3 Kutoarjo
SD Pati Kidul
Kudus District
SD Gondosari
SD 2 Papringan
Banda Aceh
MIN Rukoh
SD 69 Banda Aceh
MIN Lhong Raya
SD 57 Banda Aceh
Aceh Besar
SD Perumnas
MIN Sungai Limpah
MIN Bukoh
MIN Ba'et
Batu District
SD Tulungrejo
MIN Bustanui Ulum
SD Punten
SMP 1 Batu
Malang District
SD Turen
SMP 1 Turen
SD Wonokerso
MINWadung
Probolinggo District
SD 1 Bremi
SD Krucil
Surabaya District
SDN Meri I & II
Gunung Gedungan
Final Evaluation ofthe Managing Basic Education (MBE) Project
42
ABE-BE IQC (Contract # EDH-I-02-05-00035-00)
ANNEX IV LIST OF PEOPLE CONTACTED
1. Government ofIndonesia
• Prof. Fuad Abdul Hamied, Deputy Minister for Education and State Administration,
Coordinating Ministry for People's Welfare
• Fasli Jalal, Director General, Quality Improvement of Teachers and Educational Staff,
Ministry ofNational Education.
• A.K. Mudjito, Director, Directorate of Kindergarten and Primary School, Ministry of
National Education.
• Husaini Wardi, Directorate of Kindergarten and Primary School, Ministry of National
Education.
• Hamid Muhammad, Directorate of Junior Secondary Education, Directorate General of
Primary and Secondary Education, Ministry ofNational Education.
• Bambang Indriyanto, Secretary, Directorate General of Basic and Secondary Education,
Ministry ofNational Education.
• H. Jahja Umar, Director General, Direktorat Jenderal Pendidikan Islam, Departemen
AgamaRI. .
• Bagiono Jokosumbogo, Senior Staff, Chief of Planning and Foreign Cooperation,
Ministry ofNational Education.
• Zaenal Arifin, Kasubdin Sungram, Dinas Pendidikan Dan Kebudayaan,. Propinsi Jawa
Timur. .
• Salamun, Dinas Pendidikan and Kebudayaan, Propinsi Jawa Timur.
• Bpk Sugiman, Teacher Training consultant, Central Java, Basic and Secondary
Education, Ministry ofNational Education
2. United States Agency For International Development.
• Jim Hope, Director, Education Office, U.S. Agency for International Development
(USAID), USAID/Jakarta
• Loretta Garden, Deputy Director, Education Office, U.S. Agency for International
Development, USAID/Jakarta.
• Jill Gulliksen, Education Specialist, U.S. Agency for International Development,
USAID/Jakarta.
• Roberta Cavitt, Chief, Office of Education and Democracy, Bureau for Asia, U.S.
Agency for International Development, Washington, D. C.
• John Hatch, Senior Education Officer, Office of Education, EGAT, U. S. Agency for
International Development, Washington, D. C.
3. Managing Basic Education (MBE) Project
• Stuart Weston, Program Director, Managing Basic Education project
• Prima Setiawan, Deputy Program Director, Managing Basic Education project
Final Evaluation ofthe Managing Basic Education (MBE) Project ABE-BE IQC (Contract # EDH-I-02-05-00035-00)
43
•
•
•
•
•
B.
•
•
•
•
•
•
•
•
•
•
• Lynne Hill, International Curriculum and Teacher Training Specialist, Managing Basic
Education project
• Robert Cannon, International Education Management and Governance Specialist,
Managing Basic Education project
• Bpk Masjudi, Provincial Coordinator, Central Java, Managing Basic Education project
• Bpk Andreas Priyono, Education Management and Teacher Development Specialist,
Managing Basic Education project, Central Java
• Bpk Ferdy Rondonuwu, Education Management and Teacher Development Specialist,
Managing Basic Education project, Central Java
• Ibu Da Laela, Coordinator, Banyumas, Central Java, Managing Basic Education project
• Bpk Wahyu, Coordinator, Semarang, Central Java, Managing Basic Education project
• Ibu Nuzuli, Coordinator, NAD, Sumatra, Managing Basic Education project
• Bpk Supriyono Koes, Teacher Development Specialist, NAD and East Java, Managing
Basic Education project
• Bpk Abdur Rahman Asari, Provincial Coordinator, East Java, Managing Basic Education
project
• Ibu Fakultas Sastra, Education Management and Teacher Development Specialist,
Managing Basic Education project, East Java
4. Decentralized Basic Education Project (DBE) Project
A. DBE 1: Management and Governance
Dan Moulton, Program Director, DBE 1: Management and Governance
Basilius Bengoteku, Deputy Program Director, DBE 1: Management and Governance
Bpk Nurkolis, Education Planning and Management Specialist, DBE 1, Central Java
James Mangan, Provincial Coordinator, NAD, DBE 1: Management and Governance
Bpk Supriono Subakir, Provincial Coordinator, East Java, DBE 1: Management and
Governance
DBE 2: Teaching and Learning
Michael Calvano, Chief of Party, DBE 2: Teaching and Learning
Andrea Osborne-Smith, Program Manager, DBE 2: Teaching and Learning
Thomas Chesney, Deputy Chief of Party, DBE 2: Teaching and Learning
David O'Meara, Primary School Management Advisor, DBE 2: Teaching and Learning
Jennifer Rose, Primary School Education Advisor, DBE 2: Teaching and Learning
Arief S. Sadiman, Open University/ICT Advisor, DBE 2: Teaching and Learning
Vincent P. Costa, Provincial Coordinator, Central Java, DBE 2: Teaching and Learning
Khatib a. Latief, Provincial Coordinator, NAD, Sumatra, DBE 2: Teaching and Learning
John S. Seeger, Provincial Coordinator, East Java, DBE 2: Teaching and Learning
Ibu Silvanna Edina, Senior Program and Training Officer, East Java, DBE 2: Teaching·
and Learning
Final Evaluation ofthe Managing Basic Education (MBE) Project
44
ABE-BE IQC (Contract # EDH-I-02-05-00035-00)
C. DBE 3: Life Skills for Youth
• Lisa Laumann, Chief of Party, DBE 3: Life Skills for Youth
• Lorna Power, Formal Education Advisor, DBE 3: Life Skills for Youth
• Ajar Budi Kuncoro, Provincial Coordinator, Central Java, DBE 3: Life Skills for Youth
• Bpk Dwe, Coordinator and Business Development Specialist, Central Java, DBE 3: Life
Skills for Youth
5. Other Donors and Programs
Australia
• Wita Katoppo, Program Manager, Education, AusAID, Jakarta.
• Barry Clark, Australian Team Leader, Indonesia Australia Partnership in Basic Education
Program, Malang, East Java.
• Trina Supit, School Management and Governance advisor, Indonesia Australia
Partnership in Basic Education, Malang, East Java
European Union
• Sheila Town, Programme Manager, European Union, Jakarta
United Nations Children's Fund (UNICEF)
• Erik Bentzen, Chief, Education Programme, UNICEF/Jakarta
• Jiyono, Project Officer, Education Unit, UNICEF/Jakarta
• Bambang Irianto, Project Officer, Education Unit, UNICEF/Jakarta
World Bank
• Mae Chu Chang, Lead Educator, World Bank/Jakarta
• Susiana Iskandar, Senior Education Specialist, World Bank/Jakarta
• Andy Ragatz, Program Coordinator (Education), World Bank/Jakarta
Final Evaluation ofthe Managing Basic Education (MBE) Project ABE-BE IQC (Contract # EDH-I-02-05-00035-00)
45
ANNEX V SELECTED LIST OF DOCUMENTS CONSULTED
• AusAID (2006), IAPBE and NTT PEP Mid Term Review, 10 May 2006 (Report), Jakarta:
Australian Agency for International Development
• AusAID (2006), AusAID Indonesian Basic Education Portfolio Strategic Review, 27
September 2006 (Draft Independent Review, Report), Jakarta: Australian Agency for
International Development
• Buchori, M. (2004) Key Issues Related to Improvement of Basic Education in Indonesia
(Unpublished Report, Revision # 3: January 27, 2004)
• MONE, The World Bank, AusAID (2005), Teacher Employment and Deployment Study, .
Draft Report, 28 October 2005. Jakarta: Ministry of National Education, Directorate General
of Quality Improvement for Teacher and Education Personnel.
• Emmott, S., Bladen, 1., Suparman, Romli, Siregar, E. (2005). Creating Learning
Communitiesfor Children: Evaluation Report.
• MONE (2006), Pilot Study for Teacher Employment and Deployment, and Teacher
Certification and Quality Improvement: Progress and Plans for Implementation of Teacher
Certification Using the Teacher Law Regulations ofMay 2006, Draft, 9 June 2006 (Report), .
Jakarta: Ministry ofNational Education.
• MONE (2006), Strategic Plan (RENSTRA) 2005 - 2009, draft Translation June 27, 2006.
Jakarta: Ministry ofNational Education.
• Republic of Indonesia, (1999), Decentralisation Law. Undang-Undang Republik Indonesia
Nomor 22, Tahun 1999 Tentang Pemerintahan Daerah. Penerbit BP. Panca Usaha, Jakarta,
1999.
• Republic ofIndonesia (1999), Education Law, number 20, 2003: Undang Undang Nomor 20
Tahun 2003 tentang Sistem Pendidikan Nasional, signed by President Megawati
Sukarnoputri on July 8, 2003.
• Siregar, S. (2005). Program Evaluation, Creating Learning Communities for Children,
September 2005 (Individual Report), Jakarta: UNESCO, UNICEF.
• The World Bank (2004), Education in Indonesia: Managing the Transition to
Decentralization, August 2004. Jakarta, The World Bank.
• USAID (2004), MBE Annual Progress Monitoring September 2004 Phase 1 Districts,
(Report). Jakarta: United States Agency for International Development.
• USAID (2005), MBE Project Report and Updated Work plan, June 2005, (Report). Jakarta:
United States Agency for International Development.
• USAID (2006), DBE 2 Fiscal Year 2007 Work plan and Work plan Narrative, Final Version,
December 2006 (Report). Jakarta: United States Agency for International Development.
• USAID (2006), DBE 2 Quarterly Report Decentralized Basic Education, Indonesia,
Component 2: Teaching and Learning (Quarter 4 FY06 Report, July - September, 2006)
(Report). Jakarta: United States Agency for International Development.
Final Evaluation ofthe Managing Basic Education (MBE) Project
46
ABE-BE IQC (Contract # EDH-I-02-05-00035-00)
• USAID (2006), MBE Annual Progress Monitoring Report October 2004 - September 2005,
Phase 1 & 2 Districts, 1 February 2006 (Report). Jakarta: United States Agency for
International Development.
• USAID (2004), MBE Annual Progress Monitoring September 2004 Phase 1 Districts,
(Report). Jakarta: United States Agency for International Development.
• USAID (2006), MBE Annual Progress Monitoring Report September 2006, Phase 1, 2 & 3
Districts, 8 December 2006 (Report). Jakarta: United States Agency for International
Development.
• USAID (2006), Managing Basic Education, Work Plan, October 2006 - March 2007, Draft 5
October 2006 (Report). Jakarta: United States Agency for International Development.
• USAID (2006), Managing Basic Education, Work Plan, October 2006 - March 2007, Draft 5
October 2006 (Report). Jakarta: United States Agency for International Development.
• USAID (2006), Managing Basic Education, Assessing the Impact ofthe MBE Program on
Student Performance, May 2006 (Report). Jakarta: United States Agency for International
Development.
Final Evaluation ofthe Managing Basic Education (MBE) Project ABE-BE IQC (Contract # EDH-I-02-05-00035-00)
47
ANNEX VI AREAS FOR FURTHER RESEARCH
Within the context of lessons learned and other information gathered during the MBE Final
Evaluation, the following areas should be considered for further research.
• The KKG and MGMP have potential to assist teachers in improving their teaching skills.
An investigation should be made on the effectiveness of KKGIMGMP, providing cases
studies of good practice, and clear guidelines and practices for the effective establishment
and maintenance ofKKGIMGMP.
• There is some skepticism among teachers regarding the involvement of universities in the
delivery of on-the-:-job training to teachers in schools. Criticisms leveled at university staff
include their predominantly theoretical approach and lack of real classroom teaching
experience. A study therefore needs to be undertaken on various approaches to the delivery
ofthis training by universities, and the impact ofsuch training. .
• A number of teachers raised the issue of the PAKEM student-centered methodologies not
being supported by formal approaches to student assessment. An investigation should be
undertaken to determine the degree of alignment between student centered teaching
approaches and the local/national approaches to formal student assessment and evaluation.
Based on these results, recommendations should be made regarding improvements and
alignment ofstudent-centered teaching practices with formal student assessment.
• The MBE has produced a wealth of training and learning resources. An investigation
should be undertaken to determine their potential for wider application, and a strategy
developed for distribution.
• While the MBE focused interventions at both the school and district levels, it was clear to
the evaluation team that District Offices occupied an important link between the centralized
policy makers and schools. An investigation should be undertaken to determine strategies
and mechanisms to strengthen the capacity of District Offices to inform national policy
development.
• The MBE claims high levels of replication, for example in September 2006 it is reported
that dissemination by all MBE districts between 2004 - 2006 had reached 6,075 schools
and 51,630 participants. However details of the actual impact of these activities on the
schools is not provided, and the monitoring information provided is largely quantitative
(number of non-target schools trained, number of participants trained, number of study
visits, etc.). An investigation should be undertaken to determine (i) indicators of the actual
capacity of local authorities to manage and expand these sorts of programs, and (ii) the
actual take up rates with an assessment ofthe level of funding needed to achieve successful
implementation.
• The MBE evaluation indicated that while there was significant improvements in classroom
teaching environments, there was a need for further improvement in teaching
methodologies. Coupled with this, it was found that the monitoring of classroom teaching
was relatively soft. Similar findings occur in evaluations of theCLCC. A rigorous
investigation of the various PAKEM models and their impact on learning in the classroom
should be undertaken to determine where improvements need to be made. An additional
outcome will be the development of more rigorous classroom monitoring instruments and
associated strategies to assist teachers as a result ofthese evaluations.
Final Evaluation ofthe Managing Basic Education (MBE) Project
48
ABE-BE IQC (Contract # EDH-I-02-05-00035-00)
ANNEX VII SUMMARY OF MEETING WITH GOI AND
DONORS TO REVIEW FINDINGS
On March 8, 2007 the evaluation team met with representatives of the Gal and the international
donor community to review the findings of the MBE evaluation at the Mandarin Oriental Hotel
in Jakarta. The representatives were provided a draft copy of the full report in English and an
Executive Summary in Bahasa Indonesia two days prior to the meeting. In addition, about 12
representatives from local government and members of the MBE field staff were invited to
attend the meeting as "champions~' of the MBE approach and knowledgeable resource people.
The purpose of the meeting, as requested by the Gal, was to start a dialogue between the
interested parties, and especially between local and central government officials, about methods
for sustaining the school-based, active learning approach. About 60 people attended the meeting.
The meeting was opened by Deputy Minister Prof. Fuad Abdul Hamied of the Coordinating
Ministry for People's Welfare and by Bill Frej, Mission Director, USAID/lndonesia. Next, Prof.
M. Basri Wello presented the findings, lessons learned, and implications for the Gal/donors of
the MBE evaluation. Next the team answered detailed questions from the audience. After the
mid-meeting break, the meeting broke into two working groups to examine various possible
policy issues.
The two groups, which were evenly distributed with samples ofthe group at large discussed the
issue of how to ensure the sustainability of MBE-like programs. One group focused on
management issues and the other group focused on teaching and learning issues. They made the
following recommendations.
Group One Recommendations
Central Government:
• Evaluate the existing programs and products (modules, best practices) and, using
approved criteria, select programs and products as "Best Practices" resources.
• Create a compilation of Best Practices in MBE or from any other activities, distribute to
whoever needs it.
• Increase coordination among central government institutions (MaNE, MORA, KESRA)
and donors. To support this activity a data bank and information management system
needs to be designed in order to record any activities and products created by donors and
other activities.
• Involve LPMP in running any activity related to education training program.
Local Government:
• In order to avoid neglecting a "good and proved program" like MBE, and the effective
deployment of trained staff (teachers/ facilitators), it is important to institutionalize the
good and proved activities into local government regulation or the DEO by decree or law.
This activity will need the involvement and commitment of local Parliament, DEO, local
Planning Agency, local Education Board, and other education stakeholders. This step
should strengthen and acknowledge the responsibility of the DEO staff, principals,
facilitators, and teachers.
Final Evaluation ofthe Managing Basic Education (MBE) Project ABE-BE IQC (Contract# EDH-I-02-05-00035-00)
49
• Local governments should be more proactive in dealing with the donor community to get
more/other assistance related to MBE activities.
• Donor programs usually last only 2-3 years, so when stakeholders think a program is
good and needed, they should sit withthe donor to prepare an exit strategy.
• Create more coordination with other local governments by having a "same program
club."
• Local governments should collect any best practices that have been implemented in the.
local government, including: training, study visits, mentoring, modules, participatory
approaches designed by MBE or other projects.
• Many local governments are eager to provide matching grants to support and disseminate
good programs like MBE. In order to allocate a matching grant from the local budget,
DEO should ask the donor/program about the budget in running the activity.
Donors:
• To reduce the dependability of local governments on any donor's program, donors and
related parties (central government, local government, and DEOs) should design an exit
strategy.
• More coordination is needed in the donor community to avoid any duplication and
overlapping among related programs.
• Provide information on program budgets to central and local government, in order to let
the central and local .government prepare a matching grant to continue funding to
disseminate any good program, such as, MBE.
Group Two Recommendations
Empowerment ofKKG and MGMP
Empowering KKGIMGMP is very essential to facilitate various innovations to improve the
quality of education. This can be done with the following:
• In 2006 LPMP (Institution for Education Quality Assurance) provided Block Grant funds
to support the KKG and MGMP activities. Such support needs to be continued on a
competitive base, but allocation for food and transportation should be left out since such
cost can be covered with BOS fund.
• So far KKG and MGMP programs have been monotonous, uninteresting, and not focused
to support better teaching. They need assistance for developing more focused annual
programs. Therefore, the government should develop guidelines for more effective
KKGIMGMP programs that would enhance the teaching and learning process at schools.
• KKGIMGMP activities in fostering better teaching and learning should be acknowledged
in teacher certification context.
• Facilitators need to be involved as tutors in the KKGIMGMP activities. However, they
need official appointment from DEO (endorsed by Head of District! Mayor). Facilitators'
activities should be part of their required teaching hours to qualify for their professional
allowance, otherwise sustainability for any teaching and learning innovation could be
ineffective.
Final Evaluation ofthe Managing Basic Education (MBE) Project
50
ABE-BE IQC (Contract # EDH-I-02-05-00035-00)
• Many teachers found that the activities of KKG/MGMP have been ineffective and were
not responsive to their needs. Consequently; many of them did not come to the program
regularly. Therefore, the programs need to be redesigned to be more meaningful and
contextually responsive to increase the teachers' needs and participation. Invitations for
attending the activities should be sent to their school principals. Once they find the
activities provide knowledge and skills necessary for them to improve their
professionalism, they will attend the program regularly.
Institutionalizing PAKEM
PAKEM practice should be continuously performed in the classroom and so a standard
process for each subject is required. For this context GOI should immediately pass a decree
for the teaching/learning process standard which is now being developed.
RoleofLPMP
In most districts across the country many facilitators have been trained through the GOI and
donors' projects. Therefore, the GOI, in this case LPMP, should take a strategic role to unite
them in a forum where they can share ideas and skills, or make plans for better improvement
ofteaching and learning. LPMP needs to gather information about facilitators who have been
trained by MBE, CLCC, IAPBE, and by other projects in order to make them available
resources for similar training across the country.
Design ofGOI training
Training designed and provided by GOI projects should be in workshops rather than in
lectures. The GOI resource persons should be practitioners rather than bureaucratic people.
Should bureaucrats be involved, they need to do more school visits and practice teaching at
schools to make them more exposed to the existing situation of the teaching and learning
process before providing training to teachers.
Clearinghouse
We need a kind of "clearinghouse" to mediate between the training providers/facilitators
available and those who seek training. The LPMP could/should play this strategic role.
Final Evaluation ofthe Managing Basic Education (MBE) Project ABE-BE IQC (Contract # EDH-I-02-05-00035-00)
51
U5AID/Education Framework for Performance Management Plan
SO 18: Improved Quality of Decentralized Basic Education
1. Percent of students in targeted primary schools 2. Percent of junior secondary school students in achieving or surpassing minimum level on Indonesian - - targeted schools who have satisfactorily developed a reading, writing, and math achievement tests. (DBE2) predetermined set of life skill competencies. (DBE3)
3. Percent of primary schools in targeted clusters 4. Percent of targeted districts that developed long-
- term District Education Development Plans that meet where all teachers use active-learning methods and f-- a threshold of key criteria. (DBE1) practices. (DBE2)
IR 1: More Effective
Decentralized Management and
Governance of Schools
1.1: Percent of targeted schools
that developed long-term School
Development Plans that meet a
threshold of key criteria. (DBE1)
1.2: Number of non-targeted
schools that have developed
School Development Plans that
meet a threshold of key criteria.
(DBE1)
1.3: Percent of targeted schools
that disseminated Annual School
Budget in at least two venues.
(DBE1 )
1.4: Percent of targeted districts in
which all four of the key elements
of governance were involved in
developing District Education
Development Plan. (DBE1)
1.5: Percent of targeted districts
with improved resource and asset
management. (DBE1)
IR 2: Improved Quality of
Teaching and Learning
2.1: Number of targeted learners
completing fifth-grade in USAID￾supported primary orequivalent
non-school-based settings. ·C.L
26.1.1· (DBE2)
2.2: Number of targeted schools
producing annual School Quality
Report cards. (DBE2)
2.3: Number of active teachingl
learning exchanges in classrooms
of targeted schools. (DBE2)
2.4: Percent of students from
targeted kindergartens enrolling in
grade 2 two years after completing
kindergarten. (DBE2)
2.5: Increased capacity of higher
education institutions to contribute
to development. (DBE2)· C.L27.1
2.6: Number of teachers in
targeted schools using continuous
assessment practices. (DBE2)
2.7: Value of cash and in-kind
contributions from profit or non￾profit organizations mobilized to
support improved quality of
teaching and learning in targeted
schools. (DBE2)
IR 3: Increased Education
Relevance, Workforce and Life
Skills for Youth
3.1: Decrease in junior-secondary
school dropout rates in targeted
schools. (DBE3)
3.2: Value of cash and in-kind
contributions from profit and non￾profit organizations to support
youth education programs. (DBE3)
3.3: Junior secondary completion
rate in junior-secondary schools
and equivalent non-school settings
in target areas. (DBE3)
3.4: Number ofyouth in USAID
workforce skills programs who gain
training-related employment. ••
(Chevron)
3.5: Number of non-targeted junior
secondary schools in targeted
districts that are implementing
youth life skills toolkits. (DBE3)
3.6: Number of target teacher
networks that develop life skills
materials for youth. (DBE3)
3.7: Number of teachers in non￾targeted schools in target districts
trained to provide opportunities for
youth to develop life skills. (DBE3)
3.8: Number of non-formal
education learners using USAID￾produced junior secondary
equivalency education materials.
(DBE3)
IR 2: Improved Quality of
Teaching and Learning,
continued
2.8: Number of teachers trained in
inclusive education principles and
using them in the classroom. (HKI)
2.9: Number of non-targeted
schools in targeted districts that
have received training in active
learning practices. (DBE2)
2.10: Percent of teachers at
targeted schools reported by their
principals to have developed
teaching materials at Cluster
Resource Centers in the last
month. (DBE2)
2.11: Number of teachers using
ICT in the implementation of their
professional activities. (DBE2)
2
Other Common Agency
Indicators
Millennium Challenge
Indicators. and Millennium
Development Goals
( Other EDU Indicators ]
C.I. 8,Numberof persons
completing vocationejlftechnical
... education programs orequivalent
programs.** (Chevron)
C.1.8: Number of people trained
as a part of assistanceto create
or improve mechanisms for
citizens to engage their local
governments.* (MBE, DBE1)
C.1. 8: Number ofpeopletrained
as apart of assistance to local
governments to improve service
delivery and make those services
financiallyviable.* (MBE, DBE1)
.Percentage of national GDP spent
on primary education.*- (DBE1)
Percentage of girls completing
primary education.*** (EDU)
I Other Tsunami Indicators )
Number of Community Buildings
Reconstructed: classrooms.
Number of Community Buildings
Repaired: classrooms. (EDU)
Number of schools supported
through USAID Basic Ed
Programs. (MBE, DBE1, DBE3)
Number of students enrolled at
USAID-supported schools. (MBE,
DBE1, DBE3)
Number of person-days of
training. (MBE, DBE1, DBE2,
DBE3, HKI)
C.1. 26.1.1: Number of learners
enrolled in U$AID-supported
primary schools or equivalent non￾school-based settings (basic
educationonly).* (MBE, DBE1)
CA. 26.2.1: Number of learners
enrolledin lJSAID~supported
lower-secondary schools or
equivalent non-school-based
settin.gs.* (DBE3)·
CJ. 26.9: NLlmberofpeople
annual sentfor long~term training
(e.g. university) to the U.S, in
excess of one month under
programs for achieving·equitable
access to basic education.*
C.L27.2: Number of host country
higher education institutions
assisting in the implementation of
USAID programs.* (DBE2)
C.1. 26.8: Number of educators
(teachers and administrators)
trained through USAID Basic Ed
Programs. (MBE. DBE1, DBE3)
Number of Agencies that received
technical support. (EDU)
Speciai Interest. Dollar value of
assistance dedicated to
education. (EDU)
Special Interest. Number of
classroom beneficiaries (rebuilt
classrooms). (EDU)
Special Interest. Doliarvalue of
assistance for economic
opportunities for women. (EDU)
,,-------------------------~ ,
,
" Key to symbols * Indicator required for Agency
reporting under Basic Education
** Indicator required for Agency
reporting under Workforce through
VoclTec Ed
*** Indicator required for Joint State￾USAID (MDG goals) reporting
Indicators of program impact
Other indicators requested by
Washington
\
,
, '- w ,
3
, ,
,
\
\
I ,
,
so - Level Indicators ---------------------------
Performance Indicator Reference Sheet
Strategic Objective (SO): Improved Quality ofDecentralized Basic Education
Indicator 1: Percent ofstudents in targeted primary schools achieving or surpassing minimum level on Indonesia reading, writing, and
math achievement tests.
DESCRIPTION
Precise Definition(s): The percent oftotal number of qualifying students based on thresholds oftests instruments ofthe total number
of primary students tested.
Unit of Measure: Percent ofPrimary students
Disaggregated by: Province, district, school, school type, sex, grade level and subject
JustificationlManagement Utility: Quantitative measures ofstudent learning are the highest level of assessing quality education.
Student performance is an impact indicator the Agency prefers, where possible, to assess reading, writing and math achievement using
national or international testing instruments
Data Collection Method: Data will be collected by testing students in designated subjects in grades 3 and 6 in stratified sample of
targeted project-assisted schools associated with three cohorts.
Method ofAcquisition by USAID: DBE 2 contractor (EDC) will administer and score the specially-developed test instruments,
tabulate the results, and prepare report.
Data Source(s): DBE 2 student test data base, developed for sample and control schools.
Frequencyffiming of Data Acquisition: Twice annually over Life of Strategy; beginning in August 2006 (as a pre test) and July
2007 (as a post test) thru 2009.
Estimated Cost of Data Acquisition: Medium-high - baseline and sample data must be collected and analyzed depending on analysis
ofnational assessment tests, baseline collection, and sample size.
Responsible Individual(s) at USAlD: CTa (DBE2)
I)A.TAQUALITYISSUES
Date ofInitial Data Quality Assessment: Initial Baseline carried out in January 2006 and baseline new instrument anticipated August
2006. DQA TBD.!
Known Data Limitations and Significance (if any): Initial baseline established profile ofschools while new instrument in
development. .
Actions Taken or Planned to Address Data Limitations: DBE 2 reviewed the national assessment instruments and development of
new instruments in process.
Date of Future Data Quality Assessments: Annually in August through 2009
Procedures for Future Data Quality Assessments: TBD. An external contractor will be hired to carry out data quality assessments.
&·REPORTING
Data Analysis: The DBE 2 contractor will provide primary analysis ofsample results.
Presentation ofData: Charts, graphs, and narrative broken out by district-wise, by school, by school type, by sex, by grade and by
subject
Review ofData: DBE 2 contractor will review the data after each sample data collection
Reporting of Data: Data reported at least twice for Annual Report.
! Size: School sample
4
,~;,{i);,; ••·<r;.:;:;•. '··?l';; , OTHERNOTES ,
<,\,.•. ,.;.. ., ..•.'.',..•.... :r.·..•... '(
.'
..,...•' .... ( .,
Notes on Baselines/Targets: TBD
Baseline, Planned/Target And Achievement!Actual Values
Location: Recap National
Year (Fiscal) Planned Actual
Male Female Total Male Female Total
2004
2005
2006 a Not yet Not yet
available! available!
2007 b
Grade 3: 45%; Grade 3: 45%; Grade 3: 45%; Grade 6: Grade 6: Grade 6: 47.5% 47.5% 47.5%
Grade Grade Grade 3:47.5%; 2008 c 3:47.5%; 3:47.5%;
Grade 6: 50.% Grade 6: 50.% Grade 6: 50.%
2009 d Grade 3: 50%; Grade 3: 50%; Grade 3: 50%;
Grade6: 52.5% Grade6: Grade6: 52.5% 52.5%
2010 e
a. Cohort 1 covers 483 primary schools and 26 districts, does not include Aceh and Jakarta schools
b. Cohort 1 and 2 included
c. Cohorts 1, 2 and 3 included
d. Cohorts 1, 2 and 3 included
e. Cohorts 2 and 3 still remain, however no planned targets due to project completion
f. DBE 2 conducted pretest, date TBD
Note: Numbers are based on rough estimates and not upon a baseline assessment, actual number maybe
significantly different than those listed'on this page.
Location: Province North Sumatera
Year (Fiscal) Planned Actual
Male Female Total Male Female Total
2004
2005
2006 a Not yet Not yet
available! available!
2007 b
Grade 3: 45%; Grade 3: 45%; Grade 3: 45%; Grade 6: Grade 6: Grade 6: 47.5% 47.5% 47.5%
Grade Grade Grade 3:47.5%; 2008 c 3:47.5%; 3:47.5%;
Grade 6: 50.% Grade 6: 50.% Grade 6: 50.%
2009 d Grade 3: 50%; Grade 3: 50%; Grade 3: 50%; Grade6: Grade6: Grade6: 52.5% 52.5% 52.5% '.
2010 e
a. Cohort 1 covers 483 primary schools and 26 districts, does not include Aceh and Jakarta schools
b. Cohort 1 and 2 included
5
c. Cohorts 1, 2 and 3 included
d. Cohorts 1, 2 and 3 included
e. Cohorts 2 and 3 still remain, however no planned targets due to project completion
f. DBE 2 will conduct pretest, date TBD
Note: Numbers are based on rough estimates and not upon a baseline assessment, actual number maybe
significantly different than those listed on this page.
Location: Province West Java and Banten
Year (Fiscal) Planned Actual
Male Female Total Male Female Total
2004
2005
2006 a Not yet Not yet
available! available!
2007 b
Grade 3: 45%; Grade 3: 45%; Grade 3: 45%; Grade 6: Grade 6: Grade 6: 47.5% 47.5% 47.5%
Grade Grade Grade 3:47.5%; 2008 c 3:47.5%; 3:47.5%;
Grade 6: 50.% Grade 6: 50.% Grade 6: 50.%
2009 d Grade 3: 50%; Grade 3: 50%; Grade 3: 50%; Grade6: Grade6: Grade6: 52.5% 52.5% 52.5%
2010 e
a. Cohort 1 covers 483 primary schools and 26 districts, does not include Aceh and Jakarta schools
b. Cohort 1 and 2 included
c. Cohorts 1, 2 and 3 included
d. Cohorts 1, 2 and 3 included
e. Cohorts 2 and 3 still remain, however no planned targets due to project completion
f. DBE 2 will conduct pretest, date TBD
Note: Numbers are based on rough estimates and not upon a baseline assessment, actual number maybe
significantly different than those listed on this page.
Location: Province Central Java
Year (Fiscal) Planned Actual
Male Female Total Male Female Total
2004
2005
2006 a Not yet Not yet
available! available!
2007 b
Grade 3: 45%; Grade 3: 45%; Grade 3: 45%; Grade 6: Grade 6:
47.5% 47.5% Grade 6: 47.5%
Grade Grade Grade 3:47.5%; 2008 c 3:47.5%; 3:47.5%;
Grade 6: 50.% Grade 6: 50.% Grade 6: 50.%
2009 d Grade 3: 50%; Grade 3: 50%; Grade 3: 50%; Grade6: Grade6: Grade6: 52.5% 52.5% 52.5%
2010 e
a. Cohort 1 covers 483 pnmary schools and 26 districts, does not mclude Aceh and Jakarta schools
b. Cohort 1 and 2 included
c. Cohorts 1, 2 and 3 included
d. Cohorts 1, 2 and 3 included
e. Cohorts 2 and 3 still remain, however no planned targets due to project completion
6
f. DBE 2 will conduct pretest, date TBD
Note: Numbers are based on rough estimates and not upon a baseline assessment, actual number maybe
significantly different than those listed on this page.
Location: Province East Java
Year (Fiscal) Planned Actual
Male Female Total Male Female Total
2004
2005
2006 a Not yet Not yet
available! available!
2007 b Grade 3: 45%; Grade 3: 45%; Grade 3: 45%; Grade 6: Grade 6: Grade 6: 47.5% 47.5% 47.5%
Grade Grade Grade 3:47.5%; 2008 c 3:47.5%; 3:47.5%;
Grade 6: 50.% Grade 6: 50.% Grade 6: 50.%
2009 d
Grade 3: 50%; Grade 3: 50%; Grade 3: 50%; Grade6: Grade6: Grade6: 52.5% 52.5% 52.5%
2010 e
a. Cohort 1 covers 483 primary schools and 26 districts, does not include Aceh and Jakarta schools
b. Cohort 1 and 2 included
c. Cohorts 1, 2 and 3 included
d. Cohorts 1, 2 and 3 included
e. Cohorts 2 and 3 still remain, however no planned targets due to project completion
f. DBE 2 will conduct pretest, date TBD
Note: Numbers are based on rough estimates and not upon a baseline assessment, actual number maybe
significantly different than those listed on this page
Location: Province South Sulawesi
Year (Fiscal) Planned Actual
Male Female Total Male Female Total
2004
2005
2006 a Not yet Not yet
available! available!
2007 b
Grade 3: 45%; Grade 3: 45%; Grade 3: 45%; Grade 6: Grade 6: Grade 6: 47.5% 47.5% 47.5%
Grade Grade Grade 3:47.5%; 2008 c 3:47.5%; 3:47.5%;
Grade 6: 50.% Grade 6: 50.% Grade 6: 50.%
2009 d Grade 3: 50%; Grade 3: 50%; Grade 3: 50%; Grade6: Grade6: Grade6: 52.5% 52.5% 52.5%
2010 e
a. Cohort 1 covers 483 primary schools and 26 districts, does not include Aceh and Jakarta schools
b. Cohort 1 and 2 included
c. Cohorts 1,2 and 3 included
d. Cohorts 1, 2 and 3 included
e. Cohorts 2 and 3 still remain, however no planned targets due to project completion
f. DBE 2 will conduct pretest, date TBD
Note: Numbers are based on rough estimates and not upon a baseline assessment, actual number maybe
significantly different than those listed on this page
7
Performance Indicator Reference Sheet
Strategic Objective (SO): Improved Quality ofDecentralized Basic Education
Indicator 2: Percentage ofjunior secondary school students in target schools who have satisfactorily developed a .1 set oflife skill competencies.
•. ~Y·' i ..• '>t'····.···~." ... • .' .... :: ': .......... '{ ..• DESCRIP]ION ....
F" .:. . . .....
Precise Definition(s): The percentage ofjunior secondary school students who satisfactorily pass a life skills performance assessment.
The percentage is to be calculated as the number ofjunior secondary school students passing the test divided by the total number of
students assessed.
Unit of Measure: Student
Disaggregated by: Cohort, province, district, subdistrict, school, school type, sex
JustificationlManagement Utility: Acquisition of life skills is a core objective ofthe project and Gal curriculum changes, and a key
indicator ofthe project's results.
··>FiF:·"••·,•....... ' ':'. :.·'i .......... PLi\NFOl{DATA ACQV~$ITIONBYUSA!D
.
•••
..•..:...: ";:'Y.•i ...... ,".' i"
Data Collection Method: Review of performance assessment records
Method ofAcquisition by USAID: Performance assessment ofstudents (grade 9 cohort).
Data Source(s): Performance assessment records.
Frequencyrriming of Data Acquisition: Annually, once for each graduating cohort, at the end ofthe school year
Estimated Cost of Data Acquisition: Medium
Responsible Individual(s) at USAID:. CTO (DBE 3) and/ or M&E Activity Manager
I·:.)}.•i ..ji:" F· ..:Fi .'. :'.'::. . .... Di\.TAQUALI'fY ISSlJltS '. .......
I·· ..... ',..;.::./., •.• ..' .FFF . '"
Date ofInitial Data Quality Assessment: 2007 (on a pilot group)
Known Data Limitations and Significance (if any): The performance assessment neds to be developed and tested. The passing
grade/level needs to be determined.
Actions Taken or Planned to Address Data Limitations: TBD, pilot the assessment, train assessors well
Date of Future Data Quality Assessments: TBD
Procedures for Future Data Quality Assessments:. TBD
: ····::.·:••·•.••·.i(:ri,. ·..·.:.."..i: .. : :······.··.·:J.>pANli'AullA.TAA . . .:..... ... .. ~.. '.. .:. '.' <
.F: ....,: ..R..£i.£ U;l':,L .lL~"" ••7:.F,. .,.:. ~"""'.'~i"t" "
" ..'
Data Analysis: DBE3 reports
Presentation of Data: Graphs, charts and narratives
Review ofData: DBE 3 in collaboration with schools.
Reporting of Data: To provincial offices, districts, schools, School Committees, community, MaNE, and annual report.
:.. .......: .. ~:;..... .', .' /i'" .••..• OTHER NOTES' ..
.: .... ..:....i,.. .: ........•.:......:......:......... ...:.... .. . •... :.·.0 .
Notes on Baselines/Targets:
Baseline, Planned/Target And Achievement!Actual Values
Location: National
Year (Fiscal) Planned Actual
2004 NA
2005 NA
2006
2007
2008
2009
2010
8
Precise Definition(s): The total number primary schools instituting a minimum number of active-learning methods and practices,
including use of active, gender sensitive teaching methods divided by number of primary schools in targeted clusters
Unit of Measure: Number ofschools
Disaggregated by: Province, district, school and type,
Justification/Management Utility: This is an impact indicator that reflects the essence ofimproved quality ofinstruction in primary
education.
Data Collection Method: DBE 2 will develop and or compile observation instruments. Data will be collected through teacher and
classroom observation instrument(s) that identifies and measures key active learning practices, administered to selected grade 3 and 6
in stratified sample ofproject-assisted schools associated with three cohorts.
Method. ofAcquisition by USAID: DBE 2 contractor (EDe) will administer and score the specially-developed observation
instruments, tabulate the results, and prepare report.
Data Source(s): Existing and developed testing and observation instruments. DBE2 teacher and classroom observation data base.
Frequencyrriming of Data Acquisition: Annually in July 2006, July 07, July 08, July 09 and July 2009.
Estimated Cost of Data Acquisition: Low-medium.
Responsible Individual(s) at USAID: DBE 2 CTa
Date ofInitial Data Quality Assessment: July, 2006
Known Data Limitations and Significance (if any): Unknown
Actions Taken or Planned to Address Data Limitations:
Date ofFuture Data Quality Assessments: July through 2009
Procedures for Future Data Assessments:
Notes on Baselines/Targets:
Baseline, Plannedrrarget And Achievement/Actual Values·
Location: Recap National
Year (Fiscal)
2004
2005
2006 a
2007 b
2008 c
2009 d
2010 e
Planned
o
40%
50%
65%
Actual
a. Cohort 1 covers 483 primary schools and 26 districts, does not include Aceh and Jakarta schools
b. Cohort 1 and 2 included
c. Cohorts 1, 2 and 3 included
d. Cohorts 1, 2 and 3 included
9
e. Cohorts 2 and 3 still remain, however no planned targets due to project completion
Note: Numbers are based on rough estimates and not upon a baseline assessment, actual number maybe
significantly different than those listed on this pages
Location: North Sumatera
Year (Fiscal) Planned Actual
2004
2005
2006 a 0
2007 b 40%
2008 c 50%
2009 d
65%
2010 e
a. Cohort 1 covers 483 primary schools and 26 districts, does not include Aceh and Jakarta schools
b. Cohort 1 and 2 included
c. Cohorts 1, 2 and 3 included
d. Cohorts 1, 2 and 3 included
e. Cohorts 2 and 3 still remain, hQwever no planned targets due to project completion
Note: Numbers are based on rough estimates and not upon a baseline assessment, actual number maybe
significantly different than those listed on this page.
Location: West Java & Banten
Year (Fiscal) Planned Actual
2004
2005
2006 a 0
2007 b 40%
2008 c 50%
2009 d
65%
2010 e
a. Cohort 1 covers 483 primary schools and 26 districts, does not include Aceh and Jakarta schools
b. Cohort 1 and 2 included
c. Cohorts 1, 2 and 3 included
d. Cohorts 1, 2 and 3 included
e. Cohorts 2 and 3 still remain, however no planned targets due to project completion
Note: Numbers are based on rough estimates and not upon a baseline assessment, actual number 'maybe
significantly different than those listed on this page.
Location: Central Java
Year (Fiscal) Planned Actual
2004
2005
2006 a 0
2007 b 40%
2008 c 50%
2009 d
65%
2010 e
10
a. Cohort 1 covers 483 primary schools and 26 districts, does not include Aceh and Jakarta schools
b. Cohort 1 and 2 included
c. Cohorts 1, 2 and 3 included
d. Cohorts 1,2 and 3 included
e. Cohorts 2 and 3 still remain, however no planned targets due to project completion
Note: Numbers are based on rough estimates and not upon a baseline assessment, actual number maybe
significantly different than those listed on this page.
Location: East Java
Year (Fiscal) Planned Actual
2004
2005
2006 a 0
2007b
40%
2008 c 50%
2009 d
,65%
2010 e
a. Cohort 1 covers 483 primary schools and 26 districts, does not include Aceh and Jakarta schools
b. Cohort 1 and 2 included
c. Cohorts 1, 2 and 3 included
d. Cohorts 1,2 and 3 included
e. Cohorts 2 and3still remain, however no planned targets due to project completion
Note: Numbers are based on rough estimates and not upon a baseline assessment, actual number maybe
significantly different than those listed on this page.
Location: South Sulawesi
Year (Fiscal) Planned Actual
2004
2005
2006 a 0
2007 b 40%
2008 c 50%
2009 d
65%
2010 e
a. Cohort 1 covers 483 primary schools and 26 districts, does not include Aceh and Jakarta schools
b. Cohort 1 and 2 included
c~ Cohorts 1, 2 and 3 included
d. Cohorts 1, 2 and 3 included
e. Cohorts 2 and 3 still remain, however no planned targets due to project completion
Note: Numbers are based on rough estimates and not upon a baseline assessment, actual number maybe
significantly different than those listed on this page.
11
Performance Indicator Reference Sheet
Strategic Objective (SO): Improved Quality ofbecentralized Basic Education
Indicator 4: Percent oftargeted districts that developed long-term District Education Development Plans that meet a threshold
of key criteria.
DESCRIP'l;lON
Precise Definition(s):
• "Key criteria" refers to a set of criteria that will be finalized by DBEI in June 2006. The criteria will be similar in
complexity and depth to those used to measure the quality ofschool development plans (see indicator 1.1), and will include
aspects such as: how well the 12 streams of education funding are included in the planning process, whether or not it is a
multi-year plan that connects long-term needs with specific act
• ivities each year, and whether or not community input is reflected in the product.
• "District Education Development Plan" is Rencana Pengembangan Pendidikan Kabupaten/Kota (RPPK).
• "Threshhold" will be determined, probably about half ofthe criteria.
Unit of Measure: Percent of districts
Disaggregated by: Province
Justification and history ofindicator: This indicator measures the results ofthe DBEI project at the district level and reflects
the results ofmany-although not all---ofthe areas oftechnical assistance under the project. Some ofthe criteria will identify
how well the district-level plan reflects the needs expressed by the school-level plans in that district, and to what degree
participation ofschool level actors is encouraged and reflected in the district level plan. Additional criteria will identify how
well the district education office has involved key elements of governance in the process and content ofthe district plan (see
indicator 104 for more discussion of governance). Lastly, the district level plan will reflect the technical assistance the DBEI
project will provide directly to the education offices (DINAS Pendidikan) on planning, EMIS, school based management,
community participation, and asset and resource management.
DBEI is the only project reporting against this indicator. MBE reports district level impact in a different way.
Data Collection Method: DBEI will finalize (in June 2006) a series of questions that assess a district development plan against
approximately 20-30 objective criteria. Likely to include DBEI staff plus district and school personnel using a standardized
instrument or questionnaire.
Data Source(s):
Frequencyffiming ofData Acquisition: Baseline in June 2006. Regular collection in October 2006 and then annually every
October.
Method of Acquisition by USAID: Reports from DBE 1
Estimated Cost of Data Acquisition: low
Responsible Individual(s) at USAfD Elizabeth Sunindyo (DBEl)
Date ofInitial Data Quality Assessment: no assessment conducted yet
Known Data Limitations and Significance (if any):
Actions Taken or Planned to Address Data Limitations:
Date of Future Data Quality Assessments: no assessment planned yet
Procedures for Future Data Quality Assessments: Review district records; Site visits to schools; conversations with District
officials
12
Data Analysis: Suggested questions for analysis:
Is there a trend in terms ofJava vs. Non-Java; or city (kota) versus district (kabupaten)? Does size ofdistrict seem to make
a difference?
Once a district meets half ofthe criteria, is further improvement likely?
Are many district plans weak in the same areas?
Is the quality ofthe district plan related to the overall capacity ofthe district (as related to the DBEI technical assistance on
district management capacity).
Some ofthe districts may have started at a "lower" point than others before working with the DBEI technical teams. Does
the starting point seem to make a difference?
Does presence ofLGSP appear to make a difference?
Presentation ofData: TBD (Graphs, charts and narratives?)
Review ofData: USAID to review data when submitted, and in preparation for Annual Report
Reporting of Data: USAID Office ofEducation will include this indicator in Annual Report
...
.
2006 DBE Aceh 0 of2 Districts (phase I) _ of 2 Districts (Phase I) Not expected to see results in 2006
because DBEI assistance to Aceh
focusing on school level in 2006 and
adding district components in 2007.
DBE N. Sumatra
DBEBanten
DBE Jakarta
DBE W. Java
DBE C. Java
DBE E. Java
DBE S. Sulawesi
SUMMARY
1 of 5 Districts (Phase I)
oof 2 Districts (Phase 1)
oof 1 Districts (Phase 1)
1 of 3 Districts (Phase 1)
2 of 5 Districts (phase I)
2 of 5 Districts (Phase I)
2 of 5 Districts (Phase 1)
oof 27 Districts (Phase I)
of 5 Districts (phase I)
of 2 Districts (Phase I)
of 1 Districts (Phase I)
of 3 Districts (Phase I)
of 5 Districts (Phase I)
of 5 Districts (Phase I)
of 5 Districts (Phase I)
_ of27 Districts (Phase I) All USAID-supported districts will
receive some assistance on district level
planning in 2006. All will produce a
district plan but we do not expect that all
plans will be adequate when measured
against the criteria until later years ofthe
program.
2007 DBEAceh 1 of 2 Districts (phase I)
oofTBD Districts (Phase II)
DBE N. Sumatra 3 of 5 Districts (phase I)
oofTBD Districts (Phase II)
DBE Banten 1 of 2 Districts (phase I)
oofTBD Districts (Phase II)
DBEJakarta 1 of 1 Districts (phase 1)
oofTBD Districts (Phase II)
DBE W. Java 2 of 3 Districts (phase I)
oofTBD Districts (Phase II)
DBE C. Java 3 of 5 Districts (phase 1)
oofTBD Districts (Phase II)
DBE E. Java 3 of 5 Districts (phase I)
oofTBD Districts (Phase II)
DBE S. Sulawesi 3 of 5 Districts (phase I)
oofTBD Districts (phase II)
SUMMARY 17 of27 Districts (Phase I)
oof 25 Districts (Phase II)
Overall, we expect to see about half of
the Phase I districts make adequate plans
(nearly all in 2008) and very few ofthe
Phase II districts.
13
2008 DBEAceh 2 of 2 Districts (phase I)
half ofTBD Districts (Phase II) ..
oofTBD Districts (Phase III)
DBE N. Sumatra 5 of 5 Districts (phase I)
half ofTBD Districts (Phase II)
oofTBD Districts (Phase III)
DBE Banten 2 of 2 Districts (phase I)
half ofTBD Districts (phase II)
oofTBD Districts (Phase III)
DBE Jakarta 1 of 1 Districts (phase I)
half ofTBD Districts (Phase II)
oof TBD Districts (Phase III)
DBE W. Java 3 of 3 Districts (phase I)
half ofTBD Districts (Phase II)
oofTBD Districts (Phase III)
DBE C. Java 5 of 5 Districts (phase I)
half ofTBD Districts (Phase II)
oofTBD Districts (Phase III)
DBE E. Java 5 of 5 Districts (phase I)
half ofTBD Districts (Phase II)
oofTBD Districts (Phase III)
DBE S. Sulawesi 5 of 5 Districts (phase I)
half ofTBD Districts (Phase II)
oofTBD Districts (Phase III)
SUMMARY 27 of 27 Districts (Phase I) Overall, we expect to see nearly all ofthe
12 of 25 Districts (Phase II) Phase I districts make adequate plans in
oof 50 Districts (Phase III) 2008, about half ofthe Phase II, and very
few ofthe Phase III districts. After 2008
we will no longer report on Phase I.
2009 DBEAceh all ofTBD Districts (Phase II)
half ofTBD Districts (Phase III)
DBE N. Sumatra all ofTBD Districts (Phase II)
half ofTBD Districts (Phase III)
DBEBanten all ofTBD Districts (Phase II)
half ofTBD Districts (Phase III)
DBE Jakarta all ofTBD Districts (Phase II)
half ofTBD Districts (Phase III)
DBE W. Java all ofTBD Districts (Phase II)
half ofTBD Districts (Phase III)
DBE C. Java all ofTBD Districts (Phase II)
half ofTBD Districts (Phase III)
DBE E. Java all ofTBD Districts (Phase II)
half ofTBD Districts (Phase III)
DBE S. Sulawesi all ofTBD Districts (phase II)
half ofTBD Districts (Phase III)
SUMMARY 23 of 25 Districts (Phase II) We are no longer reporting on Phase I in
25 of 50 Districts (Phase III) 2009. Overall in 2009, we expect to see
nearly all ofthe Phase II make adequate
progress on district plans, and about half of
the Phase III districts.
14
Year Province ,
Planned Actual Notes ......
2010 DBEAceh all ofTBD Districts (Phase III)
DBE N. Sumatra all ofTBD Districts (Phase III)
DBEBanten all ofTBD Districts (phase III)
DBE Jakarta all ofTBD Districts (Phase III)
DBE W. Java all ofTBD Districts (Phase III)
DBE C. Java all ofTBD Districts (Phase III)
DBEE.Java all ofTBD Districts (phase III)
DBE S. Sulawesi all ofTBD Districts (phase III)
SUMMARY 48 of 50 Districts (phase III) Weare no longer reporting on Phase I or
Phase II in 2010. Overall in 2010, we I expect to see nearly all ofthe Phase III "
districts produce adequate district plans.
15
-------------------------~--~--~----------~---------- -------------
Pe...·ormance Indicator Reference Sheet
Strategic Objective (SO): Improved Quality ofDecentralized Basic Education
Intermediate Result 1: More effective decentralized management and governance ofschools
Indicator 1.1: Percent oftargeted schools that developed long-term School Development Plans that meet athreshold of key
criteria.
DESCRIPTION
Precise Definition(s):
"Key criteria" refers to criteria in a measurement instrument (developed by the DBEI project) that measures the quality and
relevance of a school development plan against 32 objective criteria, in four main categories: (1) including multiple years,
(2) developed with community participation, (3) regularly updated, and (4) based on data about the school ("profile"), an
analysis ofthe schools needs, and linking those with proposed programs. (For more information, see DBE1 program
documents.)
"School Development Plan" means Rencana Pengembangan Sekolah (RPS).
"Threshold" means 9 or more ofthe 32 criteria on the DBE1 instrument.
Unit of Measure: Percent ofschools·
Disaggregated by: Province, district, school, and school type
Justification and history ofindicator: In 2005, the National Ministry ofEducation (MONE) passed govemmentregulation 19
which mandated all schools (including madrasah reporting to MORA) to complete school development plans. The law requires
the plans to be multi-year and therefore to provide a set oflonger-term priorities against which annual school budgets (RAPBS)
can be developed and implemented. Anecdotally and based on MBE program experience, USAID found that most school
development plans were perfunctory documents that often were created by a school principal alone, in order to say the
requirement was completed, and then not used. The impetus for adequate school development plans increased in 2006, however,
when MONE stipulated that an adequate school development plan was a requirement before schools could receive money from
the Biaya Operasional Sekolah (BOS), a grant program that was expanded in 2006 based on oil revenues, and which often
doubled the amount of budget available to an individual school. The MONE regulations do not specify the format or detailed
contents of a school development plan, but MONE and MORA officials have been involved in the development ofthe MBE and
DBE1 methodologies and training materials. The USAID MBE and DBE1 programs are one ofonly a few that provide technical
assistance to schools in developing a useful plan.
This indicator measures the extent to which USAID-supported schools are managing resources at the school level effectively and
transparently.
DA'rA'QUALITY ISSUli:S
Data Collection Method: DBE1 district coordinators work with district officials and use a standardized instrument. The
instrument and the data are maintained by the DBE1 project. (In DBE1 project documents, this is "Indicator 1".)
Data Source(s): School data and records, conversations with school officials and school committee members, and some direct
observation
Frequencyffiming of Data Acquisition: Baseline in March 2006. Regular collection in June 2006, and subsequently every six
months in June and December.
Method ofAcquisition by USAID: Reports from DBE1 project.
Estimated Cost of Data Acquisition: Low to medium depending on how much ofthe data is self-reported vs collected
Individual(s) at USAID: Elizabeth Sunindyo (eTO DBE1)
MBE and DBE both provide technical assistance on school development plans. This indicator measures the progress with the
DBE1 as the MBE project uses a slightly different indicator.
Date oflnitial Data Quality Assessment: no assessment conducted yet
Known Data Limitations and Significance (if any):
Actions Taken or Planned to Address Data Limitations:
Date of Future Data Quality Assessments: no assessment planned yet
Procedures for Future Data Quality Assessments: Review school records, make site-visits, and interview or survey school
personnel.
16
Aceh baseline data to be
collected later in 2006.
Notes
TBD of 40 Schools (Phase I)
Actual
DBE Aceh
Data Analysis: Suggested questions for analysis:
How do religious MI compare to secular SD? (The baseline report in April 2006 does not disaggregate the data specifically
by type ofschool, but future monitoring reports will analyze by type ofschool.)
In the baseline, which criteria were most often met (and not met) by schools that had not yet received our assistance?
In subsequent years, what are the areas offastest improvement? Slowest improvement?
Presentation of Data: Graphs, charts and narratives.
Review ofData: USAID to review data when submitted, and in preparation for Annual Report
Reporting of Data: USAID Office ofEducation will NOTinclude this indicator in Annual Report
DBE N. Sumatra
DBEBanten
34 of 100 Schools (Phase I)
xx of 50 Schools (Phase I)
Baseline was collected March
2006
DBEJakarta
DBE W. Java
TBD of 7 Schools (Phase I)
xx of61 Schools (Phase I)
Jakarta baseline data to be
collected later in 2006.
DBE C. Java 6 of 106 Schools (Phase I) Baseline was collected March
2006
DBE E. Java 12 of 84 Schools (Phase I) Baseline was collected March
2006
DBE S. Sulawesi 17 of 88 Schools (Phase I) Baseline was collected March
2006
SUMMARY 154 of 536 Schools (Phase I) All USAID-supported schools
will receive some assistance on
school-level planning in 2006.
Many schools already have a
RPS and/or will produce an
improved RPS with assistance
from DBEl. We do not expect
that all plans will be adequate
when measured against the
criteria until later years ofthe
program. Baseline in March
2006 confirmed that only 393 of
536 targeted schools had a RPS,
and that only 154 ofthem met
the threshold (9 or more of the
32 criteria). This number will
increase later in 2006 when
Jakarta and Aceh baseline data is
collected.
17
)*~ir<> cp -;- ... >
~ruv111~1;; Planned Actual ... Notes
2007 DBEAceh 15 of 40 Schools (Phase I)
XX ofTBD Schools (Phase II)
DBE N. Sumatra 50 of 100 Schools (Phase I)
XX ofTBD Schools (phase II)
DBEBanten 90 of 111 Schools (Phase I)
XX ofTBD Schools (Phase II)
DBE Jakarta 3 of 7 Schools (phase I)
XX ofTBD Schools (phase II)
DBE W. Java TBD of 61 Schools (Phase I)
XX ofTBD Schools (Phase II)
DBE C. Java 25 of 106 Schools (Phase I)
XX ofTBD Schools (Phase II)
DBE E. Java 30 of 84 Schools (Phase I)
XX ofTBD Schools (Phase II)
DBE S. Sulawesi 35 of 88 Schools (Phase I)
XX ofTBD Schools (Phase II)
SUMMARY 248 of 536 Schools (Phase I) Overall, we expect to see about
100 of 500 Schools (Phase II) half ofthe Phase I schools make
adequate plans (nearly all in
2008) and very few ofthe Phase
II schools.
2008 DBEAceh 30 of 40 Schools (Phase I)
XX ofTBD Schools (Phase II)
XX ofTBD Schools (Phase III)
DBE N. Sumatra 80 of 100 Schools (Phase I)
XX ofTBD Schools {Phase II)
XX ofTBD Schools (phase III)
DBEBanten 100 of 111 Schools (Phase I)
XX ofTBD Schools (Phase II)
XX ofTBD Schools (Phase III)
DBE Jakarta 6 of 7 Schools (phase I)
XX ofTBD Schools (phase II)
XX ofTBD Schools (phase III)
DBE W. Java TBD ofTBD Schools (Phase I)
XX ofTBD Schools (Phase II)
XX ofTBD Schools (Phase III)
DBE C. Java 60 of 106 Schools (Phase I)
XX ofTBD Schools (phase II)
XX ofTBD Schools (phase III)
DBE E. Java 60 of 84 Schools (Phase I)
XX ofTBD Schools (Phase II)
XX ofTBD Schools (Phase III)
DBE S. Sulawesi 60 of 88 Schools (Phase I)
XX ofTBD Schools (Phase II)
XX ofTBD Schools (Phase III)
SUMMARY 450 of 536 Schools (Phase I) Overall, we expect to see nearly
300 of 500 Schools (Phase II) all ofthe Phase I schools make
200 of 1,000 Schools (Phase III) adequate plans in 2008, about
half ofthe Phase II, and very
few ofthe Phase III schools.
After 2008 we will no longer
report on Phase 1.
18
Year Province Platlll~d·' ,',' "
Actual Notes ,
2009 DBEAceh all ofTBD Schools (Phase II)
half ofTBD Schools (Phase III)
DBE N. Sumatra all ofTBD Schools (Phase II)
half ofTBD Schools (Phase III)
DBE Banten all ofTBD Schools (Phase II)
half ofTBD Schools (Phase III)
DBE Jakarta all ofTBD Schools (Phase II)
half ofTBD Schools (Phase III)
DBE W. Java all ofTBD Schools (Phase II)
halfofTBD Schools (Phase III)
DBE C. Java all ofTBD Schools (Phase II)
half ofTBD Schools (Phase III)
DBEE. Java all ofTBD Schools (Phase II)
half ofTBD Schools (Phase III)
DBE S. Sulawesi all ofIBD Schools (phase II)
half ofTBD Schools (Phase III)
SUMMARY 400 of 500 Schools (Phase II) Weare no longer reporting on
500 of 1,000 Schools (Phase III) Phase I in 2009. Overall in
2009, we expect to see nearly all
ofthe Phase II schools make
adequate plans, and about half of
the Phase III schools.
2010 DBE Aceh all ofTBD Schools (Phase III)
DBE N. Sumatra all ofTBD Schools (Phase III)
DBEBanten all ofTBD Schools (Phase III)
DBE Jakarta all ofTBD Schools (Phase III)
DBE W. Java all ofTBD Schools (Phase III)
DBE C. Java . all ofTBD Schools (Phase III)
DBEE. Java all ofTBD Schools (Phase III)
DBE S. Sulawesi all ofTBD Schools (Phase III)
SUMMARY 800 of 1,000 Schools (Phase III) Weare no longer reporting on
Phase lor Phase II in 2010.
Overall in 2010, we expect to
see nearly all ofthe Phase III
schools produce adequate plans.
19
---------------~--~~------------------------
Ped'o.omance Indicator Reference Sheet
Strategic Objective (SO): Improved Quality of Decentralized Basic Education
Intermediate Result 1: Improved quality ofteaching and learning
Indicator 1.2: The number of non-targeted schools that have prepared School Development Plans that meet a threshold ofkey
criteria.
Precise Definition(s):
"Key criteria" refers to criteria in a measurement instrument (developed by the DBEI project) that measures the quality and
relevance of a school development plan against 32 objective criteria, in four main categories: (1) including multiple years,
(2) developed with community participation, (3) regularly updated, and (4) based on data about the school ("profile"), an
analysis ofthe schools needs, and linking those with proposed programs. (For more information, see DBEI program
documents.) [Same as Indicator 1.1]
"School Development Plan" means Rencana Pengembangan Sekolah (RPS). [Same as Indicator 1.1]
"Threshold" means 9 or more ofthe 32 criteria on the DBE1 instrument. [Same as Indicator 1.1]
In order for a school to be counted in this indicator, DBEI staffwould have to assess the School Development Plan against
the same criteria and with the same data collection instrument as in Indicator 1.1.
Unit of Measure: number ofschools
Disaggregated by: Province, and school type
Justification and history ofindicator: See Indicator 1.1.
This indicator measures replication ofthe DBEI methodologies for developing school development plan. This indicator
demonstrates part ofthe demand for, and perceived usefulness of, the approaches developed under DBE1.
(There is no direct support under DBEI to fund programs outside ofthe targeted areas. There is limited funding and support to
facilitate the initial dissemination oftools and approaches to other interested schools and districts, whether they be intargeted
provinces or in other areas ofthe country.)
DATA-QUALITY ISSUES
Data Collection Method: DBEI staff assess the plan usingthe same instrument as in Indicator 1.1. The assessment could be
conducted in person or long-distance.
Data Source(s): N/A
Frequencyrriming of Data Acquisition: ad hoc, but to be summarized every year by implementer and reported to USAID in
September annual report.
Method ofAcquisition by USAID: September annual report
Estimated Cost of Data Acquisition: Low to medium, depending on information available at District level
Responsible Individual(s) at USAID: CTO ofDBEI (Elizabeth Sunindyo)
Date ofInitial Data Quality Assessment: no assessment conducted yet
Known Data Limitations and Significance (if any):
Actions Taken or Planned to Address Data Limitations:
Date of Future Data Quality Assessments: no assessment planned yet
Procedures for Future Data Quality Assessments:
Data Analysis: Suggested questions for analysis:
Is there higher demand with schools in the target districts and/or provinces as opposed to provinces that are far away?
How do other schools learn about the DBEI resources and how do they determine ifthey want to replicate them at home?
Presentation ofData: TBD (Graphs, charts and narratives?)
Review of Data: USAID to review data when submitted, and in preparation for Annual Report
Reporting of Data: USAID Office ofEducation will NOT include this indicator in Annual Report
20
------------------------------ ----------~---------------
Pcdonnancc Indicatol" Rcfcl'cnce Shect
Strategic Objective (SO): Improved Quality ofDecentralized Basic Education
Intermediate Result 1: More effective decentralized management and governance ofschools
Indicator 1.3: Percent schools that disseminated Annual School Budget in at least two venues,
DElSCllIPTION
Precise Definition(s):
• "Venues" means three kinds of places where schools budgets tend to be posted: somewhere inside the school compound,
somewhere outside the school compound such as at the village office, or in a letter sent to parents.
• "Annual School Budget" means Rencana Anggaran Pendapatan dan Belanja Sekolah (RAPBS), which is a mandatory
document to document the planned funding and expenditure for just one year of one school. The Annual School Budget is
one part ofthe larger multi-year School Development Plan (see Indicator 1.1).
Unit of Measure: percent ofschools
Disaggregated by: Province and school type
Justification and history ofindicator: The DBEI and MBE projects are trying to encourage greater financial accountability
and transparently at local levels. The assumption ofthe project is that posting the Annual School Budget encourages greater
transparency from multiple sides. It encourages school principals and school committees to consolidate the financial information
about the school and to be held accountable to the community. Posting budgets also encourages parents and community
members to be aware of how school funds are supposed to be used and how they are ultimately used. Experience in collecting
the baseline data for the first five provinces in early 2006 for this indicator found that 55% ofthe schools did not post the budget
at all, and that only 9% disseminated in two venues. Since posting a budget within the school could be an easy one-off
accomplishment, the project will track how many schools achieve posting the budget in two places, which requires greater
coordination and commitment.
One ofthe issues is that some school plans do not fully account for the multiple sources offunding in their budget and this
severely limits financial accountability. This indicator about posting a budget therefore is also related to the criteria within
Indicator 1.1 about the uali of a school develo ment Ian.
Data Collection Method: School visits and interviews with school committees, conducted by DBEI district coordinators, and
accompanied by members ofBAPPEDA and DINAS. Even though the membership ofthe group collecting data varied slightly
depending on stakeholders in each province, a standardized instrument was used (see DBEI project files for "Indicator 5").
, Data Source(s): members ofschool committee and direct observation of postings
Frequencyrriming ofData Acquisition: Baseline data collected in March 2006 for the first five provinces, with additional
collection for Jakarta and Aceh reported by July 2006. Regular collection in June 2006, and subsequently every six months in
June and December. .
Method ofAcquisition by USAID: DBEI reports.
Estimated Cost of Data Acquisition: Low to medium depending on how much ofthe data is self-reported vs collected
Responsible Individual(s) at USAID: Elizabeth Sunindyo (eTO DBEI)
DA1r:AQlJALITY.ISSUES
Date ofInitial Data Quality Assessment: in June 2006
Known Data.Limitations and Significance (if any):
Actions Taken or Planned to Address Data Limitations:
Date of Future Data Quality Assessments: no assessment planned yet
Procedures for Future Data Quality Assessments: Review school records, make site-visits, and interview or survey school
pI.;:ANlfOltI>AT:AJ\NALYSIS, REVIEW, & REPORTING
Data Analysis: Suggested questions for analysis:
How do religious Ml compare to secular SD?
In the baseline, which venue was most often used by schools that had not yet received our assistance?
Presentation ofData: Graphs, charts and narratives.
Review ofData: USAID to review data when submitted, and in preparation for Annual Report
Reporting of Data: USAID Office ofEducation will NOT include this indicator in Annual Report
21
Year Province ···Plannecl Actual Notes ...
2006 Aceh TBD of 40 Schools (phase I) Aceh baseline data to be collected
later in 2006.
N. Sumatra XX of 100 Schools (Phase I) Baseline was collected March 2006
Banten XX of 50 Schools (Phase I)
Jakarta XX of7 Schools (Phase I) Jakarta baseline data to be collected
later in 2006.
W. Java XX of 61 Schools (Phase I)
C. Java XX of 106 Schools (phase 1) Baseline was collected March 2006
E. Java XX of 84 Schools (Phase I) Baseline was collected March 2006
S. Sulawesi XX of 88 Schools (Phase I) Baseline was collected March 2006
SUMMARY 46 of 536 Schools (Phase I)
2007 Aceh XX of 40 Schools (Phase I)
XX of TBD Schools (Phase II)
N. Sumatra XX of 100 Schools (Phase I)
XX ofTBD Schools (Phase II)
Banten XX of 111 Schools (Phase I)
XX ofTBD Schools (phase II)
Jakarta XX of7 Schools (Phase I)
XX ofTBD Schools (Phase II)
W.Java XX ofTBD Schools (Phase 1)
XX ofTBD Schools (Phase II)
C. Java XX of 106 Schools (Phase I)
XX ofTBD Schools (Phase II)
E. Java X of 84 Schools (Phase I)
XX ofTBD Schools (phase II)
S. Sulawesi XX of 88 Schools (Phase I) .
XX ofTBD Schools (Phase II)
SUMMARY XX of 536 Schools (Phase I)
XX of500 Schools (Phase II)
2008 Aceh 30 of 40 Schools (Phase 1)
XX ofTBD Schools (Phase II)
XX ofTBD Schools (Phase III)
N. Sumatra 80 of 100 Schools (Phase I)
XX ofTBD Schools (Phase II)
XX ofTBD Schools (Phase III)
Banten 100 of 111 Schools (Phase 1)
XX ofTBD Schools (Phase II)
XX ofTBD Schools (Phase III)
Jakarta 6 of 7 Schools (phase I)
XX ofTBD Schools (Phase II)
XX ofTBD Schools (Phase III)
W. Java TBD ofTBD Schools (Phase 1)
XX ofTBD Schools (Phase II)
XX ofTBD Schools (Phase III)
C. Java 60 of 106 Schools (Phase 1)
XX ofTBDSchools (Phase II)
XX ofTBD Schools (Phase III)
E. Java 60 of 84 Schools (phase I)
XX ofTBD Schools (Phase II)
XX ofTBD Schools (Phase III)
S. Sulawesi 60 of 88 Schools (Phase I)
XX ofTBD Schools (Phase II)
XX ofTBD Schools (Phase III)
SUMMARY 450 of 536 Schools (Phase 1)
300 of 500 Schools (Phase II)
200 of 1,000 Schools (Phase III)
22
I<x~~, 1·~rfl\f>4fce, .<........... Planned .Actual Notes ." '., . ." '.
2009 Aceh an ofTBD Schools (Phase II)
half ofTBD Schools (Phase III)
N. Sumatra all ofTBD Schools (Phase II)
half ofTBD Schools (Phase III)
Banten all ofTBD Schools (Phase II)
half ofTBD Schools (Phase III)
Jakarta all ofTBD Schools (Phase II)
half ofTBD Schools (Phase III)
W.Java all ofTBD Schools (Phase II)
half ofTBD Schools (Phase III)
C. Java all ofTBD Schools (Phase II)
half ofTBD Schools (phase III)
E. Java all ofTBD Schools (Phase II)
half ofTBD Schools (Phase III)
S. Sulawesi all ofTBD Schools (Phase II)
half ofTBD Schools (Phase III)
SUMMARY 400 of 500 Schools (Phase II)
500 of 1,000 Schools (Phase III)
2010 Aceh all ofTBD Schools (phase III) .:
N. Sumatra all ofTBD Schools (Phase III)
Banten an ofTBD Schools (Phase III)
Jakarta all ofTBD Schools (Phase III)
W. Java all ofTBD Schools (Phase III)
C. Java all ofTBD Schools (Phase III)
E. Java all ofTBD Schools (Phase III)
S. Sulawesi all ofTBD Schools (Phase III)
SUMMARY 800 of 1,000 Schools (Phase III)
23