Do Early Warning Systems and Student Engagement Activities 
Reduce Dropout?
Findings from the School Dropout Prevention Pilot Program Impact 
Evaluation in Timor-Leste
Volume 1: Main Findings
Contract No. EDH-I-00-05-00029-00
Task Order AID-OAA-TO-10-00010
September 2015
This document was produced for review by the United States Agency for International 
Development. It was prepared by Creative Associates International. 

Do Early Warning Systems and Student Engagement 
Activities Reduce Dropout? 
Findings from the School Dropout Prevention Pilot 
Program Impact Evaluation in Timor-Leste 
Volume 1: Main Findings
Submitted to: 
United States Agency for International Development 
Washington, DC 
Submitted by: 
Creative Associates International, Inc. 
Washington, DC 
September 2015 
This report was made possible by the American people through the United States Agency for 
International Development (USAID). The contents of this report are the sole responsibility of 
Creative Associates International and do not necessarily reflect the views of USAID or the United 
States Government.
DEC Submission Requirements 
a. USAID Award Number Contract No. EDH-I-00-05-00029-00
Task Order AID-OAA-TO-10-00010
b. USAID Objective Title 
and Number Investing in People (IIP)
c.
USAID Project Title and 
Number
USAID Asia and Middle East Regional School Dropout 
Prevention Pilot (SDPP) Program
d. USAID Program Area and 
Program Element
Education (program area 3.2)
Basic Education (program element 3.2.1)
e. Descriptive Title
Findings from the School Dropout Prevention Pilot 
Program Impact Evaluation in Timor-Leste, Volume 1: 
Main Findings
f. Author Name(s) Creative Associates International and Mathematica 
Policy Research
g. Contractor name
Creative Associates International, Inc.
5301 Wisconsin Avenue, NW, Suite 700
Washington, DC 20015
Telephone: 202 966 5804 Fax: 202 363 4771
Contact: KarenT@creativedc.com
h. Sponsoring USAID 
Operating Unit and COTR
Asia/TS
Eric Bergthold, COTR
i. Date of Publication September 2015
j. Language of Document English, Portuguese, Tetun
Findings from the School Dropout Prevention Pilot Program Impact Evaluation in Timor-Leste Page i
TABLE OF CONTENTS 
List of Tables ................................................................................................................................. iii
List of Figures ................................................................................................................................ iv
Acronyms ...................................................................................................................................... vii
Acknowledgements ...................................................................................................................... viii
Executive Summary ....................................................................................................................... ix
I. Introduction ..........................................................................................................................1
II. SDPP Program in Timor-Leste ............................................................................................4
A. Targeting grades and geographic areas for intervention ................................................4
B. Interventions ..................................................................................................................5
1. Selecting the SDPP interventions ..............................................................................5
2. Early warning system .................................................................................................9
3. Extra-curricular activities program ..........................................................................10
C. Program implementation ..............................................................................................10
D. Fidelity of implementation ...........................................................................................13
III. Evaluation Design ..............................................................................................................16
A. SDPP theory of change ................................................................................................16
B. Research questions .......................................................................................................17
C. Evaluation design .........................................................................................................17
1. Study eligibility ......................................................................................................18
2. Primary impact analysis .........................................................................................18
3. Impact analysis for at-risk students........................................................................19
4. Additional subgroup analyses ................................................................................20
5. Primary and additional measures of SDPP’s effectiveness ...................................21
IV. Sampling and Data Collection ...........................................................................................23
V. Characteristics of the Sample Prior to Implementation .....................................................25
VI. Impacts of SDPP ................................................................................................................28
A. Impacts on teacher outcomes .......................................................................................29
1. Impact on teacher take-up of dropout prevention practices ...................................29
2. Impact on additional teacher outcomes..................................................................33
3. Impact on additional school administrator outcomes ............................................35
B. Impacts on at-risk students’ attitudes toward school ...................................................40
Findings from the School Dropout Prevention Pilot Program Impact Evaluation in Timor-Leste Page ii
1. Impact on primary measures of student attitudes toward school ...........................40
2. Impact on additional measures of student attitudes toward school........................48
C. Impacts on student engagement in school ...................................................................50
1. Impact on student’s daily attendance in school .....................................................50
2. Impact on additional measures of student engagement in school ..........................55
D. Impacts on school dropout ...........................................................................................57
1. Impact on primary measure of school dropout ......................................................57
2. Impact on additional measures of school dropout .................................................60
VII. School-Level Dropout Trends...........................................................................................64
VIII. Discussion ..........................................................................................................................67
A. Overview of findings ...................................................................................................67
B. Contextual factors related to findings ..........................................................................70
C. Conclusion ...................................................................................................................72
References ......................................................................................................................................73
Findings from the School Dropout Prevention Pilot Program Impact Evaluation in Timor-Leste Page iii
LIST OF TABLES 
Table ES.1. SDPP Program impacts on primary measures of program effectiveness in 
Timor-Leste................................................................................................................ xix
Table ES.2. SDPP Program impacts on additional outcome measures in Timor-Leste .............. xix
Table III.C.1. Primary and additional subgroup analyses ..............................................................21
Table III.C.2. Primary and additional measures of Timor-Leste SDPP Program 
effectiveness .................................................................................................................22
Table IV.1. Study sample sizes ......................................................................................................24
Table V.1. Average school and target grade teacher characteristics prior to intervention 
(grades 4–6, SY 2012) (percentage unless indicated otherwise) .................................26
Table V.2. Average student characteristics prior to intervention (grades 3–6, SY 2012) 
(percentages of students unless otherwise indicated) ..................................................27
Table VIII.A.1. SDPP Program impacts on primary measures of program effectiveness in 
Timor-Leste..................................................................................................................69
Table VIII.A.2. SDPP Program impacts on additional outcome measures in Timor-Leste ..........69
Findings from the School Dropout Prevention Pilot Program Impact Evaluation in Timor-Leste Page iv
LIST OF FIGURES 
Figure ES.1. Impacts of the Timor-Leste SDPP Program on teacher dropout prevention 
practices (SY 2013 and SY 2014) .............................................................................. xiii
Figure ES.2. Impacts of the Timor-Leste SDPP Program on at-risk student attitudes 
toward school (SY 2013 and SY 2014) .......................................................................xv
Figure ES.3. Impacts of the Timor-Leste SDPP Program on daily attendance, overall and 
by at-risk status (SY 2013 and SY 2014)................................................................... xvi
Figure ES.4. Impacts of the Timor-Leste SDPP Program on school dropout, overall and 
by at-risk status (SY 2013 and SY 2014).................................................................. xvii
Figure II.A.1. National dropout rates by grade (2009–2010) .........................................................4
Figure II.A.2. Target regions of the SDPP program ........................................................................5
Figure II.B.1. Reported causes of dropout ......................................................................................6
Figure II.B.2. Absenteeism reported by dropouts and at-risk students ............................................7
Figure II.B.3. Treatment by teachers reported by at-risk students ...................................................8
Figure II.B.4. At-risk student engagement in school .......................................................................8
Figure II.C.1. Rollout of the interventions in Timor-Leste............................................................12
Figure II.C.2. Grades by academic year for SDPP students in Timor-Leste .................................13
Figure II.D.1. EWS: Proportion of schools meeting or exceeding 80% FOI threshold ................14
Figure II.D.2. EWS: Proportion of schools meeting or exceeding FOI threshold by 
component ....................................................................................................................14
Figure II.D.3. ECA: Proportion of Schools Meeting or Exceeding 80% FOI Threshold ..............15
Figure II.D.4. ECA: Proportion of Schools Meeting or Exceeding 80% FOI Threshold, by 
Component ...................................................................................................................15
Figure III.A.1. Timor-Leste SDPP conceptual model ...................................................................16
Figure III.C.1. SDPP Randomized Control Trial design in Timor-Leste ......................................18
Figure VI.A.1. SDPP Program impacts on teacher dropout prevention practices (SY 2013 
and SY 2014) ...............................................................................................................30
Figure VI.A.2. SDPP Program impacts on teacher dropout prevention practices, by subgroup 
(SY 2013 and SY 2014) ................................................................................................31
Figure VI.A.3. SDPP Program impact on teacher sense of self-efficacy (SY 2013 and SY 
2014) ............................................................................................................................34
Figure VI.A.4. SDPP Program impact on teacher sense of responsibility (SY 2013 and 
SY 2014) ......................................................................................................................35
Figure VI.A.5. SDPP Program impact on administrators’ dropout prevention practices 
scale (SY 2013 and SY 2014) ......................................................................................36
Findings from the School Dropout Prevention Pilot Program Impact Evaluation in Timor-Leste Page v
Figure VI.A.6. SDPP Program impact on administrators’ sense of self-efficacy scale (SY 
2013 and SY 2014) ......................................................................................................37
Figure VI.A.7. SDPP Program impact on administrators’ sense of responsibility for at￾risk students scale (SY 2013 and SY 2014) .................................................................38
Figure VI.B.1. SDPP Program impacts on at-risk students’ emotional attitudes toward 
school (SY 2013 and SY 2014) ...................................................................................41
Figure VI.B.2. SDPP Program impacts on students’ emotional attitudes toward school, by 
subgroup .......................................................................................................................42
Figure VI.B.3. SDPP Program impacts on at-risk students’ cognitive attitudes toward 
school (SY 2013 and SY 2014) ...................................................................................43
Figure VI.B.4. SDPP Program impacts on students’ cognitive attitudes toward school, by 
subgroup (SY 2013 and SY 2014) ...............................................................................44
Figure VI.B.5. SDPP Program impacts on at-risk students’ behavioral attitudes toward 
school (SY 2013 and SY 2014) ...................................................................................45
Figure VI.B.6. SDPP Program impacts on students’ behavioral attitudes toward school, 
by subgroup (SY 2013 and SY 2014) ..........................................................................46
Figure VI.B.7. SDPP Program impact on students’ perceptions of teacher support (SY 
2013 and SY 2014) ......................................................................................................49
Figure VI.B.8. SDPP Program impact on students’ perceptions of parental support (SY 
2013 and SY 2014) ......................................................................................................50
Figure VI.C.1. SDPP Program impact on daily attendance, overall and by at-risk status 
(SY 2013 and SY 2014) ...............................................................................................52
Figure VI.C.2. SDPP Program impacts on daily attendance for students overall, by 
subgroup .......................................................................................................................53
Figure VI.C.3. SDPP Program impact on academic performance (SY 2013 and SY 2014) .........56
Figure VI.D.1. SDPP Program impacts on school dropout, overall and by at-risk status 
(SY 2013 and SY 2014) ...............................................................................................58
Figure VI.D.2. SDPP Program impacts on school dropout, by cohort (SY 2013 and SY 
2014) ............................................................................................................................59
Figure VI.D.3. SDPP Program impacts on school dropout, by subgroup (SY 2013 and SY 
2014) ............................................................................................................................60
Figure VI.D.4. SDPP Program impact on student progression in school at endline, overall 
and by at-risk status (percentage) (SY 2013 and SY 2014) .........................................61
Figure VII.1. School between-grade dropout rates, by grade and school year ..............................65
Figure VII.2. School within-grade dropout rates, by grade and school year .................................66
Figure VIII.A.1. SDPP conceptual model with program impacts .................................................68
Findings from the School Dropout Prevention Pilot Program Impact Evaluation in Timor-Leste Page vi
VOLUME 2: TECHNICAL APPENDICES 
APPENDIX A. Evaluation Design .............................................................................................. A.1
APPENDIX B. Analytic Methods ............................................................................................. A.12
APPENDIX C. Analyzing Teacher and School Administrator Outcomes ................................ A.19
APPENDIX D. Analyzing At-Risk Student Attitudes ............................................................... A.27
APPENDIX E. Analyzing School Engagement.........................................................................A.30
APPENDIX F. Analyzing School Dropout ............................................................................... A.32
APPENDIX G. Exposure Analysis ............................................................................................ A.36
APPENDIX H. Supplemental Tables ........................................................................................ A.39
APPENDIX I. School-Level Trend Analysis ............................................................................A.51
APPENDIX J. Propensity Score Analysis of Students Identified by the EWS ......................... A.59
APPENDIX K. Data Collection Instruments .............................................................................A.73
Findings from the School Dropout Prevention Pilot Program Impact Evaluation in Timor-Leste Page vii
Acronyms 
CARE Care International 
EMIS Education Management Information System 
EWS Early Warning System 
ECA Extra-Curricular Activities 
FOI Fidelity of Implementation 
ITT Intent-to-Treat 
KAPE Kampuchean Action for Primary Education 
MDI Minimum Detectable Impact 
QUEST Quest Alliance 
RCT Randomized Control Trial 
SDPP School Dropout Prevention Pilot Program 
STS School-to-School International 
SY School Year 
TOT Treatment-on-the-Treated 
USAID United States Agency for International Development 
Findings from the School Dropout Prevention Pilot Program Impact Evaluation in Timor-Leste Page viii
Acknowledgements 
These findings are the result of the work of the collective SDPP team in the United States and 
India in its stages from research design and instrument development to data collection and entry 
to analysis and reporting. The following persons played a major role in the process: 
Karen Tietjen (Creative Associates) 
Nancy Murray (Mathematica Policy Research) 
Quinn Moore (Mathematica Policy Research) 
Emilie Bagby (Mathematica Policy Research) 
Ali Protik (Mathematica Policy Research) 
Mark Strayer (Mathematica Policy Research) 
Kristine Johnston (Mathematica Policy Research) 
Kathy Buek (formerly Mathematica Policy Research) 
Larissa Campuzano (Mathematica Policy Research) 
Jane Fortson (Mathematica Policy Research) 
Ebo Dawson-Andoh (Mathematica Policy Research) 
Owen Schochet (Mathematica Policy Research) 
Ron Palanca (Mathematica Policy Research) 
Jeremy Page (Mathematica Policy Research) 
Linda Molinari (Mathematica Policy Research) 
Serge Lukashanets (Mathematica Policy Research) 
Brendan Kirwan (Mathematica Policy Research) 
Carol Razafindrakoto (Mathematica Policy Research) 
Jonathan McCay (Mathematica Policy Research) 
Ivonne Padilla 
 Espinosa (Mathematica Policy Research)
Lotte Renault (CARE) 
Shoaib Danish (CARE) 
The final report was prepared by Nancy Murray, Quinn Moore, Emilie Bagby, Ali Protik, Anitha 
Sivasankaran, Mark Strayer, Ebo Dawson-Andoh, Jonathan McCay, Owen Schochet and Karen 
Tietjen. 
Findings from the School Dropout Prevention Pilot Evaluation in Timor-Leste Page ix
Executive Summary 
Over the past two decades, considerable progress has been made in increasing school enrollment. 
However, many children do not complete primary or secondary cycles once they enroll. In many 
countries and regions, a greater percentage of out-of-school children have dropped out of school 
than have never enrolled in school. Interventions have been conducted in the United States and 
abroad to prevent dropout, but there is limited evidence on how well they work, particularly in 
developing countries. 
The School Dropout Prevention Pilot (SDPP) Program, a five-year multi-country program funded 
by the U.S. Agency for International Development (USAID), is designed to identify successful 
means of decreasing student dropout rates in primary and secondary schools.1
 Its goal is to pilot 
and test the effectiveness of dropout prevention interventions in four countries—Cambodia, India, 
Tajikistan, and Timor-Leste—to generate evidence-based programming guidance for USAID 
missions and countries in Asia and the Middle East. In all four countries, SDPP introduced an early 
warning system (EWS) and a student engagement intervention to motivate greater student 
engagement, better attendance and desire to stay in school. This report presents findings from the 
impact evaluation of the SDPP Program in Timor-Leste. 
SDPP Program in Timor-Leste 
Timor-Leste’s SDPP Program had two main components: (1) an EWS, and (2) an Extra-Curricular 
Activities (ECA) program, featuring songs, games, crafts and small group projects (Creative 
Associates International 2012a; Creative Associates International 2012b). The SDPP Program was 
targeted to 4th-, 5th-, and 6th-grade students in five districts, as both the grades and geographic 
areas exhibited the highest dropout rates and would benefit most from a dropout prevention 
program (Shin, Jennifer, Rajani Shrestha, and Karen Tietjen 2011b).
The EWS consists of three components: (1) identification of students at risk of school dropout; 
(2) first response strategies; and (3) community engagement. For Component 1, the SDPP Program 
worked with teachers to identify at-risk students based on six measures of dropout risk. For 
Component 2, SDPP helped teachers use a “track and trigger” approach to closely monitor the 
attendance, behavior and coursework of at-risk students and initiate “first response” activities 
when students show signs of struggling, ranging from in-class attention to contact via letter, phone 
call and/or home visit, to case management meetings with school staff. Component 3 focused on 
raising community awareness about the problem of dropout, working with parent-teacher 
associations and other community groups on advocacy activities and enlisting their support for the 
first response activities, working closely with the school. 
The Extra-Curricular Activities program offered enrichment activities to all target-grade students, 
with special encouragements for students identified as at risk through the EWS (Creative 
 
1
 SDPP is implemented by Creative Associates International, with international partners Mathematica Policy Research 
and School-to-School International and local partners Kampuchean Action for Primary Education (KAPE) in 
Cambodia, Quest Alliance (QUEST) in India, and Care International (CARE) in Timor-Leste. Creative Associates has 
a corporate office in Tajikistan, which covers the responsibilities of a local partner in that country. 
Findings from the School Dropout Prevention Pilot Evaluation in Timor-Leste Page x
Associates International 2012a)2
. The activities were scheduled either before or after school, 
depending on instructional schedules. Each week, students participated in one or two sessions 
focused on promoting fun through songs, games, and craft sessions. The ECA program was 
intended to motivate children to attend and engage in school through structured activities to foster 
cooperative learning, enhance self-confidence and reinforce basic skills. The ECA program was 
initially conducted by SDPP facilitators. During the second year of program implementation, 
target-grade teachers and community members led the second weekly session. The SDPP 
facilitators monitored at-risk students’ progress, recorded participation levels and met at least 
monthly to discuss progress with homeroom teachers. The intervention was developed with cost 
and sustainability considerations in mind to increase the likelihood of continued implementation 
in Timor-Leste beyond the funding period. 
The SDPP Program was active in schools during three school years (SY)—SY 2012, SY 2013, 
and SY 2014. Students and teachers of target grades received only a partial year of exposure during 
the first year of the program (SY 2012) because of the timing of the roll-out of program activities. 
Students and teachers of the target grades during the second and third school year received the 
program for the full school year.3
During three school years of program activity, the SDPP Program targeted students in grades 4, 5 
and 6 and their teachers. Students in grades 4 and 5 during SY 2012 and SY 2013 received the 
program for more than one year (grade 6 is the final year of primary school). Specifically, the 
evaluation follows: (1) SY 2012 5th graders who continued to receive the program in SY 2013, in 
their 6th grade; (2) SY 2012 4th-graders who continued to receive the program in SY 2013 and 
SY 2014, in their 5th and 6th grades, respectively, (3) SY 2013 4th grade students, who continued 
to receive the program in SY 2014, in their 5th grade; and (4) SY 2014 4th grade students.4 The 
analyses focus on the latest point in time for which these students and teachers are observed across 
these multiple school years. The latest observation point occurs in SY 2013 or SY 2014 for all 
students followed by the evaluation. 
SDPP evaluated the fidelity of implementation (FOI) of the EWS and after-school activity 
components during field visits conducted in May, October, and November 2014. Overall, schools 
appeared to implement the EWS unevenly: the proportion of schools meeting threshold levels for 
identifying at-risk students and tracking attendance was high (93 percent for both), but the 
proportion of schools meeting threshold levels for communicating with parents and taking follow￾up actions were very low (at 8 percent and 26 percent, respectively) (Creative Associates 
International and School-to-School International 2015). The FOI results suggested that the after￾school activities were fairly well-implemented, though there was room for improvement, 
particularly in the frequency with which the after-school activity program was held. Only 76 
 
2
 During the first year of SDPP implementation, one session was offered per week. SDPP transitioned to two weekly 
sessions at the start of the second year of implementation. 
3
 The teachers and school administrators were first trained in August 2012, which is the middle of SY 2012. The EWS 
began in September 2012 and the after-school activities began in October 2012 for entering 4th-, 5th-, and 6th-grade 
students in SY 2012. For entering 4th-, 5th-, and 6th-grade students in SY 2013 and SY 2014, both the EWS and after￾school activities began in January. 
4
 The evaluation does not focus on SY 2012 6th graders because they did not receive the program for a full school 
year. 
Findings from the School Dropout Prevention Pilot Evaluation in Timor-Leste Page xi
percent of schools reported holding the after-school activities weekly, as intended (Creative 
Associates International and School-to-School International 2015). 
Evaluation Design 
SDPP hypothesized that academic and social support, combined with additional enrichment 
activities for at-risk students and changes in teacher practices, would improve student attitudes 
and behavior, translating into increased student engagement and reduced school dropout. Guided 
by this conceptual model, SDPP designed the impact evaluation to address five primary research 
questions: 
1. Does SDPP improve teacher behavior and attitudes? 
2. Does SDPP improve student attitudes? 
3. Does SDPP improve student engagement in school associated with retention, such as 
attendance? 
4. Does SDPP improve the dropout rate? 
5. What are SDPP’s impacts for students most at risk of dropping out of school? 
The SDPP evaluation addressed these research questions using a Randomized Control Trial (RCT) 
in which the outcomes of students and teachers in 97 schools randomly assigned to provide SDPP 
services were compared to those of teachers and students in 94 schools randomly assigned to a 
control group providing business-as-usual services.5 With random assignment, exposure to SDPP 
is not directly related to the choices or pre-existing characteristics of study participants, allowing 
attribution of any observed treatment-control differences in outcomes to the program. 
Data Collection 
The data used in this report was collected from school records and through interviews with teachers 
and at-risk students. SDPP collected data five times from all 191 schools in the study over four 
school years, between SY 2012 and SY 2015, to follow the four cohorts of students who were 
exposed to the SDPP intervention for at least one full school year and included in the analyses. 
SDPP collected information from school records for 37,861 students, and conducted interviews 
with 7,387 at-risk students and 1,444 teachers and administrators. 6 All final outcomes were 
measured during SY 2013 or 2014 for the four cohorts followed (SY 2012 5th graders, SY 2012 
4th graders, SY 2013 4th graders and SY 2014 4th graders). 7
SDPP and control group schools had comparable characteristics at baseline during SY 2012, with 
no evidence of systematic differences between the groups. The typical SDPP group school enrolls 
about 251 students, compared with about 260 students in control group schools. The average SDPP 
group school has 41 4th-grade students, 34 5th-grade students, and 30 6th-grade students, 
 
5
 Of the 94 schools randomly assigned to the control group, 1 school dropped out of the study sample during the SY 
2013.
6
 Please see Appendix A for further details about data collection. An eligible subset of the total students and teachers 
collected was used in the analysis. 
7
 An eligible subset of the total students and teachers was used in the analysis. 
Findings from the School Dropout Prevention Pilot Evaluation in Timor-Leste Page xii
compared with 42, 40, and 33 students in control group schools, respectively. SDPP group schools 
had an average of 9 teachers, compared with 10 teachers in the control group schools. Most study 
schools have school feeding programs, but few have other active external assistance programs. 
Impacts of SDPP
A conceptual model of SDPP Program activities and how they might affect student and teacher 
outcomes guided the design of the program and the impact evaluation. This model posits that 
teacher and parent knowledge and practices—shared and reinforced by the larger community—
are inputs into students’ attitudes toward school and educational aspirations. These student 
attitudes translate into student engagement in school, including their attendance, behavior, and 
academic performance. The complex, cumulative interactions of these factors are inputs into the 
student’s ability, desire and decision to remain in school or drop out. 
The evaluation estimated SDPP’s impacts on teacher outcomes, student attitudes, student 
engagement in school, and school dropout. In each domain, SDPP focuses on a small set of key 
outcomes, identified before the analysis began. 
The text boxes and figures below summarize the primary measures of effectiveness and findings 
for them in domains related to teacher practices, at-risk students’ attitudes toward school, student 
engagement in school, and school dropout. Estimates of the impact of the SDPP Program are based 
on differences in average outcomes for SDPP and control group students and teachers. These 
impact estimates represent the difference in the outcome of interest at endline that is attributable 
to the SDPP Program relative to the status quo. The estimates are expressed as percentage point 
differences between the treatment and control group; we also present percentage increases or 
decreases in the primary outcomes across the treatment and control groups. These “percentage 
changes” should not be interpreted as the percentage “change” that might be calculated in a pre￾post or baseline/endline change, but rather, as the increase or decrease in the treatment group’s 
outcome measure in relation to the control group at endline. 
Statistical tests were conducted to assess whether each impact is significantly different from zero. 
Impacts estimates are described as statistically significant if there is less than a 5 percent 
probability that it is due to chance (and not to the SDPP Program). Impact estimates are described 
as marginally significant if the probability that it is due to chance (and not to the SDPP Program) 
is between 5 and 10 percent.
Findings from the School Dropout Prevention Pilot Evaluation in Timor-Leste Page xiii
SDPP had a statistically significant impact on teacher dropout prevention practices (Figure 
ES.1), but no impact on administrator dropout prevention practices. Despite the high scores 
on this scale—which measures whether teachers report engaging in behavior that might help at￾risk students succeed in school—in both the SDPP (7.87 out of 8) and the control groups (7.72 out 
of 8), the difference between groups is statistically significant. Even without SDPP, teachers may 
be employing most of the dropout prevention practices measured by this scale, and SDPP only 
changed them slightly; or, the scale scores could be artificially high in both groups because they 
are based on self-reports rather than direct observation. 
Figure ES.1. Impacts of the Timor-Leste SDPP Program on teacher dropout prevention practices (SY 
2013 and SY 2014) 
Sources: SDPP baseline and follow-up teacher self-administered questionnaires and school records data collection; May 2012, 
May 2013, and September 2014. 
7.87*** 7.72
0
1
2
3
4
5
6
7
8
Score on an eight-point scale
SDPP group Control group
SDPP effectiveness in influencing teacher outcomes
Primary research question
Did SDPP affect teacher dropout prevention practices?
Primary measure of SDPP Program’s effectiveness
 Teacher take-up of dropout prevention practices
Additional measures of SDPP Program’s effectiveness
 Teacher sense of self-efficacy
 Teacher sense of responsibility
 Administrator dropout prevention practices, sense of self-efficacy and sense of 
responsibility
Findings from the School Dropout Prevention Pilot Evaluation in Timor-Leste Page xiv
Note: The analysis is based on 4th-, 5th-, and 6th-grade homeroom, math, and language teachers during SY 2013 and SY 
2014. The sample includes 412 teachers for the SDPP group and 406 for the control group 
 Differences between SDPP and control group means were tested using two-tailed t-tests. Mean values are adjusted for 
baseline characteristics. The analysis accounts for clustering of teachers within schools and school-year and grade fixed 
effects. For a tabular presentation of these findings, see Appendix Table H.5. 
***/**/* Impact estimate is statistically significant at the .01/.05/.10 level. 
The evaluation also examined teachers’ and administrators’ sense of self-efficacy (ability to 
respond to factors related to dropout) and sense of responsibility (whether they perceived it was 
their role to prevent at-risk students from dropping out). The SDPP Program had a statistically 
significant positive impact on teachers’ sense of self-efficacy, but not on teacher sense of 
responsibility. 
SDPP had a positive, statistically significant impact on at-risk students’ behavioral attitudes 
toward school, but had no impacts on cognitive or emotional attitudes toward school (Figure 
ES.2). These measures of student attitudes toward school—which were captured by surveying 
students identified as being at risk of school dropout based on baseline characteristics—could have 
changed because of changes in teacher or parent attitudes and practices or due to the intervention 
activities.8 On the behavioral attitudes toward school scale, at-risk students in SDPP schools 
agreed to 80 percent of the questions, compared with 76 percent for students in control schools, a 
statistically significant difference.9
 
8
 The three measures of student attitudes are constructed from responses to a survey administered to a sample of at￾risk students in each cohort. This survey is explained in further detail in the Technical Appendix – Section A. 
9
 Agreement indicate more positive feelings and actions towards school. 
SDPP effectiveness in influencing student attitude outcomes
Primary research question 
Did SDPP affect at-risk students’ attitudes toward school?
Primary measures of SDPP Program’s effectiveness
 Emotional attitudes toward school (e.g., Student likes school)
 Cognitive attitudes toward school (e.g., Student adapts better study habits)
 Behavioral attitudes toward school (e.g., Student observes school requirements and rules)
Additional measures of SDPP Program’s effectiveness
 Students’ perceptions of teacher support
 Students’ perceptions of parental support
Findings from the School Dropout Prevention Pilot Evaluation in Timor-Leste Page xv
Figure ES.2. Impacts of the Timor-Leste SDPP Program on at-risk student attitudes toward school (SY 
2013 and SY 2014) 
Sources: SDPP baseline and follow-up student surveys, May 2013 and September 2014; baseline and follow-up school records 
data collection, May 2012, May 2013, March 2014, September 2014, and January 2015. 
Note: The analysis is based on SY 2012 4th and 5th grade at-risk students, and SY 2013 and 2014 4th grade at-risk 
students. The sample includes 3,226 at-risk students for the SDPP group and 3,006 at-risk students for the control group. 
 Differences between SDPP and control group means were tested using two-tailed t-tests. Mean values are adjusted for 
baseline characteristics. The analysis accounts for clustering of students within schools and cohort fixed effects. Impacts 
were adjusted for multiple comparisons using the Benjamini-Hochberg method. For a tabular presentation of these 
findings, see Appendix Table H.5. 
***/**/* Impact estimate is statistically significant at the .01/.05/.10 level. 
SDPP also measured students’ perceptions of the support provided by their parents and their 
teachers and found no impacts on these secondary measures of student attitudes in Timor-Leste. 
SDPP improved attendance for students exposed to the program, and particularly had 
impact on at-risk students (Figure ES.3). The difference in the average rate of daily 
85.0
71.3
79.5***
85.9
71.7
76.3
0.0
10.0
20.0
30.0
40.0
50.0
60.0
70.0
80.0
90.0
100.0
Emotional attitudes toward
school
Cognitive attitudes toward
school
Behavioral attitudes toward
school
Percentage of scale items with which 
student agreed
SDPP group Control group
SDPP effectiveness in influencing student engagement
Primary research question 
Did SDPP affect attendance, overall or for at-risk students?
Primary measure of SDPP Program’s effectiveness
 Student average daily attendance for the year
Additional measures of SDPP Program’s effectiveness
 Student performance in school
 Student behavior in school
Findings from the School Dropout Prevention Pilot Evaluation in Timor-Leste Page xvi
attendance—82 percent for target-grade students in the SDPP group compared with 80.3 percent 
for students in the control group—was statistically significant. For students at-risk of dropping 
out of school, these rates were lower—78.9 percent for students in the SDPP group compared to 
76.6 percent for students in the control group; this difference was also statistically significant. 
Both elements of SDPP—the EWS and ECA—might have contributed toward SDPP’s impacts on 
student behavioral attitudes toward school and attendance. Fun, recreational activities might have 
attracted students to attend school more frequently than they would without them. Contacting 
parents if students were absent may have induced at-risk students to attend school more often. 
Figure ES.3. Impacts of the Timor-Leste SDPP Program on daily attendance, overall and by at-risk 
status (SY 2013 and SY 2014) 
Sources: SDPP baseline and follow-up school records data collection, May 2012, May 2013, March 2014, September 2014, and 
January 2015. 
Note: The analysis is based on SY 2012 4th- and 5th-grade students, and SY 2013 and 2014 4th-grade students. The sample 
includes 13,645 students for the SDPP group (5,884 at risk and 6,233 not at risk) and 13,702 students for the control 
group (5,724 at risk and 6,681 not at risk). 
 Differences between SDPP and control group means were tested using two-tailed t-tests. Mean values are adjusted for 
baseline characteristics. The analysis accounts for clustering of students within schools and cohort fixed effects. For a 
tabular presentation of these findings, see Appendix Tables H.5 and H.6. 
***/**/* Impact estimate is statistically significant at the .01/.05/.10 level. 
SDPP also estimated impacts on students’ math, language, and behavioral performance, measured 
at the end of the school year. The SDPP Program did not affect these outcomes in Timor-Leste. 
82.0** 78.9***
84.3
80.3 76.6
82.9
0.0
10.0
20.0
30.0
40.0
50.0
60.0
70.0
80.0
90.0
100.0
All Students At-Risk Students Not-At-Risk Students
Percentage of days attended
SDPP group Control group
Findings from the School Dropout Prevention Pilot Evaluation in Timor-Leste Page xvii
SDPP had no impact on dropout for the students exposed to the program (Figure ES.4).10
Students in SDPP schools dropped out at a rate of 15.5 percent, compared with 16.3 percent for 
students in control schools, but this difference was not statistically significant. At-risk students 
dropped out at a higher rate, at 17.7 percent in both SDPP and control schools; this difference was 
not statistically significant. 
Figure ES.4. Impacts of the Timor-Leste SDPP Program on school dropout, overall and by at-risk status 
(SY 2013 and SY 2014) 
Sources: SDPP baseline and follow-up school records data collection, May 2012, May 2013, March 2014, September 2014, and 
January 2015. 
 
10 SDPP operated in three target grades (grades 4, 5, and 6), including the final year of primary school (grade 6). 
Students were considered dropouts if they were no longer continuing their education at the last possible time we 
observed them. Students who started the program in grades 4 and 5 in the first year of implementation were considered 
to have dropped out if they did not complete their final grade 6 examinations in subsequent years. Students who started 
the program in grade 4 in the second and third years of implementation were considered to have dropped out if they 
did not enroll in school for the 2015 school year (as grade 5 and 6 students, respectively). 
16.3 17.7 15.5 15.3 17.7
14.0
0.0
10.0
20.0
30.0
40.0
50.0
60.0
70.0
80.0
90.0
100.0
All Students At-Risk Students Not-At-Risk Students
Percentage of students dropped out
SDPP group Control group
SDPP effectiveness in influencing school dropout
Primary research question 
Did SDPP affect school dropout, overall or for at-risk students?
Primary measure of SDPP Program’s effectiveness
 School dropout
Additional measure of SDPP Program’s effectiveness
 Student progression in school
Findings from the School Dropout Prevention Pilot Evaluation in Timor-Leste Page xviii
Note: The analysis is based on SY 2012 4th- and 5th-grade students, and SY 2013 and 2014 4th-grade students. The sample 
includes 14,045 students for the SDPP group (6,015 at risk and 8,030 not at risk) and 14,458 students for the control 
group (5,950 at risk and 8,508 not at risk). 
 Differences between SDPP and control group means were tested using two-tailed t-tests. Mean values are adjusted for 
baseline characteristics. The analysis accounts for clustering of students within schools and cohort fixed effects. For a 
tabular presentation of these findings, see Appendix Tables H.5 and H.6. 
***/**/* Impact estimate is statistically significant at the .01/.05/.10 level. 
SDPP also measured grade progression, or whether a student enrolled in the next grade or higher 
in the following school year. Grade progression is different than school dropout in that a student 
who repeated the grade he/she was in would not be considered to have progressed to the next grade 
or higher, but such a student would not be considered a school dropout. For example, if a student 
repeated 5th grade, he/she would not be considered a between-grade dropout, but he/she would 
not be considered as having progressed to the next grade or higher. The SDPP Program did not 
affect grade progression in Timor-Leste. 
What do beneficiaries say about the SDPP interventions?
Insight into how beneficiaries interacted with the SDPP interventions was obtained through a 
Qualitative Research Study. Responses from parents, students, teachers and administrators was very 
positive about both the Early Warning System (EWS) and the Extra-Curricular Activities (ECA) 
Program. 
EWS: About one third of at-risk students and nearly two-thirds of dropouts skipped school without 
their parents’ knowledge. About 40 percent of at-risk students and 80 percent of dropouts confirmed 
their parents had received a warning postcard from the school and/or home visit. Nearly all parents 
contacted spoke positively about the experience, although they also said they were initially surprised, 
ashamed, and disappointed in their child. The “Stay in School” committee members said that parents 
received the visit well once they understood its purpose was to help, not blame. Afterwards, their 
children—with the exception of dropout boys—confirmed their parents were more supportive: parents 
were more likely to monitor their attendance, talk to them about school, and provide material support. 
Teachers (73%) felt EWS made their job easier, in part because the EWS procedures and forms were 
easy to follow. Teachers, school directors and community members all felt the increased and improved 
contact with the families was a particularly positive feature of the EWS, helping both families and 
teachers better understand the issues the child faced. Ninety percent of school personnel hope to 
continue using the EWS.
ECA: The ECA Program was popular among all the students, although most felt one day per week was 
sufficient and denied it improved their attendance on the day it was held. They noted how engaging in 
games together improved their skills in collaboration and conflict resolution, and that the sessions 
helped to improve their language and math skills. Parents generally favored ECA, noting that it did 
seem to make their child more positive about school, but they remained ambivalent about its impact on 
attendance. Teachers disagreed. They strongly felt the ECA had a positive impact on improved 
attendance, behavior and general performance. The teachers also felt they had benefitted from the ECA 
training, noting that it helped them to use more interactive and hands-on instructional techniques in 
their regular classes. While teachers were concerned that the lack of supplies might hinder their ability 
to continue the ECA Program, school directors worried whether the teachers were adequately trained 
to lead the ECA Program on their own, and suggested additional training and guides.
.
Findings from the School Dropout Prevention Pilot Evaluation in Timor-Leste Page xix
Discussion
SDPP had a positive impact on teachers’ dropout prevention practices and their sense of self￾efficacy. Overall, SDPP affected improvements in key student attitudes and behaviors that can 
lead to dropout: at-risk students’ behavioral attitudes toward school, and daily attendance overall 
and among at-risk students. Despite improvement in student engagement to school as measured 
through attitudes and attendance, SDPP did not reduce the primary measure of school dropout. 
Table ES.1. SDPP Program impacts on primary measures of program effectiveness in Timor-Leste 
Impacts
Teacher dropout prevention practices +++
At-risk student attitudes toward school
Emotional attitudes toward school ○
Cognitive attitudes toward school ○
Behavioral attitudes toward school +++
Attendance
Overall ++
At-risk +++
Dropout
Overall ○
At-risk ○
+ + +/+ +/+ Statistically significant positive impact at the .01/.05/.10 level. 
— — —/— —/— Statistically significant negative impact at the .01/.05/.10 level. 
○ Impact is not statistically significant. 
Table ES.2. SDPP Program impacts on additional outcome measures in Timor-Leste 
Impacts
Teacher outcomes
Teacher self-efficacy ++
Teacher sense of responsibility ○
Administrator dropout prevention practices ○
Administrator self-efficacy ○
Administrator sense of responsibility ○
Student attitudes toward school
At-risk student perceptions of parent support ○
At-risk student perceptions of teacher support ○
Student engagement
Math performance ○
Portuguese ○
Tetun ○
Behavior ○
Dropout
Progression ○
+ + +/+ +/+ Statistically significant positive impact at the .01/.05/.10 level. 
— — —/— —/— Statistically significant negative impact at the .01/.05/.10 level. 
○ Impact is not statistically significant. 
There are several factors that might explain why SDPP did not reduce dropout in Timor-Leste. 
Each of these factors is discussed briefly below. 
Findings from the School Dropout Prevention Pilot Evaluation in Timor-Leste Page xx
Inconsistent implementation of SDPP in Timor-Leste. According to SDPP’s fidelity-of￾implementation (FOI) analysis, the vast majority of its treatment schools in Cambodia, India and 
Tajikistan had met the threshold established to indicate that the SDPP interventions had been 
implemented in congruence with their design. However, in Timor-Leste the FOI study found that 
schools appeared to implement the EWS inconsistently, particularly in terms of communicating 
with parents and taking follow-up actions. The implementation of extra-curricular activities in 
Timor-Leste was somewhat better, although only about 71 percent of schools reported that they 
conducted after-school activities weekly, which was the intended frequency (Creative Associates 
International and School-To-School International 2015). Although the SDPP interventions have 
improved attitudes and engagement toward school, the inconsistent implementation of follow-up 
actions by teachers and communication with parents may have decreased the program’s ability to 
affect dropout. 
Duration of exposure. SDPP was active in schools for two full school years and worked with 
grades 4, 5 and 6. Exposure time for individual students varied from one to two years depending 
upon their grade when they first received SDPP. While the EWS was a part of school itself, the 
ECA Program was offered to students once a week outside of school hours. From an operational
standpoint, exposure time was even less than intended, being limited by interruptions in the school 
calendar, including school closures due to rain, unscheduled and extended holidays, and national 
teacher training activities that pulled teachers from school for several weeks. Student attitudes 
toward school and behavior patterns are formed over time, starting with initial 
enrollment. Introducing SDPP interventions--which combine targeted adult attention and support, 
a welcoming school environment and activities that build student engagement—at the start of a 
student’s academic career, as well as continuing throughout their schooling, could help create 
positive perceptions of school and better behaviors. That the program had its intended effect on 
intermediate outcomes suggests that it would have its intended effect on longer-term outcomes 
with longer or more intense exposure to the intervention.11 Findings suggest that with additional 
exposure, program impacts might be more pronounced. 
Complexity of factors related to dropout. Given the complex pathways and inputs that lead to 
school dropout, it is possible that SDPP services alone may not be sufficient to influence the 
dropout decision for the majority of the students dropping out. In particular, economic factors that 
SDPP was contractually prohibited from addressing directly—such as school-related expenses, 
and the need to supplement income through household chores or domestic work—were found to 
be particularly relevant in decisions to drop out of school (Creative Associates International 
2014).12 However, that SDPP was able to show impact on key teacher and student behaviors shown 
to influence decisions to dropout demonstrates that the EWS and ECA Program interventions 
should be included among the multiple strategies needed to reduce dropout in Timor-Leste. 
This study shows that the SDPP Program in Timor-Leste was successful in improving some 
intermediate outcomes— particularly teachers’ dropout prevention practices, students’ behavioral 
attitudes towards school, and daily attendance—but not in reducing dropout. It is useful to compare 
the impacts of SDPP in Timor-Leste to other countries where SDPP was able to target students in 
 
11 We explore the question of the role of exposure more fully in Appendix J of Volume 2 of the report. 
12 The SDPP dropout prevention interventions were contractually prohibited from including conditional cash transfers 
or economic incentives, which have already been demonstrated to be effective by prior research. 
Findings from the School Dropout Prevention Pilot Evaluation in Timor-Leste Page xxi
contexts where dropout was much higher, and where SDPP was implemented with more fidelity. 
Additional discussion of the impacts of the SDPP Program across all SDPP countries is presented 
in a separate, four-country summary report (Creative Associates International and Mathematica 
Policy Research 2015). 
Findings from the School Dropout Prevention Pilot Evaluation in Timor-Leste Page xxii
Findings from the School Dropout Prevention Pilot Program Impact Evaluation in Timor-Leste Page 1
I. Introduction
Over the past two decades, considerable progress has been made in increasing school enrollment. 
From 2000 to 2011, the number of children out of school worldwide decreased from 102 million 
to 57 million—a reduction of almost 50 percent (Millennium Development Goals Report 2013). 
This effort has been supported by extensive research evaluating interventions aimed at increasing 
access to schooling in developing countries (Petrosino, Anthony, Claire Morgan, Trevor Fronius, 
Emily Tanner-Smith, and Robert Boruch 2012). However, many children do not complete primary 
or secondary cycles once they enroll; out of the 137 million children worldwide who entered first 
grade in 2011, 34 million are likely to leave school before reaching the last grade in primary school 
(Millennium Development Goals Report 2013). In many countries and regions, a greater 
percentage of out-of-school children have dropped out of school than have never enrolled in 
school. Interventions have been conducted in the United States and abroad to prevent dropout, 
however, there is limited evidence on how well they work, particularly in developing countries. 
The School Dropout Prevention Pilot (SDPP) Program, a five-year multi-country program funded 
by the U.S. Agency for International Development (USAID), is designed to identify ways to reduce 
student dropout in primary and secondary schools. 13 Its goal is to provide evidence-based 
programming guidance to USAID missions and countries in Asia and the Middle East (AME) on 
student dropout prevention by piloting and testing the effectiveness of dropout prevention 
interventions in four countries: Cambodia, India, Tajikistan, and Timor-Leste. To better 
understand ways of mitigating dropout in these countries, SDPP used a three-stage process: 
(1) reviewing the literature to identify international best practices in preventing dropout; (2) 
analyzing dropout trends and identifying risk factors and conditions associated with dropout in 
each country as part of a situational analysis; and (3) designing, implementing, and evaluating 
interventions to keep students at risk of dropout in school. Earlier reports describe findings from 
the first two stages of the project (Brush, Lorie, Jennifer Shin, Rajani Shrestha, and Karen Tietjen 
2011; Creative Associates International 2014a, 2014b; Shin, Jennifer, Rajani Shrestha, and Karen 
Tietjen 2011a, 2011b; Shrestha, Rajani, Jennifer Shin, and Karen Tietjen 2011a, 2011b). 
Based on the findings from the literature review and situational analyses, as well as input from key 
stakeholders in the four countries, SDPP worked with the Ministry of Education in each country 
to identify two interventions to address dropout. In all four countries, SDPP introduced an Early 
Warning System (EWS) and a student engagement intervention to motivate students to stay in 
school. 
Early Warning Systems (EWS) are a dropout prevention strategy that has shown promise in the 
United States, but for which there is very little evidence internationally. 14 EWS use teacher 
assessments and student records data to identify students who are at risk of dropping out of school 
and to target them for assistance. In reviews of dropout prevention research conducted by the What 
 
13 SDPP is implemented by Creative Associates International, with international partners Mathematica Policy 
Research and School-to-School International and local partners Kampuchean Action for Primary Education (KAPE) 
in Cambodia, Quest Alliance (QUEST) in India, and Care International (CARE) in Timor-Leste. Creative Associates 
has a corporate office in Tajikistan, which covers the responsibilities of a local partner in that country. 
14 The SDPP dropout prevention interventions were contractually prohibited from including conditional cash transfers 
or economic incentives, which have already been demonstrated to be effective by prior research. 
Findings from the School Dropout Prevention Pilot Program Impact Evaluation in Timor-Leste Page 2
Works Clearinghouse, U.S. Department of Education, two versions of EWS, the Check & Connect 
program and ALAS program, were found to help keep middle and high school students from 
dropping out and potentially help them progress in school (American Institutes of Research 2006a, 
2006b). However, existing research does not tell us whether EWS would have similar impacts in 
developing countries, given the different educational and cultural contexts in these countries. 
Similarly, the effectiveness of these types of programs on students of younger ages and lower 
grades is not known. 
In developing countries, there is evidence that other types of interventions designed to mitigate the 
factors that affect dropout, such as cash transfers (in specific contexts), scholarships and school 
construction, can be successful in improving enrollment, attendance, and retention (see, for 
example, An Angrist, Joshua, Eric Bettinger, Erik Bloom, Elizabeth King, and Michael Kremer 
2002; Levy, Dan, Matt Sloan, Leigh Linden, and Harounan Kazianga 2009; Schultz, T. Paul 2001). 
Interventions that target specific groups of students, such as girls (see, for example, Friedman, 
Willa, Michael Kremer, Edward Miguel, and Rebecca Thornton 2011; Oster, Emily, and Rebecca 
Thornton 2011) and students that live in rural areas (see, for example, Miguel, Edward, and 
Michael Kremer 2004) have also proven to be successful. However, to our knowledge, the SDPP 
evaluations present the first rigorous evidence of EWS in the developing country context. Studies 
of interventions that have incorporated academic activities, such as tutoring, computer labs, and 
other after-school activities, have had mixed results, though there is little rigorous evidence from 
evaluations that focus specifically on these activities (Banerjee, Abhijit, Shawn Cole, Esther Duflo, 
and Leigh Linden 2007; Brush, Lorie, Jennifer Shin, Rajani Shrestha, and Karen Tietjen 2011). 
The SDPP Program for each of the four project countries included the EWS combined with 
additional activities in the schools that varied depending on the country.15 The additional activities 
were designed to motivate greater student engagement, better attendance, and increase the desire 
to stay in school. The interventions were rolled out to the target grades in each country at various 
times during 2012. 
Although all of the target country programs included the EWS, they were distinct enough to merit 
independent evaluations in each country. The implementation of the EWS in four diverse countries 
allowed experimental evaluation of the effectiveness of these interventions in several contexts with 
different populations, strengthening the external validity of SDPP’s findings. 
SDPP conducted rigorous evaluations of the effectiveness of the SDPP Program in each of the four 
countries. In each country, schools that were eligible to receive the program were identified, 
recruited, and asked to consent to participate in the study. Eligible schools in targeted regions were 
then randomly assigned to either an SDPP treatment group, which offered the SDPP intervention 
package, or a control group, which did not. For each country, the evaluation estimated program 
effects by comparing the outcomes of students and teachers in SDPP schools with the outcomes 
of those in control schools. 
This report presents findings from the impact evaluation of the SDPP Program in Timor-Leste. In 
Timor-Leste, the program included an EWS in combination with an Extra-Curricular Activities 
 
15 The grades targeted in each country are as follows: grades 7, 8, and 9 in Cambodia; grade 5 in India; grade 9 in 
Tajikistan; and grades 4, 5, and 6 in Timor-Leste. 
Findings from the School Dropout Prevention Pilot Program Impact Evaluation in Timor-Leste Page 3
(ECA) activities once to twice a week focused on promoting fun through songs, games, and craft 
sessions. The impact evaluation draws on school records and survey data collected in 97 SDPP 
schools and 93 control schools across five districts.16 The student sample includes 4th-, 5th-, and 
6th-grade students from each of the two complete school years (SY) (2013 and 2014) during which 
the SDPP Program was in effect. 
The findings and information on the impact evaluation for SDPP in Timor-Leste are presented in 
two volumes. Volume 1 presents the impact evaluation findings and Volume 2 details the 
methodology used. 
This report—Volume 1—is organized as follows. Section II describes the SDPP Program and its 
implementation in Timor-Leste. Section III discusses the impact evaluation design and describes 
the types of outcome domains used to evaluate the program. Section IV discusses the sample and 
data collection, and Section V describes the characteristics of the sample prior to implementation. 
Section VI presents the impacts of the program on teacher outcomes, attitudes toward school, 
engagement in school, and school dropout. Section VII presents school-level dropout trends. 
Section VIII discusses the findings and conclusions. 
Volume 2—a technical appendix—provides more detail on the study, including further detail on 
the sample frame, data collection, estimation procedures, subgroup analyses, robustness checks, 
and additional exploratory analyses. 
 
16 Of the 94 schools randomly assigned to the control group, 1 school dropped out of the study sample during the SY 
2013.
Findings from the School Dropout Prevention Pilot Program Impact Evaluation in Timor-Leste Page 4
II. SDPP Program in Timor-Leste 
The SDPP Program in Timor-Leste had two main components: (1) an EWS, and (2) an Extra￾Curricular Activities (ECA) program (Creative Associates International 2012a, 2012b). To the 
extent possible, these activities built on existing Ministry of Education procedures to facilitate 
sustainability after the project ends. 
A. Targeting grades and geographic areas for intervention 
Using data from the national education management information system, SDPP identified the 
population for whom dropout was most prevalent and who would benefit most from a dropout 
prevention program. In-grade dropout was consistently most acute at the primary level, with an 
average dropout rate of about 6 percent compared to average rates closer to 3 percent at the pre￾secondary and secondary levels.17
 Grades 1, 4, and 5 had the highest dropout rates at about 7 
percent. Because the Grade 1 dropout rate was largely explained by high underage enrollment, 
resulting in later re-enrollment, it was not a suitable target grade for SDPP interventions. In 
contrast, the Grade 6 dropout was low, as the national EMIS did not report students who leave 
school after completing a cycle as “dropouts.” However, the transition rate of 82 percent between 
primary and pre-secondary cycles indicated that 18 percent of Grade 6 students do not reach Grade 
7. (Shrestha, Rajani, Jennifer Shin, Karen Tietjen 2011b). Consequently, SDPP focused on the 
upper primary grades 4, 5, and 6 (Figure II.A.1). Based on the severity of dropout coupled with 
security and accessibility considerations, the five target districts of Bobonaro, Ermera, Liquica, 
Manatuto and Viqueque were selected (Figure II.A.2).18
Figure II.A.1. National dropout rates by grade (2009–2010) 
Sources: Creative Associates International (2014b) 
 
17 The between-grade dropout rate is the proportion of students who no longer enroll in the following year. The in￾grade dropout rate is the proportion of students who drop out of the school before the end of the academic year. 
18 The dropout rate overall for 4th, 5th, and 6th graders in these districts according to Ministry records was about 7 
percent, ranging from 4.75 percent to 8.1 percent. SDPP data reported in Section VI shows notably higher rates. 
Primary: Grades 1-6 Pre-secondary: Grades 7-9 Secondary: Grades 10-12
Findings from the School Dropout Prevention Pilot Program Impact Evaluation in Timor-Leste Page 5
Figure II.A.2. Target regions of the SDPP Program 
Source: Creative Associates International (2014a). 
B. Interventions 
1. Selecting the SDPP interventions 
SDPP selected dropout prevention interventions to be implemented on the basis of: (1) a review 
of the existing domestic and international evidence on interventions designed to decrease school 
dropout; (2) an analysis of existing policies and programs in each country that could affect dropout; 
(3) situational analyses of the factors and conditions associated with school dropout in each 
country; and (4) input from key stakeholders in each country. The literature review found little 
rigorous evidence on dropout prevention interventions in an international context.19 Conditional 
 
19 See Brush, Lorie, Jennifer Shin, Rajani Shrestha, and Karen Tietjen (2011) for complete findings from the literature 
review. 
Findings from the School Dropout Prevention Pilot Program Impact Evaluation in Timor-Leste Page 6
cash transfer interventions showed consistently positive impacts on school dropout, but other 
evidence was mixed or focused on U.S.-based interventions. 
SDPP conducted primary research focused on 4th, 5th and 6th grades in Bobonaro, Ermera and 
Liquicia, in order to identify key factors and conditions associated with school dropout in Timor￾Leste. 20 The situational analysis collected data from at-risk students, dropouts, their 
parents/guardians, school administrators and teachers, community members and local education 
officials in 30 school-communities. 
Findings from the SDPP Situational Analysis indicate that the top causes of student dropout among 
4th-, 5th-, and 6th-grade students in the target districts fell into three categories: (1) economic, (2) 
academic or school-related, and (3) health. (Figure II.B.1). 21 Economic reasons are most 
commonly cited by children and their families: about 60 percent of the at-risk students and parents, 
and about 40 percent of the dropout children and parents cited the inability to pay for school 
expenses, and 20 to 30 percent pointed to the need to work to earn money. About 25 percent of 
at-risk students and 40 percent of dropouts cited domestic chores. Nearly one-third indicated health 
reasons. However, about 20 percent of Grade 4, 5, and 6 students drop out for academic and 
school-related reasons: they performed poorly in school and/or simply do not like school. About 
50 percent of at-risk parents and 65 percent of dropout parents reported their child had not mastered 
basic reading and math, and nearly 60 percent of the at-risk pupils and dropouts said they had 
failed a subject. 
Figure II.B.1. Reported causes of dropout 
Source: Creative Associates International (2014a) 
 
20 The other two districts—Manatuto and Viqueque--were added later to meet sampling requirements. 
21 See Creative Associates International (2014b) for complete findings from the situational analysis. 
0 10 20 30 40 50 60 70
Did not like school
Poor academic performance
Work to earn money
Help with chores/family business
Illness
Can't pay school-related expenses
Percent of respondents
At-risk students
At-risk parents
Dropouts
Dropout parents
Findings from the School Dropout Prevention Pilot Program Impact Evaluation in Timor-Leste Page 7
These reasons led to or were exacerbated by high student absenteeism (Figure II.B.2). Nearly half 
of at-risk students and more than half of dropouts missed two or more days of school per month; 
over 40 percent of dropouts and 30 percent of at-risk students have missed more than 15 
consecutive days of school. Only 40 percent of at-risk parents indicated they were aware of their 
child’s school attendance patterns, and 7 percent had no idea whether or not their child was 
regularly attending school. Almost half of the parents reported keeping their child home from 
school to help with domestic chores. 
Figure II.B.2. Absenteeism reported by dropouts and at-risk students 
Source: Creative Associates International (2014a) 
Many at-risk students reported poor treatment by teachers and an unsupportive school environment 
(Figure II.B.3). Student engagement is low, with only slightly more than one-third of at-risk 
students completing homework assignments, less than a quarter participating in school activities 
and more than a quarter reporting being bullied (Figure II.B.4). Between 10 to 20 percent of 
dropouts and at-risk students experienced some form of behavioral problem, ranging from 
disruptive behavior and conflicts with teachers to breaking school rules and getting into trouble. 
58
42
45
30
0
10
20
30
40
50
60
70
Absent 2+ days/month Absent 15+ consecutive days
Percent of respondents
Dropouts At-risk students
Findings from the School Dropout Prevention Pilot Program Impact Evaluation in Timor-Leste Page 8
Figure II.B.3. Treatment by teachers reported by at-risk students 
Source: Creative Associates International (2014a). 
Figure II.B.4. At-risk student engagement in school 
Source: SDPP situational analysis data collection; 2011
68
43
8
11
44
30
0
10
20
30
40
50
60
70
80
Criticizes student
for wrong answer
Uses physical
punishment on
student
Does not organize
study
groups/extra
tutoring
Teacher does not
think I am/was
intelligent
Teacher treats
some students
better than others
Teacher does not
care about my
problems
Percent of Respondents
37
33
0
22
28
0
5
10
15
20
25
30
35
40
Rarely or never
completes required
homework
Spends less than
1 hour on homework
each night
School is not fun No participation
in school activities
Is bullied or picked on
Percent of respondents
Findings from the School Dropout Prevention Pilot Program Impact Evaluation in Timor-Leste Page 9
In November 2011, SDPP convened a consultation workshop on program design in Timor-Leste 
to solicit ideas for and determine the school-based dropout prevention interventions with the 
greatest chance of both success and of sustainability. SDPP led stakeholders—including 
nongovernmental organizations and education authorities—through the main findings from the 
literature review and situational analyses, and discussed intervention options for the SDPP 
Program. The SDPP contract parameters—which excluded economic subsidies, vocational 
training, construction/infrastructure improvements and general teacher training—were reviewed.22
Design workshop participants ranked a set of intervention options. SDPP then selected the 
interventions for Timor-Leste using these rankings along with additional program considerations. 
Two complementary inventions were planned, based on power calculations, sample size 
parameters and timeline.23
One of the complementary interventions was aimed at reducing negative student behaviors 
associated with dropout, such as attendance, and the other aimed at student motivation, based on 
the findings of the situational analysis, which found low student engagement. Stakeholders in 
Timor-Leste selected EWS as the primary component and enrichment and recreational activities 
as the supplemental component.24
2. Early Warning System (EWS) 
The purpose of the EWS was to identify and provide targeted support to students at risk of dropping 
out of school. The EWS used existing data in schools as well as teacher input to identify at-risk 
students, closely monitor them, and target them for additional support. It was intended to enhance 
the capacity of schools to address the needs of at-risk pupils, strengthen the partnership between 
the parents/guardians and school personnel to monitor and improve school attendance and 
performance, and raise awareness among parents/guardians and the community about the value of 
children staying in school and what parents/guardians can do to support their children. The EWS 
consisted of three components: (1) identification of students at risk of school dropout, (2) first 
response strategies, and (3) community engagement. 
In Component 1, the SDPP Program in Timor-Leste worked with teachers to identify at-risk 
students based on six measures of dropout risk. These included the globally recognized ABCs of 
dropout – attendance, behavior, and coursework—and were augmented with other contextually 
specific indicators—how often the student completes his or her homework, and how often the 
student is tardy or leaves early. (Creative Associates International 2014b). SDPP worked with 
 
22 While USAID’s AME Regional Bureau recognized the role that economic, infrastructure/construction, and school 
quality barriers could play in families’ decisions not to send their children to school, these types of interventions were 
excluded from experimentation for a variety of reasons. A solid research base already existed for economic subsidies 
and cost alleviation measures. Infrastructure improvements and construction exceeded USAID manageable interests. 
Desire to focus on dropout-specific interventions eliminated general teacher training for instructional improvement, 
which was already funded under other programs. Finally, funding for USAID Basic Education prohibited expenditure 
of SDPP budget on vocational education activities. 
23 The SDPP contract specified two recommendations for interventions. SDPP was originally a three-year program, 
and did not provide sufficient time to design, develop, and implement multiple interventions in each country. 
24 See Prouty, D. and K. Tietjen, SDPP Pilot Design Plan: Timor-Leste (February 2012).
Findings from the School Dropout Prevention Pilot Program Impact Evaluation in Timor-Leste Page 10
teachers to score and rank students’ relative risk level. Students with the highest score were deemed 
at-risk students.25
Component 2 used a “track and trigger” approach to closely monitor the progress of at-risk students 
and initiate “first response” activities when students showed signs of struggling. Teachers recorded 
and tracked key student behaviors, such as attendance, behavior and coursework. Signs of 
problems—for example, frequent absences or failed classes—triggered an immediate set of 
response actions, ranging from in-class attention to contact via letter and/or home visit with parents 
to case management meetings with school staff to develop an individualized program of 
intervention. 
Component 3 focused on raising awareness within the community about the importance of 
schooling and the problem of dropout. SDPP worked with parent-teacher associations and other 
community groups on advocacy activities and enlisted their support in implementing some of the 
first response activities, working closely with the school. For example, contact with parents was 
conducted through “Stay in School” groups made up of community members, who delivered 
illustrated “postcards” from the school to households to notify parents of a problem and follow up 
with home visits to discuss issues and solutions. For many school communities, this was the first 
time that community or school organizations and their members engaged in student support 
activities not limited to fund raising or infrastructure improvement. This component also included 
outreach activities, such as school events to discuss how parents can support their child in school. 
These activities were designed to directly change the behaviors of community members, parents, 
and students themselves.
3. Extra-curricular activities program 
Extra-curricular activities, which consisted of one or two in-school sessions per week, were offered 
to all target-grade students, with special encouragements for students identified as at-risk through 
the EWS (Creative Associates International 2012a).26 The activities were scheduled either before 
or after school, depending on instructional schedules. Each week, students participated in one or 
two sessions focused on promoting fun through songs, games, and craft sessions. The ECA 
program was intended to motivate children to attend and engage in school through structured 
activities to foster cooperative learning, enhance self-confidence and reinforce basic skills. The 
ECA program was initially conducted by SDPP facilitators, and during the second year of program 
implementation, target-grade teachers and community members were encouraged to lead a second 
weekly session. The SDPP facilitators monitored at-risk students’ progress, recorded participation 
levels and met at least monthly to discuss progress with homeroom teachers. 
C. Program implementation 
The SDPP Program was active in schools during three school years—SY 2012, SY 2013, and SY 
2014 (Figure II.C.1). Students and teachers of target grades received only a partial year of exposure 
 
25 Please see Appendix A for more details on the SDPP determination of dropout risk. 
26 During the first year of SDPP implementation, one session was offered per week. SDPP encouraged schools to 
transition to two weekly sessions at the start of the second year of implementation, with the second session led by 
teachers and community members. 
Findings from the School Dropout Prevention Pilot Program Impact Evaluation in Timor-Leste Page 11
during the first year of the program (SY 2012) because of the timing of the roll-out of program 
activities. Students and teachers of the target grades during the second and third school year 
received the program for full school years.27 The teachers and school administrators were first 
trained in August 2012 during a break in the 2012 school year. The EWS began in September 
2012, followed by the after-school activities in October 2012 for entering 4th-, 5th-, and 6th-grade 
students in the 2012 school year. 
During three school years of program activity, the SDPP Program targeted students in grades 4, 5,
and 6 and their teachers. Students in grades 4 and 5 during SY 2012 and SY 2013 received the 
program for more than one year (Figure II.C.2); grade 6 is the final year of primary school. 
Specifically, the evaluation follows: (1) SY 2012 5th graders who continued to receive the program 
in SY 2013, in their 6th grade; (2) SY 2012 4th-graders who continued to receive the program in 
SY 2013 and SY 2014, in their 5th and 6th grades, respectively; (3) SY 2013 4th grade students, 
who continued to receive the program in SY 2014, in their 5th grade; and (4) SY 2014 4th grade 
students.28 SDPP focuses the analyses on the latest point in time for which these students and 
teachers are observed across these multiple school years. The latest observation point occurs in SY 
2013 or SY 2014 for all students followed by the evaluation. 
 
27 The teachers and school administrators were first trained in August 2012, which is the middle of SY 2012. The 
EWS began in September 2012 and the after-school activities began in October 2012 for entering 4th-, 5th-, and 6th￾grade students in SY 2012. For entering 4th-, 5th-, and 6th-grade students in SY 2013 and SY 2014, both the EWS 
and after-school activities began in January. 
28 The evaluation does not focus on SY 2012 6th graders because they did not receive the program for a full school 
year. 
Findings from the School Dropout Prevention Pilot Program Impact Evaluation in Timor-Leste Page 12
Figure II.C.1. Rollout of the interventions in Timor-Leste 
2012 2013 2014
M J J A S O N D J F M A M J J A S O N D J F M A M J J A S O N D J
Program rollout T E A
E, 
A
T, 
E, 
A
Data collection X X X X
E
O
A
X
Note: The school year in Timor-Leste lasts from January to October/November. 
T = teacher and school administrator training begins; E = EWS intervention rolled out to students; A = after-school enrichment activities rolled out to students; EOA = end of 
activities; X = Impact evaluation data collection in schools. 
2012 school year
2013 school year
2014 school year
2015 school year
Findings from the School Dropout Prevention Pilot Program Impact Evaluation in Timor-Leste Page 13
Figure II.C.2. Grades by academic year for SDPP students in Timor-Leste 
D. Fidelity of Implementation
As part of the research design, Fidelity of Implementation (FOI) was measured to determine the 
extent to which the SDPP interventions were being implemented as designed. SDPP evaluated the 
fidelity of implementation of the EWS and after-school components in May, October, and 
November 2014. 
To assess the fidelity of the EWS implementation, SDPP examined how well schools and teachers 
identified at-risk students, tracked their attendance, communicated with parents, and provided 
follow-up support. In the second year of implementation, two rounds of FOI data were collected 
and analyzed. 29
 
Overall, schools appeared to implement the EWS unevenly (Creative Associates International and 
School-to-School International 2015). During the first round of FOI data collection and analysis, 
only 45 percent of the schools met or exceeded the threshold score of 80 percent for the EWS 
intervention (with an average total score of 39%) (Figure II.D.1). With the exception of 
Component 2 (“tracking research students”), the percentage of schools fell below the threshold 
score on the three other components: 66 percent of schools met the threshold score for identifying 
research students, but only 9 percent met the threshold for communicating with parents and only 
 
29 Fidelity of implementation was monitored throughout intervention implementation, but only in the second year of 
implementation were standardized instruments used for the data reported in this report. 
Findings from the School Dropout Prevention Pilot Program Impact Evaluation in Timor-Leste Page 14
8 percent met the threshold for follow-up action, primarily case management meetings among 
school personnel (Figure II.D.2). 
Results from the second round showed notable improvement, with 69 percent of schools meeting 
or exceeding the threshold score. By component, 93 percent of schools met or surpassed the 
threshold score for identifying and tracking research students and 26 percent met it for taking 
follow-up action. However, only 8 percent of schools met the threshold for communicating with 
parents, indicating the difficulty of mobilizing the “Stay in School” community groups (Figure 
II.D.2). 
Figure II.D.1. EWS: Proportion of schools meeting or exceeding 80% FOI threshold 
Source: Creative Associates International and School-to-School International (2015). 
Figure II.D.2. EWS: Proportion of schools meeting or exceeding FOI threshold by component 
Source: Creative Associates International and School-to-School International (2015). 
45%
69%
Early Warning System Round 1
(n=93)
Early Warning System Round 2
(n=94)
66%
81%
9% 8%
93% 93%
8%
26%
Component 1
(Identify Research
Students)
Component 2
(Track Research
Students)
Component 3
(Communication with
Parents)
Component 4
(Follow-up Action)
Round 1 (n=93) Round 2 (n=94)
Findings from the School Dropout Prevention Pilot Program Impact Evaluation in Timor-Leste Page 15
To assess the implementation of the ECA Program, SDPP looked at whether: the program was 
held weekly; schools maintained attendance lists and lesson plans; students and teachers were 
actively and positively engaged; and key activity components were incorporated into the program. 
The results suggested that the after-school activities were fairly well-implemented, with 71 percent 
of the schools meeting or exceeding the threshold score (Figure II.D.3). There was room for 
improvement, particularly in the frequency with which the after-school activity program was held. 
Only 71 percent of schools reported holding the after-school activities weekly, as intended (Figure 
II.D.4). (Creative Associates International and School-to-School International 2015).
Figure II.D.3. ECA: Proportion of Schools Meeting or Exceeding 80% FOI Threshold 
Source: Creative Associates International and School-to-School International (2015). 
Figure II.D.4. ECA: Proportion of Schools Meeting or Exceeding 80% FOI Threshold, by Component 
Source: Creative Associates International and School-to-School International (2015).
71%
n/a
Extra-Curriculur Activities
Round 1 (n=93)
Extra-Curriculur Activities
Round 2
71% 77%
44%
81%
Component 1
(Weekly ECA held)
Component 2
(Required Lists and
Plans)
Component 3
(Student
Participation/Facilitator
Behavior)
Component 4
(ECA Key Components
Present)
Round 1 (n=93)
Findings from the School Dropout Prevention Pilot Program Impact Evaluation in Timor-Leste Page 16
III. Evaluation design
A. SDPP Theory of Change 
A conceptual model of SDPP Program activities and how they might affect student and teacher 
outcomes guided the design of the impact evaluation (Figure III.A.1). This model posits that 
teacher and parent knowledge and practices—shared and reinforced by the larger community—
are inputs into students’ attitudes toward school and educational aspirations. These student 
attitudes translate into student engagement in school, including their attendance, behavior, and 
academic performance. The complex, cumulative interaction of these factors are inputs into the 
student’s ability, desire and decision to remain in school or drop out. 
Figure III.A.1. Timor-Leste SDPP conceptual model 
The ultimate goal of these activities is to reduce school dropout, and SDPP activities involved 
working directly with teachers, administrators, and parents with this goal in mind. Teachers 
received extensive training to influence their attitudes toward and practices used with at-risk 
students, as well as instruction on how to use a new EWS and how to engage students in extra￾curricular activities using active learning and child-centered methods. The EWS was designed to 
improve student attendance and attitudes toward school, both directly (through interactions with 
students) and indirectly (through interactions with teachers and parents). The idea behind this 
Teacher, Parent, and Community Knowledge, and Awareness
Parent 
Support
Teacher Prevention 
Practices and 
Support
Student 
Attitudes and 
Aspirations
Behavior Performance
Dropout
Attendance
School Dropout
Student Engagement 
in School 
Student Attitudes
Teacher Behavior and 
Attitudinal Outcomes
Findings from the School Dropout Prevention Pilot Program Impact Evaluation in Timor-Leste Page 17
system is that, by changing teacher, administrator, and parent knowledge and behaviors toward 
students, student attitudes toward school should change. This improvement in attitudes should lead 
to more student engagement in and attachment to school, which in turn should reduce school 
dropout. Likewise, the ECA Program was designed to improve teacher-student interactions and, 
as a consequence, student attitudes toward school by helping students improve their academic 
performance and offering recreational activities to encourage attendance. 
Recognizing the complex processes that lead to dropping out, mechanisms through which the 
SDPP interventions aim to reduce dropout are varied and focused on influencing intermediate 
outcomes—specifically the attitudes, practices, and behaviors of teachers, parents, and students—
that that may be related to dropout. 
B. Research questions 
Based on this conceptual model, SDPP designed the impact evaluation to address five 
primary research questions: 30
 Does SDPP improve teacher behavior and attitudes? 
 Does SDPP improve student attitudes toward school? 
 Does SDPP improve student engagement in school associated with retention, such as 
attendance? 
 Does SDPP improve the dropout rate? 
 What are SDPP’s impacts for students most at risk of dropping out of school? 
C. Evaluation Design 
To answer these questions, SDPP used a randomized control trial design, as depicted in Figure 
III.C.1. SDPP randomly assigned schools to either 
an SDPP group that provided the SDPP Program, or 
a control group that operated as usual. SDPP 
estimated program impacts by comparing relevant 
outcomes for students and teachers in schools 
offering the SDPP Program (the SDPP group) to the 
outcomes of students and teachers in schools 
offering regular services (the control group) 
A randomly assigned control group is a crucial 
element of a rigorous impact evaluation because it 
allows the evaluator to estimate what would have 
happened in the absence of the program. With well￾implemented random assignment, the students and 
teachers in treatment schools will be similar to those 
in control schools in terms of their pre-existing 
 
30 See Murray, Nancy, Quinn Moore, Larissa Campuzano, Kathy Buek, Emilie Bagby, and Mark Strayer 2012 for 
details on the evaluation design. 
What is an impact evaluation?
 Goal: an evaluation designed to 
provide answers to policy or 
program effectiveness, aims to 
generate rigorous evidence to answer 
questions about program impact
 Purpose: establish the causal effect 
of an intervention through the use of 
a counterfactual
 Mechanism: comparison of 
outcomes for those randomly
assigned to SDPP and control groups
Findings from the School Dropout Prevention Pilot Program Impact Evaluation in Timor-Leste Page 18
characteristics. The only systematic difference between these groups is that the students and 
teachers in the treatment group were offered the SDPP Program, and the students and teachers in 
the control group were not. The result is that any observed treatment-control differences in 
outcomes can be attributed to the SDPP Program and not to pre-existing differences in the 
characteristics of students, teachers, and schools in the sample. 
Figure III.C.1. SDPP Randomized Control Trial design in Timor-Leste 
From 190 eligible schools in Timor-Leste, 93 schools were randomly assigned to the control group 
and 97 schools were randomly assigned to the SDPP group. Over the course of the project, data 
was collected on 19,255 and 1,216 individual students and teachers in control schools, and 18,606 
and 1,182 individual students and teachers in SDPP schools.
1. Study eligibility 
To be eligible for the evaluation, schools had to: (1) be located in the five target districts; (2) offer 
4th, 5th, and 6th grade; (3) not be a technical or religious school; (4) not be receiving major 
assistance from other donors or organizations, such as UNICEF, Childfund, and CARE; and (5) 
agree to participate in the evaluation. SDPP identified 190 schools that met these criteria;31 97 
were randomly assigned to the SDPP group and 93 were randomly assigned to the control group.32
SDPP implemented the random assignment, stratifying by district. 
2. Primary impact analysis 
Given the RCT design, the assessment of the SDPP Program’s effectiveness focuses on the 
difference in average outcomes at our final follow-up between students and teachers randomly 
assigned to the SDPP group and those randomly assigned to the control group. Because random 
assignment means that there should be no systematic differences in baseline characteristics 
between the SDPP and control groups, a simple difference in outcomes across groups provides a 
rigorous, unbiased estimate of the SDPP Program’s impact. However, we are able to increase the 
precision of the impact estimates and our ability to identify impacts as statistically significant by 
using statistical models that adjusted for small differences in the initial characteristics of the study 
 
31 With this sample size, and a 7.5 percent dropout rate, power calculations indicated that the study would be able to 
detect an impact of 4 percentage points or higher, with 80 percent power. Please see Appendix A for more information. 
32 Originally, 194 schools were randomly assigned. However, one SDPP school and three control schools dropped out 
of the study because they no longer offered 6th grade, a condition for participation in the evaluation. 
Eligible Schools
SDPP EWS + After School
Enrichment group Control group
Random Assignment
Findings from the School Dropout Prevention Pilot Program Impact Evaluation in Timor-Leste Page 19
groups that may have arisen by chance or because of survey nonresponse.33 In this way, the impact 
estimates adjust for baseline values of the outcomes of interest, as well as individual and school￾level characteristics. RCT impact estimates are considered the gold standard in evaluating program 
effectiveness.34
These impact estimates represent the difference in the outcome of interest at endline that is 
attributable to the SDPP Program relative to the status quo. The estimates are expressed as 
percentage point differences between the treatment and control group. The impact estimates 
reported in this study should be interpreted as the difference in outcomes that resulted from 
exposure to SDPP. For example, an “X” percentage point favorable impact on school dropout 
indicates that, on average, the dropout rate under SDPP is “X” percentage points lower than it 
would have been under business-as-usual operations. We also present percentage increases or 
decreases in the primary outcomes across the treatment and control groups. These “percentage 
changes” should not be interpreted as the percentage “change” that might be calculated in a pre￾post measure or baseline/endline change, but rather, the increase or decrease in the treatment 
group’s outcome measure in relation to the control group at endline. 
The impact analysis includes all students in the targeted grades in SDPP and control schools, 
regardless of whether the students in the SDPP schools actually participated in SDPP activities.
Therefore, these estimates represent the average impact of SDPP on all students in the enrolled 
schools. These are called intent-to-treat (ITT) estimates—they reflect the fact that not every school 
or student intended to be treated (via program services) actually participated in the program. The 
ITT estimates therefore answer the policy-relevant question—do programs make a difference for 
schools that choose to enroll? These types of estimates are widely used in large-scale evaluations 
and preserve the integrity of the random assignment design. 
SDPP worked with 4th, 5th and 6th graders from SY 2012, SY 2013 and SY 2014. Therefore, 
SDPP worked with: (1) SY 2012 5th graders who continued to receive the program in SY 2013, 
in their 6th grade; (2) SY 2012 4th-graders who continued to receive the program in SY 2013 and 
SY 2014, in their 5th and 6th grades, respectively; (3) SY 2013 4th grade students, who continued 
to receive the program in SY 2014, in their 5th grade; and (4) SY 2014 4th grade students. The 
analyses look at outcomes for these cohorts of students. SDPP schools and control school outcomes 
are compared, and differences are estimated while controlling for school level and individual level 
information regarding the school year during which they received SDPP. 
3. Impact analysis for at-risk students 
The SDPP Program was intended to affect outcomes more strongly for students at risk of dropout 
than for students not at risk, since the goal of the EWS was to train teachers to identify and work 
with students that are at risk of dropping out of school. Therefore, SDPP analyzed the impacts on 
students at risk of dropout in two ways. First, students in the SDPP Program were identified in 
control schools as at risk based on student characteristics before they entered the target grades, 
mimicking the EWS identification process as closely as possible with available data. SDPP also 
conducted a subgroup analysis of the students which were identified as at-risk. This subgroup 
 
33 Statistical significance is explained in Section IV; see text box on “statistical significance”. 
34 Please see Appendix B for more details on the impact estimation methods. 
Findings from the School Dropout Prevention Pilot Program Impact Evaluation in Timor-Leste Page 20
analysis is part of the primary assessment of whether the SDPP Program effectively accomplished 
its goals (or was effective). 
Importantly, the students identified as at risk for the subgroup analysis in SDPP schools based on 
baseline characteristics are not necessarily the same as those identified by the EWS in SDPP 
schools. However, there is substantial overlap in these groups. SDPP focused the primary analysis 
on the subgroup of students identified as at risk of dropout based on their baseline characteristics 
because it would allow us to identify at-risk students in the control group using an analogous 
process. More critically, preserving the integrity of the random assignment design requires that 
subgroup analysis be based on baseline characteristics observed before receiving program services, 
and students in both SDPP and control schools are thus identified as being at-risk in the same 
manner. 
4. Additional subgroup analyses 
In addition to looking at impacts separately for at-risk students, SDPP also conducted several 
additional subgroup analyses. The SDPP Program was not designed to have different impacts for 
different subgroups of students (other than students at-risk of dropout); however, the literature 
suggests that outcomes and impacts might vary for different types of students. There might be 
differential impacts for students with certain characteristics (such as gender or being over age for 
their grade), for different types of schools (such as the percentage of at-risk students in a school, 
and the remoteness of the school), or for different types of teachers (such as differences by gender 
or by full-time teaching status) (Table III.C.1). These subgroup analyses provide interesting 
context for interpretation of the main findings, but are not part of SDPP’s primary assessment of 
program effectiveness. Because these subgroup analyses were exploratory, SDPP did not adjust 
statistical significance thresholds for multiple comparisons when there were multiple subgroup 
comparisons being made. These additional subgroup analyses provide interesting context for our 
interpretation of the main findings and insight into the groups for which the SDPP Program may 
be more or less successful.35
 
35 Because these subgroup analyses were exploratory, SDPP did not adjust statistical significance thresholds for 
multiple comparisons when there were multiple subgroup comparisons being made (Schochet, Peter Z. 2009). 
Findings from the School Dropout Prevention Pilot Program Impact Evaluation in Timor-Leste Page 21
Table III.C.1. Primary and additional subgroup analyses 
Subgroup Subgroup 
Type Definition Analysis
Type
At-risk students / Not￾at-risk students
Students Students were identified as being most at risk of dropping 
out based on information on their characteristics available 
in school records at baseline before entering the target 
grade.
Primary
Female/male Students; 
Teachers 
Student and teacher sex was determined from school 
records. 
Additional
Full-time teaching 
status
Teachers Teachers were determined to be full-time or not full-time 
(part-time, contract, volunteer, or other types of teachers 
that are not full time employees)
Additional
Over-age / not over￾age
Students A student is considered to be over-age if he or she is two 
years older than the appropriate age for his or her grade, 
compared to those who were within two years of the
appropriate age. In Timor-Leste, 4th-grade students were 
defined as over-age if they were 13 or older, 5th-grade 
students as over-age if they were 15 or older, and 6th￾grade students as over-age if they were 17 or older.
Additional
High % at-risk/low % 
at-risk
Schools SDPP divided schools into a group that was at the 70th 
percentile or higher in percentage of at-risk students at 
baseline among control group schools (the high 
percentage group) and a group that was below the 70th 
percentile (the low percentage group).
Additional
Distance to school Schools Schools are defined as remote if they could not be reached 
by all types of vehicles and schools as not remote if they 
could.
Additional
5. Primary and additional measures of SDPP’s effectiveness 
To select outcomes for the impact study, SDPP identified the key domains that were expected to 
be affected by the SDPP Program as indicated by the program theory of change: (1) teacher 
behavior and attitudes, (2) student attitudes, (3) student engagement in school, and (4) school 
dropout. Within each of these domains, SDPP identified key outcomes that the program was 
intended to affect; these primary outcomes can be used to assess whether the program achieved its 
goals. In addition to the primary outcomes, the evaluation presents findings for additional 
outcomes to provide context to the primary analysis and more detail on how students and teachers 
may have been affected by SDPP.
Findings from the School Dropout Prevention Pilot Program Impact Evaluation in Timor-Leste Page 22
Table III.C.2. Primary and additional measures of Timor-Leste SDPP Program effectiveness 
Domain
Primary measures of program 
effectiveness Secondary measures*
Primary Measures of Program Effectiveness
Teacher 
outcomes
Teacher dropout prevention practice scale 
(range: 1 to 8), SY 2013 and SY 2014, 
teachers in grades 4, 5, and 6
Teacher’s sense of responsibility for at-risk students 
scale (range: 1 to 4), SY 2013 and SY 2014, 
teachers in grades 4, 5, and 6
Teacher’s sense of self efficacy scale (range: 1 to 5), 
SY 2013 and SY 2014, teachers in grades 4, 5, 
and 6
Administrator dropout prevention practice scale 
(range: 1 to 8), SY 2013 and SY 2014
Administrator’s sense of responsibility for at-risk 
students scale (range: 1 to 4), SY 2013 and SY 
2014
Administrator’s sense of self efficacy scale (range: 1 
to 5),d SY 2013 and SY 2014, school 
administrators
At-risk student 
attitudes
Emotional attitudes toward school, SY 
2013 and SY 2014, for at-risk 
students in the SY 2012 4th- and 
5th-grade cohort and the SY 2013 
and SY 2014 4th-grade cohorts
Cognitive attitudes toward school, SY 
2013 and SY 2014, for at-risk 
students in the SY 2012 4th- and 
5th-grade cohort and the SY 2013 
and SY 2014 4th-grade cohorts
Behavioral attitudes toward school, SY 
2013 and SY 2014, for at-risk 
students in the SY 2012 4th- and 
5th-grade cohort and the SY 2013 
and SY 2014 4th-grade cohorts
Student perception of teacher support, SY 2013 and 
SY 2014, for at-risk students in the SY 2012 
4th- and 5th-grade cohort and the SY 2013 and 
SY 2014 4th-grade cohorts
Student perception of parental support, SY 2013 and 
SY 2014, for at-risk students in the SY 2012 
4th- and 5th-grade cohort and the SY 2013 and 
SY 2014 4th-grade cohorts
Engagement 
in school
Attendance, January–October, SY 2013 
for 2012 5th-grade cohort; 
January–November, SY 2014 for 
2012 4th-grade cohort, 2013 4th￾grade cohort, and 2014 4th-grade 
cohort
Performance in math, SY 2013 for the SY 2012 
cohort; SY 2014 for the SY 2012, SY 2013, 
and SY 2014 4th-grade cohorts
Performance in Portuguese language, SY 2013 for 
the SY 2012 cohort; SY 2014 for the SY 2012, 
SY 2013, and SY 2014 4th-grade cohorts
Performance in Tetun language, SY 2013 for the 
SY 2012 cohort; SY 2014 for the SY 2012, SY 
2013, and SY 2014 4th-grade cohorts
Behavior, SY 2013 for the SY 2012 cohort; SY 
2014 for the SY 2012, SY 2013, and SY 2014 
4th-grade cohorts
Dropout Global dropout measure composed of:
Between-grade dropout at start of SY 
2015 for 2013 4th-grade cohort and 
2014 4th-grade cohort
Within-grade dropout at the end of SY 
2014 for the 2012 4th-grade cohort 
and the 2012 5th-grade cohort
Progression from 4th grade to 6th grade or higher 
for the SY 2012 and SY 2013 4th-grade 
cohorts; 5th grade to 6th grade or higher for 
the SY 2012 5th-grade cohort; 4th grade to 
5th grade or higher for the SY 2014 4th-grade 
cohort
* Exploring the impact of SDPP on these additional outcomes of interest is meant to be descriptive in nature, to provide context 
and better understand the impacts on the primary measures by which SDPP’s effectiveness is determined. In these exploratory 
analyses on additional outcomes, SDPP did not adjust statistical significance thresholds for multiple comparisons when presenting 
impacts of the program. 
SY = school year. 
Findings from the School Dropout Prevention Pilot Program Impact Evaluation in Timor-Leste Page 23
Within the teacher outcomes domain, the primary outcome measure is a scale representing teacher 
dropout prevention practices (Table III.C.2). SDPP worked directly with teachers to improve their 
knowledge and practices related to preventing dropout. Teachers also received training on how 
better to deal with students through the after-school enrichment activities program. Since teachers 
inspire and shape student attitudes and behaviors, changes in teacher practices may represent 
catalysts for student change. Changes in student attitudes should then result from these changes in 
teacher attitudes and practices, and so SDPP looks at outcomes within the student attitudes domain. 
The three primary outcomes in this domain are: student emotional, cognitive, and behavioral 
attitudes toward school.36 Changes in student attitudes should then result in changes in student 
engagement in school, including in their attendance, behavior and performance. Low attendance, 
behavior, or performance may indicate that students are less active participants in their own 
education, which could be a sign that students are on their way to dropping out of school. Finally, 
improvements in attendance and in performance should result in a reduction in dropout. Therefore, 
the ultimate goal of SDPP is its own domain, and the primary outcome measure is an indicator of 
whether the student had dropped out at the time of follow-up data collection. 
In addition to the primary outcomes, the evaluation analyzes additional information to provide 
context to the primary analysis and increase understanding of the ways in which the program was 
and was not effective in influencing a particular domain. Table III.C.2 lists these additional 
outcomes, which include teacher attitudes and practices, student perceptions of teachers and 
parents, academic performance, behavior, and grade progression. 
IV. Sampling and data collection 
This report draws on data from three sources: (1) student records collected from schools, 
(2) surveys conducted with at-risk students, and (3) surveys conducted with school administrators 
and teachers teaching 4th, 5th, and 6th grade homeroom, math, and language courses. SDPP 
collected this data at five points in time, collecting data for four school years, from the first 
trimester in the school year that the program was later rolled out (SY 2012) to the school year after 
completion of the program (SY 2015) (Figure II.1). Across the five rounds of data collection, 
SDPP gathered information on the four cohorts used in the analysis, their teachers, and their 
schools.
Across all data collection rounds, SDPP gathered data from 190 schools—97 treatment and 93 
control—that are used in the impact analyses (Table IV.1). Information used in the student 
analyses from school records was gathered for 16,961 students from the SDPP group and 17,615 
students from the control group. Similarly, SDPP analyzed information collected from interviews 
with 3,750 students from the SDPP Program group and 3,581 students from the control group, and 
506 target grade teachers and administrators in the SDPP group and 511 target grade teachers and 
administrators in the control group.37
 
36 SDPP used a multiple-comparison adjustment procedure (the Benjamini-Hochberg method) for student attitudinal 
outcomes to account for the fact that there were three primary outcomes in this domain (Benjamini, Yoav and Yosef 
Hochberg 1995). 
37 Please see Appendix A for further details about data collection. An eligible subset of these total students and teachers 
collected were used in the analysis. 
Findings from the School Dropout Prevention Pilot Program Impact Evaluation in Timor-Leste Page 24
Table IV.1. Study sample sizes 
SDPP group Control group Total
Schools 97 93 190
Student records 16,961 17,615 34,576
Student survey 3,750 3,581 7,331
Teacher survey 506 511 1,017
Administrator survey 185 179 364
Sources: SDPP baseline, follow-up 1, follow-up 2, follow-up 3, and follow-up 4 student records, student questionnaire, and 
teacher self-administered questionnaire. 
Note: Sample sizes refer to the number of unique data points from each data collection source that are used in the impact 
analyses. Data on different cohorts come from different rounds of data collection. Overall sample sizes for each data 
collection effort were larger, since some students were included in the school records that are not eligible to be included 
in the analysis. 
SDPP administered school questionnaires to the school director or deputy director to gather 
information about school characteristics, enrollment, and teacher characteristics during each data 
collection round. SDPP administered teacher questionnaires to directors, deputy directors, and 4th, 
5th, and 6th-grade math, language, and homeroom teachers at the end first trimester in SY 2012, 
in the middle of SY 2013, and at the end of SY 2014. The teacher questionnaire included questions 
related to respondents’ experience and training, awareness of risk factors related to dropout, and 
attitudes toward and practices used with at-risk students. Eligible teachers responded at rates of 81 
percent, 91 percent, and 96 percent (SY 2012, SY 2013, SY 2014, respectively) at each data 
collection point.
During each round of data collection, SDPP obtained official student records to glean information 
on the attendance, school performance and demographics, and enrollment of the four cohorts. 
During three rounds of data collection, at the end of SY 2012, SY 2013, and SY 2014, SDPP also 
interviewed students who were at risk of dropping out of school (based on their baseline 
characteristics) to assess their attitudes about school. A subset of at-risk students from each cohort 
was sampled for interviews and included in the analyses. The actual sampling process in the field 
varied by cohort because of the data available to identify student risk status at the time of sampling. 
Appendix A describes this process in detail. 
As mentioned earlier, to identify at-risk students for this evaluation, SDPP only used data that were 
available in school records for all SDPP Program and control group schools. This is a different 
method than the one used to identify at-risk students via the EWS in SDPP schools. To identify 
students based on school records, the analysis used four analogs of the six at-risk components used 
in Timor-Leste’s EWS at-risk identification process: (1) attendance from the latest month available 
during the previous school year; (2) math exam score in the latest trimester of available the 
previous school year; (3) Tetun exam score in the latest trimester of available the previous school 
year; and (4) grade in behavior in the latest trimester available during the previous school year.38
 
38 For the SY 2012 4th and 5th grade cohort of at-risk students, there was no information available from the last 
trimester of the previous school year at the time of sampling students, so SDPP used 1) March 2012 attendance, (2) 
first trimester SY 2012 math exam score, (3) first trimester SY 2012 Tetun exam score, and (4) first trimester SY 2012 
behavior grade. 
Findings from the School Dropout Prevention Pilot Program Impact Evaluation in Timor-Leste Page 25
At-risk students who were selected for an interview responded to questions about demographics; 
cognitive, behavioral, and emotional attitudes toward school; and perceptions of teachers and 
parents. Sampled students responded at a rate of 86 percent at each data collection point at the end 
of each SY, 2012, SY 2013, and SY 2014. 
V. Characteristics of the sample prior to implementation
Having data on the sample members before they are exposed to the intervention is a crucial element 
of a rigorous impact evaluation; it provides information on the sample’s baseline characteristics 
and allows us to check for equivalence between the treatment and control groups. In the case of 
the SDPP in Timor-Leste, the characteristics of students and teachers gathered before the 
intervention started in schools in September 2012 serves as the reference point for all subsequent 
measurement and analysis.
SDPP Program and control group schools had comparable characteristics at baseline (SY 2012), 
with one characteristic, the mean number of students enrolled in grade 5, that differed between 
study groups by a marginally significant amount (Table V.1, top panel). Most study schools 
maintained school feeding programs throughout the intervention, with a few additional programs 
active in some schools. No schools had other active programs targeting school dropout, which 
strengthens any claims of the SDPP Program’s effectiveness. Seventy-seven percent of schools in 
the SDPP group and 72 percent of schools in the control group are accessible by all types of 
vehicles. The typical SDPP group school enrolled 251 students on average, and the typical control 
group school enrolled 260 students. The average SDPP group school had 41 4th-grade students, 
34 5th-grade students, and 30 6th-grade students, compared with 42, 40, and 33 students in those 
grades in control group schools, respectively. SDPP group schools had an average of 9 teachers, 
compared with 10 in the control group schools. 
SDPP also compared the characteristics of 4th-, 5th-, and 6th-grade teachers in the SDPP Program 
group versus the control group, and found no statistically significant differences (Table V.1, 
bottom panel). The average teacher in both groups was 42 years old. Thirty-one percent of SDPP 
group teachers were female, compared to 34 percent of control group teachers. Secondary 
schooling was the highest level of education for the majority of teachers. Fifty percent of SDPP 
group teachers had less than ten years teaching experience, compared to 53 percent of teachers in 
the control group schools. For both groups, less than 10 percent of teachers had 30 years or more 
of teaching experience. In the SDPP group schools, 45 percent of teachers had received training 
related to at-risk students, compared with 39 percent of control group teachers. Far fewer in each 
study group had received such training within the past year. Teachers scored very high on the 
teacher prevention practice scale: 7.76 and 7.80 (on an 8-point scale) for SDPP and control group 
schools, respectively. 
Findings from the School Dropout Prevention Pilot Program Impact Evaluation in Timor-Leste Page 26
Table V.1. Average school and target grade teacher characteristics prior to intervention (grades 4–6, SY 
2012) (percentage unless indicated otherwise) 
Outcome SDPP group Control group
School characteristics prior to intervention
Offer grades 1 through 6 100.0 100.0
Enrollment (mean number of students)
Grade 4 (target grade) 40.5 41.5
Grade 5 (target grade) 33.7* 39.7
Grade 6 (target grade) 29.8 33.1
Grades 1 through 6 250.8 260.3
Number of teachers 9.3 10.3
Attendance rate at time of headcount (% of students) 
Grade 4 (target grade) 79.7 78.8
Grade 5 (target grade) 81.6 79.9
Grade 6 (target grade) 83.4 82.9
Grades 4, 5, and 6 teacher attendance rate
January 95.9 93.8
February 95.0 94.1
March 94.8 94.9
Active external school programsa
No other active programs 18.6 22.3
School feeding 76.3 72.3
Health or hygiene 21.6 20.2
Other active programs 26.8 20.2
School accessible by all types of vehicles 77.3 72.3
Grade 4–6 math, language, and homeroom teacher characteristics prior to intervention
Age (years) 42.4 42.0
Female 30.9 34.4
Highest level of educationb
University degree or higher 1.1 2.2
Bacharelatoc 20.9 20.4
Teacher training institute/college (SPG) 19.9 17.2
Secondary school 55.7 55.6
Did not complete secondary school 2.5 4.7
Teaching experience overallb
Less than 10 years 50.5 52.6
10 years to less than 20 years 26.4 25.2
20 years to less than 30 years 14.1 15.6
30 years or more 9.0 6.7
Received training related to at-risk students
Ever 45.2 39.4
In the past year 15.4 16.8
Teacher dropout prevention practice scale (range: 1 to 8)d 7.7 7.8
Sample size
Schools 97 94
Teachers 282 279
Sources: SDPP baseline teacher self-administered questionnaire and school questionnaire, May 2012. 
Note: Differences between SDPP and control group means were tested using two-tailed t-tests unless otherwise indicated. The teacher analysis 
accounts for clustering of teachers within schools. Sample sizes for some characteristics may be smaller due to missing responses. 
a
 External programs are those funded and implemented by organizations other than the school system. More than one external program can 
operate in a school. Examples of other programs include teacher training programs and provision of textbooks and other materials. 
b
 Differences between SDPP and control group distributions were tested using a chi-squared test. 
c This is an associate’s degree obtained through an 18 month course of study provided by the Ministry of Education and the Portuguese Cooperation. 
d
 This scale represents the sum of teacher responses to eight items that indicate whether the teacher reports recording daily attendance, 
taking action if the student is absent three days in a month, giving weak students feedback, discussing support for weak students with other 
teachers, developing plans to support weak students, communicating with parents of weak students, meeting with weak students, and being 
willing to come early or stay late to help weak students. 
***/**/* Difference between SDPP and control group means is statistically significant at the .01/.05/.10 level. 
†††/††/† Difference between SDPP and control group distributions is statistically significant at the .01/.05/.10 level. 
Findings from the School Dropout Prevention Pilot Program Impact Evaluation in Timor-Leste Page 27
Students who were in 4th-, 5th- and 6th- grade in SY 2012 were also similar across the SDPP and 
control groups, with no statistically significant differences (Table V.2).39 Forty-nine percent of 
students in both groups were female, and about 16 to 17 percent of students in both groups were 
not appropriately aged for their grade. About 38 to 39 percent of students in both groups were at 
risk of school dropout at baseline. At-risk students had slightly lower attendance and academic 
performance in Math, and lower behavior grades than the full sample. Between 36 and 38 percent 
of at-risk students were rated good or better all four quarters of SY 2012, compared to between 59 
and 61 percent of the full sample. 
Table V.2. Average student characteristics prior to intervention (grades 3–6, SY 2012) (percentages of 
students unless otherwise indicated) 
Full sample At-risk sample
SDPP 
group
Control 
group
SDPP 
group
Control 
group
Demographic characteristics
Female 48.9 48.6 43.5 43.9
Overage for gradea 17.1 16.1 14.5 15.0
Factors related to risk of dropout
Categorized as at-risk based on baseline information, grades 3–
5 39.0 37.5 100.0 100.0
Daily attendance 1st trimester of school year 93.9 94.5 91.1 92.0
Academic performance on 1st trimester exam scores (range 1–
10)
Math 5.8 5.7 5.3 5.2
Portuguese 6.0 6.0 5.5 5.5
Tetun 6.4 6.3 5.9 5.8
Behavior rated good or better during 1st trimester b 61.3 58.7 36.0 37.6
Sample size
Schools 97 94 97 94
Grade 3 Students 4,275 4,248 1,000 960
Grade 4 Students 3,883 3,830 936 912
Grade 5 Students 3,229 3,679 891 868
Grade 6 Students 2,845 3,069 n.a. n.a.
Students Overall 14,232 14,826 2,827 2,740
Sources: SDPP baseline student survey and school records data collection, May 2012. 
Note: Differences between SDPP and control group means were tested using two-tailed t-tests. The analysis accounts for 
clustering of students within schools. Sample sizes for some characteristics may be smaller due to missing responses. 
a
 A student is considered over-age for his or her grade if he or she is two years older than the appropriate age for the grade. 
b
 Behavior rating has a range of 1–10. A rating of “good or better” is equal to a score of 8 or higher. 
***/**/* Difference between SDPP and control group means is statistically significant at the .01/.05/.10 level.
 
39 Assuming regular progression, the 3rd- 4th- and 5th-grade students in SY 2012 would have been in the 4th, 5th, and 
6th grade in SY 2013. 
Findings from the School Dropout Prevention Pilot Program Impact Evaluation in Timor-Leste Page 28
VI. Impacts of SDPP 
The evaluation estimated the SDPP Program’s impacts on teacher outcomes, student attitudes, 
student engagement in school, and school dropout. This chapter discusses the extent to which the 
SDPP Program was successful in improving the primary measures of program effectiveness in 
each of these domains. The assessment of the SDPP Program’s effectiveness focuses on program 
impacts measured as the difference in average outcomes at final follow-up between students and 
teachers randomly assigned to the SDPP group and those randomly assigned to the control group, 
adjusting for baseline characteristics. The impact estimates reported in this study should be 
interpreted as the difference in outcomes that resulted from exposure to SDPP. For example, an 
“X” percentage point favorable impact on school dropout indicates that, on average, the dropout 
rate under SDPP is “X” percentage points lower than it would have been under business-as-usual 
operations. In some places, to provide additional context we also present percentage increases or 
decreases in the primary outcomes across the treatment and control groups. These “percentage 
changes” should not be interpreted as the percentage “change” that might be calculated in a pre￾post measure or baseline/endline change, but rather, the increase or decrease in the treatment 
group’s outcome measure in relation to the control group at endline. 
The discussion is organized by outcome domain, as follows: (1) teacher behavior and attitudinal 
outcomes, (2) student attitudes, (3) student engagement in school, and (4) school dropout. Impact 
findings for primary and additional measures of program effectiveness are presented with bar 
charts corresponding to the mean outcome level by random assignment group status. Differences 
that are statistically different than zero are indicated with asterisks. 
Statistical significance
Estimates of the impact of the SDPP Program are based on differences in average outcomes for SDPP 
and control group students and teachers. In interpreting these estimates, it is important to evaluate 
whether they are sufficiently large that it is unlikely that the difference is due to chance (indicating 
that the SDPP Program did have an impact). With this in mind, statistical tests were conducted to 
assess whether each impact is significantly different than zero. Impacts estimates are described as 
statistically significant if there is less than a 5 percent probability that it is due to chance (and not to 
the SDPP Program). Impact estimates are described as marginally significant if the probability that 
it is due to chance (and not to the SDPP Program) is between 5 and 10 percent. In tables and figures, 
the statistically significant impacts at the 1 percent, 5 percent, and 10 percent levels are denoted with 
asterisks as ***, ** or *.
The chance (1%, 5% or 10%) that the reported findings are falsely reporting an impact increases as 
additional tests are conducted. Therefore, caution should be used when interpreting the meaning of 
our exploratory analyses of additional outcomes and subgroups because we do not correct for the 
total number of comparisons being made. Individual tests of these additional contrasts of program 
effects for other subgroups are provided as additional context for the main findings. 
Findings from the School Dropout Prevention Pilot Program Impact Evaluation in Timor-Leste Page 29
A. Impacts on teacher outcomes
Teachers can shape the attitudes and actions of 
students through their own actions. As shown in the 
theory of change (Figure III.A.1), a teacher’s 
actions might represent the first level of change in 
reducing school dropout. SDPP worked directly 
with teachers to improve their knowledge and 
awareness of dropout, training them to identify at￾risk students and to work with those students and 
their families to strengthen their attachment to 
school. Teachers were taught to use the EWS to 
improve their dropout prevention practices, and 
received training on how to better deal with 
students through the complementary enrichment 
program, which, in turn, should improve student 
attitudes and behaviors. Thus, changes in teacher 
actions can be early indicators of changes in student 
behaviors. 
This section discusses the impacts of SDPP on teachers’ knowledge, attitudes, and practices related 
to dropout. 
1. Impact on teacher take-up of dropout prevention practices 
The primary measure of SDPP’s influence on teacher outcomes is the teacher dropout prevention 
scale. This scale combines responses to eight questions posed to homeroom teachers, who were 
the focus of the EWS intervention, and to teachers of math and language. The questions focus on 
teacher behavior that might help at-risk students succeed in school, such as whether the teacher 
records daily attendance and whether he or she takes action if the student is absent three days in a 
month. (See Appendix C for details on the creation of the scale.) 
Teacher dropout prevention practices scale (primary measure)
Teachers responded to a questionnaire. Scoring is based on an 8-point scale corresponding to eight 
survey items that indicate whether the teacher has adopted dropout prevention practices. The items 
include:
 Recording daily attendance 
 Taking action if the student is absent three days in a month 
 Giving weak students feedback 
 Discussing support for weak students with other teachers 
 Developing plans to support weak students 
 Communicating with parents of weak students 
 Meeting with weak students 
 Willingness to arrive early or stay late to help weak students
Teacher Outcomes
Primary measure of program effectiveness
 Teacher take-up of dropout 
prevention practices 
Additional measures
 Teachers’ sense of self-efficacy in 
dealing with dropout 
 Teachers’ sense of responsibility for 
addressing dropout
 Administrators’ dropout prevention 
practices, self-efficacy and sense of 
responsibility
Findings from the School Dropout Prevention Pilot Program Impact Evaluation in Timor-Leste Page 30
SDPP had a statistically significant impact on teachers’ practices related to dropout 
prevention (Figure VI.A.1). SDPP and control group teachers both scored 7.87 and 7.72, 
respectively, on the 8-point dropout prevention scale.40 Although both groups of teachers scored 
high, this represents a two percent improvement for the SDPP group relative to the control group. 
Figure VI.A.1. SDPP Program impacts on teacher dropout prevention practices (SY 2013 and SY 2014) 
Sources: SDPP baseline and follow-up teacher self-administered questionnaires and school records data collection, May 2012, 
May 2013, and September 2014. 
Note: The analysis is based on 4th-, 5th-, and 6th-grade homeroom, math, and language teachers during SY 2013 and SY 
2014. during SY 2013 and SY 2014. The sample includes 412 teachers for the SDPP group and 406 teachers for the 
control group. 
 Differences between SDPP and control group means were tested using two-tailed t-tests. Mean values are adjusted for 
baseline characteristics. The analysis accounts for clustering of teachers within schools and school-year and grade fixed 
effects. For a tabular presentation of these findings, see Appendix Table H.5. 
***/**/* Impact estimate is statistically significant at the .01/.05/.10 level. 
Although a higher percentage of SDPP group homeroom teachers remained homeroom teachers 
for the duration of the intervention (44 percent of SY2012 homeroom teachers remained 
homeroom teachers in SY 2014) than for control group homeroom teachers (32 percent of SY 
2012 remained in SY 2014), the impact on teacher practices may have been greater had a larger 
percentage of SDPP benefitted from two full years of SDPP training and intervention 
implementation. 
 
40 Higher scores indicate better practices to prevent dropout. 
7.87*** 7.72
0
1
2
3
4
5
6
7
8
Score on an eight-point scale
SDPP group Control group
Findings from the School Dropout Prevention Pilot Program Impact Evaluation in Timor-Leste Page 31
SDPP also explored the effects of the program on additional subgroups that were not directly 
targeted by SDPP but are still of interest.41 These comparisons provide useful context and can 
suggest pathways through which the program might be working. 
SDPP examined impacts on the primary teacher outcome, the teacher dropout prevention practices 
scale, for four pairs of subgroups (Figure VI.A.2). In particular, SDPP assessed impacts separately 
by the teacher’s gender, the percentage of at-risk students in the school, the remoteness of the 
school (a school is defined as “remote” is it is not accessible by all types of vehicles), and the 
teacher’s full-time teaching status. Consistent with SDPP’s main findings, the program’s impacts 
on the teacher dropout prevention practices scale were statistically significant or marginally 
significant in the following subgroups: male teachers, teachers in schools with a low percentage 
of at-risk students, teachers in both remote and not remote schools, and full-time teachers. There 
were no statistically significant impacts on the teacher dropout prevention practices scale between 
these subgroups.42
Figure VI.A.2. SDPP Program impacts on teacher dropout prevention practices, by subgroup (SY 2013 and 
SY 2014) 
Sources: SDPP baseline and follow-up school records data collection, May 2012, May 2013, March 2014, September 2014, and 
January 2015; baseline and follow-up teacher self-administered questionnaires and school records data collection; May 
2012, May 2013, and September 2014. 
Note: The analysis is based on 4th-, 5th-, and 6th-grade homeroom, math, and language teachers during SY 2012, SY 2013, 
and SY 2014. 
 Differences between SDPP and control group means were tested using two-tailed t-tests. Mean values are adjusted for 
baseline characteristics. The analysis accounts for clustering of teachers within schools and school-year and grade fixed 
effects. Because these subgroup analyses are exploratory, statistical significance thresholds were not adjusted for 
multiple comparisons. 
***/**/* Impact estimate is statistically significant at the .01/.05/.10 level.
 
41 As described in Section III, because these analyses are exploratory, results were not adjusted for multiple 
comparisons, despite the large number of comparisons. 
42 Differences between impacts within each subgroup type were tested. Within each subgroup grouping, there were no differences 
in impacts between females and males, between full time and not full time employees, between teachers in schools with a high 
percentage of at-risk students and teachers in schools with a low percentage of at-risk students, and between teachers in remote 
schools compared to teacher not in remote schools on teacher dropout prevention practices.
0.15 0.15***
0.14***
0.18
0.1
0.18***
0.14**
0.18*
0
0.05
0.1
0.15
0.2
0.25
Female Male Full-time Not full-time High % At￾risk
Low % At￾risk
Remote
schools
Not remote
schools
Difference in SDPP and control group 
means in scores on an eight-point scale
SDPP group
Findings from the School Dropout Prevention Pilot Program Impact Evaluation in Timor-Leste Page 32
Teachers Work to Prevent Dropout in Timor-Leste
Fernando dos Santos Gonsalves teaches 
fourth grade at Bazartete, a rural school in the 
Liquica district, high in the mountains of 
Timor-Leste. In the past, many of his students 
frequently missed school to help their parents 
work in the fields. But things have changed 
since his school has been a part of the Student 
Dropout Prevention Pilot Program (SDPP). 
As part of the program, the school personnel, 
families and community leaders have learned 
about the importance of staying in school.
Fernando and other teachers at his school 
have been taught to implement an Early 
Warning System (EWS) to track key 
indicators that signal a student is at risk of 
dropping out of school. Being absent from 
school is one of those warning signs. As part 
of the SDPP Program school personnel work with community leaders to increase awareness 
about school dropout and to strengthen the school and community partnership. One of the things 
they did to strengthen the partnership was to create “Stay in School” committees that reach out 
to students’ families about their children’s school attendance.
When one of his students, Domingo, was absent from school a second day in one week, 
Fernando followed the EWS steps he was taught. He notified the school director and together 
they asked a “Stay in School” committee volunteer to hand deliver a Parent Notification Card 
to Domingo’s parents. During the visit the volunteer talked to his parents to find out why he was 
absent and remind them they needed to regularly send their son to school.
Fernando believes that the increased awareness teachers have to monitor attendance and work
with the “Stay in School” committees has made a big difference. “After the ‘Stay in School’ 
Community Group started working with the school, the local leaders held meetings, and really 
began to get people to think about the importance of keeping students in school.” 
SDPP also worked with the school to make classrooms more inviting by working with teachers 
to provide individualized attention to at-risk students and use interactive student-centered 
teaching activities in their classrooms to make learning more interesting. Now, Domingo doesn’t 
want to miss school. “Last year I didn’t like school because it wasn’t exciting. But now I like 
school… there are many interesting activities.”
Fernando sees a difference not only in Domingo’s behavior but in his attitudes toward school as 
well. “Domingo used to be very quiet and shy. By making sure that he can join in all of the 
activities, his confidence has really grown. It has had a result on his grades too—especially in 
Tetum. And because we are making school more interesting and monitoring attendance we have 
gone from 30 Parent Notification Cards last year to only 2 cards this year.”
A teacher who works to prevent dropout in Timor-Leste
Findings from the School Dropout Prevention Pilot Program Impact Evaluation in Timor-Leste Page 33
2. Impact on additional teacher outcomes 
Besides the primary measure of the program’s effect on teachers’ dropout-related practices, SDPP 
used two other scales to examine teachers’ sense of self-efficacy and sense of responsibility for at￾risk students. SDPP provides these findings here because they may be of interest and could help 
paint a more complete picture of teachers’ experiences.43 
a. Teacher sense of self-efficacy 
An additional indicator of the SDPP Program’s effectiveness in addressing dropout is influencing 
teachers’ belief that they are capable of affecting factors associated with student dropout, such as 
poor behavior, disinterest in class, and absenteeism. This outcome is measured according to the 
teacher sense of self-efficacy scale, which SDPP adapted from Tschannen-Moran and Hoy (2001). 
The questions focus on teacher beliefs that they can help at-risk students succeed in school, such 
as whether the teacher thinks he or she can encourage students to value learning and provide 
assistance to families in helping their child succeed in school.
Responses to the questions about teachers’ self-efficacy ranged from 1 (nothing) to 5 (a great deal) 
and were compiled into a scale in which higher values corresponded to a higher sense of self￾efficacy. 
The SDPP Program had a statistically significant impact on teachers’ sense of self-efficacy 
(Figure VI.A.3). Teachers in the SDPP group schools had an average score of 4.16 on the sense 
of self-efficacy scale, compared with 4.08 for teachers in the control group schools. This difference 
was statistically significant. This represents a two percent improvement for the SDPP group 
relative to the control group. This means that teachers in both SDPP and control schools felt they 
 
43 As described in Section III, because these analyses are exploratory, results were not adjusted for multiple 
comparisons, despite the large number of comparisons being made. 
Teacher sense of self-efficacy scale 
Teachers were asked how much they can do to prevent or address the following 12 factors 
associated with dropout:
 Disruptive behavior in the classroom
 Motivation of students with low interest in school
 Encouragement of students to believe they are capable of succeeding in school
 Help students value learning
 Make lessons interesting for students 
 Enforcing classroom rules
 Encouragement of active participation among students not engaged
 Identification of students needing extra support
 Student attendance
 Modification of teaching and learning activities to help weak or poorly performing students
 Assistance to families in helping their children do well in school
 Help for poor-performing students to do better in school
Findings from the School Dropout Prevention Pilot Program Impact Evaluation in Timor-Leste Page 34
could do “quite a bit” to respond to factors associated with dropout, but teachers in SDPP schools 
felt they could do slightly more. 
Figure VI.A.3. SDPP Program impact on teacher sense of self-efficacy (SY 2013 and SY 2014) 
Source: SDPP baseline and follow-up teacher self-administered questionnaires and school records data collection, May 2012, 
May 2013, and September 2014. 
Note: The analysis is based on 4th-, 5th-, and 6th-grade homeroom, math, and language teachers during SY 2013 and SY 
2014. 
 Differences between SDPP and control group means were tested using two-tailed t-tests. Mean values are adjusted for 
baseline characteristics. The analysis accounts for clustering of teachers within schools and school-year and grade fixed 
effects. Because these are additional outcomes and the analyses are exploratory, statistical significance thresholds were 
not adjusted for multiple comparisons. 
***/**/* Impact estimate is statistically significant at the .01/.05/.10 level. 
b. Teacher sense of responsibility 
In addition to teacher sense of self-efficacy scale, SDPP developed a scale to measure teachers’ 
sense of responsibility for students at risk of dropping out. The questions in this scale focus on 
teachers’ opinions about ways to prevent students from dropping out of school 
4.16** 4.08
1
2
3
4
5
Score on a
five-point scale
SDPP group Control group
Teacher sense of responsibility for at-risk students scale
This scale is based on teacher agreement with five statements about at-risk students. The scale 
items are:
 Students at risk of dropping out of school should work harder.
 Little can be done by the teacher or school to help students at risk of dropping out.
 If a student is at risk of dropping out, it is mainly the fault of the parent/guardian or family.
 At-risk students face too many challenges to succeed in school.
 At-risk students need more help than teachers have time or resources to provide. 
Findings from the School Dropout Prevention Pilot Program Impact Evaluation in Timor-Leste Page 35
Responses for this scale ranged from 1 (strongly agree) to 4 (strongly disagree). The scale score is 
the mean of the five items. Higher values correspond to a higher sense of teacher responsibility for 
at-risk students. 
Teachers in both research groups tended to have a low sense of responsibility for at-risk 
students (Figure VI.A.4). SDPP had no impact on teachers’ sense of responsibility for at-risk 
students. The average score for teachers in the SDPP and control schools was 1.89 (out of 4), 
indicating that teachers tended to agree that they bore little responsibility for at-risk students. It is 
interesting to note that teachers feel they are able to greatly influence many of the factors associated 
with dropout (self-efficacy), but they do not feel particularly responsible for doing so, as shown 
by the sense of responsibility scale. SDPP presents findings for further additional teacher outcomes 
as well as for administrators in Appendix G. 
Figure VI.A.4. SDPP Program impact on teacher sense of responsibility (SY 2013 and SY 2014) 
Source: SDPP baseline and follow-up teacher self-administered questionnaires and school records data collection, May 2012, 
May 2013, and September 2014. 
Note: The analysis is based on 4th-, 5th-, and 6th-grade homeroom, math, and language teachers during SY 2013 and SY 
2014. 
 Differences between SDPP and control group means were tested using two-tailed t-tests. Mean values are adjusted for 
baseline characteristics. The analysis accounts for clustering of teachers within schools and school-year and grade fixed 
effects. Because these are additional outcomes and the analyses are exploratory, statistical significance thresholds were 
not adjusted for multiple comparisons. 
***/**/* Impact estimate is statistically significant at the .01/.05/.10 level. 
3. Impact on additional school administrator outcomes 
Impacts were also estimated for school administrators’ dropout-prevention practices, sense of self￾efficacy, and sense of responsibility for at-risk students. Each of these outcomes was measured in 
the same way as the teacher outcomes. 
1.89 1.89
1
2
3
4
Score on a
four-point scale
SDPP group Control group
Findings from the School Dropout Prevention Pilot Program Impact Evaluation in Timor-Leste Page 36
a. Administrator dropout prevention practices 
The SDPP Program did not have a statistically significant impact on administrators’ dropout 
prevention practices in Tajikistan (7.85 versus 7.74) (Figure VI.A.5). The average score for 
both groups was quite high, which left little room for improvement. SDPP did appear to have a 
positive effect on school administrator knowledge about dropout. Administrators in SDPP schools 
identified an average of 41 percent of a list of 8 risk factors for school dropout, compared to control 
school administrators, who only identified an average of 33 percent of the risk factors. The 
difference between SDPP and control school administrators was significant. 
Figure VI.A.5. SDPP’s impact on administrators’ dropout prevention practices scale (SY 2013 and SY 
2014) 
Source: SDPP baseline and follow-up teacher self-administered questionnaires and school records data collection, May 2012, 
May 2013, and September 2014. 
Note: The analysis is based on administrators during SY 2013 and SY 2014. The sample includes 185 administrators for the 
SDPP group and 176 administrators for the control group. 
 Differences between SDPP and control group means were tested using two-tailed t-tests. Mean values are adjusted for 
baseline characteristics. The analysis accounts for clustering of administrators within schools and school-year and grade 
fixed effects. Because these are additional outcomes and the analyses are exploratory, statistical significance thresholds 
were not adjusted for multiple comparisons. 
***/**/* Impact estimate is statistically significant at the .01/.05/.10 level. 
b. Administrator sense of self-efficacy 
Contrary to the findings for teachers’ sense of self-efficacy, there was no statistically 
significant impact on administrators’ sense of self-efficacy (Figure VI.A.6). 
7.85 7.74
0
1
2
3
4
5
6
7
8
Score on an eight-point scale
SDPP group Control group
Findings from the School Dropout Prevention Pilot Program Impact Evaluation in Timor-Leste Page 37
Figure VI.A.6. SDPP Program impact on administrators’ sense of self-efficacy scale (SY 2013 and SY 
2014) 
Source: SDPP baseline and follow-up teacher self-administered questionnaires and school records data collection, May 2012, 
May 2013, and September 2014. 
Note: The analysis is based on administrators during SY 2013 and SY 2014. The sample includes 185 administrators for the 
SDPP group and 176 administrators for the control group. 
 Differences between SDPP and control group means were tested using two-tailed t-tests. Mean values are adjusted for 
baseline characteristics. The analysis accounts for clustering of administrators within schools and school-year and grade 
fixed effects. Because these are additional outcomes and the analyses are exploratory, statistical significance thresholds 
were not adjusted for multiple comparisons. 
***/**/* Impact estimate is statistically significant at the .01/.05/.10 level. 
c. Administrator sense of responsibility 
Consistent the results for teachers, SDPP had no statistically significant impact on 
administrator sense of responsibility (Figure VI.A.7). 
4.16 4.13
1
2
3
4
5
Score on a five-point scale
SDPP group Control group
Findings from the School Dropout Prevention Pilot Program Impact Evaluation in Timor-Leste Page 38
Figure VI.A.7. SDPP Program impact on administrators’ sense of responsibility for at-risk students scale 
(SY 2013 and SY 2014) 
Source: SDPP baseline and follow-up teacher self-administered questionnaires and school records data collection, May 2012, 
May 2013, and September 2014. 
Note: The analysis is based on administrators during SY 2013 and SY 2014. The sample includes 185 administrators for the 
SDPP group and 176 administrators for the control group. 
 Differences between SDPP and control group means were tested using two-tailed t-tests. Mean values are adjusted for 
baseline characteristics. The analysis accounts for clustering of administrators within schools and school-year and grade 
fixed effects. Because these are additional outcomes and the analyses are exploratory, statistical significance thresholds 
were not adjusted for multiple comparisons. 
***/**/* Impact estimate is statistically significant at the .01/.05/.10 level.
1.93 1.93
1
2
3
4
Score on a four-point scale
SDPP group Control group
Findings from the School Dropout Prevention Pilot Program Impact Evaluation in Timor-Leste Page 39
Teachers Adopt Dropout Prevention Habits
It’s not easy to change teacher attitudes and behaviors. But when teachers see how important 
new approaches are, and when they learn strategies that help them improve, it can make a 
significant difference for students in their classrooms.
Julio Amaral, an SDPP Field Officer in Manatuto, Timor-Leste, has seen this kind of change. 
“In the past, teachers did not understand the importance of regular school attendance, and 
students would miss school, or leave early whenever they felt like it. If a student’s marks were 
not very good, the teachers were not particularly bothered. But with this program, teachers are 
now working hard to check on students’ attendance on a daily basis. This means that teachers 
now understand the importance of daily attendance for their students.”
Teachers who recognize the 
importance of attendance have 
become a positive force for both 
their own students and their fellow 
teachers. 
Julio continues: “The change that 
I have noticed is that now our 
teachers interact with their 
students and pay attention to 
them. Therefore, they act as a 
good model for the students and 
other teachers to follow.” The 
teachers also realize they need to 
change what happens when the 
students do come to school – they 
need to make classrooms more 
inviting and learning more 
interesting. 
These changes in teachers have
had particular impact on Tadeo, a 4th grade student in School Maun Fahe. When Tadeo missed 
three days during the first trimester, his parents received a notification card. His teacher then 
met with his parents to identify ways to address his attendance issues and help him do better in 
school. Since the visit Tadeo has missed school only once, and his Tetum exam marks have 
risen from a 5 to a 7 out of 10. 
Julio notes that teachers want to continue using the approaches they learned from SDPP because 
they realize these strategies are making a difference in their classroom. SDPP has helped to lay 
the groundwork for teachers to develop ideas on their own that support and guide students to do 
better and stay in school. Julio notes with satisfaction the positive changes he has seen in his 
visits to schools:
“Last week I visited the schools where I had worked with the teachers; they were still checking 
attendance and creating lively activities for students – even though I was not there to support 
them. This means that teachers are able to continue using dropout prevention approaches on 
their own in the future.”
Teachers coached by SDPP field officers feel they can continue to 
use dropout prevention approaches on their own.
Findings from the School Dropout Prevention Pilot Program Impact Evaluation in Timor-Leste Page 40
B. Impacts on at-risk students’ attitudes toward school 
The SDPP theory of change (Figure III.A.1) 
suggests that a change in teacher knowledge and 
practices could lead to a change in student 
attitudes toward school. Observed impacts in 
teacher dropout prevention practices, together 
with the EWS and ECA program, could translate 
to a change in student attitudes. SDPP captured 
these attitudes by surveying students whom the 
analysis identified—via the available baseline 
data as described in Section IV—as being at risk 
of dropout. 
This section presents findings on the impact of the SDPP Program on attitudes of at-risk students. 
1. Impact on primary measures of student attitudes toward school 
As described earlier, SDPP used three measures of student attitudes to determine the effectiveness 
of the program: emotional, cognitive, and behavioral attitudes toward school. 44
The emotional attitudes toward school scale measures how a student feels toward school. The 
measure captures the percentage of responses to questions with which a student agreed. A higher 
percentage represents more positive attitudes towards school. (See Appendix D for details on the 
construction of the scales.)
There were no statistically significant impacts on emotional attitudes toward school; SDPP 
students agreed to 85.0 percent of the questions, compared with control students, who agreed 
with 85.9 and percent of the questions (Figure VI.B.1).
 
44 SDPP constructed the three measures of student attitudes from responses to a survey administered to multiple 
cohorts of students identified as at risk based on their pre-intervention characteristics. The survey was developed 
through developing and testing new questions and adapting existing student engagement, cognitive and behavioral 
attitudes surveys (see Fredericks, Jennifer A., Phyllis Blumenfeld, Jeanne Friedel, and Alison Paris, 2005; and Finlay, 
Krystina A. 2006.). For more details on the survey, see Appendix A. 
At-risk student attitude toward school 
outcomes
Primary measure of program effectiveness
 Emotional, cognitive, and behavioral 
attitudes toward school 
Additional outcomes:
 Students' perceptions of teacher support
 Students' perceptions of parental 
support
Emotional attitudes toward school
Students identified as being at risk of dropping out of school responded to a questionnaire. This scale 
is based on student responses to the following six survey items related to how students feel about 
school:
 School is a fun place to be.
 There are teachers I can talk to.
 I participate in school activities after school.
 I enjoy participating in class activities.
 I look forward to school.
 A tutoring program would help me with my studies.
Findings from the School Dropout Prevention Pilot Program Impact Evaluation in Timor-Leste Page 41
Figure VI.B.1. SDPP Program impacts on at-risk students’ emotional attitudes toward school (SY 2013 
and SY 2014) 
Sources: SDPP baseline and follow-up school records data collection, May 2012, May 2013, March 2014, September 2014, and 
January 2015; baseline and follow-up student surveys, May 2013 and September 2014. 
Note: The analysis is based on SY 2012 4th- and 5th-grade at-risk students, and SY 2013 and 2014 4th-grade at-risk 
students. The sample includes 3,226 at-risk students for the SDPP group and 3,006 at-risk students for the control group. 
 Differences between SDPP and control group means were tested using two-tailed t-tests. Mean values are adjusted for 
baseline characteristics. The analysis accounts for clustering of students within schools and cohort fixed effects. Impacts 
were adjusted for multiple comparisons using the Benjamini-Hochberg method. For a tabular presentation of these 
findings, see Appendix Table H.5. 
***/**/* Impact estimate is statistically significant at the .01/.05/.10 level. 
SDPP targeted students who were at risk of school dropout. However, there are additional 
subgroups of students for whom the understanding the effects of SDPP on student attitudes would 
provide useful further context in understanding the intervention and the potential pathways through 
which it has effects. Here, SDPP explores the impacts of the program on the attitudes of at-risk 
students using additional subgroups of interest (Figure VI.B.2).45
Consistent with SDPP’s main findings, the program’s impacts on emotional attitudes toward 
school were not statistically significant for several subgroups. The impact on at-risk students 
in schools that are not remote was marginally significant. 
 
45 As described in Section III, because these analyses are exploratory, result were not adjusted for multiple 
comparisons, despite the large number of comparisons. 
85.0 85.9
0.0
10.0
20.0
30.0
40.0
50.0
60.0
70.0
80.0
90.0
100.0
Percentage of scale items with which 
student agreed
SDPP group Control group
Findings from the School Dropout Prevention Pilot Program Impact Evaluation in Timor-Leste Page 42
Figure VI.B.2. SDPP Program impacts on students’ emotional attitudes toward school, by subgroup 
Sources: SDPP baseline and follow-up school records data collection, May 2012, May 2013, March 2014, September 2014, and 
January 2015; baseline and follow-up student surveys, May 2013 and September 2014. 
Note: The analysis is based on SY 2012 4th and 5th grade at-risk students, and SY 2013 and 2014 4th grade at-risk students. 
Sample sizes are 2,865 (female) and 3,367 (male), 598 (overage) and 5,198 (not overage), 2,223 (schools with high 
percentage of at-risk students) and 4,009 (schools with low percentage of at-risk students), and 1,339 (not remote 
schools) and 4,893 (remote schools).
 Differences between SDPP and control group means were tested using two-tailed t-tests. Mean values are adjusted for 
baseline characteristics. The analysis accounts for clustering of students within schools and cohort fixed effects. Because 
these subgroup analyses are exploratory, statistical significance thresholds were not adjusted for multiple comparisons. 
There were no statistically significant differences between subgroup impact estimates. 
***/**/* Impact estimate is statistically significant at the .01/.05/.10 level. 
The cognitive attitudes toward school scale measures how a student thinks about school and school 
work. The measure captures the percentage of responses to questions with which a student agreed. 
A higher score represents more positive attitudes toward school. 
-0.5
-1.2
0.5
-0.9
-1.6
-0.5 -0.2
-3.1*
-4
-3
-2
-1
0
1
2
3
4
Female Male Overage Not overage High % At￾risk
Low % At￾risk
Remote
schools
Not remote
schools
Percentage point difference in SDPP 
and control group percentage of scale 
items with which student agreed
SDPP group
Cognitive attitudes toward school
Students identified as being at risk of dropping out of school responded to a questionnaire. This scale 
is based on student responses to the following nine survey items related to how students think about 
school:
 I will complete the grade I'm in.
 Completing the grade will be useful to my family and me.
 Missing school affects my performance in school.
 Doing homework helps me do well in school.
 I am interested in the work I get to do in my classes.
 I check my school work for mistakes.
 I need extra help with my studies or homework.
 I have difficulty paying attention in school.
 I try to do my best at school, even if it is not perfect.
Findings from the School Dropout Prevention Pilot Program Impact Evaluation in Timor-Leste Page 43
There were no impacts on cognitive attitudes toward school; SDPP students agreed with 71.3 
percent of the questions, compared with control students who agreed with 71.7 percent of 
the questions (Figure VI.B.3). 
Figure VI.B.3. SDPP Program impacts on at-risk students’ cognitive attitudes toward school (SY 2013 
and SY 2014) 
Sources: SDPP baseline and follow-up school records data collection, May 2012, May 2013, March 2014, September 2014, and 
January 2015; baseline and follow-up student surveys, May 2013 and September 2014. 
Note: The analysis is based on SY 2012 4th- and 5th-grade at-risk students, and SY 2013 and 2014 4th-grade at-risk 
students. The sample includes 3,226 at-risk students for the SDPP group and 3,006 at-risk students for the control group. 
 Differences between SDPP and control group means were tested using two-tailed t-tests. Mean values are adjusted for 
baseline characteristics. The analysis accounts for clustering of students within schools and cohort fixed effects. Impacts 
were adjusted for multiple comparisons using the Benjamini-Hochberg method. For a tabular presentation of these 
findings, see Appendix Table H.5. 
***/**/* Impact estimate is statistically significant at the .01/.05/.10 level. 
As with emotional attitudes toward school, we explore the impacts of SDPP on the cognitive 
attitudes of at-risk students using additional subgroups of interest. Consistent with our main 
findings, there were no statistically significant impacts for any subgroup on cognitive 
attitudes toward school (Figure VI.B.4). 
71.3 71.7
0.0
10.0
20.0
30.0
40.0
50.0
60.0
70.0
80.0
90.0
100.0
Percentage of scale items with which student 
agreed
SDPP group Control group
Findings from the School Dropout Prevention Pilot Program Impact Evaluation in Timor-Leste Page 44
Figure VI.B.4. SDPP Program impacts on students’ cognitive attitudes toward school, by subgroup (SY 
2013 and SY 2014) 
Sources: SDPP baseline and follow-up school records data collection, May 2012, May 2013, March 2014, September 2014, and 
January 2015; baseline and follow-up student surveys, May 2013 and September 2014. 
Note: The analysis is based on SY 2012 4th and 5th grade at-risk students, and SY 2013 and 2014 4th grade at-risk students. 
Sample sizes are 2,865 (female) and 3,367 (male), 598 (overage) and 5,198 (not overage), 2,223 (schools with high 
percentage of at-risk students) and 4,009 (schools with low percentage of at-risk students), and 1,339 (not remote 
schools) and 4,893 (remote schools).
 Differences between SDPP and control group means were tested using two-tailed t-tests. Mean values are adjusted for 
baseline characteristics. The analysis accounts for clustering of students within schools, and cohort fixed effects. 
Because these subgroup analyses are exploratory, statistical significance thresholds were not adjusted for multiple 
comparisons. There were no statistically significant differences between subgroup impact estimates. 
***/**/* Impact estimate is statistically significant at the .01/.05/.10 level. 
The behavioral attitudes toward school scale measures how a student reports behaving in relation 
to school activities. The measure captures the percentage of responses to questions comprising the 
scale with which a student agreed. A higher score represents more positive attitudes toward school. 
-0.3
-0.5 -0.3 -0.3
0.6
-1.0
0.1
-2.1 -2.5
-2
-1.5
-1
-0.5
0
0.5
1
1.5
2
2.5
Female Male Overage Not overage High % At￾risk
Low % At￾risk
Remote
schools
Not remote
schools
Percentage point difference in SDPP and 
control group percentage of scale items with 
which student agreed
SDPP group
Behavioral attitudes toward school
Students identified as being at risk of dropping out of school responded to a questionnaire. This scale 
is based on student responses to the following 10 survey items related to how students act at school:
 I have thought about dropping out.
 I attend school regularly.
 I reach school on time.
 I stay home from school even if I am not sick.
 I skip classes during school.
 I skip school or miss classes without telling my parents.
 I do the homework assigned to me.
 I follow the rules at school.
 I get in trouble at school.
 I have difficulty getting along with other students.
Findings from the School Dropout Prevention Pilot Program Impact Evaluation in Timor-Leste Page 45
There was a statistically significant impact on behavioral attitudes toward school (Figure 
VI.B.5). At-risk students in SDPP schools agreed with 79.5 percent of the questions, compared 
with control school students, who agreed with 76.3 percent. This represents a 4.2 percent 
improvement for the SDPP group relative to the control group. 
Figure VI.B.5. SDPP Program impacts on at-risk students’ behavioral attitudes toward school (SY 2013 
and SY 2014) 
Sources: SDPP baseline and follow-up school records data collection, May 2012, May 2013, March 2014, September 2014, and 
January 2015; baseline and follow-up student surveys, May 2013 and September 2014. 
Note: The analysis is based on SY 2012 4th- and 5th-grade at-risk students, and SY 2013 and 2014 4th-grade at-risk 
students. The sample includes 3,226 at-risk students for the SDPP group and 3,006 at-risk students for the control group. 
 Differences between SDPP and control group means were tested using two-tailed t-tests. Mean values are adjusted for 
baseline characteristics. The analysis accounts for clustering of students within schools and cohort fixed effects. Impacts 
were adjusted for multiple comparisons using the Benjamini-Hochberg method. For a tabular presentation of these 
findings, see Appendix Table H.5. 
***/**/* Impact estimate is statistically significant at the .01/.05/.10 level. 
Consistent with estimates overall, the SDPP Program impact on behavioral attitudes toward 
school was statistically significant for several subgroups (Figure VI.B.6). The only subgroup 
that did not have a statistically significant or marginally significant impact was for at-risk students 
in schools with a low percentage of at-risk students.46
 
46 Differences between impacts for each subgroup were tested. There was a statistically significant difference between 
the impact estimates for over-age and not over-age students and between the impact estimates for students in schools 
with a high percentage of at-risk students and students in schools with a low percentage of at-risk students. 
79.5*** 76.3
0.0
10.0
20.0
30.0
40.0
50.0
60.0
70.0
80.0
90.0
100.0
Percentage of scale items with which 
student agreed
SDPP group Control group
Findings from the School Dropout Prevention Pilot Program Impact Evaluation in Timor-Leste Page 46
Figure VI.B.6. SDPP Program impacts on students’ behavioral attitudes toward school, by subgroup (SY 
2013 and SY 2014) 
Sources: SDPP baseline and follow-up school records data collection, May 2012, May 2013, March 2014, September 2014, and 
January 2015; baseline and follow-up student surveys, May 2013 and September 2014. 
Note: The analysis is based on SY 2012 4th and 5th grade at-risk students, and SY 2013 and 2014 4th grade at-risk students. 
Sample sizes are 2,865 (female) and 3,367 (male), 598 (overage) and 5,198 (not overage), 2,223 (schools with high 
percentage of at-risk students) and 4,009 (schools with low percentage of at-risk students), and 1,339 (schools below 
median distance to district capital) and 4,893 (schools at or above median distance to district capital). 
 Differences between SDPP and control group means were tested using two-tailed t-tests. Mean values are adjusted for 
baseline characteristics. The analysis accounts for clustering of students within schools and cohort fixed effects. Because 
these subgroup analyses are exploratory, statistical significance thresholds were not adjusted for multiple comparisons. 
***/**/* Impact estimate is statistically significant at the .01/.05/.10 level. 
†††/††/† Statistically significant difference between the subgroup impact estimates at the .01/.05/.10 level
3.4*** 3.1***
6.8***
3***
5.9***
1.8
3.2*** 3.5*
0
1
2
3
4
5
6
7
Female Male Overage Not overage High % At￾risk
Low % At￾risk
Remote
schools
Not remote
schools
Percentage point difference in SDPP 
and control group percentage of scale 
items with which student agreed
SDPP group
†† †
Findings from the School Dropout Prevention Pilot Program Impact Evaluation in Timor-Leste Page 47
Changed Behavior Puts Nelson on the “Fast Track” to Grade 7
Nelson, a Grade 6 student in Timor-Leste, didn’t like school and he didn’t mind letting people 
know it. He was frequently absent and failed to do his homework; even if he did do it he didn’t 
turn it in. He acted up in class. These were warning signs that he was on the precipitous track to 
drop out of school. However, his teacher, Honório Pinto, decided he would take aggressive 
action to derail Nelson from that track and turn those behaviors around. Pinto had received 
training on an Early Warning System (EWS) as part of the School Dropout Prevention Pilot 
(SDPP) Program. The EWS showed Honório how to identify at-risk students like Nelson; it also 
taught him actions he could take to lower the risk of students dropping out of school. 
The first step Honório took was 
making the classroom more 
inviting for Nelson. He also 
contacted Nelson’s mother to let 
her know about Nelson’s 
worrisome behaviors and invited 
her to discuss what to do to keep 
Nelson in school. She explained, 
“Sometimes Nelson doesn’t go to 
school for up to a month. As the 
oldest son he helps his father on 
the farm and tends the cattle. 
There is no one else at home to do 
this. I’ve received the cards
informing me of Nelson’s 
absences.” Despite receiving the 
warning cards, she didn’t know 
what to do.
Nelson’s mother came to the school after she received the warning card. Nelson’s teacher and 
the school director explained the long-term impact of dropping out of school and how bad it 
would be for Nelson, the family and their community. They discussed his high number of 
absences and what could be done to improve his attendance. His mother left the meeting 
determined to make sure Nelson went to school every day. “Now as his mother, I send him to 
school every morning.” Nelson also agreed to make sure he did his homework and focus on 
what he was being taught.
It didn’t take long for his teacher Honório to see a change. “Comparing now to before, there is 
a big difference in Nelson’s attendance. Before he would miss up to three days of school in a 
week, but recently he hasn’t missed at all.” Honório also noted the change in homework, 
“Nelson is now more active in school because of the support the SDPP Program provides to 
motivate students and the use of the early warning card.”
Nelson’s teacher, school director and his family took actions based on EWS steps, which 
produced positive results. His behavior changed, he worked hard and at the end of the year he 
proudly reported that he passed his exams and would continue to Grade 7.
Students in Timor-Leste improved their behavioral engagement in 
school – coming to school, being punctual, doing homework, and 
following rules.
Findings from the School Dropout Prevention Pilot Program Impact Evaluation in Timor-Leste Page 48
2. Impact on additional measures of student attitudes toward school 
In addition to measuring impacts on the three primary student attitudes described above, SDPP 
also assessed the program’s impacts on two other attitudinal outcomes for at-risk students: their 
perceptions about teachers’ and parents’ support of their success in school.47 These scales were 
also based on responses to the at-risk student survey, and higher scores on these scales represent 
more positive student perceptions of the support they receive from teachers and parents.48
SDPP found no impact on student perceptions of teacher support (Figure VI.B.7). Students 
from SDPP agreed to 80.7 percent of the questions on their perceptions of teacher support, and 
students from control schools agreed to 79.7 percent of questions. 
 
47 As described in Section III, because these analyses are exploratory, results were not adjusted for multiple 
comparisons, despite the large number of comparisons. 
48 Appendix D describes the construction of these measures. 
Student perceptions of teacher support
Students identified as being at risk of dropping out of school responded to a questionnaire. This 
scale is based on student responses to the following 11 survey items related to how students 
perceive the support provided by their teachers:
 I have had difficulty getting along with my teacher(s).
 My teacher(s) cares about how I’m doing.
 My teacher(s) talks to me about how I did on my homework and/or exams.
 My teacher(s) helps me if I am having problems with a lesson.
 I feel comfortable asking my teacher(s) for help with my lessons.
 My teacher(s) talks to me if I miss school or class.
 My teacher(s) thinks I am capable of completing my current grade.
 My teacher(s) has talked to me about my future plans.
 My teacher(s) has contacted my parents about my school work.
 My teacher(s) has contacted my parents about my attendance.
 My teacher(s) and my classmates encourage me not to drop out.
Findings from the School Dropout Prevention Pilot Program Impact Evaluation in Timor-Leste Page 49
Figure VI.B.7. SDPP Program impact on students’ perceptions of teacher support (SY 2013 and SY 2014) 
Sources: SDPP baseline and follow-up school records data collection, May 2012, May 2013, March 2014, September 2014, and 
January 2015; baseline and follow-up student surveys, May 2013 and September 2014. 
Note: The analysis is based on SY 2012 4th and 5th grade at-risk students, and SY 2013 and 2014 4th grade at-risk students. 
 Differences between SDPP and control group means were tested using two-tailed t-tests. Mean values are adjusted for 
baseline characteristics. The analysis accounts for clustering of students within schools and cohort fixed effects. Because 
these are additional outcomes and the analyses are exploratory, statistical significance thresholds were not adjusted for 
multiple comparisons. 
***/**/* Impact estimate is statistically significant at the .01/.05/.10 level. 
SDPP found no impact on student perceptions of parent support (Figure VI.B.8). Students 
from SDPP schools agreed to 86.4 percent of the questions on their perceptions of parental support, 
compared to students in control schools who agreed to 86.6 percent. 
80.7 79.7
0
10
20
30
40
50
60
70
80
90
100
Percentage of scale items with which 
student agreed
SDPP group Control group
Student perceptions of parental support
Students identified as being at risk of dropping out of school responded to a questionnaire. This scale 
is based on student responses to the following 10 survey items related to how students perceive the 
support provided by their parents:
 My parents know when I have not completed my homework and assignments.
 My parents have talked with my teacher about my exam scores or absences.
 My parents have talked with my teacher about my attendance.
 My parents make sure I go to school every day.
 It is important to my parents that I do well in school.
 My parents attend school events.
 My parents talk to me about improving my grades.
 My parents try to support me with my studies.
 My parents free up my time for schoolwork.
 My parents want me to complete my current grade.

Findings from the School Dropout Prevention Pilot Program Impact Evaluation in Timor-Leste Page 50
Figure VI.B.8. SDPP Program impact on students’ perceptions of parental support (SY 2013 and SY 
2014) 
Sources: SDPP baseline and follow-up school records data collection, May 2012, May 2013, March 2014, September 2014, and 
January 2015; baseline and follow-up student surveys, May 2013 and September 2014. 
Note: The analysis is based on SY 2012 4th and 5th grade at-risk students, and SY 2013 and 2014 4th grade at-risk students. 
 Differences between SDPP and control group means were tested using two-tailed t-tests. Mean values are adjusted for 
baseline characteristics. The analysis accounts for clustering of students within schools and cohort fixed effects. Because 
these are additional outcomes and the analyses are exploratory, statistical significance thresholds were not adjusted for 
multiple comparisons. 
***/**/* Impact estimate is statistically significant at the .01/.05/.10 level. 
C. Impacts on student engagement in school
The SDPP theory of change suggests that an 
improvement in student attitudes toward school 
will result in an improvement in their 
engagement in school. SDPP used three 
outcomes to measure student engagement in 
school: attendance, academic performance, and 
behavior. For both categories, SDPP used school 
records to construct the outcome measures. 
Because the SDPP Program was intended to 
improve student attendance, it is the primary measure of its effectiveness in this domain. 
This section presents findings on the impact of SDPP on student engagement in school.
1. Impact on student’s daily attendance in school 
Because SDPP was intended to improve student attendance, it is the primary measure used to 
determine SDPP’s effectiveness in this domain. Students at risk of dropping out may miss 
excessive amounts of school, causing them to do poorly on their school work and fall behind in 
86.4 86.6
0
10
20
30
40
50
60
70
80
90
100
Percentage of scale items with which 
student agreed
SDPP group Control group
Student engagement in school outcomes
Primary measure of program effectiveness
 Daily attendance 
Additional measures
 Student performance in school
 Student behavior in school
Findings from the School Dropout Prevention Pilot Program Impact Evaluation in Timor-Leste Page 51
their studies, discouraging and alienating them from their studies, school and classmates. SDPP 
estimated impacts on attendance for all students as well as for students identified as at risk of 
dropout based on their baseline characteristics. 
SDPP had a statistically significant impact on attendance for the students exposed to the 
program (Figure VI.C.1). Students in SDPP schools had an average daily attendance rate of 82.0 
percent, compared with 80.3 percent for students in control schools, a 2.1 percent change 
improvement in attendance for the treatment group as compared to the control group. 49 This 
improvement in engagement with school is the next level in the theory of change, following the 
improvement in students’ behavioral attitudes. 
The difference in attendance rates for students at risk of dropout in SDPP and control schools was 
statistically significant .The average daily attendance for at-risk students in SDPP schools was 78.9 
percent versus 76.6 percent for those in control schools, a three percent change improvement in 
attendance for the treatment group as compared to the control group. In contrast, average 
attendance rates differed for students not at risk of dropout in SDPP schools (84.3 percent) 
compared to control schools (82.9 percent), but this difference was not statistically significant. 
Appendix E presents findings based on alternative measures of attendance. 
 
49 This translates to an additional school days attended of up to 77,000 in Timor-Leste. This gross calculation uses 
rounded numbers and assumes that impacts would be similar for all students in the target grades and schools. Under 
alternate assumptions these numbers could vary. 
Student attendance
The attendance measure is average daily attendance for the full school year. The daily attendance 
rate is the percentage of school days a student attended during the school year, constructed by 
averaging the monthly percentages for the most recent school year.
Improved Attendance: What does this mean? 
Positive impacts on attendance mean that students were in school and able to receive instruction more 
often than they would have without SDPP. Impacts on attendance can be translated into the number of 
additional school days attended by all students who received the SDPP Program.
In Timor-Leste, schools are open for 240 days, on average, during a school year. A student typically 
misses around 19 days of school per school year. Overall, a student who participated in SDPP 
attended four additional days at school compared to a student who did not participate in SDPP. Given 
that 14,000 students were exposed to the intervention in the SDPP schools, the number of additional 
school days attended is up to 56,000 in Timor-Leste. 
Among at-risk students, the number of additional school days attended is up to 33,000 in Timor-Leste.
Findings from the School Dropout Prevention Pilot Program Impact Evaluation in Timor-Leste Page 52
Figure VI.C.1. SDPP Program impact on daily attendance, overall and by at-risk status (SY 2013 and SY 
2014) 
Sources: SDPP baseline and follow-up school records data collection, May 2012, May 2013, March 2014, September 2014, and 
January 2015. 
Note: The analysis is based on SY 2012 4th- and 5th-grade students, and SY 2013 and 2014 4th-grade students. The sample 
includes 13,645 students for the SDPP group (5,884 at risk and 6,233 not at risk) and 13,702 students for the control 
group (5,724 at risk and 6,681 not at risk). 
 Differences between SDPP and control group means were tested using two-tailed t-tests. Mean values are adjusted for 
baseline characteristics. The analysis accounts for clustering of students within schools and cohort fixed effects. For a 
tabular presentation of these findings, see Appendix Table H.5. 
***/**/* Impact estimate is statistically significant at the .01/.05/.10 level. 
The SDPP Program was not designed to 
have different impacts for different 
subgroups of students (other than 
students at-risk of dropout); however, 
the literature suggests that outcomes and 
impacts might vary for different types of 
students. Impacts of the SDPP Program 
on attendance were present in several 
subgroups (Figure VI.C.2).50
Consistent with SDPP’s main 
findings, the program had positive 
impacts on attendance which were 
statistically significant for students in several subgroups. These included females, males, 
students either at or above the appropriate age for their grade, and students from schools with a 
 
50 As described in Section III, because these analyses are exploratory, the results were not adjusted for multiple 
comparisons, despite the large number of comparisons. 
82.0** 78.9***
84.3
80.3 76.6
82.9
0.0
10.0
20.0
30.0
40.0
50.0
60.0
70.0
80.0
90.0
100.0
All Students At-Risk Students Not-At-Risk Students
Percentage of days attended
SDPP group Control group
Attendance Data
SDPP collected aggregate attendance data via 
headcounts during each data collection effort during 
mid-morning. These values were similar in SDPP and 
control schools. 
The attendance rate from this data was then compared to 
attendance measures derived from student- level data 
from school records. The attendance rate based on the 
headcount was very similar to the rate determined 
through school records in both research groups. 
Findings from the School Dropout Prevention Pilot Program Impact Evaluation in Timor-Leste Page 53
low percentage of at-risk students. The impacts were marginally significant for students from 
schools that are remote. 
Figure VI.C.2. SDPP Program impacts on daily attendance for students overall, by subgroup 
Sources: SDPP baseline and follow-up school records data collection; May 2012, May 2013, March 2014, September 2014, and 
January 2015. 
Note: The analysis is based on SY 2012 4th- and 5th-grade students, and SY 2013 and 2014 4th-grade students. 
 Differences between SDPP and control group means were tested using two-tailed t-tests. Mean values are adjusted for 
baseline characteristics. The analysis accounts for clustering of students within schools and cohort fixed effects. Because 
these subgroup analyses are exploratory, statistical significance thresholds were not adjusted for multiple comparisons. 
There were no statistically significant differences between subgroup impact estimates. 
***/**/* Impact estimate is statistically significant at the .01/.05/.10 level. 
1.7** 1.7**
2.3**
1.7**
1.1
2.1**
1.6*
1.9
0
1
2
3
Female Male Overage Not overage High % At￾risk
Low % At￾risk
Remote
schools
Not remote
schools
Percentage point difference in 
SDPP and control group percentage 
of days attended
Timor-Leste
SDPP group
Findings from the School Dropout Prevention Pilot Program Impact Evaluation in Timor-Leste Page 54
Increasing Attendance in Timor-Leste: Community Volunteers Reach Out 
to Families
The USAID-funded SDPP Program introduced intervention activities to prevent school dropout 
in the last trimester of the 2011 school year in five districts in Timor-Leste. Within three months, 
the interventions were already making a difference at the Mantane Primary School. “Earlier in 
the school year, there were 21 students who dropped out of school. In 2012 there is not one case 
of student dropout. If there is an absence, it is due to illness only, but then they return to school. 
The SDPP Program has been a real success,” stated School Principal Eugenio Guterres. 
Faustina Yustinus da Costa, an SDPP staff member working with the school added: “We heard 
stories from the years before of how many students would disappear from school before the 
exam time. Now when students aren’t there, we know through the community group members 
what their situation is.” 
Student absenteeism is a major predictor of dropout and a cause of poor academic performance. 
SDPP research shows that 30 to 40 percent of at-risk pupils and dropouts missed more than 15 
consecutive days of school and, on average, they missed 4 or more days of school a month. Yet 
only 40 percent of parents knew their child was absent, and less than 60 percent of the parents 
had been contacted by the school. Most parents do not have telephones and may live far from 
the school. 
The SDPP Program worked with community volunteers to bridge the communication gap 
between the school and parents. Romana Cáceres, a volunteer, explains: “When a student is 
absent 2 or 3 days, we follow up right to the home, even when we have to walk very far to get 
there.” The home visits, Romana says, make parents aware of the linkage between absenteeism 
and dropout. Now when students are absent, it is usually due to illness, not chores or work. 
The home visits are also a way to 
address student truancy. Principal 
Eugenio noted that the program 
can reduce the number of students 
who skip school “because they are 
concerned about the community 
volunteer’s home visit delivering a 
postcard notification from the 
school.”
In August 2013, members of the 
Mantane volunteer “Stay-in￾School” committee were awarded 
certificates in a ceremony 
recognizing their activism and 
success in reducing dropout and 
absenteeism.
. 
Members of a Stay-in-School Committee in Timor-Leste
Findings from the School Dropout Prevention Pilot Program Impact Evaluation in Timor-Leste Page 55
2. Impact on additional measures of student engagement in school 
Even though attendance is the primary outcome in the engagement domain, SDPP also examined 
the program’s impacts on academic performance, since an increase in attendance might result in 
an improvement in overall performance in school. Because SDPP did not attempt to directly 
influence academic performance, it is not a primary measure in determining program 
effectiveness.51 The measures examined Tetun, Portuguese, and math exam scores for students and 
behavior grades. Schools scored students on a 10-point scale, from 1 to 10, with higher scores 
indicating better performance. 
a. Academic performance in math and language 
The SDPP Program did not have a statistically significant impact on math, Portuguese 
language, or Tetun language performance (Figure VI.C.3). 
b. Student behavior in school 
Teachers in Timor-Leste keep records on student behavior. Behavior grades from third 
trimester for students show that SDPP had no impact on this measure of student 
performance, as assessed by their teachers (Figure VI.C.3). 
 
51 Moreover, impact estimates for academic performance may be biased if the program was successful in keeping low￾performing students in school. Also, as described in Section III, because these analyses are exploratory the results 
were not adjusted for multiple comparisons, despite the large number of comparisons. 
Math, Portuguese language, Tetun language and behavior performance
Data came from third trimester exams. Schools scored students on a scale of 1 to 10: 1 is “terrible” 
and 10 is “excellent.”
Findings from the School Dropout Prevention Pilot Program Impact Evaluation in Timor-Leste Page 56
Figure VI.C.3. SDPP Program impact on academic performance (SY 2013 and SY 2014) 
Sources: SDPP baseline and follow-up school records data collection, May 2012, May 2013, March 2014, September 2014, and 
January 2015. 
Note: The analysis is based on SY 2012 4th- and 5th-grade students, and SY 2013 and 2014 4th-grade students. 
 Differences between SDPP and control group means were tested using two-tailed t-tests. Mean values are adjusted for 
baseline characteristics. The analysis accounts for clustering of students within schools and cohort fixed effects. Because 
these are additional outcomes and the analyses are exploratory, statistical significance thresholds were not adjusted for 
multiple comparisons. 
***/**/* Impact estimate is statistically significant at the .01/.05/.10 level. 
Appendix E presents additional findings for the engagement domain based on alternative measures 
of attendance, including an alternative approach to accounting for missing values from school 
records. Findings using the alternative approaches are consistent with those presented here. 
6.30 6.56 6.88
7.42
6.22 6.47
6.84
7.50
0
1
2
3
4
5
6
7
8
9
10
Math Portuguese Tetun Behavior
Score on a ten-point scale
SDPP group Control group
Findings from the School Dropout Prevention Pilot Program Impact Evaluation in Timor-Leste Page 57
D. Impacts on school dropout 
Reducing school dropout is the primary goal of 
the SDPP intervention. Students who drop out 
of school prematurely forgo a considerable 
amount in yearly earnings over their working 
lives (Duflo, Esther 2001), limit their access to 
further education and training, reduce their 
ability to contribute to the well-being of their 
community, and add to wastage of the 
resources invested in their schooling. 
For Timor-Leste, SDPP defined the primary dropout measure as within-grade (whether a student 
completes the school year for an end of cycle target grade) or between-grade (whether a student 
enrolls back in school the year after completing a target grade) depending on the cohort.52 A 
student was considered to be a within-grade dropout if the student missed any of the three 3rd 
trimester exams, and a student was considered to be a between-grade dropout if the student was 
not enrolled in the next school year. For this analysis, SDPP uses school records to measure 
dropout from school. 
This section presents findings on the impact of SDPP on student dropout from school.
1. Impact on primary measure of school dropout 
SDPP had no statistically significant impact on dropout (Figure VI.D.1). Students in SDPP 
schools dropped out at a rate of 16.3 percent, compared with 15.5 percent for students in control 
schools, but this difference was not statistically significant. These dropout rates are much higher 
than those identified in the national EMIS data. 
At-risk students dropped out at a higher rate than not-at-risk students. In both SDPP and 
control schools, at-risk students dropped out at a rate of 17.7 percent.53 Among students not at risk 
 
52 Dropout is measured as between-grade dropout rate at the beginning of SY 2015 for the SY 2013 and SY 2014 4th￾grade student cohorts; and within-grade dropout at the end of SY 2013 for the SY 2012 5th-grade student cohort and 
at the end of SY 2014 for the SY 2012 4th-grade student cohort. 
53 We also estimated the treatment-on-the-treated (TOT) impacts for (1) schools with high fidelity of implementation 
scores, and (2) students who were identified through the EWS for SDPP services using quasi-experimental research 
techniques. The TOT estimates are consistent with the findings presented here (See Appendix J for details). 
Primary measure of program effectiveness for school dropout
Students were considered dropouts if they were no longer continuing their education at the last 
possible time we observed them. Students who started the program in grades 4 and 5 in the first 
year of implementation were considered to have dropped out if they did not complete their final 
grade 6 examinations in subsequent years. Students who started the program in grade 4 in the 
second and third years of implementation were considered to have dropped out if they did not 
enroll in school for the 2015 school year (as grade 5 and 6 students, respectively).
School dropout outcomes
Primary measure of program effectiveness
 Within-grade dropout
 Between-grade dropout
Additional measures
 Grade progression 
Findings from the School Dropout Prevention Pilot Program Impact Evaluation in Timor-Leste Page 58
of dropout, the dropout rates were lower and differed between students in SDPP schools (15.3 
percent) and control schools (14 percent), but this difference was not statistically significant. 
Figure VI.D.1. SDPP Program impacts on school dropout, overall and by at-risk status (SY 2013 and SY 
2014) 
Sources: SDPP baseline and follow-up school records data collection, May 2012, May 2013, March 2014, September 2014, and 
January 2015. 
Note: The analysis is based on SY 2012 4th- and 5th-grade students, and SY 2013 and 2014 4th-grade students. The sample 
includes 14,045 students for the SDPP group (6,015 at risk and 8,030 not at risk) and 14,458 students for the control 
group (5,950 at risk and 8,508 not at risk). 
 Differences between SDPP and control group means were tested using two-tailed t-tests. Mean values are adjusted for 
baseline characteristics. The analysis accounts for clustering of students within schools and cohort fixed effects. For a 
tabular presentation of these findings, see Appendix Table H.5. 
***/**/* Impact estimate is statistically significant at the .01/.05/.10 level. 
Because four cohorts of students received the intervention and were exposed for different periods 
of time at different points in time, SDPP also estimated dropout separately for each cohort 
(Figure VI.D.2). 54 Dropout rates differed substantially across cohorts, but there were no 
statistically significant differences between SDPP Program and control group students for any 
cohort. Students in SDPP schools in the SY 2012 4th-grade cohort, SY 2012 5th-grade cohort, SY 
2013 4th-grade cohort and SY 2014 4th-grade cohort dropped out at a rate of 17.4 percent, 30.8 
percent, 10.9 percent and 3.3 percent, respectively. Students in SDPP schools in the SY 2012 4th￾grade cohort, SY 2012 5th-grade cohort, SY 2013 4th-grade cohort and SY 2014 4th-grade cohort 
dropped out at a rate of 18.2, 25.1 percent, 12.1 percent and 4.4 percent, respectively. 
 
54 As described in Section III, because these analyses are exploratory, the results were not adjusted for multiple 
comparisons, despite the large number of comparisons. 
16.3 17.7 15.5 15.3 17.7
14.0
0.0
10.0
20.0
30.0
40.0
50.0
60.0
70.0
80.0
90.0
100.0
All Students At-Risk Students Not-At-Risk Students
Percentage of students dropped out
SDPP group Control group
Findings from the School Dropout Prevention Pilot Program Impact Evaluation in Timor-Leste Page 59
Figure VI.D.2. SDPP Program impacts on school dropout, by cohort (SY 2013 and SY 2014) 
Sources: SDPP baseline and follow-up school records data collection, May 2012, May 2013, March 2014, September 2014, and 
January 2015. 
Note: The analysis is based on four cohorts of students: SY 2012 4th- and 5th-grade students, and SY 2013 and 2014 4th￾grade students. The sample includes 14,045 students for the SDPP group (6,015 at risk and 8,030 not at risk) and 14,458 
students for the control group (5,950 at risk and 8,508 not at risk). 
Differences between SDPP and control group means were tested using two-tailed t-tests. Mean values are adjusted for 
baseline characteristics. The analysis accounts for clustering of students within schools and cohort fixed effects. For a 
tabular presentation of these findings, see Appendix Table H.7. 
***/**/* Impact estimate is statistically significant at the .01/.05/.10 level. 
SDPP also explored the program’s impacts on dropout for four additional subgroups of interest: 
gender, over-age status, whether the school has a low or high percentage of students at risk, and 
whether the school is remote or not. 55 SDPP found no effects on dropout for most of these 
subgroups (Figure VI.D.3). However, program impacts on dropout rates were statistically 
significant for students from schools with a high percentage of at-risk students. Moreover, the 
difference in impacts between students in schools with high and low percentage of at-risk students 
was also statistically significant. 
 
55 As described in Section III, because these analyses are exploratory. SDPP did not adjust the results for multiple 
comparisons, despite the large number of comparisons. 
17.4
30.8**
10.9
3.3
18.2
25.1
12.1
4.4
0
10
20
30
40
50
60
70
80
90
100
SY 2012, grade 4 SY 2012, grade 5 SY 2013, grade 4 SY 2014, grade 4
Percentage of Students Dropped Out
SDPP Group Control Group
Cohort
Findings from the School Dropout Prevention Pilot Program Impact Evaluation in Timor-Leste Page 60
Figure VI.D.3. SDPP Program impacts on school dropout, by subgroup (SY 2013 and SY 2014) 
Sources: SDPP baseline and follow-up school records data collection, May 2012, May 2013, March 2014, September 2014, and 
January 2015. 
Note: The analysis is based on SY 2012 4th- and 5th-grade students, and SY 2013 and 2014 4th-grade students. 
 Differences between SDPP and control group means were tested using two-tailed t-tests. Mean values are adjusted for 
baseline characteristics. The analysis accounts for clustering of students within schools and cohort fixed effects. Because 
these subgroup analyses are exploratory, statistical significance thresholds were not adjusted for multiple comparisons. 
***/**/* Impact estimate is statistically significant at the .01/.05/.10 level. 
†††/††/† Statistically significant difference between the subgroup impact estimates at the .01/.05/.10 level. 
2. Impact on additional measures of school dropout 
SDPP also calculated a measure of grade progression for SY 2012 4th grade, SY 2013 4th grade, 
and SY 2014 4th grade students, and expressed it in terms of whether a student enrolled in the next 
grade or higher in the year(s) following exposure to the SDPP Program. This measure provides 
additional information on students’ successful progression in school. 
As with the school dropout, SDPP had no impact on grade progression in school overall 
(Figure VI.D.4), but SDPP did improve grade progression for not-at-risk students. SDPP 
students progressed at a rate of 84.2 percent while control group students progressed at a rate of 
82.2 percent. This difference is marginally statistically significant. 
0.6 1.0
-1.2
1.5
3.9**
-0.7
1.4
-1.0
-5
-4
-3
-2
-1
0
1
2
3
4
5
Female Male Overage Not overage High % At￾risk
Low % At￾risk
Remote
schools
Not remote
schools
Percentage point difference in SDPP and 
control group school dropout rates
SDPP group
†
Grade progression
A student is considered to have progressed if they are enrolled in the next grade or higher in the 
following school year. 
Findings from the School Dropout Prevention Pilot Program Impact Evaluation in Timor-Leste Page 61
Figure VI.D.4. SDPP Program impact on student progression in school at endline, overall and by at-risk 
status (percentage) (SY 2013 and SY 2014) 
Source: SDPP baseline and follow-up school records data collection, May 2012, May 2013, March 2014, September 2014, and 
January 2015. 
Note: The analysis of grade progression is based on the SY 2012 4th-grade, the SY 2013 4th-grade, and the SY 2014 4th￾grade students. 
 Differences between SDPP and control group means were tested using two-tailed t-tests. Mean values are adjusted for 
baseline characteristics. The analysis accounts for clustering of students within schools and cohort fixed effects. Because 
this is an additional outcome and the analyses are exploratory, statistical significance thresholds were not adjusted for 
multiple comparisons. 
***/**/* Impact estimate is statistically significant at the .01/.05/.10 level. 
Appendix F provides findings related to the dropout domain based on alternative measures of 
dropout. It also provides findings for two additional measures of progression. The first measure is 
defined as a student enrolling in the next grade, but not higher, in the following school year. No 
impact was found. 
In the second measure, a student is considered to have been promoted to the next grade if their 
school records indicated that he or she completed 6th grade and was promoted to 7th grade. 
Although we were not actually able to witness whether the student enrolled in 7th grade because 
6th grade is the final target grade and students who subsequently enrolled in 7th grade frequently 
had to change schools, SDPP did have a marginally significant impact on progression from 6th 
grade to 7th grade on schedule with progression estimated at 79.7 percent among SDPP students 
and 77.7 percent among control group students. 
80.0
74.7
84.2*
78.3
73.1
82.2
0
10
20
30
40
50
60
70
80
90
100
All Students At-Risk Students Not-At-Risk Students
Percentage of students who progressed to 
the next grade or higher
SDPP group Control group
Findings from the School Dropout Prevention Pilot Program Impact Evaluation in Timor-Leste Page 62
Qualitative Research Findings in Timor-Leste
During the second year of intervention implementation in the treatment schools, the SDPP team 
conducted a Qualitative Research Study in Timor-Leste to better understand: why changes in student 
and teacher behaviors and attitudes had happened; and how beneficiaries and targets (students, teachers, 
school directors, parents and community members) responded and reacted to the interventions. Some 
findings are highlighted below. 
What do beneficiaries say about the Early Warning System?
The need to help support their families was the primary reason students considered dropping out of 
school. For those who did drop out, frequent absences and poor grades were primary factors. Despite 
these warning signs—absenteeism and poor grades—almost three-fourths (72%) of the parents 
indicated they were unaware that their child as at-risk of dropping out. Almost all of the children – both 
at-risk and those who did eventually drop out – reported that they themselves had made the decision to 
quit. This suggests students might have internalized their family’s vulnerable economic status and that 
awareness prompted their decision to leave. Dropout students also noted more could have been done to 
keep them in school, suggesting they needed more “encouragement.” But, both dropout and at-risk 
students reported their teachers had been very helpful and encouraging. 
Truancy was significant. About 30 percent of at-risk students and 60 percent of dropouts skipped school 
without their parents’ knowledge. With the exception of at-risk boys (67%), more than 80% of the 
students recalled someone communicating with their families about their absences; about 40 percent of 
at-risk students and 80 percent of dropouts confirmed their parents had received a warning postcard 
from the school and/or home visit. When asked about their parents’ response to the contact, most 
students reported their parents were either upset at them or embarrassed, but also confirmed it resulted 
in their parents becoming more supportive. They said their parents sent them back to school, punished 
them, or talked to them about why they did not want to go. Boys who dropped out reported that the 
contact from school had no impact on their parents, but many – boys and girls – mentioned the school’s 
contact with their parents made them sad they couldn’t attend. 
Nearly all (94%) of the parents contacted spoke positively about the contact, although they also said 
they were initially surprised, embarrassed, and disappointed in their child. They were more likely to 
monitor their child’s attendance and provide other support, such a buying school supplies. They noted 
they had greater awareness of the impact of missing school and how it could hinder their child’s future 
by leading to delinquency or vagrancy. There was also a greater awareness of the need to educate their 
daughters. The “Stay in School” committee members said that parents received the visit well once they 
understood its purpose was to help, not blame.
Teachers praised the structure the EWS gave them to deal with struggling students. Almost 80% of 
them realized the EWS indicators did a better job at identifying students who needed extra support. 
Three-quarters felt it made their job easier, in part because the EWS procedures and forms were easy to 
follow. When asked about the support they received from both their school directors and community 
members, the majority felt it had been a very collaborative effort. In general, teachers, school directors 
and community members all felt the increased and improved contact with the families was a particularly 
positive feature of the EWS, helping both families and teachers better understand the issues the child 
faced. As one teacher shared, “Students now pay attention to the subject material. Before they were lazy 
and sleepy. The Warning Card is one way of waking the students up.”
When asked about the impact of the EWS, especially the contact with parents, 85 percent of the teachers 
reported improvements in attendance, behavior, completion of homework, and in student performance. 
Ninety percent of school personnel hope to continue using the EWS. 
.
Findings from the School Dropout Prevention Pilot Program Impact Evaluation in Timor-Leste Page 63
What do beneficiaries say about the Extra Curricular Activities (ECA) program?
The ECA program was popular among all the students who participated in tit. Although the ECA 
sessions were offered more than once a week in some schools, attendance dropped significantly for 
students attending more than one day a week. Over half felt that once a week was fine; none of them 
suggested more than twice a week. Girls particularly liked the singing, drawing and crafts, in contrast 
to the boys, who preferred more active things like the traditional sports games, marbles, and hopscotch. 
Students unanimously agreed they benefitted from the ECA, referring to improved language and math 
skills. They also noted how engaging in games together improved their skills in collaboration and 
conflict resolution – one of the key objectives of the intervention. As one student shared, “In the past 
we weren't happy at school. There were no games, and there was no fun. School now makes us very 
happy because we can learn together. Now we feel happy, and we have fun. We feel sad when we are 
absent because we will miss the games that [the teachers] teach.” Another says, “Before, I could not 
read. Now I can read. Before I could not write or count, but now I am much better. Now I can read, 
write and count. Friends helped me with writing so that now I can write my letters well. Before I didn't 
like coming to school.” 
Although they overwhelmingly enjoyed the ECA, most students – other than a small number of at-risk 
girls – reported it did not play a role in their attendance. The majority of parents reported they knew of 
the ECA, generally favored them, and noted they did seem to make their child more positive about 
school. However, parents’ reactions were more mixed when asked if children actually went to school 
more often on days when there were extracurricular activities. Only 38 percent of mothers and 61 
percent of fathers said “yes.” When the question was rephrased to ask how they felt about their child 
taking time away from home to attend the ECA, parents were less positive about their child’s 
participation. Fathers and mothers appeared split along gender lines on their viewpoints, particularly 
when asked whether their children should spend more or less time in the ECA program: fathers 
expressed a higher level of dissatisfaction at the suggestion of additional time being allocated after 
school for more ECA.
Teachers held a different view. They strongly felt the ECA had a positive impact on improved 
attendance, behavior and general performance and pointed to school-level data to support their claim. 
They also reported students were more punctual, there was less fighting, and they were motivated to do 
well in school. They were also pleased with the increased self-confidence they observed in their students 
as a result of their participation in the ECA. The teachers also felt they had benefitted from the ECA 
training, noting that it helped them to use more interactive and hands-on instructional techniques in their 
regular classes. 
Despite the additional work involved with implementing the ECA, in general, two-thirds of the teachers 
expressed an overall positive opinion about the ECA. They cited things like the closer relationship they 
had with their students and that classroom management was much easier with happier and more 
motivated students. One of their concerns was the limited time they had within the ministry-approved 
curriculum and the challenge to add the activities to the schedule, although this may be largely driven 
by their perception about their legal work week. Teachers understand they are only required to work 24 
hours – essentially the hours that they spend in class – instead of the 40-hours defined by law. Teachers 
also doubted the capacity of their schools to continue implementing the ECA at the end of the SDPP 
due to the cost of the materials required for many of the hands-on arts and crafts activities. 
School directors had different concerns. They wondered if the teachers were adequately trained to take 
over the ECA program since teachers were in a helping role next to facilitators during the ECA. All the 
school directors except one hoped to continue doing ECA in their schools. However, in addition to more 
training, they also suggested the program might be expanded with additional guides.
Findings from the School Dropout Prevention Pilot Program Impact Evaluation in Timor-Leste Page 64
VII. School-level dropout trends 
The analyses presented thus far use student-level measures of dropout. This section presents 
different types of data to explore if alternative measures can provide additional insight or support 
to the findings presented earlier. SDPP has complementary, grade-level measures of dropout using 
data on enrollment and transfer counts from school records, as is often done in education studies.56
Constructing grade-level rates based on counts requires a much smaller data collection effort than 
gathering details about all individual students in a grade or school. School systems often report 
aggregate measures of enrollment and dropout as part of regular reporting, and stakeholders often 
find these measures to be easier to understand and interpret than other measures. Therefore, for 
readers more accustomed to these measures, they are presented in addition to the student-level 
estimates. These grade-level measures are available for 4th, 5th, and 6th grade as well as for grades 
that were not the focus of the interventions (1st–6th grades). Aggregate between-grade and within￾grade dropout rates are presented over time, across three complete school years (SY 2011 to SY 
2013) for 1st–6th grades in SDPP and control schools. 
Grade-level measures, although much easier to collect than individual-level data, typically suffer 
from substantial measurement error because it is difficult to distinguish between enrolled students, 
dropouts, transfers, repeaters, and newly enrolled students. Aggregate dropout measures actually 
include students in all of these situations. The magnitude of the aggregate dropout measure is 
therefore sensitive to whether more students transferred in or repeated the grade than dropped out 
or transferred out; the magnitude could even change signs to show a negative dropout rate. As a 
result, depending upon the levels of transfer, repetition, and dropout, there may or may not be any 
observable trends in dropout when using measures that are constructed from aggregate grade-level 
counts such as these. 
The between-grade dropout rate was generally much higher for 1st grade than for 2nd grade 
through 5th grade across both SDPP and control schools (Figure VII.1). This is seen in many 
developing countries, where schools lose many students after 1st grade. For 1st grade, between￾grade dropout rates were highest between SY 2011 and SY 2012, with an average rate of 22.9 
percent for the SDPP group and 24.2 percent for the control group. The between-grade dropout 
rate was mostly lower—between 22.9 percent and -23.4 percent—in the following two school 
years. The between-grade dropout rate for 2rd grade through 5th grade remained below 13 percent 
in all school years, and in some grades it was actually negative.57 Again, there was no clear trend 
in between-grade dropout in either the SDPP or the control group in any of the grades. No 
differences were statistically significant. 
 
56 The number of students enrolled at the start of each school year and the number of students who took exams at the 
end of each school year were collected and used to calculate between-grade dropout and within-grade. 
57 If more students enrolled in the school during the school year than dropped out or transferred out during the same 
time frame, then the dropout rate would be negative. 
Findings from the School Dropout Prevention Pilot Program Impact Evaluation in Timor-Leste Page 65
Figure VII.1. School between-grade dropout rates, by grade and school year 
Sources: SDPP baseline and follow-up school records data collection, SY 2011, 2012, 2013, and 2014. 
Note: The analysis is based on aggregate enrollment count data for SY 2011, SY 2012, and SY 2013 1st- through 6th-grade 
students. 
 Differences between SDPP and control group means were tested using two-tailed t-tests. 
***/**/* Statistically significant difference at the .01/.05/.10 level. 
There were no clear trends in within-grade dropout rates for 1st through 6th grades either. Dropout 
rates were highest in 1st grade, ranging between 5.0 percent and 13.4 percent (Figure VII.2). 
Differences between SDPP group and control group dropout rates were statistically significant in 
1st grade in SY 2012 and SY 2013. Dropout rates were generally lower in 2nd through 6th grade, 
although 2nd grade dropout rates were still somewhat elevated, ranging from 2.0 percent to 7.1 
percent. Dropout rates were between 0.8 percent and 5.4 percent from SY 2011 through SY 2013 
in 3rd through 6th grade. 
22.9 22.9
18.8
12.5
2.8 3.4
6.0
0.6
4.2 3.4
-3.5
-5.2
4.7
6.6
2.7
24.2
20.9
-23.4
6.3 4.0
-15.1
-0.1
0.7
-1.7 -0.8
2.3
-2.4
4.8 6.6 5.8
-30.0
-20.0
-10.0
0.0
10.0
20.0
30.0
2011
–2012
2012
–2013
2013
–2014
2011
–2012
2012
–2013
2013
–2014
2011
–2012
2012
–2013
2013
–2014
2011
–2012
2012
–2013
2013
–2014
2011
–2012
2012
–2013
2013
–2014
Grade 1 Grade 2 Grade 3 Grade 4 Grade 5
Percentage of students dropped out
Between-grade dropout rate
SDPP group Control group
Findings from the School Dropout Prevention Pilot Program Impact Evaluation in Timor-Leste Page 66
Figure VII.2. School within-grade dropout rates, by grade and school year 
Sources: SDPP baseline and follow-up school records data collection, SY 2011, 2012, and 2013. 
Note: The analysis is based on aggregate enrollment count data for SY 2011, SY 2012, and SY 2013 1st- through 6th-grade 
students. 
 Differences between SDPP and control group means were tested using two-tailed t-tests. 
***/**/* Statistically significant difference at the .01/.05/.10 level. 
Given the measurement issues in these dropout rates, it is challenging to draw any conclusions 
about the effectiveness of the program using grade-level estimates. These estimates can be useful 
in understanding how the level of dropout might change across grades (for example, whether it is 
higher for 1st grade than for the later grades), but they cannot be used to estimate an accurate level 
of dropout. 
SDPP also examined trends using similar information for enrollment and for attendance, as 
measured by headcounts. During each round of data collection, the SDPP team counted the 
number of students present in target grade classrooms on the day of data collection by entering 
classrooms during third period and counting the number of students present. The headcounts 
collected by SDPP during each data collection round show that the average number of target grade 
students present at the time of data collection was similar in SDPP and control schools. This 
number was then compared to attendance measures derived from student-level data in order to 
determine how similar the two data sources were. There was no clear trend between the attendance 
rate as determined through the head count compared to the attendance rate determined from school 
records, though they were typically similar. Appendix I presents these findings in more detail. 
8.6
13.4**
5.0** 4.3
7.1
2.0
3.0 3.1 3.0 3.5
5.0
2.3 1.9
3.9
1.4
2.2
3.5
2.0
10.4
7.5**
9.4**
2.4
5.9
2.7
2.3
3.6
5.4
2.5
0.8
3.5
2.0
3.6 3.6
1.3
3.1
2.1
0.0
2.0
4.0
6.0
8.0
10.0
12.0
14.0
16.0
2011 2012 2013 2011 2012 2013 2011 2012 2013 2011 2012 2013 2011 2012 2013 2011 2012 2013
Grade 1 Grade 2 Grade 3 Grade 4 Grade 5 Grade 6
Percentage of students dropped out
Within-grade dropout rate
SDPP group Control group
Findings from the School Dropout Prevention Pilot Program Impact Evaluation in Timor-Leste Page 67
VIII. Discussion
The SDPP Program in Timor-Leste aimed to reduce school dropout among students in the 4th, 5th, 
and 6th grade—the last three years of primary school in Timor-Leste. SDPP interventions included 
an EWS to identify and target support to students at risk of dropout and an ECA program to 
increase student engagement and attachment to school. SDPP hypothesized the ECA program 
activities for at-risk students, along with changes in teacher practices, involvement of parents, and 
increased community involvement, would improve student attitudes and behavior, translating into 
increased student engagement and reduction in school dropout. The SDPP evaluation in Timor￾Leste examined the effectiveness of SDPP using a randomized controlled trial in which the 
outcomes of students and teachers in 97 schools randomly assigned to provide SDPP services are 
compared to those of teachers and students in 93 schools randomly assigned to a control group 
providing business-as-usual services. The evaluation is part of a broader study of similar dropout 
prevention interventions implemented in four Asian countries. 
A. Overview of findings 
SDPP examined the effectiveness of the program on the primary and additional outcomes it was 
intended to affect related to teacher practices, at-risk students’ attitudes toward school, student 
engagement in school, and school dropout. Figure VIII.A.1 shows SDPP’s impacts according to 
its theory of change. The boxes banded in red show the outcomes on which SDPP had statistically 
significant or marginally statistically significant impacts as measured by the impact evaluation. 
Findings from the School Dropout Prevention Pilot Program Impact Evaluation in Timor-Leste Page 68
Figure VIII.A.1. SDPP conceptual model with program impacts 
Notes: The thickness of the border around each box indicates the number statistically significant or marginally significant 
favorable impacts on the primary and secondary outcomes within the given domain. Impacts on subgroups are not 
included. 
a Dropout includes the primary measure of dropout, global dropout. 
b Attendance is daily attendance averaged over the entire school year. 
c Students are assigned a behavior grade by their teachers. 
d Math and language performance is measured on a range of 1–10, with 1 being the lowest grade and 10 being the highest. 
e Student attitudes and aspirations are measured only for at-risk students and include students’ emotional, cognitive, and 
behavioral attitudes toward school. 
f Teacher and administrator prevention practices and at-risk student perceptions of teacher support. 
g Parent support is measured by the at-risk students’ perceptions of parent support scale. 
h Teacher and administrator self-efficacy and sense of responsibility. 
SDPP had a positive impact on teacher dropout prevention practices and a positive impact on 
teachers’ sense of self-efficacy, but not on teachers’ sense of responsibility for at-risk students. 
Teacher, Parent, and Community Knowledge and Awarenessh
Parent 
Supportg 
Teacher Prevention 
Practices and 
Supportf
Student 
Attitudes and 
Aspirationse 
Behaviorc Performanced 
Dropouta
Attendanceb
School Dropout
Student Engagement 
in School 
Student Attitudes
Teacher Behavior 
and Attitudinal 
Outcomes 
Statistically significant impacts for this domain were found on one outcome
Statistically significant impacts for this domain were found on two outcome
Findings from the School Dropout Prevention Pilot Program Impact Evaluation in Timor-Leste Page 69
Overall, SDPP effected improvements in key student attitudes and behaviors that can lead to 
dropout. These include positive, statistically significant changes in: 
 at-risk students’ behavioral attitudes toward school; and
 daily attendance overall and among at-risk students. 
Despite improvement in student engagement to school as measured through attitudes and 
attendance, SDPP did not reduce school dropout overall or among at-risk students. SDPP did not 
reduce dropout among students identified as at-risk based on their baseline characteristics. 
Tables VIII.A.1 and VIII.A.2 summarize the impacts. 
Table VIII.A.1. SDPP Program impacts on primary measures of program effectiveness in Timor-Leste 
Impacts
Teacher dropout prevention practices +++
At-risk student attitudes toward school
Emotional attitudes toward school ○
Cognitive attitudes toward school ○
Behavioral attitudes toward school +++
Attendance
Overall ++
At-risk +++
Dropout
Overall ○
At-risk ○
+ + +/+ +/+ Statistically significant positive impact at the .01/.05/.10 level. 
— — —/— —/— Statistically significant negative impact at the .01/.05/.10 level. 
○ Impact is not statistically significant. 
Table VIII.A.2. SDPP Program impacts on additional outcome measures in Timor-Leste 
Impacts
Teacher outcomes
Teacher self-efficacy ++
Teacher sense of responsibility ○
Administrator dropout prevention practices ○
Administrator self-efficacy ○
Administrator sense of responsibility ○
Student attitudes toward school
At-risk student perceptions of parent support ○
At-risk student perceptions of teacher support ○
Student engagement
Math performance ○
Language performance ○
Language performance 2 ○
Behavior ○
Dropout
Progression ○
+ + +/+ +/+ Statistically significant positive impact at the .01/.05/.10 level. 
— — —/— —/— Statistically significant negative impact at the .01/.05/.10 level. 
○ Impact is not statistically significant. 
Findings from the School Dropout Prevention Pilot Program Impact Evaluation in Timor-Leste Page 70
B. Contextual factors related to findings 
In interpreting findings from the evaluation, it is useful to consider the fidelity with which SDPP 
was implemented and other contextual factors that may have influenced its effectiveness. 
SDPP worked directly with teachers at school, community members affiliated with the schools, 
and parents at home to change their knowledge and awareness of dropout. This work was intended 
to improve the dropout prevention practices of teachers, which is reflected in the improved scores 
for teachers in SDPP schools on the teacher dropout prevention practice scale used by the 
evaluation. However, scores on this scale were near the maximum for teachers in both the SDPP 
and the control groups (7.87 and 7.72 out of 8, respectively). Thus, even without SDPP, teachers 
could be maintaining a high intensity of the dropout prevention practices as measured by this scale, 
and SDPP could only change them slightly, even if the difference in the teacher scores across 
groups is statistically significant. Alternatively, the scale scores might be artificially high in both 
groups because they are based on self-reports rather than direct observation. Consistent with the 
idea that SDPP school teachers performed an increased amount of duties to keep at-risk students 
from dropping out because of the EWS, findings from the qualitative analysis found that teachers 
reported doing more activities to keep students in school, such as checking attendance, issuing 
warning cards when students were absent twice in a month, and counseling them not to drop out 
(Creative Associates International 2015). 
As noted earlier, SDPP had positive impacts on at-risk students’ behavioral attitudes toward school 
and daily attendance. Both elements of SDPP—the EWS and after-school enrichment activities— 
are likely to have contributed toward SDPP Program impacts on these outcomes. Almost all 
students included in the qualitative analysis reported receiving help from homeroom teachers when 
they had trouble in school (Creative Associates International 2015). Additionally, it is possible 
that the after-school activities that attempted to foster cooperative learning and enhance self￾confidence improved students’ behavioral attitudes toward school and motivated them to attend 
school more frequently than they would have without after-school enrichment activities. Also, as 
part of the EWS, homeroom teachers, parents, and community members were asked to monitor at￾risk students’ attendance and contact parents if students were consistently absent. This part of the 
EWS could have induced at-risk students to attend school more than they would have in the 
absence of the program, leading to the positive impact on attendance, particularly for at-risk 
students. 
There are several factors that might explain why SDPP was not successful in reducing dropout in 
Timor-Leste. Each of these factors is discussed in more detail below. 
Inconsistent implementation of SDPP in Timor-Leste. According to SDPP’s fidelity-of￾implementation (FOI) analysis, the vast majority of its treatment schools in Cambodia, India and 
Tajikistan had met the threshold established to indicate that the SDPP interventions had been 
implemented in congruence with their design. However, in Timor-Leste the FOI study found that 
schools appeared to implement the EWS unevenly, particularly in terms of communicating with 
parents and taking follow-up actions. The implementation of extra-curricular activities in Timor￾Leste was somewhat better, although only about 76 percent of schools reported that they conducted 
after-school activities weekly, which was the intended frequency (Creative Associates 
International and School-To-School International 2015). Although the SDPP interventions have 
Findings from the School Dropout Prevention Pilot Program Impact Evaluation in Timor-Leste Page 71
improved attitudes and engagement toward school, the inconsistent implementation of follow-up 
actions by teachers and communication with parents may have decreased the program’s ability to 
affect dropout. 
Duration of exposure. Although SDPP did not reduce dropout, it was successful in improving 
three key intermediate outcomes—teacher dropout prevention practices, student behavioral 
attitudes toward school and attendance. SDPP was active in schools for two full school years and 
worked with grades 4, 5 and 6. Therefore, exposure time for individual students varied from one 
to two years depending upon their grade when they first received SDPP. Two cohorts, SY 2012 
4th graders and SY 2013 4th graders, were exposed to the intervention for two full school years, 
and two cohorts, SY 2012 5th graders and SY 2014 4th graders were exposed for one full school 
year. 
While the EWS was a part of school program itself, the ECA program sessions were offered to 
students once a week out of school hours. From an operational standpoint, exposure time was even 
less than intended, being limited by interruptions in the school calendar, including school closures 
due to rain, unscheduled and extended holidays, and national teacher training activities that pulled 
teachers from school for weeks at a time, leaving classrooms unstaffed. That the program had its 
intended effect on intermediate outcomes suggests that it could have its intended effect on longer￾term outcomes with longer or more intense exposure to the intervention. 
This evaluation examined outcomes after students were exposed to the program for one to two 
years. It should be noted that student attitudes toward school and behavior patterns are formed 
over time, starting with initial enrollment. Introducing SDPP interventions—which combine 
targeted adult attention and support, a welcoming school environment and activities that build 
student engagement—at the start of a student’s academic career, as well as continuing throughout 
their schooling, could help create positive perceptions of school and better behaviors. In Appendix 
G we explore the question of the role of exposure more fully. Findings suggest that with additional 
exposure, program impacts might be more pronounced. After at least two years of exposure to 
SDPP, SDPP group students had a marginally significant higher rate of attendance than control 
group students. SDPP group students also had a lower rate of school dropout, although this 
difference was not statistically significant. Future research should assess whether SDPP’s positive 
impacts on intermediate outcomes translate into impacts on dropout with more intense and/or 
extended exposure to the intervention. 
Complexity of factors related to dropout. Given the complex pathways and inputs that lead to 
school dropout, it is possible that SDPP services alone may not be sufficient to influence the 
dropout decision for the majority of the students dropping out. In particular, economic factors that 
SDPP was contractually prohibited from addressing directly—such as school-related expenses, 
and the need to supplement income through household chores or domestic work—were found to 
be particularly relevant in decisions to drop out of school (Creative Associates International 
2014).58 However, that SDPP was able to show impact on key teacher and student behaviors shown 
to influence decisions to drop out demonstrates that the EWS and ECA program intervention 
should be included among the multiple strategies needed to reduce dropout in Timor-Leste. 
 
58 The SDPP dropout prevention interventions were contractually prohibited from including conditional cash transfers 
or economic incentives, which have already been demonstrated to be effective by prior research. 
Findings from the School Dropout Prevention Pilot Program Impact Evaluation in Timor-Leste Page 72
C. Conclusion
This study shows that the SDPP Program in Timor-Leste was successful in improving some 
important intermediate outcomes— particularly teachers’ dropout prevention practices, students’ 
behavioral attitudes towards school, and daily attendance—but not in reducing dropout. It is useful 
to compare the impacts of SDPP in Timor-Leste to other countries where SDPP was able to target 
students in contexts where dropout was much higher, and where students were older, and where 
SDPP was implemented with more fidelity. Additional discussion of the impacts of the SDPP 
Program across all SDPP countries is presented in a separate, four-country summary report 
(Creative Associates International and Mathematica Policy Research 2015).
Findings from the School Dropout Prevention Pilot Program Impact Evaluation in Timor-Leste Page 73
References
American Institutes of Research. “ALAS: Intervention Report.” Retrieved from 
http://ies.ed.gov/ncee/wwc/interventionreport.aspx?sid=22, 2006a.
American Institutes of Research. “Check & Connect: Intervention Report.” Retrieved from 
http://ies.ed.gov/ncee/wwc/interventionreport.aspx?sid=78, 2006b.
Angrist, Joshua, Eric Bettinger, Erik Bloom, Elizabeth King, and Michael Kremer. “Vouchers for 
private schooling in Colombia: Evidence from a randomized natural experiment.” The 
American Economic Review, vol. 92, no. 5, 2002, pp. 1535-1558.
Banerjee, Abhijit, Shawn Cole, Esther Duflo, and Leigh Linden. “Remedying Education: Evidence 
from Two Randomized Experiments in India.” Quarterly Journal of Economics, vol. 122, no. 
3, 2007, pp. 1235–1264. 
Benjamini, Yoav and Yosef Hochberg. “Controlling the false discovery rate: A practical and 
powerful approach to multiple testing.” Journal of the Royal Statistical Society Series B 
Methodological, vol. 57, no. 1, 1995, pp. 289–300. 
Bloom, Howard S. “Accounting for No-Shows in Experimental Evaluation Designs.” Evaluation 
Review, vol. 8, 1984, pp. 225–246.
Brush, Lorie, Jennifer Shin, Rajani Shrestha, and Karen Tietjen. “School Dropout Prevention Pilot 
Program Review of the Literature.” Report submitted to the U.S. Agency for International 
Development. Washington, DC: Creative Associates International, Inc., 2011. 
Creative Associates International. “School Dropout Prevention Pilot Program; Timor-Leste Early 
Warning and Response System: School Manual.” Report submitted to the U.S. Agency for 
International Development. Washington, DC: Creative Associates International, Inc., 2012a. 
Creative Associates International. “School Dropout Prevention Pilot Program; Timor-Leste 
Extracurricular activities: Facilitators Manual.” Report submitted to the U.S. Agency for 
International Development. Washington, DC: Creative Associates International, Inc., 2012b. 
Creative Associates International. “Decision Rules for the Cut Point for At-Risk Students.” 
Memorandum to SDPP Local Country Offices, 2012c. 
Creative Associates International. “At-Risk Criteria for Early Warning System in the Pilot 
Countries.” Memorandum to SDPP Local Country Offices, 2012d. 
Creative Associates International. “School Dropout Prevention Pilot Program; Findings From the 
Situational Analysis: Factors and Conditions that Affect Dropout in Cambodia, India, 
Tajikistan, and Timor-Leste.” Report submitted to the U.S. Agency for International 
Development. Washington, DC: Creative Associates International, Inc., 2014a. 
Findings from the School Dropout Prevention Pilot Program Impact Evaluation in Timor-Leste Page 74
Creative Associates International. “Timor-Leste Situational Analysis: Factors and Conditions that 
Affect Dropout.” Report submitted to the U.S. Agency for International Development. 
Washington, DC: Creative Associates International, Inc., 2014b. 
Creative Associates International. “What Works: Beneficiary Perspectives on dropout Prevention 
Interventions in Timor-Leste” Report submitted to the U.S. Agency for International 
Development. Washington, DC: Creative Associates International, Inc., 2015.
Creative Associates International and Mathematica Policy Research. “Findings from the 
Four-Country School Dropout Prevention Pilot Program Impact Evaluation.” Report 
submitted to the U.S. Agency for International Development. Washington, DC: Creative 
Associates International, Inc., 2015. 
Creative Associates International and School-to-School International. “School Dropout Prevention 
Pilot Program: Summary of Findings, Fidelity of Implementation (FOI) of SDPP Interventions 
in Cambodia, India, Tajikistan and Timor Leste.” Report submitted to the U.S. Agency for 
International Development. Washington, DC: Creative Associates International, Inc., 2015. 
Duflo, Esther. "Schooling and Labor Market Consequences of School Construction in Indonesia: 
Evidence from an Unusual Policy Experiment." American Economic Review, vol. 91, no. 4, 
2001, pp. 795-813. 
Finlay, Krystina A. “Quantifying School Engagement: Research Report.” National Center for 
School Engagement, 2006. 
Fredericks, Jennifer A., Phyllis Blumenfeld, Jeanne Friedel, and Alison Paris. “School 
engagement.” In What do children need to flourish?: Conceptualizing and measuring 
indicators of positive development, edited by Kristin Anderson Moore and Laura H. Lippman. 
New York, NY: Springer Science and Business Media, 2005. 
Friedman, Willa, Michael Kremer, Edward Miguel, and Rebecca Thornton. “Education as 
Liberation?” No. w16939. National Bureau of Economic Research, 2011. 
Levy, Dan, Matt Sloan, Leigh Linden, and Harounan Kazianga. “Impact evaluation of Burkina 
Faso’s BRIGHT program. Final Report.” Washington, DC: Mathematica Policy Research, 
Inc. submitted to Millennium Challenge Corporation, 2009. 
Miguel, Edward, and Michael Kremer. “Worms: Identifying impacts on education and health in 
the presence of treatment externalities.” Econometrica, vol. 72, no. 1, 2004, pp. 159-217.
The Millennium Development Goals Report: 2013. New York, New York: United Nations, 2013. 
Murray, Nancy, Quinn Moore, Larissa Campuzano, Kathy Buek, Emilie Bagby, and Mark Strayer.
“School Dropout Prevention Pilot Program: Impact Evaluation Research Plan.” Report 
submitted to the U.S. Agency for International Development. Washington, DC: Creative 
Associates International, Inc., 2012.
Findings from the School Dropout Prevention Pilot Program Impact Evaluation in Timor-Leste Page 75
Oster, Emily, and Rebecca Thornton. “Menstruation, sanitary products, and school attendance: 
Evidence from a randomized evaluation.” American Economic Journal: Applied Economics, 
2011, pp. 91-100. 
Petrosino, Anthony, Claire Morgan, Trevor Fronius, Emily Tanner-Smith, and Robert Boruch. 
“Interventions in Developing Nations for Improving Primary and Secondary School 
Enrollment of Children: A Systematic Review.” Campbell Systematic Reviews, vol. 8, no. 19, 
2012. 
Prouty, Diane and Karen Tietjen. “School Dropout Prevention Pilot Program Pilot Design Plan: 
Timor-Leste.” Report submitted to the U.S. Agency for International Development. 
Washington, DC: Creative Associates International, Inc., 2012.
Schochet, Peter Z. “An Approach for Addressing the Multiple Testing Problem in Social Policy 
Impact Evaluations.” Evaluation Review, vol. 33, no. 6, 2009.
Schochet, Peter Z., and John Burghardt. “Using Propensity Scoring to Estimate Program-Related 
Subgroup Impacts in Experimental Program Evaluations.” Evaluation Review, vol. 31, no. 
2, 2007, pp. 95-120. 
Schultz, T. Paul. “School subsidies for the poor: Evaluating a Mexican strategy for reducing 
poverty.” Washington, DC: International Food Policy Research Institute, 2001. 
Shin, Jennifer, Rajani Shrestha, and Karen Tietjen. “School Dropout Prevention Pilot Program 
Dropout Trend Analysis: India.” Final report submitted to the U.S. Agency for International 
Development. Washington, DC: Creative Associates International, Inc., 2011a.
Shin, Jennifer, Rajani Shrestha, and Karen Tietjen. “School Dropout Prevention Pilot Program 
Dropout Trend Analysis: Timor-Leste.” Final report submitted to the U.S. Agency for 
International Development. Washington, DC: Creative Associates International, Inc., 2011b.
Shrestha, Rajani, Jennifer Shin, and Karen Tietjen. “School Dropout Prevention Pilot Program 
Dropout Trend Analysis: Cambodia.” Final report submitted to the U.S. Agency for 
International Development. Washington, DC: Creative Associates International, Inc., 2011a. 
Shrestha, Rajani, Jennifer Shin, and Karen Tietjen “School Dropout Prevention Pilot Program 
Dropout Trend Analysis: Tajikistan.” Final report submitted to the U.S. Agency for 
International Development. Washington, DC: Creative Associates International, Inc., 2011b. 
Tietjen, Karen. “School Dropout Prevention Pilot Program: Quarterly Report, January 1, 2012–
March 31, 2012.” Final report submitted to the U.S. Agency for International Development. 
Washington, DC: Creative Associates International, Inc., 2012. 
Tschannen-Moran, Megan and Anita Woolfolk Hoy. “Teacher Efficacy: Capturing an Elusive 
Construct.” Teaching and Teacher Education, vol. 17, 2001, pp. 783-805.
Findings from the School Dropout Prevention Pilot Program Impact Evaluation in Timor-Leste Page 76
Tsigilis, Nikolaos, Athanasios Koustelios, and Vasilios Grammatikopoulos. “Psychometric 
Properties of the Teachers’ Sense of Efficacy Scale Within the Greek Educational 
Context.” Journal of Psychoeducational Assessment, vol. 28, no. 2, 2010, pp. 153-162.