IMPAQ International, LLC Page i Food For Education Baseline Report McGovern-Dole International Food for Education LEAPS II Project – Performance and Impact Evaluations 2016 – 2021 Lao PDR Performance Evaluation Baseline Report May 2017 Authors: Elnaz Safarha, M.S. Maria DiFuccia, M.A. Elizabeth Saint George, Ph.D. Michaela Gulemetova, Ph.D. Submitted by: Project Director: Elnaz Safarha IMPAQ International, LLC Submitted to: Chief of Party: Jessica Garrels Catholic Relief Services – Laos McGovern-Dole International Food for Education FY 2016 LEAPS II Project in Lao PDR Baseline Evaluation Program: McGovern-Dole International Food for Education and Child Nutrition Agreement Number: FFE-439-2016/009-00 Funding Year: Fiscal Year 2016 Project Duration: 2016-2021 Implemented by: Catholic Relief Services Evaluation Authored by: IMPAQ International LLC Elnaz Safarha DISCLAIMER: This evaluation was conducted by an independent third party firm. The views expressed in this report do not necessarily reflect the views of the United States Department of Agriculture or the United States Government. Accessibility Note: An accessible version of this report can be made available upon request by emailing FAS.monitoring.evaluation@usda.gov. IMPAQ International, LLC Page ii Food For Education Baseline Report TABLE OF CONTENTS ACKNOWLEDGEMENTS............................................................................................................. i TABLE OF CONTENTS ............................................................................................................... ii TABLE OF EXHIBITS .................................................................................................................. v ACRONYM LIST ....................................................................................................................... vi EXECUTIVE SUMMARY............................................................................................................vii Outcomes for Key Indicators ....................................................................................................viii Additional Key Findings.............................................................................................................viii Recommendations for the Project............................................................................................viii Evaluation-Specific Recommendations ......................................................................................ix Evaluation-Specific Recommendations with Budget Implications..............................................x SECTION 1. INTRODUCTION..................................................................................................... 1 Background on the LEAPS II Intervention................................................................................... 1 Evaluation Background ............................................................................................................... 1 SECTION 2. EVALUATION METHODOLOGY............................................................................... 3 2.1 Quantitative Approach.......................................................................................................... 3 2.1.1 Research Questions .....................................................................................................................3 2.1.2 Sampling Design...........................................................................................................................3 2.1.3 Data Sources ................................................................................................................................6 2.1.3.1 Student Survey and LBRA..........................................................................................................6 2.1.3.2 Classroom Observations ...........................................................................................................7 2.2 Qualitative Assessment......................................................................................................... 7 2.2.1 Qualitative Sampling Design2.2.1.1 Identification of Schools.....................................................8 2.2.1.2 Identification of Individuals ......................................................................................................8 SECTION 3. FIELD WORK AND ANALYSIS .................................................................................. 9 3.1 Field Work ............................................................................................................................. 9 3.1.1 Quantitative Data Collection........................................................................................................9 3.1.2 Qualitative Data Collection ........................................................................................................10 3.2 Quantitative Analysis.......................................................................................................... 10 3.3 Qualitative Analysis............................................................................................................. 10 SECTION 4. EVALUATION SAMPLES........................................................................................ 11 IMPAQ International, LLC Page iii Food For Education Baseline Report 4.1 Schools................................................................................................................................ 11 4.2 Students.............................................................................................................................. 12 4.3 Household Environment..................................................................................................... 13 4.3.1 Household Size...........................................................................................................................13 4.3.2 Language Spoken at Home ........................................................................................................13 4.3.3 Socioeconomic Status................................................................................................................14 SECTION 5. QUANTITATIVE OUTCOMES................................................................................. 15 5.1 School Environment............................................................................................................ 16 5.1.1 Student Attitudes toward Schooling..........................................................................................16 5.1.2 School Libraries..........................................................................................................................16 5.1.3 Classroom Activities...................................................................................................................16 5.2 Household Environment..................................................................................................... 17 5.2.1 Access to Reading Materials......................................................................................................17 5.2.2 Home Literacy Environment ......................................................................................................18 5.3 Students’ Outcomes............................................................................................................ 19 5.3.1 Student’s Reading Assessment Outcomes.................................................................................19 5.3.1.1 Letter Knowledge....................................................................................................................21 5.3.1.2 Reading Outcomes..................................................................................................................22 5.3.1.3 Comprehension.......................................................................................................................23 5.3.2 Attentiveness.............................................................................................................................24 5.3.3 Food Security .............................................................................................................................26 5.3.4 Health.........................................................................................................................................27 SECTION 6. QUALITATIVE OUTCOMES ................................................................................... 29 6.1 Value of Education and School Attendance ....................................................................... 29 6.1.1 Motivation for Parents to Send their Children to School ..........................................................29 6.1.2 Disincentives for Parents ...........................................................................................................30 6.1.3 Aspirations of Parents................................................................................................................31 6.1.4 Motivation for Students to Attend ............................................................................................31 6.1.5 Gender and Age Factors.............................................................................................................32 6.2 Incentivizing VEDC Performance ........................................................................................ 33 6.2.1 VEDC Rolesand Responsibilities.................................................................................................33 6.2.2 VEDC Motivation........................................................................................................................33 IMPAQ International, LLC Page iv Food For Education Baseline Report 6.3 Motivation of School Staff and Volunteers......................................................................... 35 6.3.1 Teachers.....................................................................................................................................35 6.3.2 Cooks..........................................................................................................................................35 6.3.3 Storekeepers..............................................................................................................................36 6.4 Other Emergent Themes..................................................................................................... 37 6.4.1 Requests for Assistance .............................................................................................................37 6.4.2 Concern about the Future of the Program ................................................................................37 6.4.3 Role of Community Mobilizers...................................................................................................37 6.4.4 Discrepancies in Food Allocation ...............................................................................................37 6.4.5 Community Spirit from School Feeding .....................................................................................38 6.5 USDA Feedback ................................................................................................................... 38 SECTION 7. CONCLUSIONS ..................................................................................................... 39 7.1 Key Findings ........................................................................................................................ 39 7.2 Limitations........................................................................................................................... 40 7.3 Recommendations.............................................................................................................. 40 7.3.1 Recommendations for the Project.............................................................................................40 7.3.2 Evaluation-Specific Recommendations......................................................................................41 7.3.3 Evaluation-Specific Recommendations with Budget Implications ............................................42 APPENDICES .......................................................................................................................... 43 APPENDIX 1: LEAPS II Evaluation Indicators........................................................................... 44 APPENDIX 2: References........................................................................................................ 60 APPENDIX 3: List of Stakeholders .......................................................................................... 62 APPENDIX 4: Additional Tables and Complementary Outcomes............................................ 64 APPENDIX 5: Other Subtests of Reading Assessment............................................................. 75 APPENDIX 6: Inter-rater Reliability ........................................................................................ 78 APPENDIX 7: Survey Instruments........................................................................................... 79 APPENDIX 8: Qualitative Protocols...................................................................................... 102 APPENDIX 9: Evaluation SOW & TOR................................................................................... 114 IMPAQ International, LLC Page v Food For Education Baseline Report TABLE OF EXHIBITS EXHIBIT 1: KEY PERFORMANCE INDICATORS FOR LEAPS II PERFORMANCE INDICATOR ..................................................................3 EXHIBIT 2: LEAPS II TARGETED DISTRICTS FROM THE SAVANNAKHET PROVINCE ..........................................................................4 EXHIBIT 3: SAMPLE COMPOSITION BY SCHOOL TYPE ................................................................................................................5 EXHIBIT 4: SAMPLING RULES FOR SELECTING STUDENTS BY SCHOOL, GRADE, AND GENDER............................................................6 EXHIBIT 5: QUALITATIVE ELEMENTS FOR THE BASELINE PERFORMANCE EVALUATION.....................................................................7 EXHIBIT 6: RESPONDENTS BY TYPE AND GENDER .....................................................................................................................8 EXHIBIT 7: SAMPLE DISTRIBUTION BY DISTRICT AND TYPE OF RESPONDENT ...............................................................................11 EXHIBIT 8: STUDENT GENDER DISTRIBUTION BY GRADE..........................................................................................................12 EXHIBIT 9: STUDENTS’ AGE DISTRIBUTION BY GRADE.............................................................................................................12 EXHIBIT 10: DISTRIBUTION OF MAIN LANGUAGE SPOKEN AT HOME .........................................................................................13 EXHIBIT 11: BASELINE LEVELS FOR MCGOVERN-DOLE PERFORMANCE INDICATORS.....................................................................15 EXHIBIT 12: CLASSROOM ACTIVITIES IN THE LAST WEEK .........................................................................................................17 EXHIBIT 13: READING MATERIALS AT HOME BY MAIN LANGUAGE ...........................................................................................18 EXHIBIT 14: HOME LITERACY ENVIRONMENT BY MAIN LANGUAGE AT HOME.............................................................................19 EXHIBIT 15: SECOND GRADE STUDENTS’ LITERACY SKILLS OVERVIEW .......................................................................................20 EXHIBIT 16: LETTER KNOWLEDGE BY LANGUAGE ...................................................................................................................21 EXHIBIT 17: DISTRIBUTION OF LETTER SCORES ......................................................................................................................22 EXHIBIT 18: FLUENCY AND ACCURACY BY MAIN LANGUAGE AT HOME ......................................................................................23 EXHIBIT 19: COMPREHENSION QUESTIONS ANSWERED CORRECTLY BY READERS AND NONREADERS ..............................................23 EXHIBIT 20: READING PROFICIENCY ....................................................................................................................................24 EXHIBIT21: STUDENT ACTIVITIES OBSERVED.........................................................................................................................25 EXHIBIT 22: STUDENT ATTENTIVENESS BY STUDENTS’ ACTIVITY ...............................................................................................25 EXHIBIT 23: STUDENT ATTENTIVENESS BY TEACHERS’ ACTIVITY................................................................................................26 EXHIBIT 24: STUDENT ATTENTIVENESS BY SUBJECT ................................................................................................................26 EXHIBIT 25: STUDENT SELF-REPORTED FOOD INTAKE .............................................................................................................27 EXHIBIT 26: PROPORTION OF SICK STUDENTS AND THEIR ILLNESSES..........................................................................................28 EXHIBIT 27: SCHOOL COMPLETION BY DISTRICT ....................................................................................................................31 EXHIBIT 28: OPTIONS CHOSEN BY VEDC MEMBERS...............................................................................................................34 EXHIBIT 29: LEAPS II PERFORMANCE INDICATORS ................................................................................................................44 EXHIBIT 30: KEY INFORMANTS AT SCHOOL LEVEL ..................................................................................................................62 EXHIBIT 31: KEY REQUESTS AND PRIORITIES FROM SCHOOLS FOR IMPROVEMENT .......................................................................64 EXHIBIT 32: SOCIOECONOMIC STATUS BY DISTRICT................................................................................................................68 EXHIBIT 33: SOCIOECONOMIC STATUS BY CONSUMER DURABLE GOODS ...................................................................................68 EXHIBIT 34: NON-PRIMARY LANGUAGES SPOKEN IN HOUSEHOLD ............................................................................................68 EXHIBIT 35: REASONS STUDENTS LIKED GOING TO SCHOOL .....................................................................................................69 EXHIBIT 36: ACCESS TO READING MATERIAL AT HOME ..........................................................................................................69 EXHIBIT 37: HOME LITERACY ENVIRONMENT........................................................................................................................70 EXHIBIT 38: LITERACY SKILLS BY MAIN LANGUAGE AT HOME...................................................................................................71 EXHIBIT 39: DIFFERENT TYPES OF COMPREHENSION QUESTIONS BY READERS AND LANGUAGE......................................................72 EXHIBIT 40: DIFFERENT TYPES OF COMPREHENSION QUESTIONS BY NON-READERS AND LANGUAGE ..............................................72 EXHIBIT 41: TEACHER NARRATING A STORY/READING A POEM IN CLASS ..................................................................................73 EXHIBIT 42: TEACHERS ASKING ABOUT THE STORY NARRATED/POEM READ IN CLASS THE PAST WEEK .........................................73 EXHIBIT 43: STUDENTS PLAYING A GAME IN THE CLASSROOM AROUND ALPHABETS ...................................................................73 EXHIBIT 44: ACCESS TO READING MATERIALS BY DISTRICT......................................................................................................74 EXHIBIT 45: NUMBER OF DAYS CHILDREN MISSED SCHOOL DUE TO SICKNESS ............................................................................74 EXHIBIT 46: EXPRESSIVE VOCABULARY SKILLS BY LANGUAGE...................................................................................................75 EXHIBIT 47: PHONEMIC AWARENESS BY LANGUAGE ..............................................................................................................76 EXHIBIT 48: MOST COMMON USED WORDS DISTRIBUTION BY MAIN LANGUAGE.......................................................................77 EXHIBIT 49: INTER-RATER RELIABILITY BY LITERACY SKILL SUBTESTS..........................................................................................78 IMPAQ International, LLC Page vi Food for Education Baseline Report ACRONYM LIST ANOVA Analysis of Variance CM Community Mobilizer CRS Catholic Relief Services ECD Early Childhood Development EGRA Early Grade Reading Assessment EMC Emerging Market Consultants FANTA Food and Nutrition Technical Assistance FFE Food for Education FGD Focus Group Discussion KII Key Informant Interview LB Literacy Boost LBRA Literacy Boost Reading Assessment MEAL Monitoring, Evaluation, Accountability, and Learning MGD McGovern-Dole PESS Provincial Education and Sports Service SCI TOR Save the Children International Terms of Reference USDA United States Department of Agriculture VEDC Village Education Development Committee IMPAQ International, LLC Page vii Food for Education Baseline Report EXECUTIVE SUMMARY This report describes the baseline findings of the performance evaluation of the Learning and Engaging All in Primary School (LEAPS) II project in Lao PDR. LEAPS II is a five-year project (2016 – 2021) funded by the United States Department of Agriculture McGovern-Dole International Food for Education program. LEAPS II is being implemented by Catholic Relief Services (CRS) together with government partners, Save the Children International (SCI), and the University of Oregon. CRS selected IMPAQ International (IMPAQ) to design and conduct a performance and an impact evaluation of the project. The purpose of this report is to document our approach and present baseline values for key variables for the performance evaluation. These values will be critical as they will serve as a basis for measuring the overall success of the project. The overarching objective of LEAPS II is to improve the literacy skills of over 86,000 new and continuing students in 350 schools across seven educationally disadvantaged districts of Laos’ Savannakhet province. The goals of the intervention include the following: (1) improve the quality of instruction with an intensive model of training, (2) increase enrollment of children in schools through greater inclusion of children with disabilities and other traditionally excluded groups, and (3) maintain strong attendance by ensuring a healthy and hunger-free learning environment for all students. The five key research questions for the baseline performance evaluation are:  What are the baseline levels of letter identification among second graders?  What are the baseline levels of reading and understanding of second grade level text?  What are the baseline levels of student classroom attentiveness?  What are the baseline levels of prevalence of afternoon hunger among students?  What are the baseline levels of student health-related absences? To answer the above evaluation questions, IMPAQ conducted: 1) key informant interviews (KIIs), 2) focus groups discussions (FGDs), 3) student surveys, including reading assessments, and 4) classroom observations. The student survey collected information on students’ background, food security, health, and learning environment. The IMPAQ team complemented the student survey with the Literacy Boost Reading Assessment (LBRA). The LBRA was developed by SCI as a modification of the Early Grade Reading Assessment (EGRA) to measure a variety of grade-level reading and literacy skills. We also created a classroom observation tool to measure student attentiveness, using a time-sampling technique to observe the percentage of attentive students at classroom level and to capture factors that might contribute to attentiveness. Finally, our evaluation approach verifies project design assumptions and identifies potential threats to implementation. We supplement the KII and FGD discussions guides with questions about the attitudes and motivations of the stakeholders. In March 2017, IMPAQ collected data from 1,962 students from 87 schools, including 496 second-graders who also took the LBRA. We also collected data from 40 mothers, 35 fathers, 30 village education development committee (VEDC) members, 17 teachers, 7 principals, 17 cooks, and 7 storekeepers. Some key findings from the baseline data analysis are outlined below. IMPAQ International, LLC Page viii Food for Education Baseline Report Outcomes for Key Indicators  While 47 percent of students demonstrated proficiency in identifying letters by the end of two grades of primary schooling, only 3 percent of students demonstrated that they could read and understand grade-level text. This finding suggests that nearly all students are behind the attainment level they are expected to reach by the end of second grade.  The student attentiveness rate was 84 percent, with girls reported to be slightly more attentive (87 percent) than boys (82 percent). Students seemed more attentive during educative games, dictation, repetition, story time, and discussion.  Missing lunch and feeling hungry were not as prevalent as anticipated. Almost all students said that they ate breakfast (96 percent), and only 3 percent said they could have eaten more. In the afternoon, 85 percent of students reported that they ate lunch, but only four out of 10 did so in school1 . Six out of 10 students reported having lunch outside of school. Only 7 percent of students indicated that they were “somewhat” or “very” hungry during their afternoon class.  Students reported often missing school due to health-related absences. The most cited illness was fever, followed by headaches. Approximately 1 in 3 students reported being sick during the previous week and reported they missed 2 school days, on average, because of their illness. Additional Key Findings  Community spirit and engagement were highly valued and programs were likely to be more successful when they reinforced a sense of community.  The vast majority of parents valued schooling for their children, but cited the cost of schooling as a major barrier to their children’s continued education.  Cooks and storekeepers in particular appreciated the take-home rations as recognition and material support for their participation in the program. Teachers and principals also appreciated the take-home rations; however, except in the case of volunteer teachers, they did not see the rations as vital. Based on lessons learned from our experience in the field and after analyzing the baseline data, we developed the following recommendations for CRS. Recommendations for the Project  Provide additional training for rotating cooks. In our focus groups with cooks, there was lower satisfaction among rotating cooks with the types of food provided and more requests for additional types of food. In these schools, we recommend training be provided on a more periodic basis rather than once an academic year. Periodic training reinforces the training material for a larger number of cooks, and ensures adequate knowledge of food preparation and methods for preparing and supplementing rice and lentils.  Ensure storekeepers and other relevant stakeholders understand how to use supporting materials for program delivery. There was a discrepancy in the amount of rice storekeepers and others believed should be allocated for each child. The project should ensure that the simple chart created by CRS is displayed in the storeroom. The chart provides the number of kilos of rice, lentils, and oil to be allocated according to the number of children at the school. All relevant 1At the baseline data collection, schools had not yet received LEAPS II food, and were cooking their remaining commodities from LEAPS I. IMPAQ International, LLC Page ix Food for Education Baseline Report stakeholders (teachers, storekeepers and others engaged in ration distribution) should be able to understand the chart.  Explore additional opportunities to supplement the school rations with locally grown nutritional food. Some schools had a well-organized system of supplementing the commodities, others were not clear on how to do this. Locally grown foods may be easier for cooks to prepare and more palatable for the children.2  Reduce the administrative burden on teachers. The qualitative findings showed that teachers are often drawn into different administrative aspects of the program to supplement village capacity. To ensure that the school feeding program does not create an additional burden for teachers, CRS should explore approaches on a case-by-case basis to reduce the school feeding program tasks undertaken by teachers and encourage villages to have a back-up option when the community member responsible is unable to undertake their duties.  The use of non-monetary performance incentives may help to increase VEDC performance. VEDC and other community members reported non-monetary incentives such as seeing the village working together in a spirit of cooperation as the most motivating element of their work as VEDC members. For example, CRS could consider annual village awards or recognition to whole villages achieving the greatest improvement in students’ attendance, establishing the most attractive area for eating lunch or developing the most delicious methods for cooking lentils. Money was considered important for being able to complete an activity.  LEAPS II activities should be implemented from a holistic community perspective. Building on the recommendation above, it is rare for any activity to be undertaken by a single group of stakeholders. Receipt of the food delivery in the school, distribution of the food, review of monthly ledgers are all activities that are likely to engage several stakeholder groups within the village. All VEDC activities require close cooperation between teachers, parents and VEDC members to be successful. Incentives should ensure that they reinforce the collaboration between these groups.  Explore opportunities for improving the outcomes of speakers of Lao as a second language. The findings showed that children whose language at home was not Lao consistently had lower literacy outcomes during the literacy testing. Children whose main language at home is not Lao are exposed to teaching at school in a language they are not familiar with. There are a wide variety of interventions that may help to close the gap between these children and those whose main language spoken at home is Lao, such as improving teaching skills, increasing the availability of appropriate school materials, additional preparatory schooling for students, etc. Evaluation-Specific Recommendations  Sufficient time is needed to prepare for data collection field activities. More time (two to three weeks) is needed to prepare for data collection activities, anticipate challenges in the field, consult with all partners, and come up with solutions. Our experience shows that additional days for training and time to explain all lingering questions to enumerators results in minimizing errors in the field and ensuring greater consistency in the collected data. This additional time allows for more practice and comparison between the enumerators. This can increase the interrater 2 Note that this recommendation is based on interviews and focus group discussions with stakeholders at baseline – before the start of LEAPS II. CRS plans to promote community contributions and school gardens to provide fruits and vegetables to complement the meals. IMPAQ International, LLC Page x Food for Education Baseline Report reliability across the fielded instruments, including classroom observations and the LBRA. As we learn about the local landscape, additional time allows for more optimal logistical planning.  A longer window is necessary to complete data collection in the field. In order to randomly select students and create unique student identifiers during school visits, the data collection partner needs more time (at least a month) to ensure the rigor of the process. Additionally, future data collection rounds may occur during the rainy season. These rounds may require more time to overcome challenges in the field such as muddy roads and inaccessible schools.  The LBRA needs additional cognitive testing for the Lao context. Our field experience showed that the passage used in the reading assessment was not tested prior to fielding, so some words appeared difficult to students. We recommend that SCI cognitively test and pilot the tool before fielding it again for the impact evaluation to ensure that we are capturing valid and reliable reading data.  A proportional number of schools where Lao is not the primary language should be added to the qualitative sample. Future qualitative samples should include more schools in villages where Lao is not spoken as the first language, in proportion to the total number of these villages receiving the program.  Additional stakeholder observations should be included in further rounds of data collection. Future qualitative studies should include observation of school feeding activities, including food preparation and other stakeholder participation in the program.  The enrollment list at participating schools should be updated on a more regular basis. During baseline data collection, we found more schools than previously expected that did not have all five grades, and some schools had different numbers of students from the numbers in the school lists. Collecting and quality-checking monitoring data more frequently (monthly or quarterly) would ensure that school lists are up to date. Evaluation-Specific Recommendations with Budget Implications  Data from parents and teachers on other factors that affect children’s literacy should be collected. Characteristics such as parent’s educational attainment, home accessibility to water, and presence of a latrine at home can be important drivers of child outcomes. Similarly, data from teachers on their educational background, years of experience, and ethnicity may be significant predictors of success. If added to the scope of work and budget, we will control for these characteristics in future rounds of data collection and analysis.  The scope of work and budget should be amended to include surveying mothers at midline and endline. Collecting data from young children is often unreliable (e.g., a large number of children reported being absent due to illness; many children were confused about whether or not their school had a library; etc.). Being able to triangulate this data with mothers’ responses will help ensure more accurate data. IMPAQ International, LLC Page 1 Food for Education Baseline Report SECTION 1. INTRODUCTION This report describes the baseline findings of the performance evaluation of the LEAPS II project in Lao PDR. LEAPS II is a five-year project (2016 – 2021) funded by the USDA McGovern-Dole International Food for Education program. The introduction section provides a brief overview of the program context for the baseline performance evaluation. Section 2 outlines our evaluation methodology, including research questions, sampling design, and data tools. In Section 3, we describe field work for data collection and our data analysis. Section 4 describes the samples and their key characteristics. In Sections 5 and 6, we present the quantitative and qualitative outcomes. Section 7 concludes with lessons learned, study limitations, and recommendations. Background on the LEAPS II Intervention LEAPS II is implemented by CRS together with government partners, SCI, and the University of Oregon. The objective of LEAPS II is to improve the literacy skills of over 86,000 new and continuing students in 350 schools across seven educationally disadvantaged districts of Laos’ Savannakhet province. The purpose of the interventions being implemented under LEAPS II is to ensure that schools have access to resources and community support. The goals include the following: (1) improve quality of instruction with an intensive model of training, (2) increase enrollment of children in schools through greater inclusion of children with disabilities and other traditionally excluded groups, and (3) maintain strong attendance by ensuring a healthy and hunger-free learning environment for all students. For LEAPS II, CRS is building on the gains already achieved in its earlier project, LEAPS I. During 2012 – 2016, LEAPS I reached over 36,000 students in 310 schools in Savannakhet province. The evaluation of LEAPS I showed that the program has increased boys’ and girls’ attendance by 38 percent and has enhanced reading skills through approaches that empower government, parents, teachers, and community leaders. LEAPS II is set to continue the effective interventions from LEAPS I (i.e., school feeding with teacher training), while also incorporating new strategies built on lessons learned and identified needs. Some of the new activities include: (1) expanding project coverage to approximately 3,400 additional students in 46 newly targeted schools in educationally disadvantaged Nong district; (2) intensifying and scaling-up efforts to develop the pedagogical skills of teachers and administrators in literacy instruction through the Literacy Boost program; (3) building a supportive community environment for language and learning; (4) strengthening the capacities of government, school, and community actors in inclusive education practices and techniques; and (5) improving access to water and sanitation in target schools. CRS designed LEAPS II with a focus on sustainability. CRS is working with government and community stakeholders to phase-in key interventions, strengthen key capacities, commitments, and resources to promote student success after the end of the program. Evaluation Background CRS selected IMPAQ to design and conduct performance and impact evaluations of the project. While the performance and impact evaluations were designed in parallel to maximize comparability in the outcome indicators and findings, they follow slightly different timelines and will be discussed in separate reports. The impact evaluation, which measures the causal effect of the project’s Literacy Boost and/or school feeding interventions on literacy of school-aged children, will span only two data collection periods – baseline and follow up – while the performance evaluation spans three– baseline, midline, and endline. IMPAQ International, LLC Page 2 Food for Education Baseline Report The purpose of this report is to document our approach and present baseline values for key variables of the performance evaluation. These values will be critical as they will serve as a basis for measuring the overall success of the project. The baseline, midline, and endline rounds of the performance evaluation are structured and sequenced to measure the changes in outcomes over time and to inform the overall evaluation results on LEAPS II core activities. To accurately capture program performance over time, IMPAQ will measure the same program indicators at all three data collection points. The three core objectives of the baseline evaluation are: 1) To set benchmark values for performance indicators, 2) To verify project design assumptions and identify potential threats to implementation, and 3) To allow longitudinal assessment of project outcomes and impacts. To address these objectives, IMPAQ collected quantitative survey data and classroom observations as well as qualitative interview and focus group data. The same quantitative indicators will be collected and reported over time, but some qualitative performance evaluation questions about lessons learned and sustainability will be different at baseline, midline, and endline. At baseline, questions focused on verifying project design assumptions and identifying potential threats to implementation, including formative feedback on planned activities. At midline, questions will focus on learning what has occurred to date, including a formative assessment and suggestions for program process improvement. At endline, questions will focus on learning more about the potential for program sustainability and about promising practices and lessons learned. IMPAQ International, LLC Page 3 Food for Education Baseline Report SECTION 2. EVALUATION METHODOLOGY 2.1 Quantitative Approach The performance evaluation measures changes in desired outcomes related to LEAPS II core activities over the life of the project. To accurately reflect program performance over time, IMPAQ will measure the same program indicators using the same methodology at all three data collection points - baseline, midline, and endline. 2.1.1 Research Questions The five key research questions for the baseline performance evaluation are: • What are the baseline levels in reading and understanding of second-grade-level text? • What are the baseline levels in letter identification among second graders? • What are the baseline levels in student classroom attentiveness? • What are the baseline levels in prevalence of afternoon hunger among students? • What are the baseline levels in student health-related absences? To answer the evaluation questions, the performance evaluation needs to provide evidence addressing the key indicators described in Exhibit 1. Exhibit 1: Key Performance Indicators for LEAPS II Performance Indicator Data Source Percent of students who, by the end of two grades of primary schooling, demonstrate that they can read and understand grade￾level text Literacy Boost Reading Assessment (Grade 2) Percent of students who, by the end of two grades of primary schooling, demonstrate proficiency in identifying letters Literacy Boost Reading Assessment (Grade 2) Percent of students who are attentive in the classroom Classroom observations of (Grades 1 – 5) Percent of students reporting that they are “somewhat” or “very” hungry during their afternoon class Student survey of (Grades 1 – 5) Percent of students in target schools reporting health-related absences Student survey of (Grades 1 – 5) Source: CRS Terms of Reference 2.1.2 Sampling Design For the performance evaluation, we applied two sampling schemes for two distinct target populations: the first is to collect data from a sample of students in Grades 1 through 5 across 380 schools initially considered to be part of LEAPS II in all seven districts of Savannakhet province and the second is to collect data from a sample of Grade 2 students across a subset of 186 schools considered to receive the Literacy IMPAQ International, LLC Page 4 Food for Education Baseline Report Boost intervention in six districts of Savannakhet.3 Exhibit 2 shows the seven districts in Savannakhet province that are included in the evaluation. Exhibit 2: LEAPS II Targeted Districts from the Savannakhet Province Because the 186 schools that considered to receive the Literacy Boost (LB) intervention are not a random sample of the 380 schools,4 they may differ in observable and unobservable characteristics from the rest of the schools. To select our sample of schools for the performance evaluation, we drew from these two distinct sampling frames. It was necessary to select a sample of schools which will be representative of the 186 LB schools and a sample that is representative of all 380 schools receiving the school feeding activities. Per CRS initial calculations and in accordance with the Food and Nutrition Technical Assistance (FANTA) sampling guidance, using power calculations, sufficient sample sizes were determined to be: 490 students from Grade 2 in 49 schools out of all LB schools for an expected design effect of 3.15, and 1,525 students from Grades 1 through 5 in 61 out of all schools receiving feeding activities schools for an expected design effect of 2.00.5 The other parameters included in the power calculation include an alpha of 0.05 (for a 95% confidence level), a desired power of 80%, and a contingency factor of 10% for non-response. 3 IMPAQ received a database of 380 schools including 186 potential LB schools and 194 potential schools with feeding activities only on January 25, 2017. Eventually, CRS selected 350 schools (of the initial 380) for implementations including 180 LB schools and 170 SF only schools. 4 In two districts where we will implement the impact evaluation, Sepon and Atsaphone, schools were randomly assigned to LB intervention and school feeding activities. We will discuss this in more detail in the impact evaluation report. 5 The design effect accounts for the use of cluster sampling instead of simple random sampling. The sample size for the student survey is computed using a default design effect of 2. The expected design effect for the LB assessment sample is set to 3.15 based on recommendation provided in the TOR. The TOR specified an expected design effect of 3.15 based on SCI’s experience conducting the LB assessment in Laos. IMPAQ International, LLC Page 5 Food for Education Baseline Report We used a two-stage sampling approach for the performance evaluation. Because students are nested within schools, in the first sampling stage, we drew a number of schools to be sampled; in the second stage, we drew a number of students to be surveyed within each selected school. In the first stage, we selected the schools with a probability-proportional-to-size clustered sampling approach, using the total number of students per school as a school size measure. The procedure for selecting the schools was systematic-random sampling, which involves choosing schools from a list using a sampling interval. Starting from the school on the top of the list, we selected subsequent schools by adding the sampling interval to the cumulative measure of school size. In order to minimize the costs related to travel and time to cover all selected schools, we developed an algorithm to check for the rate of overlap between the two samples. The rate of overlap ranges between 0 percent (when none of the 49 LB schools are among the 61 schoolssampled to assess other performance indicators) and 100 percent (when all 49 LB schools are included among the 61 schools). As the systematic￾random sampling is sensitive to the school ordering, we performed over 5,000 simulations with different school orders to retain the iteration that had the highest rate of overlap. This way, we are certain that the final sample is representative of all 380 schools and of the 186 LB schools, so that this sample is the most cost-effective given limited resources for data collection. Therefore, the schools in our final sample can be seen as three separate groups of school feeding (SF) and LB schools: SF + LB schools for the Literacy Boost sample only, SF + LB schools to be included in both samples, and SF schools for the overall sample only (See Exhibit 3 of overlap). The final sample after the simulations include a total of 87 schools. It consists of 38 schools to be part of the SF sample only, 26 schools to be part of the LB sample only, and 23 schools to be part of both samples. In the second stage, at each sampled school, we selected a sample of students by physically lining up boys and girls separately for each grade in their classrooms. To identify the n th student for random selection, we used a simple rule as follows: 𝑛 𝑡ℎ𝑔𝑖𝑟𝑙 𝑜𝑟 𝑏𝑜𝑦 𝑡𝑜 𝑠𝑎𝑚𝑝𝑙𝑒 = 𝑇𝑜𝑡𝑎𝑙 𝑛𝑢𝑚𝑏𝑒𝑟 𝑜𝑓 𝑔𝑖𝑟𝑙𝑠 𝑜𝑟 𝑏𝑜𝑦𝑠 𝑖𝑛 𝑒𝑎𝑐ℎ 𝑔𝑟𝑎𝑑𝑒 𝑇𝑜𝑡𝑎𝑙 𝑛𝑢𝑚𝑏𝑒𝑟 𝑜𝑓 𝑔𝑖𝑟𝑙𝑠 𝑜𝑟 𝑏𝑜𝑦𝑠 𝑡𝑜 𝑏𝑒 𝑠𝑒𝑙𝑒𝑐𝑡𝑒𝑑 For example, if there were 10 female second graders and we required five for the study (as in SF + LB schools for the Literacy Boost sample only ), then we selected every other student from the line of second￾grade girls (10/5 = 2). We applied the same rule to select students systematically from all sampled schools and grades by gender (Exhibit 4). In the absence of electronic class lists, this approach ensured sampling consistency across schools and achieved a random sample of students who were present on the day of data collection. However, the possibility of systematic absences might induce a risk of sampling bias by Exhibit 3: Sample Composition by School Type IMPAQ International, LLC Page 6 Food for Education Baseline Report selecting only present students. For example, such a bias can arise from excluding information on children who are more likely to be absent from school due to health issues. Exhibit 4: Sampling Rules for Selecting Students by School, Grade, and Gender School Sample Group Number of Students to be Selected Number of Boys to be Selected Number of Girls to be Selected SF + LB schools for Literacy Boost sample only 5 boys and 5 girls from Grade 2 Every nth boys to select 5 boys Every nth girls to select 5 girls SF + LB schools for both samples 15 boys and 15 girls from Grades 1 – 5 Every nth boys to select 2 boys in Grade 1 and 4 Every nth boys to select 3 boys in Grade 3 and 5 Every nth boys to select 5 boys in Grade 2 Every nth girls to select 3 girls in Grade 1 and 4 Every nth girls to select 2 girls in Grade 3 and 5 Every nth girls to select 5 girls in Grade 2 SF schools for the overall sample 12 boys and 13 girls from Grades 1 – 5 Every nth boys to select 2 boys in Grade 1, 3 and 5 Every nth boys to select 4 boys in Grade 2 and 4 Every nth girls to select 3 girls in Grade 1, 3 and 5 Every nth girls to select 2 girls in Grade 2 and 4 2.1.3 Data Sources To answer the research questions for the baseline performance evaluation and to set benchmark values for performance indicators shown in Exhibit 1, we collected and analyzed data from two sources (included in Appendix 7): a student survey including the LBRA, and a classroom observation tool. 2.1.3.1 Student Survey and LBRA The student survey collected data on students’ background, food security, health, and school and household environments. We included survey questions that have already been field-tested and approved for other evaluations by USDA as well as new items specifically designed for this evaluation. We adapted the questions to the Lao context through cognitive interviews, prior to data collection, so that the questions are appropriate for local conditions and the results can be compared to other national and international data. In collaboration with CRS, we conducted the cognitive interviews in school areas where students’ main language at home was Lao and non-Lao to identify, in the local context and in different languages, what works, what does not work, and why. During the cognitive testing, the interviewers discussed the meaning of each item with students to assess the clarity of the question and the appropriateness of the proposed categories. To measure the reading skills of children, as part of the student survey, we implemented the LBRA, developed and adapted in Lao context by SCI. After the survey instrument was developed, we pre-tested the instrument in two non-LEAPS schools in Savannakhet that were not participating in either the performance evaluation or the impact study. Pre-testing was designed to ensure that the survey IMPAQ International, LLC Page 7 Food for Education Baseline Report instructions and wording are appropriate and understandable for students who are in the same age range and from the same ethnic minorities as those in the study group. As well, the pre-testing helped enumerators with hands-on practice to prepare for the data collection. After the pre-testing process was completed, the survey instrument was revised to increase the reliability and validity. 2.1.3.2 Classroom Observations To measure student attentiveness, we used a time-sampling technique based on the Stallings “snapshot” method (Stallings et al. 1988). We created a tool to measure the percentage of attentive students and to capture potential factors of attentiveness (other than school feeding) such as class size, subject, class arrangement, and activities. Every three minutes, enumerators documented the classroom activity by recording what both teachers and students were doing. They then went clockwise around the room to count attentive students and distracted students, disaggregated by gender. Each observation lasted 35 minutes to allow completion of 10 snapshots. To determine the percentage of attentive students, we calculated the number of attentive students over the total number of students for all 10 snapshots. 2.2 Qualitative Assessment Our qualitative study complements the quantitative approach by addressing some of its limitations in answering the research questions and by providing context for its results. The qualitative study provides insight into the relevance, effectiveness, and sustainability of LEAPS II — issues that will be addressed primarily by the midline and endline data collection. For the baseline qualitative evaluation, we are guided by Objective 2 in the CRS terms of reference (TOR): “To verify project design assumptions and identify potential threats to implementation.” We have supplemented the evaluation questions with additional questions about the attitudes and motivations of the stakeholders (see Exhibit 5 below). Exhibit 5: Qualitative Elements for the Baseline Performance Evaluation Qualitative Element Data Source 1. How do parents and communities value education? And what are the challenges and supports that schools and families face in sending children to school? Focus group discussions: Parents and VEDC members Key informant interviews: Principals, teachers 2. Will the VEDC grant scheme in LEAPS II incentivize better performance by VEDC members? Focus group discussion: VEDC members, principals 3. What incentives other than take-home rations motivate teachers, cooks, and storekeepers? Key informant interviews: Principals, teachers, cooks, and storekeepers 4. What data would USDA be interested in collecting for the midline and endline qualitative component? Key informant interviews: USDA staff Source: LEAPS II TOR IMPAQ International, LLC Page 8 Food for Education Baseline Report 2.2.1 Qualitative Sampling Design 2.2.1.1 Identification of Schools We collaborated with CRS to identify one school in each of the seven LEAPS II districts for qualitative data collection.6 The schools selected were ones already included in the quantitative sample. We sought to achieve a balance between schools within easy access of the district center and more remote schools. To address the additional research questions listed above, we conducted semi-structured focus group discussions with mothers, with fathers, with VEDC members, with teachers, and with cooks. We also conducted key informant interviews with the principal and the storekeeper at each school. Although the initial plan was to interview only one teacher in each school, teachers expressed greater comfort and provided richer feedback in groups, as they were able to exchange their opinions during the discussion. 2.2.1.2 Identification of Individuals The district official responsible for the school visit contacted the school principal one or two days in advance of the interviews. The principal then coordinated the attendance of relevant individuals with the village head. The selection of individuals was dictated, in the case of the principal and the storekeeper(s), by their position. Following data collection at the first two schools, all teachers were asked to join focus group discussions at the remaining schools. For cooks, VEDC members, fathers, and mothers, the participants self-selected on the basis of their availability and friendship groups, following the request from the district and principal. Some cooks were also mothers at the school, so they participated in both focus group discussions. Some fathers were also VEDC members. Thus there was some crossover in membership among the focus groups. In a number of focus groups with mothers and fathers, grandmothers and grandfathers attended instead. Exhibit 30 in Appendix 4 summarizes the sample and provides notes on the composition of the sample. A summary of the respondents by gender is provided in Exhibit 6 below. Exhibit 6: Respondents by Type and Gender Interview Type Men Women Total Mothers focus group discussion 0 40 40 Fathers focus group discussion 35 0 35 VEDC focus group discussion 24 6 30 Teachers focus group discussion 4 13 17 Cook focus group discussion 0 17 17 Principal key informant interview 3 4 7 Storekeeper key informant interview 7 0 7 Total 73 80 153 In addition, we spoke with three USDA staff to get their feedback on what they would like to be included in future qualitative data collection efforts. 6 Nong was included but had no experience with school feeding or the project yet since they were not part of LEAPS I. IMPAQ International, LLC Page 9 Food for Education Baseline Report SECTION 3. FIELD WORK AND ANALYSIS 3.1 Field Work Prior to collecting data, IMPAQ submitted protocol documents and received approval from Chesapeake Institutional Review Board (IRB) on February 28, 2017 (Pro00020840). The IRB submission was to ensure there are no ethical issues with any component of LEAPS II evaluation. The approved documentation for this evaluation includes:  Student survey and LBRA,  Classroom Observation tool,  Informed consent forms,  Quantitative protocol, and  Qualitative protocol. We used the IRB approved instruments to collect data. 3.1.1 Quantitative Data Collection In collaboration with SCI, we trained 12 enumerators, recruited by our data collection partner Emerging Market Consulting (EMC) in February and March 2017. The training consisted of three parts. SCI took the lead on the first four days to train enumerators on the LBRA, including pilot testing and a debriefing session. The IMPAQ classroom observation expert took the lead on the second part to train enumerators on classroom observations for three days. This part consisted of two days of theoretical training in real classrooms, one day of pilot testing at a school in Phalanxai district, and one day of debriefing and instrument refinement. Because of the complexity of classroom observation data collection, the nine most capable enumerators, including the three team leaders, received the training. After this, the IMPAQ quantitative specialist took the lead for another three days to train enumerators on other student survey sections. Enumerators received training on how to collect data on paper and tablets, but they used tablets to conduct the in-person surveys and electronically submitted the surveys periodically during the field work. EMC organized the enumerators into three teams of five7 individuals, including one team leader per team. Two Mon-Khmer speaking enumerators were included in the data collection teams, with primary responsibility for interviewing Grade 2 students whose Lao language skills were not as strong. The IMPAQ country expert and fieldwork manager closely followed the teams of enumerators to oversee data quality and provide enumerators with technical support. The CRS Monitoring, Evaluation, Accountability, and Learning (MEAL) officer and a Provincial Education and Sports Service (PESS) representative accompanied the fieldwork team to facilitate informal meetings between EMC and the District Education and Sports Bureaus (DESBs) as well as coordinate school visits in each target district to support the field visits. All enumerators regrouped with their supervisory teams in their village several times during the data collection to debrief, submit daily data collection logs, submit electronic surveys, and review and plan for the next days of data collection. The team completed field work in 15 working days. 7 EMC added three more enumerators at the end to increase the pace of the data collection. The new enumerators did not collect classroom observations, since they were not part of the original training. IMPAQ International, LLC Page 10 Food for Education Baseline Report 3.1.2 Qualitative Data Collection Two IMPAQ researchers collected data in seven schools in Savannakhet in March. Both interviewers took notes during the interviews and coordinated with each other to ask questions. This approach provided the dual advantage of keeping the conversation flowing naturally with respondents and allowing interviewers to cross-check their understandings as they jointly wrote up the interviews. The interviews were also recorded, when respondents were comfortable with this approach, in order to add a further layer of certainty that all key information was captured. The responses were then organized thematically according to the key questions of interest to CRS, with the addition of issues that emerged during the interviews. The thematic organization of notes was summarized directly into the final baseline report. 3.2 Quantitative Analysis This baseline report provides summary statistics, as well as constructed outcomes (percentages and averages) using individual or multiple survey items through programming Stata. In addition, the team conducted subgroup analyses by grade, student gender, and district, highlighting emerging patterns by running t-test and using p-values. 3.3 Qualitative Analysis To analyze the interview and focus group notes, we used a structured summary form paralleling the structure of the interview guides. Our summary (Section 6) synthesizes the major themes from the interview and focus group sessions that address the key evaluation questions. We also include verbatim quotes of particular interest. IMPAQ International, LLC Page 11 Food for Education Baseline Report SECTION 4. EVALUATION SAMPLES In this section we provide summary statistics about:  School composition  Student composition  Household environment characteristics 4.1 Schools To set benchmark values for performance indicators, shown in Exhibit 1, Section 2 and to measure progress toward the desired outcomes over time, we selected students from 61 schools targeted for LEAPS II generally and 49 schools targeted for Literacy Boostspecifically. As explained in Section 2.1.2 with overlapped samples, we surveyed students in 87 schools in seven districts of Savannakhet: Atsaphone, Nong, Outhoumphone, Phalanxai, Phin, Vilabouly, and Sepon. We surveyed five students in Grades 1 through 5 in each of 38 schools in the LEAPS II sample and 10 second-grade students in the 26 Literacy Boost schools. In 23 schools included in both samples, we surveyed 10 second-grade students and five students in Grades 1, 3, 4, and 5. While planning the school visits with CRS’ Community Mobilizers (CMs) before the data collection, we realized that two of the 87 schools were inaccessible because of road conditions. As soon as field operations started, we also realized that more schools lacked Grades 4 and/or 5 than indicated on the school list shared by CRS. To ensure a large enough sample, we oversampled other schools in the same treatment conditions following the same random selection rule. With our oversampling strategy, we ended up with a sample of 1,962 students, including 496 second graders who also took the LBRA. All teachers gave their written consent for the children to be surveyed. We also asked students’ verbal assent; only seven students refused8 . Exhibit 7 shows the distribution of sampled respondents by district. Exhibit 7: Sample Distribution by District and Type of Respondent District Number of Schools Surveyed Number of Students Surveyed Atsaphone 19 423 Nong 7 161 Outhoumphone 14 370 Phalanxai 13 255 Phin 14 366 Vilabouly 12 213 Sepon 8 174 Total 87 1962 Source: Student survey; Authors’ calculations 8 In accordance the U.S. Department of Health and Human Services guidelines on Human Subjects Research (45 C.F.R. § 46), we asked all respondents for their consent to proceed with the survey. Human Subject Regulations Decision Charts. (2016, February 16). Retrieved from http://www.hhs.gov/ohrp/policy/checklists/decisioncharts.html IMPAQ International, LLC Page 12 Food for Education Baseline Report 4.2 Students We randomly selected five girls and five boys from second grade in each Literacy Boost targeted school. In addition, in other LEAPS II targeted school (SF overall and SF + LB both samples), we randomly selected a balanced number of girls and boys to reflect the population of beneficiary students (see Exhibit 4 in Section 2.1.2 for gender composition). This sample enabled us to disaggregate the data by students’ gender and to explore differences across grade levels. The 1,962 students surveyed included 1,008 females and 964 males in Grades 1 through 5. Exhibits 8 and 9 show the composition of the student sample in terms of grade, gender, and average age. Although in general the proportion of girls to boys is balanced, there were slightly more female students in first and third grades (57 percent) and more male students in fourth grade (56 percent). Exhibit 8: Student Gender Distribution by Grade Gender Male Female Total Grade Percent Observations Percent Observations 1 st Grade 43% 132 57% 172 304 2 nd Grade 49% 342 51% 351 693 3 rd Grade 43% 135 57% 178 313 4 th Grade 56% 179 44% 141 320 5 th Grade 50% 166 50% 166 332 Total 49% 954 51% 1008 1962 Source: Student survey; Authors’ calculations On average, students were in the correct age in each grade as can be seen by the grade level averages and medians. Students, however, reported a wide range of ages in each grade. Some of this age variation can be explained by not knowing exact age, by grade repetition, and by late entry. Of the students in the sample, 19 percent did not know how old they were. Even among those who said they knew their age, there appear to be some inconsistencies (for example it is unlikely that a 5th grade student can be 6 years old). The presence of older students can be often explained by grade repetition. 40 percent of the sample reported that they had repeated a grade at one point. The prevalence of grade repetition was more or less the same across different grades (15 to 17 percent), however second graders repeated their grade level the most (36 percent). Across the districts, Nong students had the lowest repetition rate (9 percent), while students in Outhoumphone had the highest rate (21 percent). Among all students, almost half (46 percent) said they had attended early childhood development or preschool. Exhibit 9: Students’ Age Distribution by Grade Grade Mean Age Median Age Range of Ages Observations 1 st Grade 7.1 7 5-14 165 2 nd Grade 8.4 8 5-14 525 3 rd Grade 9.7 9 5-15 265 4 th Grade 11.1 11 7-15 293 5 th Grade 12.4 12 6-17 326 Total 9.8 10 5-17 1574 Source: Student survey; Authors’ calculations. IMPAQ International, LLC Page 13 Food for Education Baseline Report 4.3 Household Environment To capture children’s background, we asked them about key household characteristics, including:  Household size  Language spoken at home  Household socioeconomic status 4.3.1 Household Size The average size of households was more or less the same across districts: seven people. The widest range, from 2 to 20 people, was found in Sepon, Phin, Phalanxai, and Outhoumphone, and the smallest range, from 3 to 13, was in Vilabouly. 4.3.2 Language Spoken at Home We asked children about their primary language and other languages, if any, they spoke at home. The diversity of languages in Lao PDR means that home language plays an important role in children’s literacy outcomes as school instruction is in Lao. Exhibit 10 shows the distribution of the main languages children spoke at home. If students reported their primary language assomething other than the listed categories, 9 then enumerators would choose “other” as a response. The main language of approximately half of the sample (45 percent) was Lao, 17 percent was Phoutay, 28 percent were languages such as Makong, Tri, and Katang, and 15 percent reported other languages which were not among the listed options. Moreover, 10 percent of the sample did not know their primary language spoken at home. Of those, 77 percent were first and second graders which were more likely not to know the name of their primary language at home. Just over half of the students (51 percent) reported that they spoke only one language at home. Among the multilingual students, 27 percent said that, besides their main language, they also spoke Lao as a secondary language at home (see Exhibit 33, Appendix 4). Exhibit 10: Distribution of Main Language Spoken at Home N=1,962 Source: Student survey; Authors’ calculation 9 Enumerators were not allowed to read the list to the children. 45% 17% 15% 10% 8% 3% 1% 0.20% 0.05% Lao Phoutay Other Don't Know/No response Makong Katang Tri Taoy Thai IMPAQ International, LLC Page 14 Food for Education Baseline Report 4.3.3 Socioeconomic Status In order to get a sense of socioeconomic status, the survey asked children if their household possessed any of eight durable goods or services: electricity, refrigerator, bicycle, motorbike, tok tok (tractor), television, mobile phone, and car. On average, students said their households possessed five of the eight consumer goods. Only 1 percent of the sample had none of the eight. A large proportion of the sample (86 percent) had electricity at home and 76 percent had a mobile phone. Almost all households (93 percent) had at least some type of vehicles such as a motorbike, a tok tok, a bicycle or a car; 14 percent specifically had a car. When we compared the socioeconomic status of the sample across all districts, we found that Nong tended to have the lowest rates of possession of consumer goods, and Outhoumphone had the highest rates. (See Exhibit 32 in Appendix 4 for distribution of socioeconomic status, disaggregated by district.) IMPAQ International, LLC Page 15 Food for Education Baseline Report SECTION 5. QUANTITATIVE OUTCOMES Below we analyze data from the student survey (including the LBRA) and from the classroom observations. We examine the data by gender, grade, district and main language spoken at home (if relevant) but highlight only when the differences generally exceeded about 5 percent. Referring to Exhibit 10 in Section 4.3.2, we classified the primary language spoken at home by Lao, Phoutay, Miscellaneous, 10 and Unknown (i.e., those students who did not know their main language) for the analysis. Appendices 4 and 5 provide additional details. Self-reported data, especially those on culturally and socially sensitive topics such as food security, should be interpreted with caution due to social desirability bias. Exhibit 11 reports the baseline levels of the key McGovern-Dole evaluation performance indicators, as required by the LEAPS II TOR. Exhibit 28 in Appendix 1 provides the full table of these McGovern-Dole evaluation indicators derived from the project monitoring data. Exhibit 11: Baseline Levels for McGovern-Dole Performance Indicators McGovern-Dole Indicators Data Source Baseline Percentage by Sex Baseline Percentage Number of Observations (Students) 95% Confidence Interval Percent of students who, by the end of two grades of primary schooling, demonstrate that they can read and understand grade level text LBRA Girls: 3% 3% 495 1 2%-5% Boys: Percent of students who, by the end of two grades of primary schooling, demonstrate proficiency in identifying letters. Girls: 47% 47% 4961 42%-51% Boys: 46% Percent of students who are attentive in the classroom. Classroom observation Girls: 87% 84% 1,4702 83%-85% Boys: 82% Percent of students reporting that they are ‘somewhat’ or ‘very’ hungry during their afternoon class. Student Survey Girls: 7% 7% 7113 5% - 9% Boys: 7% Percent of students in target schools reporting health-related absences. Student Survey Girls: 32% 33% 1,9494 30% -35% Boys: 33% Source: Student survey; Authors’ calculation Note: 1Only second graders took the LBRA (49611 out of 1,962); 2We observed 85 classrooms with 1,470 total students – during each visit, each individual student was observed a total of 10 times; 3 This indicator is available only for those surveyed in the afternoon; 4 Excluded were students who did not know the answer or refused to answer. 10 This category includes Mon-Khmer group such as Makong, Katang, Tri, and Taoy, as we well as “other” and “Thai” options. 11 496 second graders took the LBRA, but only 495 of them finished the reading comprehension. One student refused to continue with the reading comprehension section. IMPAQ International, LLC Page 16 Food for Education Baseline Report 5.1 School Environment This section presents baseline outcomes about the school environment, including:  Student attitudes towards schooling,  School libraries, and  Classroom activities. School environment characteristics are important because they can shed light on students’ outcomes, such as student attendance or student performance. To measure the schools’ environment, we looked at how students felt about their schools, whether there were storybooks for them to borrow, and how frequently they borrowed the books. We also asked students about the frequency of various types of activities teachers practiced in their classroom 5.1.1 Student Attitudes toward Schooling Almost all students (98 percent) reported that they liked going to school. Students were also asked what they liked about their school12; up to half of the sample (39 to 48 percent) reported they liked going to school to read, to learn new things, and to write. Presenting ideas in class was the least popular option. Our cognitive testing also suggested that Lao students were not very comfortable sharing their opinions. See Exhibit 34 in Appendix 4 for details on reasons students liked going to school. 5.1.2 School Libraries Nearly half of all students (45 percent) said that their school had storybooks for them to take home. Of those, 70 percent reported taking a storybook home at least once a week. However, these results must be interpreted with caution. Cognitive testing suggested that students had difficulty understanding the purpose of a school library or the concept of borrowing storybooks. 5.1.3 Classroom Activities We asked students about the frequency of class activities: playing alphabet games, hearing a story or poem, and answering questions about the story or poem. Frequencies were roughly similar for all three activities (Exhibit 12). Almost half of students (41 to 55 percent) most often reported that the teacher “never” did any of the three activities (i.e., telling a story, asking about the story or playing game around the alphabet). Hearing a story seemed to be the most frequent activity of the three. More students at lower grades (first and second) reported their teachers told them a story or ask them about the story every day. There were also some slight differences across districts for each activity. Many students in Outhoumphone reported that that their teacher never told them a story (44 percent). Of those students in Outhoumphone that were told a story, 60 percent indicated the teacher did not ask them questions about the story. Large proportions of students (67 percent) in Nong and Vilabouly reported that their teacher never played a game in the classroom around the alphabet. Please see the breakdown of each activity by district in Exhibits 40 to 42 in Appendix 4. 12 Students could freely name as many reasons as they wanted and the enumerator was supposed to mark the right answer among the listed options on his/her tablet. However, enumerators were not allowed to read the options to the children. IMPAQ International, LLC Page 17 Food for Education Baseline Report Exhibit 12: Classroom Activities in the Last Week Source: Students’ Survey; Authors’ calculations N = 1,786. Excluded observations were 176 students who missed all five school days during the previous week. 5.2 Household Environment This section presents baseline outcomes about household literacy practices, including:  Access to reading materials, and  Home literacy environment. Household literacy practices can illuminate the level of children’s exposure to learning outside of school. A student who is exposed to literacy activities at home is more likely to have better opportunities for literacy acquisition (Young-Suk, 2007). Having reading materials at home and seeing household members use them can encourage children to practice their reading skills. 5.2.1 Access to Reading Materials We asked students about reading materials that they had at home. In general, most children (89 percent) reported that they had textbooks at home. About half of the sample (50 percent) had drawing or coloring books, and only a few (seven percent) had magazines at home. However, seven percent of the sample did not have access to any reading materials at home. Exhibit 13 shows children’s access to reading material at home by their main language. Except for textbooks, students whose main language was Lao had more reading materials at home than students with other languages. These differences were statistically significant between students whose main language was Lao and Phoutay or other miscellaneous languages. Across different districts, there were slight differences in accessibility to reading materials at home. Atsaphone had the highest rate of not having any reading materials (13 percent) at home (see Exhibit 43 in Appendix 4). 41% 14% 36% 10% 55% 13% 27% 6% 51% 13% 28% 8% Never Once during the week A few times during the week Every day How often in the last week did you play a game in the classroom around the alphabet? How often in the last week did the teacher ask you about the story s/he told? How often in the last week did the teacher tell a story or read a poem to the class? IMPAQ International, LLC Page 18 Food for Education Baseline Report Exhibit 13: Reading Materials at Home by Main Language Source: Student Survey; Authors’ calculations. N = 1,765. Excluded are students who refused to answer the language question or who did not know. *** indicates significant difference between Lao and Phoutay at the 1 percent significance level, ** at the 5 percent level, * at the 10 percent level. ^^^ indicates significant difference between Lao and miscellaneous languages at the 1 percent level, ^^ at the 5 percent level, ^ at the 10 percent level. ϯϯϯ indicates significant difference between Phoutay and miscellaneous languages at the 1 percent level, ϯϯ at the 5 percent level, ϯ at the 10 percent level. 5.2.2 Home Literacy Environment Following SCI suggestions, adapted from Hess et al. (1984), we captured the level of family involvement in literacy activities. We asked students if anyone in their household encourages them to study, reads to them, tells them a story, or asks them questions about the stories. We also asked if they saw anyone reading at home. Most students, 81 percent, reported that at least one person in their household encouraged them to study. Fewer students reported that at least one person in their household told them a story (42 percent) or read to them (57 percent) (See Exhibit 36 in Appendix 4). Only 33 percent of the sample reported that their parents or other family members asked questions about stories that they had told or read to the children. Exhibit 14 shows that students whose main language at home was either Lao or Phoutay had slightly more active home literacy environments. Lao speakers read more to their children, while families whose main language was not Lao told their children more stories. These results are consistent with the notion that most of reading materials in Laos are printed in Lao. 88% 5% 3% 21% 45% 9% 84%^^^ 6%^^ 12%^ 20% 50%^^ 10%^^^ 94%ϯϯϯ 3% 8% 21%ϯϯ 42% 4%ϯϯϯ 91% 10% *** 15%*** 25% 54%*** 5% 0% 20% 40% 60% 80% 100% Textbooks Magazines Newspapers Stories or comic books Coloring or drawing books No reading materials Lao Phoutay Miscellaneous Unknown IMPAQ International, LLC Page 19 Food for Education Baseline Report Exhibit 14: Home Literacy Environment by Main Language at Home Source: Students’ survey; Author’s calculations. N = 1,765. Excluded are students who refused to answer the language question or who did not know. Note: *** indicates significant difference between Lao and Phoutay at the 1 percent significance level, ** at the 5 percent level, * at the 10 percent level. ^^^ indicates significant difference between Phoutay and miscellaneous languages at the 1 percent level, ^^ at the 5 percent level, ^ at the 10 percent level. ϯϯϯ indicates significant difference between Lao and miscellaneous languages at the 1 percent level, ϯϯ at the 5 percent level, ϯ at the 10 percent level. 5.3 Students’ Outcomes This section presents students’ baseline outcomes in four key areas:  Student’s reading assessment outcomes,  Attentiveness,  Food security, and  Health. 5.3.1 Student’s Reading Assessment Outcomes To measure second-grade students’ literacy skills, we implemented the LBRA, which was a modified version of the EGRA developed and tested in the Lao context by SCI. This LBRA consists of seven subtests: 1. Expressive Vocabulary - Total number of animals and foods that the child could name in Lao 2. Phonemic Awareness - Number of word pairs identified, out of three, based on similar first￾letter sounds 3. Letter Knowledge - Number of letters/sounds known out of 33 4. Word Recognition - Number of words read correctly out of 20 most-used words from leveled textbooks 5. Word to Picture Matching - Number of objects matched with their corresponding pictures, out of nine 6. Reading a Short Story (134 words) 47% 74% 32% 49% 59% ϯϯ 77% ϯϯϯ 46% 52% ϯϯϯ 67% ^^ 84% ^^ 35% ^^^ 58% ^ 65% 85% 44% *** 62% 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% See anyone reading Anyone encourage you to study Anyone tell you a story Anyone read to you Lao Phoutay Miscellaneous Unknown IMPAQ International, LLC Page 20 Food for Education Baseline Report o Fluency - Number of words in a 134 word short story read correctly in a minute o Accuracy - Percentage of words in the same short story read correctly (untimed) 7. Comprehension - Eight comprehension questions related to the short story were asked in one of two ways: o Reading comprehension, which applied to children who could read at least five words in the story correctly in 30 seconds. These children were identified as “readers.” o Listening comprehension, which applied to children who could not read five words in the story correctly in 30 seconds. The enumerator read the story aloud to these children, identified as “nonreaders.” All subtests were administered in Lao, which is the official language of instruction, although instructions for completing the test were provided in the local language to assist the child as necessary. The two Mon￾Khmer speaking enumerators’ responsibility was to interview Grade 2 students whose Lao language skills were not as strong. Where they were not available, the data collection team sometimes asked older children for informal assistance. Exhibit 15 provides an overview of second grade students’ literacy skills in LB targeted schools at baseline. Overall, children did better in expressive vocabulary, phonemic awareness, and letter knowledge relative to other subtests. However, only 18 percent13 of students were classified as readers, as defined above. The listening comprehension scores of the nonreaders were slightly lower than the reading comprehension scores of the readers. However, in general non-readers were able to answer at least 75 percent of the comprehension questions (20 percent) slightly more than readers (17 percent). Students whose main language at home was Lao or Phoutay did somewhat better than students who primarily spoke other languages at home. The differences in all literacy skills between students whose main language at home was Phoutay and miscellaneous languages are statistically significant except in the case of reading independently (See Exhibit 37 in Appendix 4 for a detailed breakdown of literacy skills by language). Exhibit 15: Second Grade Students’ Literacy Skills Overview Literacy Skills Outcome Expressive Vocabulary (# out of 20) 14 Expressive Vocabulary (%) 71% Phonemic Awareness (word pairs correct out of 3) 2 Phonemic Awareness (%) 67% Foundational Literacy Skills Letter Knowledge (# correct out of 33) 21 Letter Knowledge (% correct) 64% Word Recognition (# correct out of 20) 5 Word Recognition (% correct) 25% Object to Picture Matching (# correct out of 9) 4 Object to Picture Matching (%) 44% 13 With 15%-22% confidence interval. IMPAQ International, LLC Page 21 Food for Education Baseline Report Reading Skills Students classified as readers (5+ words correct in 30 seconds) 18% Accuracy (% words correct in passage), readers only 87% Fluency (words correct per minute), readers only 29 Comprehension Skills % reading comprehension questions correct, readers only 50% % listening comprehension questions correct, nonreaders only 47% Source: Student Survey; Authors’ calculation; Overall N= 496; Reading comprehension N = 8714; Listening comprehension N = 404 For Exhibit 15, we present in details the key reading outcomes from LEAPS II performance indicators only. We provide a brief overview of other subtests and their desired outcomes in Appendix 5 (Other subtest Reading Assessment). 5.3.1.1 Letter Knowledge To measure letter knowledge, students were shown a chart of 33 letters in Lao and asked to name the letter. On average, students were able to identify the sound of 21 letters (63 percent). Almost half of the sample (47 percent) identified at least 75 percent of the letters. The most difficult letters were “ຫວ” and “ຫຍ,” which are compound consonants that might be expected to cause difficulty; only 19 percent of students identified these correctly. The easiest letter was “ກ”; 91 percent of students were able to identify it. In general, Lao speakers performed better than other students (Exhibit 16). Exhibit 16: Letter Knowledge by Language Outcomes Lao Phoutay Miscellaneous Unknown Average number of letters identified correctly 24 21 20 18 Students who were able to identify at least 75% of the letters 59% 44% 33% 30% Source: Student Survey; Authors’ calculations; N=496 A few children (3 percent) did not pronounce any letters correctly, while only 5 percent of the sample identified all 33 letters correctly. Exhibit 17 shows the distribution of letter identification scores by language. The distribution is more skewed to the right showing most of children were able to identify 18 to 25 letters. 14 Four students were excluded who correctly identified as readers but enumerators mistakenly treated them as non-readers and did not let them read the passage. IMPAQ International, LLC Page 22 Food for Education Baseline Report Exhibit 17: Distribution of Letter Scores Source: Student Survey; Authors’ calculations; N=496 5.3.1.2 Reading Outcomes Of the 496 second graders who took the LBRA, only 18 percent were classified as readers (i.e., students who could read at least five words in the story correctly in 30 seconds). More girls (20 percent) were identified as readers than boys (17 percent). Students whose home language was Lao were identified as readers more than non-Lao speakers. The same passage was used to measure the fluency and accuracy of students classified as readers15 . On average, these students read 33 words per minute with 86 percent accuracy. Girls did slightly better (36 words per minute) than boys (29 words per minute) in reading with fluency. Students who did not know their main language did better in fluency and accuracy than other students, including Lao speakers. Exhibit 18 shows students’ fluency and accuracy levels by language. 15 We excluded four readers from all reading analyses. Although these students were correctly identified as readers, apparently they were mistakenly treated as nonreaders, and the passage was read to them. 0% 10% 20% 30% 40% 50% 60% None Identified 1-9 letters Identified 10-17 letters Identified 18-25 letters Identified 26-33 letters Lao Phoutay Miscellaneous Unknown IMPAQ International, LLC Page 23 Food for Education Baseline Report Exhibit 18: Fluency and Accuracy by Main Language at Home Source: Student Survey; Authors’ calculations; N=87 readers 5.3.1.3 Comprehension After readers read or nonreaders listened to the whole passage, they were asked eight comprehension questions: one summary, 16 five literal, 17 one inferential, 18 and one evaluative19 . Readers and nonreaders performed approximately the same on the comprehension questions. Both groups, on average, answered four questions correctly out of eight. Exhibit 19 shows that readers did better in answering literal questions, while nonreaders did better on almost all other types of questions, especially the evaluative question. Readers whose main language at home was Phoutay did very well on inferential and literal questions compared to their peers. See Exhibit 38 and 39 in Appendix 4 for the comprehension questions by language for readers and nonreaders. Exhibit 19: Comprehension Questions Answered Correctly by Readers and Nonreaders Comprehension Summary Literal Inferential Evaluative Reading comprehension (readers) 9% 41% 46% 25% Listening comprehension (non-readers) 12% 35% 48% 45% Source: Student Survey; Authors’ calculations We define grade-level reading competency on this assessment as the reader’s ability to answer at least 75 percent of the reading comprehension questions correctly. Among all-second grade students, only 16 This is a type of question that tests students ability to identify the main ideas of a reading passage. 17 This is a type of question that its answer is clearly and explicitly stated in the passage. 18 This is a type of question that its answers is not clearly stated in the passage, but is usually implied by the author. 19 This is a type of question that requires some levels of cognitive and/or emotional judgment. To answer such a question, a child needs to use his/her personal opinion. 31 35 33 35 # of Correct words Read per Minute Fluency Lao Phoutay Miscellaneous Unknown 87% 78% 85% 90% 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% Accuracy Lao Phoutay Miscellaneous Unknown IMPAQ International, LLC Page 24 Food for Education Baseline Report three percent showed they could read with comprehension. Lao speakers showed that they could read at their grade level (five percent) more than non-Lao speakers or those who did not know their primarily language at home (one percent). Classifying students based on their reading proficiencies (Exhibit 20), in our sample we have:  82 percent nonreaders  18 percent readers, classified as: o Beginning readers: 15 percent of readers scored less than 75 percent on comprehension. o Grade-level readers: three percent of readers scored at least 75 percent on comprehension. Exhibit 20: Reading Proficiency Source: Student Survey; Authors’ calculations; N=495 5.3.2 Attentiveness To capture information regarding students’ attentiveness, we observed classrooms in 85 out of the 87 primary schools in our sample. Every three minutes, we captured the number of attentive students and the number of distracted students in the room. We did this for 30 minutes total, leading to 10 “snapshots” of attentiveness for each classroom. To calculate the attentiveness rate, we divided the total number of attentive students by the total number of students in the classroom. The student attentiveness rate was 84 percent, which is close to the anticipated baseline level of LEAPS II performance indicators (86 percent) outlined in the TOR. Student attentiveness ranged from 21% in one classroom to 100% in five classrooms. Girls were slightly more attentive (87 percent) than boys (82 percent). Exhibits 22, 23, and 24 show student attentiveness rates by students’ activities, by teachers’ activities, and by subject area. However, these results are illustrative only. The small number of observations means caution should be applied in imputing meaning to these breakouts. Over half of the activities observed in the classrooms involved students practicing, copying, reading aloud, or teachers lecturing (Exhibit 21). There was slight variation in attentiveness based on the classroom activities, with students being more attentive to dictation, repetition, story time, and discussion. 82% 15% 3% Non-reader Beginning reader Reader with comprehension IMPAQ International, LLC Page 25 Food for Education Baseline Report Exhibit21: Student Activities Observed Source: Classroom Observations Tool; author’s calculation Note: Number of observations is 850: 10 snapshot observations in each of 85 classrooms Exhibit 22: Student Attentiveness by Students’ Activity Activity Observations Students Paying Attention Practicing (individual, in group, or at the blackboard) 161 85% Copying 129 80% Reading aloud (collective or one by one) 112 86% Commands, lecture 105 89% Questioning/Answering 78 89% Students are left with no guidance or direction about what to do 71 68% Repetition 33 95% Production/expression 27 74% Silent reading 26 85% Other 24 88% Problem solving, debating, discussing in group 24 72% Dictation 21 96% The teacher is reading a story or another text 15 91% Discussion 13 90% Classroom management (materials distribution, transition between activities) 10 81% Educative games * * Source: Classroom Observations Tool; author’s calculation; Note: Number of observations is 850: 10 snapshot observations in each of 85 classrooms;* Indicates less than 5 observations – data is suppressed 19% 15% 13% 12% 9% 8% 4% 3% 3% 3% 3% 2% 2%2% 1% Practicing Copying Reading aloud Commands, lecture Questioning/Answering No guidance Repetition Production/expression Silent reading Other Problem solving Dictation Teacher reading Discussion Classroom management Educative games IMPAQ International, LLC Page 26 Food for Education Baseline Report There was also slight variation in attentiveness based on the teacher’s activities. The lowest proportion of children paid attention when their teacher was focused on another grade in a multi-grade classroom. By contrast, students were highly attentive when their teacher was engaged in disciplining a child. Exhibit 23: Student Attentiveness by Teachers’ Activity Activity Number of Observations % students paying attention The teacher is engaged in the same activity as the students 362 85% The teacher is engaged in classroom management (materials distribution, transition between activities) 256 87% The teacher is engaged with another grade 128 72% The teacher is engaged in disciplining a child 54 90% The teacher is outside of the classroom or does something unrelated to the classroom 50 77% Source: Classroom Observation tool; Authors’ calculation Note: Number of observations is 850: 10 snapshot observations in each of 85 classrooms. The Lao language was the subject that was observed the most during classroom observations. Exhibit 24: Student Attentiveness by Subject Subject Number of Observations % students paying attention Lao Language 40 86% Math 23 83% The World Around Us 9 82% Drawing 5 70% English * * Art (Song and Dance) * * Source: Classroom Observation tool. N= 10 snapshot observations in each of 85 classrooms. * Indicates less than 5 observations – data is suppressed 5.3.3 Food Security To measure students’ hunger during school, we asked about their food intake in the morning and afternoon. We asked all students whether they ate breakfast and felt full after consuming it. For the surveys implemented in the afternoon (36 percent), we asked children whether they ate lunch or the school meal and how they liked it. We also simply asked them if they felt hungry. As Exhibit 24 shows, almost all of the students said they had eaten breakfast (96 percent). Of those, only 3 percent said that they could have eaten more. Of the children surveyed in the afternoon, 85 percent reported that they ate lunch; 42 percent stated that school lunch had been served that day. This finding is consistent with the fact that, at time of baseline data collection, schools had not yet received LEAPS II food. While some schools were cooking their remaining commodities from LEAPS I, other schools had IMPAQ International, LLC Page 27 Food for Education Baseline Report already cooked all their commodities and were not able to prepare lunch. Almost all of the students (96 percent) who were served lunch ate the school meal, and 86 percent of them liked its taste. Only seven percent of children surveyed in the afternoon reported that they were hungry. Slightly more of the students who ate the school meal reported feeling hungry (nine percent) than those who ate lunch elsewhere (five percent). Exhibit 25: Student Self-reported Food Intake Food Intake Percentage by Sex Percentage Total Observations Children ate breakfast. Girls: 96% 96% 1,961 Boys: 95% Children could have eaten more after eating breakfast. Girls: 3% 3% 1,873* Boys: 4% Children ate during lunch break. Girls: 85% 85% 711** Boys: 84% Children were “somewhat” or “very” hungry during their afternoon class. Girls: 7% 7% 711** Boys: 7% Source: Student survey; Authors’ calculation * This indicator is only available for those who indicated eating breakfast. **This indicator is available for those surveyed in the afternoon. 5.3.4 Health To capture information on students’ health and the effects on school attendance, we asked students whether they had fallen ill in the past week, and, ifso, whether they missed school because of their illness. Of the students surveyed, 41 percent said they were sick in the past week. The most cited illness was fever, followed by headaches (Exhibit 25). One-third (33 percent) of all studentssurveyed said they missed school because of their illness. Of those who missed school, almost 80 percent said they missed between one and three days of school (on average two days). Exhibit 49 in Appendix 4 shows the breakdown of days students missed due to sickness. In addition, 27 percent of all students reported missing school in the past week for a reason other than being sick. This one-week-absence rate seems to be a little high, compared to March 2016 attendance rate from CRS’ monitoring data. However, we should interpret these results with caution since they are self-reported data from young children. 20 We recommend adding a parent survey to the midline scope of work to help triangulate the information from student to parent surveys. This could be achieved by including similar health related questions and probing through interviews about potential health/nutrition issues. 20 During cognitive testing we had to update this question to make it more understandable for children. IMPAQ International, LLC Page 28 Food for Education Baseline Report Exhibit 26: Proportion of Sick Students and their Illnesses N= 1,953 for graph on left, N=797 for graph on right. Source: Student Survey; Authors’ calculations Note: Excluded students were those who did not know the answer or refused to answer IMPAQ International, LLC Page 29 Food for Education Baseline Report SECTION 6. QUALITATIVE OUTCOMES Below is a summary of the main findings for our qualitative research questions. More detail regarding these results is elaborated in the following narrative. Summary of Main Qualitative Findings 1. How do parents and communities value education? And what are the challenges and supports that schools and families face in sending children to school? Overall, parents and community members placed a high value on education for both boys and girls. Parents, teachers, principals, and VEDC members reported that school feeding had a positive impact on parents deciding to send their children to school, although it was notable that irregular absences were still common. Key reasons for persistent non-attendance included the cost of transport, poverty, or parent attitudes (as reported by parents and teachers). For older children, especially boys, parents reported struggling to make sure that their children did not skip school. 2. Will the VEDC grant scheme in LEAPS II incentivize better performance by VEDC members? Perhaps, although most VEDC members reported being motivated by seeing the community work together on an activity. When asked how much money would encourage them to work harder in the VEDC, there was confusion from members around this question. Principals reported that they expected VEDCs would work harder if there was more money available for the activities that needed doing around the school. An incentive scheme that recognizes cooperation among different stakeholders in the village may be more likely to directly impact on motivation. 3. What incentives other than take-home rations motivate teachers, cooks, and storekeepers? The discussions with respondents suggest that eliminating take-home rations would likely most impact storekeepers, who were most motivated to participate because of the take-home rations. Some cooks, also, would be less likely to participate without the take-home rations, although many enjoyed the work and said they would continue regardless. Responses from teachers and principals suggested that, other than for volunteer teachers, the take-home rations did not have an impact on their teaching, but may have impacted positively on their willingness to ‘fill in the gaps’ where the village members missed their duties under the school feeding program. 4. What data would USDA be interested in collecting for the midline and endline qualitative component? USDA offered the following suggestions for future evaluation questions:  Sustainability – when take-home rations are taken away, will volunteers stay? Will it affect literacy?  What are national stakeholders’/decision makers’ perspectives on the program?  Are there unique factors that should be considered in schools where the main language spoken at home is not Lao?  What are some of the potentially negative unintended consequences of the program? 6.1 Value of Education and School Attendance 6.1.1 Motivation for Parents to Send their Children to School The parents interviewed were overwhelmingly positive about sending their children to school. This response needs to be treated with some caution, as only parents interested in the school and their children’s education were likely to volunteer for the interview. However, principals and teachers in every school could generally point to only two or three families who consistently did not send their children to school, suggesting that the importance of schooling was largely accepted. IMPAQ International, LLC Page 30 Food for Education Baseline Report he most common reasons parents cited for sending their children to school were to gain knowledge, to obtain a better job, and to improve their behavior. Parents noted that children who did not go to school were illiterate, “knew nothing,” and were misbehaved. In some villages, the parents emphasized the benefit children’s schooling brought to their family because children could read notices, signs, and bills for the family and could hold their heads high in the village because their children were in school. Other parents emphasized that education would give their children independence for their own families and would enable them to visit new places, talk to people, have friends, and know how to share. Parents regularly cited benefits to the village from their children going to school related to school activities such as cleaning the village, collecting rubbish, greeting guests, and contributing to the development of the village and country. In Vilabouly, the principal noted that, 10 years ago, the parents did not see the importance of education, but things had changed considerably. The village had developed, and families no longer needed to go into the forest as before. They saw other villages developing and other children going to school and getting good jobs, so they wanted to follow suit. In two villages, parents described how families killed buffaloes or pigs to ensure they had enough money for their children to attend school. All but one village appeared to have a well-functioning system to encourage parents to ensure school attendance; either teachers or VEDC members had primary responsibility for following up with parents if students did not attend. One village even had a loudspeaker system to announce student absences so that those working in the fields could make sure their children attended. Village pressure to attend was a strong incentive. 6.1.2 Disincentives for Parents The greatest disincentive expressed in every school was the cost of schooling: transportation for those living far from the school, textbooks (at one school), uniforms, pens, and writing books. Respondents in one school also cited the opportunity cost of children not being available to help their parents earn money. Other problems occurred; for example, the father had left the family, or the parents were using drugs and not taking care of their children. Two villages contributed minor support in the form of rice, pens, or writing books for families who needed them. Children who did not attend school included those who were orphans, those whose parents lived in the forest and did not communicate with the village, and those whose parents did not see the value of education. In one village, the principal complained that the village head did not follow up on student absences. In the same village, mothers and fathers expressed a lack of conviction about the usefulness of schooling. Sending children to higher levels of education was costly, and they did not have the money to pay for their children to get jobs at the end, so they did not consider that more education would help their children get better jobs. In a different school, one mother asserted that she would withdraw her son at the end of Grade 5 unless he received a scholarship. She did not believe the cost was worth it. Lack of opportunity impacts on parents’ attitude to schooling “We already have a village head and teachers so there is no reason for them to study higher – there will not be a job for them to do.” -Mother in Sepon district Parents’ attitudes influence student attendance “Really the problem is not the money but the parents wanting to send their children or not. They are illiterate themselves and don’t see education as important. The parents’ attitude is the most important. They think that if there is already a village head, then what will their own child do with an education?” -Teacher in Atsaphone district Education offers a better life “If the children don’t go to school regularly then they will be ordinary people. They will not go anywhere and will look after the pigs and buffaloes. There are no factories or private businesses here so they’re just at home.” -Mother in Nong district IMPAQ International, LLC Page 31 Food for Education Baseline Report 6.1.3 Aspirations of Parents Parents’ aspirations for education were shaped by their wider experience of what other children and villages had achieved through schooling, their children’s ability, and the cost of progressively higher levels of education. In all villages, parents overwhelmingly looked forward to their boys and girls having an education so that they could “pen chau pen nai” (become village heads and leaders). The civil service was the preferred job, so that children could have a permanent income for the rest of their lives. Preferred occupations included doctors, nurses, teachers, village heads, or office workers; in some locations, army or police positions were included. In only one village did respondents mention private-sector positions. Fathers and/or grandfathers in Atsaphone noted that factories and hotels in nearby districts were hiring educated people and that these positions offered a good income. In three districts, both mothers and fathers had aspirations for their children to reach Grade 12 and higher if they could afford it and children had the ability. In three other districts, fathers were more likely to have higher aspirations for their children. While mothers in Outhoumphone, Vilabouly, and Atsaphone mentioned wanting children to study to Grade 12, fathers in the same villages discussed their children achieving bachelor’s degrees or studying overseas. However, parents were mostly unwilling to express firm aspirations for their children because of the costs of studying beyond Grade 5. Every interviewee in Atsaphone mentioned the cost of travel to the lower secondary school, which was seven kilometers away. Atsaphone parents said that students in Grade 6 were not old enough to ride a motorbike, even if their parents could afford it. Fathers added that the road was muddy during the wet season, and students did not like arriving at school dirty. If they could not use the road for a period, it was difficult to persuade them to go back. Parents estimated that more than 90 percent of students finished Grade 5 and most continued to lower secondary school. However, they suggested much lower rates of completion of lower secondary school, with parent group estimates ranging from 30 percent to 70 percent. Parent views ranged widely about the dropout rates. Exhibit 27: School Completion by District Location Level Parents said Most Children complete Nong Lower secondary Sepon Primary Phin A few years of lower secondary Atsaphone A few years of lower secondary Outhoumphone Lower secondary Phalanxai Lower secondary Vilabouly A few years of lower secondary Source: Parent focus groups 6.1.4 Motivation for Students to Attend Parents, principals, teachers, and VEDC members highlighted motivating factors for children to attend school: seeing friends; joining in school activities, especially sports; and, for the older ones, gaining knowledge and keeping up with friends who also attended school. Respondents in four villages specifically mentioned school lunches as a reason children attended more regularly than before: Children did not “lose their way” after returning home for lunch or looking for lunch in the forest. In Sepon, where the lunches had finished a month earlier, a teacher noted, “At the moment they are not coming in the afternoon, because there is no lunch for them and they go home to eat.” IMPAQ International, LLC Page 32 Food for Education Baseline Report The key reasons respondents suggested for students not wishing to attend related to poverty: not having money for clothes and school material and so being embarrassed before their friends. Teachers noted irregular attendance due to family duties and the need to look for food and money. All of the principals interviewed said that the school feeding program had increased regular student attendance. They gave several reasons:  arents were happier to leave their children at school for the day when they knew children would get something to eat at lunchtime,  Students did not get distracted at lunchtime and forget to come back to school in the afternoon, and  Students were not so hungry, so they did not go away at lunchtime to hunt for their own food. Another benefit of the school feeding program, mentioned particularly by principals, was a much closer relationship between the school and the village, generating a feeling of “community spirit”. According to one principal, the school feeding program brought parents to the school more often than before, and parents now understood the work of the school better. One principal said, “This village had meetings before, but we are happier to do the meetings now that CRS has come in. Now we have a reason for discussing things all the time...” 6.1.5 Gender and Age Factors In all schools, all respondents agreed that both mothers and fathers had primary — and equal — responsibility for ensuring that their children attended school. The only exception to this pattern was in one school where the principal and teachers suggested that, even though parents might decide equally, in fact fathers looked after the money, so, if fathers decided schooling cost too much, then their children would not be able to attend. Respondents in all villages also agreed that girls and boys should attend equally, although parents in six villages agreed that girls studied better than boys. Only one respondent argued that girls’ brains were slower than those of boys. While most parents thought that girls and boys had equal opportunities after graduating, some noted that boys were more likely to enter the army or police and girls were more likely to be teachers or doctors, although they said that the choice depended on students’ interests and skills. One father noted that it could be more difficult for girls to study at higher levels because school dormitories were not secure, but others complained that boys were much more likely to play with their friends and get into trouble if they studied at higher levels outside the village. In all schools, teachers and principals mentioned that girls were more likely to miss school so that they could look after younger siblings during planting and harvesting seasons, help with chores, or assist in preparing for celebrations. Younger girls missed school when their families took them to the fields to look after them while they worked. The time missed in this way varied from one or two weeks up to 30 or 40 days for those whose family fields were far from the village. Four schools also reported a significant problem with boys missing school in Grades 4 and 5. Depending on the school, the boys missed class in order to hunt in the forest or play sports with their friends. In the week we visited, one teacher of Grade 4 and 5 reported having 20 boys missing from a total of 50 (it was not clear if this referred only to his own class). Although teachers notified parents, they reported that parents seemed not to know how to get their children to attend again. Teachers also felt constrained in the methods of discipline they could use. School lunches have multiple benefits “Since the rice from CRS has come it has been easier. The children can read at school while they are waiting. Then they just go home at the end of the day. It is not such a problem of parents taking their children into the rice fields.” Principal from Outhoumphone district IMPAQ International, LLC Page 33 Food for Education Baseline Report In a different school, the school allocated school chores, such as fetching water or cleaning the school, to students who missed class. However, a teacher noted that children might not come at all if they received a punishment more than once or twice. 6.2 Incentivizing VEDC Performance 6.2.1 VEDC Roles and Responsibilities Most VEDCs included the village head and at least one deputy. The majority of VEDC members usually also served on the broader village committee. In one case, the village committee was the VEDC and did not recognize a separation between the two. Instead, the committee members viewed education as an extra responsibility the village committee had taken on, particularly since the school feeding program had commenced. The VEDC members in this committee only discussed their VEDC responsibilities in terms of the school feeding program and were unclear about other VEDC responsibilities. Other activities (school maintenance, school attendance, etc.) were considered part of village committee responsibilities, though they reported they had attended two rounds of training on VEDC responsibilities. For the school feeding program, the VEDCs, and more specifically the village heads, reported that their duties included:  Organizing the cooks,  Nominating the storekeepers, and  Resolving disputes among teachers, cooks, storekeepers, and parents in the small number of cases where disagreements arose. In addition to these school feeding program responsibilities that all villages reported undertaking, some VEDCs also:  Collected money from households to buy condiments, cooking equipment, or materials for seating,  Organized labor to build the storeroom, kitchen, and seating areas, and  Approved monthly reports from the storekeeper. In addition to responsibilities for the school feeding program, all VEDCs believed they had an important role in encouraging parents to send their children to school and in following up on absences; however, in two schools, this responsibility mainly devolved to teachers. All the VEDCs reported mobilizing community labor to help with maintaining the school (cleaning, fixing tables and chairs, repairing roofs and fences, and so on), although the actual level of activity varied from school to school. The next most common responsibility VEDC members cited, for six schools, was checking on the attendance and work of the teachers, usually in a monthly meeting. VEDCs appeared to have differing levels of engagement in developing the annual school plan. Some saw it as the responsibility of the teachers, while others had firm ownership and leadership of the plan and its implementation. 6.2.2 VEDC Motivation VEDC members were asked to choose among five proposed options and select the option that they felt was most important to them if they did a good job or select ‘Other’ for a different response. A representation of each option was drawn on a colored piece of paper and each VEDC member was given two stones to place on the drawing(s) that they felt best represented their greatest source of motivation. The overall responses are provided in Exhibit 28. IMPAQ International, LLC Page 34 Food for Education Baseline Report Exhibit 28: Options chosen by VEDC Members Money Community spirit More children attend school Recognition on the radio Certificate of appreciation Other 14 24 16 3 7 2 Source: VEDC interviews; N=33 respondents As shown in Exhibit 28, by far the most important motivational factor for VEDC members was “community spirit” (khoam samakhykan) which referred in practice to seeing the village working together on an activity and having strong relationships. Respondents noted that community engagement was necessary to get anything done successfully, from getting children to come to school to working on school repairs. Community engagement and recognition of the value of their work were also what made VEDC members enthusiastic about participating in the VEDC. The role described above, encouraging student attendance, was the second most popular motivation for VEDC participation. Those who chose money as the most important reward for their work noted that it was difficult to do anything if money was not available. One women’s union leader noted that having money for the school was like having money for the family. Respondents who selected this money option referred to particular activities that would need money to move forward. Recognition through the radio or a certificate of appreciation was considered important by many, either because the respondents felt publicity for their achievements meant that they had genuinely been successful or because they wanted neighboring villages and authorities to recognize their work and status as a “model” school or VEDC. Discussions with the VEDCs also highlighted the importance members placed on district, provincial, or Communist Party officials asking them to join the VEDC. Other motivations included believing in the role of education for the development of the village and pride in the visible signs of the work they achieved. VEDC members were asked to consider how much money would encourage them to work harder in the VEDC. There was some confusion around this question in which some asked whether the reward would be for the person or for the school. On being told it would be for the school, the VEDC members referred to the cost of existing priority activities. They were not able or willing to identify a monetary reward separate from the activities they had planned. As one principal noted, “If VEDCs have more money, they will do more, because there will be more activities to do.” From the interviewers’ perspective, the most active VEDCs were constrained not only by money, but also by their engagement in a large number of activities in their villages. We were able to meet with only two heads of village from seven locations because village heads had competing responsibilities. VEDCs were not always clear on the priorities for the school, and targets that were not met were transferred from one year to the next for completion once money became available. Principals reported that they expected VEDCs would work harder if there was more money available for the activities that needed doing around the school. One principal suggested that additional money would only work if the VEDC received the money and the principal signed for its disbursement or vice versa (i.e. there needed to be a check in the system). Another principal asked that if there was more money available for the school then it should be entirely managed by the VEDC as the principal was too busy to manage the additional burden. IMPAQ International, LLC Page 35 Food for Education Baseline Report 6.3 Motivation of School Staff and Volunteers In our interviews at the schools, respondents described receiving rice, lentils, and green split peas for daily school lunches; as well as rice and oil as monthly THR for cooks, storekeepers, and teachers working at the school. While this was a motivating factor for some respondents, most gave other or additional reasons that motivated their performance. In addition, some schools had developed school vegetable gardens, fish ponds or a chicken coop to supplement the rations provided by CRS, as well as funding collection mechanisms to buy condiments (this was done above and beyond the requirements of the program). Schools had undertaken further complementary activities to the food rations, the most common were storehouses to house the rations, a seating area for eating, and a cooking area for cooking the food. 6.3.1 Teachers The primary motivation expressed by teachers for doing a good job was their commitment to students’ learning, gaining new knowledge, and their love of the profession. Some teachers were more directly motivated by particular student attitudes, such as being attentive in class, attending regularly, learning quickly, or following rules and displaying appropriate behavior. In four schools, teachers also emphasized the importance of fulfilling their duties as a teacher as a reason for doing their job. Important duties citied, for example, included their responsibility for assisting with the development of the village through education, being a role model for the country, and being a good representative of the profession. Principals (as well as some teachers) noted that teachers were obliged to do a good job because of checks in the system such as a salary based on their attendance; regular checks by the VEDC and authorities; and, in one school, lesson plan reviews by the principal. Finally, some teachers and principals referred specifically to the take-home rations as a motivating factor. For one teacher, receiving a monthly THR made her feel that her work was considered important by outsiders, a feeling that made her happy to do the work. In three schools, the rice21 enabled the teachers to stay at school during the lunch break; in two schools, it improved teachers’ ability to do lesson planning during the lunch break. One teacherspecifically cited lesson planning as a favorite activity because she could now do it at lunchtime with her colleagues. Previous studies have shown that lesson planning is usually one of the least favorite activities of Laotian teachers (LADLF, 2016). However, even though most expressed motivators outside of THR, teachers did mentioned the benefit of the rations as a useful supplement at the end of the month when their salary had not arrived, or as something they could sell in order to buy extra food such as eggs. Difficulties most commonly mentioned by teachers included lack of materials and textbooks, lack of in-service training, student behavior, and parent attitudes. Note that teachers were not asked in depth about the difficulties they faced as teachers. 6.3.2 Cooks Villages in the six schools in the qualitative sample reported the school feeding program was organized according to two different patterns. In three of the schools, three women were nominated to be the regular, permanent cooks. The other schools operated a rotation of volunteer cooks from the mothers of the village, organized by the village head or the women’s union. One of the villages had begun with two full-time cooks, but there had been disagreements in the village about the cooks notserving some children enough food and not sharing the allowance available for being a cook. The village committee decided to 21 It was not clear if this was referring to the THR or the provision of school meals. IMPAQ International, LLC Page 36 Food for Education Baseline Report ask all school families to contribute a cook on a rotational basis or pay 50,000 Lao kip per year for their child to receive food. Each school that had a permanent cook also reported having systems to supplement the rice and lentils with additional food. Schools with rotating cooks made the firmest requests for CRS to contribute additional food and condiments, and were least interested in the rice and lentils/green split peas. The cooks from those three schools reported serving only steamed rice with salt and boiled lentils/green split peas, while the three schools with permanent cooks alternated between rice porridge and fried rice or steamed rice with additional food. Cooks from each school reported the same pattern of arriving around 8:00 every morning and finishing their work around 11 a.m. Responsibilities included fetching water, lighting the fire for cooking, steaming the rice, washing and cooking the lentils, and then, in all but one school, serving the children and washing up. In one school, the teachers served the food, and the Grade 5 students did the washing up. All schools had monthly meetings in which the storekeepers and/or teachers finalized the ledger in which the reporting on the distribution of commodities was kept, and received their THR. In addition, in villages with more permanent cooks, the cooks were also responsible, to different extents, for activities such as sourcing complementary food (condiments, garden vegetables, eggs) and preparing it. In two villages, the cooks might also assist with additional food preparation, for example, for festivals or teachers’ lunches, as a resource for the village. Most cooks reported having been appointed by the village head or village committee (including the women’s union) or having been encouraged by friends. Many, but not all, had children or grandchildren at the school. Overall, most women said they preferred no particular activities associated with being a cook, but the work was generally easy (with a few exceptions) and familiar. The factors most frequently cited as leading to job satisfaction were seeing the children eat until they were full and contributing to the children’s education by ensuring they did not miss school and had more time to study. The cooks’ contribution also gave parents more time to work in the fields. Cooks did not highlight the THR as an important motivational factor, although one principal asserted that it was the only reason that cooks came. In one village, the cooks did not receive THR22; in two other villages practicing rotation of cooks, the ration was shared by up to ten families sharing the cooking that month. Two of the three permanent cooks expressed a preference for a salary if at all possible, noting the number of hours they worked. This request was supported by teachers and/or community members in both schools. Difficulties encountered by the cooks included the need to transport water some distance for cooking in two villages, the difficulty of lighting the cooking fires during the rainy season with damp wood, and having insufficient bowls and spoons for students23 . 6.3.3 Storekeepers Storekeepers spoke about receiving and counting the rice from the CRS delivery, dividing up the commodities for the cooks each day, and managing the monthly accounts and distribution of take-home rations. The role of the storekeepers varied depending on the location. Storekeepers in some locations required assistance from teachers to measure out the daily commodities and manage the ledger, and in one school assisted the cooks with food preparation and distribution to the students. 22 In this village, the cooks made other arrangements with the community to not receive THR. 23 The MOU between communities and CRS stated that CRS/USDA would provide these items only one time. IMPAQ International, LLC Page 37 Food for Education Baseline Report Storekeepers were more likely to emphasize their personal selection by the head of the village as their motivation for taking the job. They emphasized the take-home ration as a motivation for continuing the work. Several suggested that, without the take-home ration, they would have to do extra work elsewhere and rotate the role of storekeeper with others. One storekeeper said, “I would not do the work if there was not the rice to take home. This job does not come with any prestige. If there is a job with a salary, then I would take that job instead.” Two storekeepers, however, said that they would do the job even with no take-home ration because they felt the work was important the village and for the children and because the village had entrusted them with this responsibility. 6.4 Other Emergent Themes 6.4.1 Requests for Assistance Respondents raised a number of requests for assistance, or noted priorities for their school. Principals and VEDCs referred to the priorities stated in the school plan. Six schools needed teaching materials, textbooks, and more storybooks. Five schools were interested in sports equipment and three of them wanted to build a sports field or playground to make the school more attractive and interesting to students to attend. Four schools had difficulties with water and were looking for a pump, or to move the kitchen to have better access to water. Other schools had plans to repair buildings, or build new buildings, often to accommodate newly established pre-school classes. More details on schools’ requests are outlined in Exhibit 30, Appendix 3. 6.4.2 Concern about the Future of the Program The rice for lunches under LEAPS I had stopped being available about a month before data collection in all but one school; consequently, respondents had questions about whether the program would be continuing, when and how. All of the school respondents expressed appreciation for the program. 6.4.3 Role of Community Mobilizers It became apparent as interviews progressed that CMs play a vital link in connecting the school and village to wider possibilities for school improvement. One school principal of a very active and engaged school expressed his thanks to the CM for his recognition of the school’s work and suggestions for activities that the school had undertaken. This contrasted starkly with another village, where the VEDC expressed interest in having a better school for the village but had not done any of the activities of other schools (i.e. vegetable garden and seating for children, etc.) and did not appear to have any ideas of what they could do to improve the school further. This school reported that they did not often see the CM. 6.4.4 Discrepancies in Food Allocation There was a big discrepancy in the amount of rice storekeepers believed should be allocated for each child, ranging from 7 grams to 130 grams24 of rice per child and other respondents also mentioned big variations when discussing the quantity of rice; the lower amount came from the only school where parents and cooks complained that daily rice allowances were insufficient and that children who ate last each day did not get enough to eat. There were also differences in the reported division of rice and oil among teachers, cooks, and storekeepers. Usually teachers and storekeepers reported that 20 kilograms of rice and one jug of oil were allocated per month as take-home rations, but in one school volunteer 24 130g is the correct amount IMPAQ International, LLC Page 38 Food for Education Baseline Report teachers did not receive anything.25 In another school, the cooks suggested that the village had agreed to pay them an allowance at the end of the school year instead of rice, but they had not yet received this yet; sometimes additional rice was distributed to needy families. 6.4.5 Community Spirit from School Feeding One teacher in Atsaphone said, “Since we received the rice, the school feels like our family, and we even think about coming to the school on Saturday, Sunday, and holidays.” The school had established a functional seating area for eating lunch, several rest areas for community or village members to sit in the shade, a vegetable garden, and a chicken coop. Despite the school having run out of commodities earlier that month, during the lunch break we observed that villagers and teachers from the nearby cluster school came to eat together and chat with their colleagues. It was not possible to observe the lunch period at other schools, given that the rice supply had finished at those schools. The feeling of community spirit engendered by the school feeding program in some schools was a large motivational factor for teachers in feeling enthusiastic about coming to school and teaching. 6.5 USDA Feedback We spoke with the current and former USDA program analysts, as well as a Monitoring and Evaluation staff member to see what they would want the evaluation to address through qualitative methods at midline and endline. Several of their comments were already incorporated in the baseline or planned for midterm and endline. For example, USDA staff members emphasized the importance of focusing on sustainability. They wanted to know if the volunteers would continue their community involvement in school feeding once the take-home rations ended. They also wanted to hear feedback on the program from a broader range of constituents/community members, such as national stakeholders and local decision makers. USDA staff members also wanted the qualitative analysis to further investigate factors from LEAPS I. For instance, a disproportionate number of schools that dropped out of LEAPS I were schools where the main language spoken at the home of most of its students is not Lao. Because these schools are generally correlated with lower-income families and students with lower literacy outcomes, the evaluation should investigate if there are any unique factors that should be considered for program implementation in schools where Lao is not the primary language spoken at home. USDA staff would like the midline and endline evaluations to include any unintended outcomes, especially those that are potentially negative. For example, USDA has heard that in some schools participating in LEAPS, the teachers are selling the produce from the school garden for personal profit, so would like to know how the program can ensure that the funds generated benefit the school. 25 We were unable to ascertain why teachers did not receive THR, it could be for intentional reasons (e.g. no food left to distribute or volunteers were new/not supposed to receive THR) or a programmatic failure. IMPAQ International, LLC Page 39 Food for Education Baseline Report SECTION 7. CONCLUSIONS This report provides the baseline levels of the performance evaluation of the LEAPS II project in Lao PDR. For this baseline report, we employed a variety of data collection methods including: 1) a student survey and reading assessment, 2) classroom observations, 3) focus group discussions, and 4) key informant interviews. We collected data from 1,962 primary school students, 40 mothers, 35 fathers, 30 VEDC members, 17 teachers, 7 principals, 17 cooks, and 7 storekeepers. This section summarizes key findings in response to the main research questions, highlights study limitations, and provides recommendations for the project and for the evaluation. Each village presented a unique picture of the possibilities and difficulties presented by the school feeding program. Each village demonstrated the importance of having all stakeholders play an active role so the program could achieve the best possible learning outcomes. Examples ranged from a school where the principal and the village clearly had a difficult relationship, school attendance appeared to be low, and little had been done to improve the school environment; to a school where all respondents referred to long lists of activities they had done together including the school pond, vegetable garden, eating area, tree planting, community events, and regular planning meetings. Even these positives did not stop reports of large numbers of students skipping school to play with their friends, however. The qualitative study revealed substantive benefits of the school feeding program such as community motivation, engagement in schooling and village-school relationships. These outcomes are likely to have contributed to student attendance and teacher motivation. However, schools showed marked differences in the benefits they realized from the program. We observed that the program seemed most effective when the school and village welcomed it fully and did not see it as an imposition. 7.1 Key Findings  While 47 percent of students demonstrated proficiency in identifying letters by the end of two grades of primary schooling, only 3 percent of students demonstrated that they could read and understand grade-level text. This finding suggests that nearly all of students are behind the attainment level they are expected to reach by the end of second grade.  The student attentiveness rate was 84 percent, with girls reported to be slightly more attentive (87 percent) than boys (82 percent). Students seemed more attentive during educative games, dictation, repetition, story time, and discussion.  Missing lunch and feeling hungry were not as prevalent as anticipated. Almost all students said that they ate breakfast (96 percent), and only 3 percent said they could have eaten more. In the afternoon, 85 percent of students reported that they ate lunch, but only four out of 10 did so in school26 . Six out of 10 students surveyed reported having lunch outside of school. Only 7 percent of students indicated that they were “somewhat” or “very” hungry during their afternoon class.  Students reported missing school often due to health-related absences. The most cited illness was fever, followed by headaches. Approximately 1 in 3 students were sick during the previous week and reported they missed 2 school days, on average, because of their illness. 26At the baseline data collection, schools had not yet received LEAPS II food, and were cooking their remaining commodities from LEAPS I. IMPAQ International, LLC Page 40 Food for Education Baseline Report  Respondents placed a high value on community spirit and engagement suggesting the importance of passing a lens over all activities to see how they impact on all community stakeholders and not only the immediately targeted beneficiaries.  The vast majority of parents valued schooling for their children, but cited the cost of schooling as a major barrier to their children’s continued education.  Cooks and storekeepers in particular appreciated the take-home rations as recognition and material support for their participation in the program. Teachers and principals also appreciated the take-home rations. In one school they enabled teachers to spend more time on classroom preparation; however, except in the case of volunteer teachers, they did not see the rations as vital to their continued performance as teachers. 7.2 Limitations Several limitations of the study are worth mentioning here. An important limitation of our study is that it relied on self-reported data from children for a number of socially and culturally sensitive subjects such as food security or health-related absences from school. This self-reported data should be interpreted with caution and is particularly susceptible to social desirability bias. To help counter such biases, we devoted considerable attention to cognitive tests with children of our survey questionnaire to improve the reliability of the data. A second limitation arises from sampling students who are present at school rather than drawing a sample from full classroom lists. The possibility of systematic student absences might induce a risk of sampling bias by selecting only present students. Another limitation yet stems from the fact that only one school in a non–Lao-speaking area was included in the qualitative sample. This school had very different dynamics from the Lao-speaking schools. Finally, the parents who were the most motivated about their children’s attendance are most likely to have participated in the interviews, which were voluntary. Therefore, parents’ enthusiasm for schooling is likely over-represented. 7.3 Recommendations We present the following recommendations to CRS based on lessons learned from our experience in the field at baseline and findings after analyzing the data that we collected. 7.3.1 Recommendations for the Project  Provide additional training for rotating cooks. In our focus groups with cooks, there was lower satisfaction among rotating cooks with the types of food provided and more requests for additional types of food. . In these schools, we recommend training be provided on a more periodic basis rather than once an academic year. Periodic training reinforces the training material for a larger number of cooks, and ensures adequate knowledge of food preparation and methods for preparing and supplementing rice and lentils.  Ensure storekeepers and other relevant stakeholders understand how to use supporting materials for program delivery. There was a discrepancy in the amount of rice storekeepers believed should be allocated for each child. The project should ensure that the simple chart created by CRS is displayed in the storeroom. The chart provides the number of kilos of rice, lentils, and oil to be allocated according to the number of children at the school as well as amounts for distribution of take-home rations. All relevant stakeholders (teachers, storekeepers and others engaged in ration distribution) should be able to understand the chart. IMPAQ International, LLC Page 41 Food for Education Baseline Report  Explore additional opportunities to supplement the school rations with locally grown nutritional food. Some schools had a well-organized system of supplementing the commodities, others were not clear on how to do this. Locally grown foods may be easier for cooks to prepare and more palatable for the children.27  Reduce the administrative burden on teachers. The qualitative findings showed that teachers are often drawn into different administrative aspects of the program to supplement village capacity. To ensure that the school feeding program does not create an additional burden for teachers, CRS should explore approaches on a case-by-case basis to reduce the school feeding program tasks undertaken by teachers and encourage villages to have a back-up option when the community member responsible is unable to undertake their duties.  The use of non-monetary performance incentives may help to increase VEDC performance. VEDC and other community members reported non-monetary incentives such as seeing the village working together in a spirit of cooperation as the most motivating element of their work as VEDC members. For example, CRS could consider annual village awards or recognition to whole villages achieving the greatest improvement in students’ attendance, establishing the most attractive area for eating lunch or developing the most delicious methods for cooking lentils. Money was considered important for being able to complete an activity.  LEAPS II activities should be implemented from a holistic community perspective. Building on the recommendation above, it is rare for any activity to be undertaken by a single group of stakeholders. Receipt of the food delivery in the school, distribution of the food, review of monthly ledgers are all activities that are likely to engage several stakeholder groups within the village. All VEDC activities require close cooperation between teachers, parents and VEDC members to be successful. Incentives should ensure that they reinforce the collaboration between these groups.  Explore opportunities for improving the outcomes of speakers of Lao as a second language. The findings showed that children whose language at home was not Lao consistently had lower literacy outcomes during the literacy testing. Children whose main language at home is not Lao are exposed to teaching at school in a language they are not familiar with. There are a wide variety of interventions that may help to close the gap between these children and those whose main language spoken at home is Lao, such as improving teaching skills, increasing the availability of appropriate school materials, additional preparatory schooling for students, etc. 7.3.2 Evaluation-Specific Recommendations  Sufficient time is needed to prepare for data collection field activities. More time (two to three weeks) is needed to prepare for data collection activities, anticipate challenges in the field, consult with all partners, and come up with solutions. Our experience shows that additional days for training and time to explain all lingering questions to enumerators results in minimizing errors in the field and ensuring greater consistency in the collected data. This additional time allows for more practice and comparison between the enumerators. This can increase the interrater reliability across the fielded instruments, including classroom observations and the LBRA. As we learn about the local landscape, additional time allows for more optimal logistical planning.  A longer window is necessary to complete data collection in the field. In order to randomly select students and create unique student identifiers during school visits, the data collection partner 27 Note that this recommendation is based on interviews and focus group discussions with stakeholders at baseline – before the start of LEAPS II. CRS plans to promote community contributions and school gardens to provide fruits and vegetables to complement the meals. IMPAQ International, LLC Page 42 Food for Education Baseline Report needs more time (at least a month) to ensure the rigor of the process. Additionally, future data collection rounds may occur during the rainy season. These rounds may require more time to overcome challenges in the field such as muddy roads and inaccessible schools.  The LBRA needs additional cognitive testing for the Lao context. Our field experience showed that the passage used in the reading assessment was not tested prior to fielding, so some words appeared difficult to students. We recommend that SCI cognitively test and pilot the tool before fielding it again for the impact evaluation to ensure that we are capturing valid and reliable reading data.  A proportional number of schools where Lao is not the primary language should be added to the qualitative sample. Future qualitative samples should include more schools in villages where Lao is not spoken as the first language, in proportion to the total number of these villages receiving the program. Additional stakeholder observations should be included in further rounds of data collection. Future qualitative studies should include observation of school feeding activities, including food preparation and other stakeholder participation in the program.  The enrollment list at participating schools should be updated on a more regular basis. During baseline data collection, we found more schools than previously expected that did not have all five grades, and some schools had different numbers of students from the numbers in the school lists. Collecting and quality-checking monitoring data more frequently (monthly or quarterly) would ensure that school lists are up to date. 7.3.3 Evaluation-Specific Recommendations with Budget Implications  Data from parents and teachers on other factors that affect children’s literacy should be collected. Characteristics such as parent’s educational attainment, home accessibility to water, and presence of a latrine at home can be important drivers of child outcomes. Similarly, data from teachers on their educational background, years of experience, and ethnicity may be significant predictors of success. If added to the scope of work and budget, we will control for these characteristics in future rounds of data collection and analysis.  The scope of work and budget should be amended to include surveying mothers at midline and endline. Collecting data from young children is often unreliable (e.g., a large number of children reported being absent due to illness; many children were confused about whether or not their school had a library; etc.). Being able to triangulate this data with mothers’ responses will help ensure more accurate data. IMPAQ International, LLC Page 43 Food for Education Baseline Report APPENDICES 1. LEAPS II Evaluation Indicators 2. References 3. List of Stakeholder Groups 4. Additional Tables and Complementary Outcomes 5. Other Subtests of Reading Assessment 6. Inter-rater Reliability 7. Survey Instruments 8. Qualitative Protocols 9. Evaluation SOW IMPAQ International, LLC Page 44 Food for Education Baseline Report APPENDIX 1: LEAPS II EVALUATION INDICATORS Exhibit 29: LEAPS II Performance Indicators Targets McGovern-Dole Performance Indicator Indicator Number Data Collection Method Data Source Baseline Midline Endline Life of Project Target Percent of students who, by the end of two grades of primary schooling, demonstrate that they can read and understand the meaning of grade level text Standard #26 Evaluation LBRA 3% 26% Percent of students who, by the end of two grades of primary schooling, demonstrate proficiency in identifying letters Custom Evaluation LBRA 47% 55% Number of individuals benefiting directly from USDA-funded interventions Standard #27 CRS/ Monitoring CRS 0 91,833 IMPAQ International, LLC Page 45 Food for Education Baseline Report Targets McGovern-Dole Performance Indicator Indicator Number Data Collection Method Data Source Baseline Midline Endline Life of Project Target Number of individuals benefiting indirectly from USDA-funded interventions Standard #28 CRS/ Monitoring CRS 0 257,214 Number of teachers/educators/teaching assistants in target schools who demonstrate use of new and quality teaching techniques or tools Standard #5 CRS/ Monitoring CRS 0 518 Percent of teachers in attendance on day of school visit at USDA supported schools Custom CRS/ Monitoring CRS 88% 95% Number of textbooks and other teaching and learning materials provided as a result of USDA assistance Standard #2 CRS/ Monitoring CRS 0 84,052 IMPAQ International, LLC Page 46 Food for Education Baseline Report Targets McGovern-Dole Performance Indicator Indicator Number Data Collection Method Data Source Baseline Midline Endline Life of Project Target Number of schools with improved literacy instructional materials as a result of USDA assistance. Custom CRS/ Monitoring CRS 0 180 Number of teachers/educators/teaching assistants trained or certified as a result of USDA assistance Standard #6 CRS/ Monitoring CRS 0 590 Number of school administrators in targeted schools who demonstrate use of new techniques or tools Standard #3 CRS/ Monitoring CRS 0 274 Number of school administrators and officials trained or certified as a result of USDA assistance Standard #4 CRS/ Monitoring CRS 0 280 IMPAQ International, LLC Page 47 Food for Education Baseline Report Targets McGovern-Dole Performance Indicator Indicator Number Data Collection Method Data Source Baseline Midline Endline Life of Project Target Percent of students who are attentive in the classroom Custom Evaluation Classroom Observation 84% 92% Percent of students reporting that they are 'somewhat' or 'very' hungry during their afternoon class Custom Evaluation Student Survey 7% 5% Number of school-aged children receiving daily school meals (lunch) as a result of USDA assistance Standard #16 CRS/ Monitoring CRS 0 86,049 Number of daily school meals (lunch) provided to school-age children as a result of USDA assistance Standard #15 CRS/ Monitoring CRS 0 32,705,154 IMPAQ International, LLC Page 48 Food for Education Baseline Report Targets McGovern-Dole Performance Indicator Indicator Number Data Collection Method Data Source Baseline Midline Endline Life of Project Target Number of take-home rations provided as a result of USDA assistance Standard #13 CRS/ Monitoring CRS 0 94,177 Number of individuals receiving take-home rations as a result of USDA assistance Standard #14 CRS/ Monitoring CRS 0 2,704 Number of people trained in child health and nutrition as a result of USDA assistance Standard #18 CRS/ Monitoring CRS 0 134 Number of students (males/females) regularly (80%) attending USDA supported classrooms/schools Standard #1 CRS/ Monitoring CRS 0 79,165 IMPAQ International, LLC Page 49 Food for Education Baseline Report Targets McGovern-Dole Performance Indicator Indicator Number Data Collection Method Data Source Baseline Midline Endline Life of Project Target Number of social assistance beneficiaries participating in productive safety nets as a result of USDA assistance Standard #17 CRS/ Monitoring CRS 0 88,753 Percent of students in target schools with health-related absences. Custom CRS/ Monitoring CRS 33% 20% Number of educational facilities (i.e. school buildings, classrooms, and latrines) rehabilitated/constructed as a result of USDA assistance Standard #7 CRS/ Monitoring CRS 0 212 Number of students enrolled in schools receiving USDA assistance Standard #8 CRS/ Monitoring CRS 0 40,429 IMPAQ International, LLC Page 50 Food for Education Baseline Report Targets McGovern-Dole Performance Indicator Indicator Number Data Collection Method Data Source Baseline Midline Endline Life of Project Target Percent of government staff trained in the literacy ToT who go on to conduct literacy trainings Custom CRS/ Monitoring CRS 0% 80% Value of public and private sector investments leveraged as a result of USDA assistance Standard #11 CRS/ Monitoring CRS $0.00 $882,046.94 Number of public-pivate partnerships formed as a result of USDA assistance. Standard #10 CRS/ Monitoring CRS 0 1 Number of Parent-Teacher Associations or similar “school” governance structures supported as a result of USDA assistance Standard #9 CRS/ Monitoring CRS 0 350 IMPAQ International, LLC Page 51 Food for Education Baseline Report Targets McGovern-Dole Performance Indicator Indicator Number Data Collection Method Data Source Baseline Midline Endline Life of Project Target Percent of VEDCs that are 'high functioning' Custom CRS/ Monitoring CRS 11% 35% Number of individuals who demonstrate use of new child health and nutrition practices as a result of USDA assistance Standard #19 CRS/ Monitoring CRS 0 465 Number of individuals who demonstrate use of new safe food preparation and storage practices as a result of USDA assistance Standard #21 CRS/ Monitoring CRS 0 710 Percent of schools targeted for WASH interventions that achieve at least a minimum standard Custom CRS/ Monitoring CRS 0% 87% IMPAQ International, LLC Page 52 Food for Education Baseline Report Targets McGovern-Dole Performance Indicator Indicator Number Data Collection Method Data Source Baseline Midline Endline Life of Project Target Number of individuals trained in safe food preparation and storage as a result of USDA assistance Standard #20 CRS/ Monitoring CRS 0 1,000 Number of schools using an improved water source Standard #22 CRS/ Monitoring CRS 178 208 Number of schools with improved sanitary facilities Standard #23 CRS/ Monitoring CRS 160 190 Percent of water user committees collecting user fees for maintenance of water points provided through USDA assistance Custom CRS/ Monitoring CRS 0% 90% IMPAQ International, LLC Page 53 Food for Education Baseline Report Targets McGovern-Dole Performance Indicator Indicator Number Data Collection Method Data Source Baseline Midline Endline Life of Project Target Number of teachers receiving teacher recognition awards Custom CRS/ Monitoring CRS 0 280 Number of libraries established Custom CRS/ Monitoring CRS 0 465 Number of books and supplementary materials created (by type) as a result of USDA assistance Custom CRS/ Monitoring CRS 0 79,252 Number of teachers in target schools trained on Literacy Custom CRS/ Monitoring CRS 0 360 IMPAQ International, LLC Page 54 Food for Education Baseline Report Targets McGovern-Dole Performance Indicator Indicator Number Data Collection Method Data Source Baseline Midline Endline Life of Project Target Number of teachers in target schools trained on Inclusive Education Custom CRS/ Monitoring CRS 0 460 Number of school administrators trained on Literacy Custom CRS/ Monitoring CRS 0 150 Number of school administrators trained on Inclusive Education Custom CRS/ Monitoring CRS 0 71 Number of daily school meals (lunch) provided to school-age children as a result of USDA assistance Standard #15 CRS/ Monitoring CRS 0 32,705,154 IMPAQ International, LLC Page 55 Food for Education Baseline Report Targets McGovern-Dole Performance Indicator Indicator Number Data Collection Method Data Source Baseline Midline Endline Life of Project Target Number of take-home rations provided as a result of USDA assistance Standard #13 CRS/ Monitoring CRS 0 94,177 Number of school gardens established Custom CRS/ Monitoring CRS 0 150 Number of educational facilities (i.e. school buildings, classrooms, and latrines) rehabilitated/constructed as a result of USDA assistance (kitchens) Standard #7 CRS/ Monitoring CRS 0 112 Number of educational facilities (i.e. school buildings, classrooms, and latrines) rehabilitated/constructed as a result of USDA assistance (storerooms) Standard #7 CRS/ Monitoring CRS 0 70 IMPAQ International, LLC Page 56 Food for Education Baseline Report Targets McGovern-Dole Performance Indicator Indicator Number Data Collection Method Data Source Baseline Midline Endline Life of Project Target Number of CBR trainings conducted in communities Custom CRS/ Monitoring CRS 0 8 Number of children screened for disabilities Custom CRS/ Monitoring CRS 0 4,440 Number of children receiving assistive devices Custom CRS/ Monitoring CRS 0 196 Number of days Pedagogical Advisors supported to conduct in￾school literacy coaching with teachers Custom CRS/ Monitoring CRS 0 300 IMPAQ International, LLC Page 57 Food for Education Baseline Report Targets McGovern-Dole Performance Indicator Indicator Number Data Collection Method Data Source Baseline Midline Endline Life of Project Target Number of steering committee visits Custom CRS/ Monitoring CRS 0 5 Number of Literacy Boost advocacy meetings Custom CRS/ Monitoring CRS 0 5 Number of literacy promotion activities completed (by type) as a result of USDA assistance Custom CRS/ Monitoring CRS 0 6,428 Number of VEDCs trained Custom CRS/ Monitoring CRS 0 360 IMPAQ International, LLC Page 58 Food for Education Baseline Report Targets McGovern-Dole Performance Indicator Indicator Number Data Collection Method Data Source Baseline Midline Endline Life of Project Target Number of VEDC graduation grants provided Custom CRS/ Monitoring CRS 0 126 Value (USD) of VEDC graduation grants provided Custom CRS/ Monitoring CRS 0 75,600 Number of student WASH ambassadors Custom CRS/ Monitoring CRS 0 150 Number of VEDC members/food preparers trained on food preparation and storage practices Custom CRS/ Monitoring CRS 0 3,377 IMPAQ International, LLC Page 59 Food for Education Baseline Report Targets McGovern-Dole Performance Indicator Indicator Number Data Collection Method Data Source Baseline Midline Endline Life of Project Target Number of wells and water stations/systems built or rehabilitated Custom CRS/ Monitoring CRS 0 30 Number of educational facilities (i.e. school buildings, classrooms, and latrines) rehabilitated/constructed as a result of USDA assistance (latrines) Standard #7 CRS/ Monitoring CRS 0 30 Number of active water user committees trained Custom CRS/ Monitoring CRS 0 30 IMPAQ International, LLC Page 60 Food for Education Baseline Report APPENDIX 2: References Center for International Education (CIE). (2016). Mali: Teacher Preparation and Continuing Professional Development in Africa (TPA). Brighton, England: University of Sussex. Retrieved from: http://www.sussex.ac.uk/cie/projects/completed/tpa/mali. Don Dillman, D.A. (2007). Mail and Internet Surveys: The Tailored Design Method (2007 update with new Internet, visual, and mixed-mode guide), 2nd ed. New York: John Wiley. Economic Research Service, USDA. (2012). U.S. Household Food Security Survey Module: Six-Item Short Form (Tech.). Washington, DC: USDA. Eichberg, S. and Hart, J. (2013). The Truth and the Facts: Food Inequality on Long Island. Garden City, NY: Center for Health Innovation, Adelphi University. Retrieved from: http://www.adelphi.edu/wp-content/blogs.dir/3/files/2013/04/Food-Inequality-Report￾2013.pdf?t=1365537911-1632184 Fleiss, J. L., & Cohen, J. (1973). The equivalence of weighted kappa and the intraclass correlation coefficient as measures of reliability. Educational and psychological measurement, 33(3), 613-619. Food and Agriculture Organization. (2010). Guidelines for Measuring Household and Individual Dietary Diversity. Rome, Italy: United Nations. Harding, J., Morris, P., and Hughes D. The Relationship Between Maternal Education and Children's Academic Outcomes: A Theoretical Framework. Journal of Marriage and Family, vol. 77, no. 1, 2015, pp. 60-76. DOI: 10.1111/jomf.12156. Hess, R. D., & Holloway, S. D. (1984). Family and school as educational-institutions. Review of child development research, 7, 179-222. Human Subject Regulations Decision Charts. (2016, February 16). Retrieved from http://www.hhs.gov/ohrp/policy/checklists/decisioncharts.html. Leer, J. (2016). Learn Literacy Boost Baseline 2016 (Publication). Save the Children Magnani, R. (1999). Food and Nutrition Technical Assistance III Project (FANTA)- Sampling Guide (Publication).Fhi360.doi:https://www.fantaproject.org/sites/default/files/resources/ Sampling-1999-Addendum-2012-ENG_0.pdf IMPAQ International, LLC Page 61 Food for Education Baseline Report Paxson, C. and Norbert, S. (2007). Cognitive Development among Young Children in Ecuador: The Roles of Wealth, Health, and Parenting. Journal of Human Resources, vol. XLII, no. 1, pp. 49- 84. doi:10.3368/jhr.XLII.1.49. Stallings, J. A., & Mohlman, G. G. (1988). Classroom observation techniques. In J. P. Keeves (Ed.), Educational research, methodology, and measurement: An International handbook (pp. 469- 474). Oxford, England: Pergamon. The readiness and motivation of teachers and principals to change basic education teaching practices in Lao PDR (Publication). (2016). Laos-Australia development Learning Facility. doi:http://ladlf.org/images/publications/Readiness/081116_LADLF_Evaluative_Study_Teac her_Readiness.pdf United States Department of Agriculture, Economics Research Service. (2012, September). U.S. Household Food Security Survey Module: Six-Item Short Form. United States Department of Agriculture, Foreign Agricultural Service. (2014, July). Food for Progress and McGovern-Dole Indicators and Definitions. Food Assistance Division, Office of Capacity Building and Development. Walker, S. et al. (2011). Inequality in Early Childhood: Risk and Protective Factors for Early Child Development. The Lancet, vol. 378, no. 9799, pp. 1325-1338. doi:10.3368/jhr.XLII.1.49. Kim, Y. S. (2009). The relationship between home literacy practices and developmental trajectories of emergent literacy and conventional literacy skills for Korean children. Reading and Writing, 22(1), 57-84. World Food Program (2016, March). Rapport de Synthèse : Enquête Nationale sur la Sécurité Alimentaire et Nutritionnelle (ENSAN Mali). Rome, Italy: United Nations. http://documents.wfp.org/stellent/groups/public/documents/ena/wfp284183.pdf?_ga=1. 241287938.1421946729.1471897724. IMPAQ International, LLC Page 62 Food for Education Baseline Report APPENDIX 3: LIST OF STAKEHOLDERS Exhibit 30: Key Informants at School Level School/ District Type Group Composition Notes 1. School Nong District Date: 13 March 2017 Mothers FGD 6 mothers Fathers 3 fathers VEDC FGD 3 men 3 women Teacher FGD 3 women Principal KII 1 woman Cook FGD N/A Activity was not undertaken in Nong district because the school lunch program had not yet commenced Storekeeper KII N/A Activity was not undertaken in Nong district because the school lunch program had not yet commenced. 2. School Sepon District Date: 14 March 2017 Mother FGD 7 mothers 3 mothers had children who were not yet in school, the child of one mother had already graduated. Father FGD 5 fathers VEDC FGD 1 man Others were busy Teacher FGD 2 women Principal KII 1 woman Cook FGD 3 women The village operates a rotational system of cooks. The interviewees were mothers from the mothers focus group who changed in and out during the duration of the FGD. One average there were 3 participating in the discussion. Storekeeper KII 2 men 3. School Phin district Date:16 March 2017 Mothers FGD 4 mothers Fathers FGC 4 fathers VEDC FGD 5 men Teacher FGD 1 woman Principal KII 1 woman Cook FGD 2 women Storekeeper KII 1 man 4. School Atsaphone district Date: 22 March 2017 Mothers FGD 4 mothers Mothers all had children in Grade 1. Fathers FGD 10 fathers 5 fathers had grandchildren at the school; 3 had children who had already graduated from the school and 2 had children in the school cluster, of which one at this school. VEDC FGD 3 men Included Deputy head of village Teacher FGD 2 women 1 man Years of service: 28 years, 17 years, 5 years Principal KII 1 man Cook FGD 3 women Storekeeper KII 1 man IMPAQ International, LLC Page 63 Food for Education Baseline Report 5. School Outhomphone district Date: 27 March 2017 Mothers FGD 10 mothers 1 did not have children at the school. The rest had a total of 14 children at the school between them. Fathers FGD 6 fathers Only one father had a child at the school, 2 had grandchildren at the school. The rest had children who had already left the school. VEDC FGD 4 men 1 woman Included head and deputy head of village and head of women’s union. Teacher FGD 2 women 1 men Principal KII 1 man Cook FGD 3 women Storekeeper KII 1 man 6. school Phalanxay district Date: 28 March 2017 Mothers FGD 3 mothers Fathers FGD 2 fathers VEDC FGD 4 men 1 women 3 members were VEDC for this school, and the other 2 from the VEDC cluster school nearby. Teacher FGD 2 women 1 man Principal KII 1 woman Cook FDG 3 women Led by Women’s Union leader Storekeeper KII 1 man 7. school Vilabouli district Date: 29 March 2017 Mothers FDG 6 mothers Together, they were mothers of 11 children at the school. Fathers FDG 5 fathers Together they were fathers of 8 children in the school VEDC FDG 4 men 1 woman Included head of village, 2 deputy heads, and head of youth union. Teacher FDG 1 man 1 woman Principal KII 1 man Cook FDG 3 women Storekeeper KII 1 man IMPAQ International, LLC Page 64 Food for Education Baseline Report APPENDIX 4: ADDITIONAL TABLES AND COMPLEMENTARY OUTCOMES Exhibit 31: Key Requests and Priorities from Schools for Improvement District School Infrastructure School Equipment Learning materials and supplements Entertainment Other requests Notes District 1  Electricity  Regular water provision for garden  Roof repairs  Shade for children to play  Bathroom for students  Door repairs  Textbooks and books for children to read  Sports equipment  Music equipment  Playground  Games for children to play  Provide more teaching experiences  From parents: getting free textbooks District 2  Fence for school so trees don’t get eaten by animals  Rebuilding storeroom  New kitchen  Need nails, metal, roofing  Flatten the ground so it is even  Sports equipment  Sports field  Playground  Slides and swings  Other food District 3  A plan to build a new secondary school so there is more room,  Electricity  Need upper secondary near village  Tables, chairs, blackboards  Pre-school materials  Create a pre￾school  Need for textbooks  Need more pens and pencils  Reading and writing books  Footballs, kato (a small kick volleyball)  Bowls and spoons; questions about how to help really poor students  Develop extra activities so IMPAQ International, LLC Page 65 Food for Education Baseline Report District School Infrastructure School Equipment Learning materials and supplements Entertainment Other requests Notes  Scholarships for students to study at lower secondary children use spare time well District 4  Priorities for green school, grow vegetables  Roof for kitchen  Build a road outside school  More places to sit in yard  Bigger kitchen  New pre-school room  Electricity  Floor and ceiling for P5 class  Tap at front of school  Electric pump  Build secondary school closer to the village  Build a pre-school in center of village  More seats to eat together  Teacher training District 5  More clean water – water pump to help with water for cooking  More stoves for cooking  Plant trees for fruit  Barbed wire for vegetable garden  Money to do community hall  Build a new school building  Teaching materials  Pens  Writing and reading books  Improved curriculum (i.e. with local examples)  Letter cards for teaching  Stove, pots, pans (different sizes)  Well trained teachers  Enter green school competition  Extra food (e.g., eggs, vegetables, meat) IMPAQ International, LLC Page 66 Food for Education Baseline Report District School Infrastructure School Equipment Learning materials and supplements Entertainment Other requests Notes District 6  Entrance for school  Fence for smaller school (other campus)  Front wall  Bigger buildings  Eating place for children and roofing for it  A place for sitting to eat (agreed with VEDC a long time ago but didn’t happen)  Repair school building  Nails to fix tables and chairs  Materials for teaching  Textbooks  Swings  Sports equipment: footballs  Scholarships for students to study at higher levels  More teachers District 7  Electricity  Water for school  New school building for pre-school  Eating space for children  Move storeroom closer to water source and away from flooding  New pump for clean water  Water for toilet  Fixing teachers’ offices, many repairs needed  Plan to fix tables and chairs  Teaching materials  Textbooks grades 1-3  Toys for children  Sports equipment  Football field  Loudspeaker for entertainment days  Provide clothes for children  More teachers  Bowls and spoons  Work for their children IMPAQ International, LLC Page 67 Food for Education Baseline Report Overall comments: Key things common across (almost) all locations visited:  Equality between girls and boys  Request for different food to supplement the lunch meal – dislike of the green lentils (they smell!)  All the schools wanted to know what was happening with the program and until when it would continue. All but one of the schools had run out of rice since the beginning of March, but this did not seem to have impacted on children’s attendance, despite everyone saying children missed school less since the rice program started. Key things of interest – differences between locations:  Different relationships between the VEDC and the school  Different remuneration in rice and oil in different villages  Different abilities to be pro-active; different abilities to manage the accounts.  Schools that stated they didn’t have problems (school attendance, other difficulties) were the ones more likely to want different types of food at lunchtime. A conclusion could be that once a village reaches a certain level of development, parents want more than just stopping the children from being hungry.  Different beliefs in who should be supplementing the food. While a couple of locations set up systems to supplement the rice and were very active in the program, others believed that it was CRS’ responsibility to provide other types of food IMPAQ International, LLC Page 68 Food For Education Baseline Report Exhibit 32: Socioeconomic Status by District Does your home have a… Atsaphone Nong Outhoumphone Phalanxai Phin Vilabouly Sepon Mobile 80% 57% 88% 80% 67% 70% 76% Electricity 85% 87% 97% 88% 77% 87% 83% Refrigerator 42% 26% 62% 35% 28% 58% 48% Bicycle 58% 35% 58% 60% 33% 55% 40% TV 70% 64% 86% 74% 57% 78% 74% Motorbike 80% 84% 86% 75% 69% 83% 84% Car 11% 12% 12% 6% 16% 26% 21% Tractor [TOK TOK] 88% 22% 80% 85% 78% 70% 45% None 0% 4% 0% 0% 3% 4% 1% Don't know 0% 0% 0% 0% 0% 1% 0% Total Observations 423 161 370 255 366 213 174 Source: Student Survey; Authors’ calculations Exhibit 33: Socioeconomic Status by Consumer Durable Goods Source: Student Survey; Authors’ calculations N=1,962 Exhibit 34: Non-primary Languages Spoken in Household 86% 79% 76% 73% 72% 50% 43% 14% 1% 0% Electricity Motorbike Mobile Tractor [TOK TOK] TV Bicycle Refrigerator Car None Don't know IMPAQ International, LLC Page 69 Food For Education Baseline Report Source: Student Survey; Authors’ calculations N=1,962. Note: These categories do not sum to 100% as respondents could select more than one option. Exhibit 35: Reasons Students Liked going to School Source: Student Survey; Authors’ calculations N=1,962 Exhibit 36: Access to Reading Material at Home 51% 27% 10% 9% 8% 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% None Lao Miscellaneous Don' tKnow/No response Phoutay 48% 42% 39% 32% 31% 14% 9% 7% 7% 4% 3% 3% 3% 1% Reading Learning new things Writing Other Being with my friends Physical education/sports at school Participating in class activities/games Teacher reading Don't Know/Refused Dancing Working in groups Answering questions Food is provided Presenting ideas in class IMPAQ International, LLC Page 70 Food For Education Baseline Report Source: Student Survey; Authors’ calculations N=1,962 Exhibit 37: Home Literacy Environment Source: Student Survey; Authors’ calculations N=1,959. Note: Excluded are students who did not know the answer or refused to answer. 89% 50% 22% 12% 7% 7% Textbooks Coloring/Drawing books Storybooks/Comics Newspapers Magazines None of these 0% 20% 40% 60% 80% 100% Textbooks Coloring/Drawing books Storybooks/Comics Newspapers Magazines None of these IMPAQ International, LLC Page 71 Food For Education Baseline Report Exhibit 38: Literacy Skills by Main Language at Home Source: Student Survey; Authors’ calculations N= 496; Reading comprehension; N = 87 Listening comprehension; N = 404 *** indicates significant difference between Lao and Phoutay at the 1 percent significance level, ** at the 5 percent level, * at the 10 percent level. ^^^ indicates significant difference between Phoutay and miscellaneous languages at the 1 percent level, ^^ at the 5 percent level, ^ at the 10 percent level. ϯϯϯ indicates significant difference between Lao and miscellaneous languages at the 1 percent level, ϯϯ at the 5 percent level, ϯ at the 10 percent level. Note: Excluded students were those who did not know the answer or refused to answer 29% 24% 23% 52% 33% 72% 67% 81% 28% ^^^ 29% ^^^ 10% 27% ^^ 16% ^ 64% ^^^ 67% ^^^ 75% ^ 11% ϯϯϯ 8% ϯϯϯ 17% 43% ϯ 27% 39% ϯϯϯ 33% ϯϯϯ 62% ϯϯϯ 13% 0% 16% ** 36% *** 23% *** 54% 33% 60% Listening comprehension (Non-readers) Reading comprehension (Readers only) Independent readers Matching words Most used common words Letter Knowledge Letter sound recognition Expressive vocabulary % Correct Unknown Miscellaneous Phouthay Lao IMPAQ International, LLC Page 72 Food For Education Baseline Report Exhibit 39: Different Types of Comprehension Questions by Readers and Language Source: Student survey; Authors’ calculations N= 87 Exhibit 40: Different Types of Comprehension Questions by Non-readers and Language Source: Student survey; Authors’ calculations N= 404 Note: Excluded students were those who did not know the answer or refused to answer 16% 46% 56% 23% 0% 67% 67% 33% 0% 38% 38% 27% 8% 17% 16% 25% Summary Literal Inferential Evaluative Unknown Miscellaneous Phoutay Lao 18% 46% 46% 36% 15% 52% 52% 25% 5% 23% 39% 55% 6% 16% 56% 66% Summary Literal Inferential Evaluative Unknown Miscellaneous Phoutay Lao IMPAQ International, LLC Page 73 Food For Education Baseline Report Exhibit 41: Teacher Narrating a Story/Reading a Poem in Class District Frequency Atsaphone Nong Outhoumphone Phalanxai Phin Vilabouly Sepon Every day 8% 15% 7% 10% 13% 10% 12% A few times during the week 40% 33% 31% 42% 36% 33% 37% Once during the week 13% 11% 18% 14% 10% 15% 13% Never 40% 40% 45% 35% 41% 44% 38% Source: Student survey; Authors’ calculations Exhibit 42: Teachers Asking About the Story Narrated/Poem Read In Class the Past Week District Frequency Atsaphone Nong Outhoumphone Phalanxai Phin Vilabouly Sepon Every day 5% 11% 3% 5% 8% 7% 8% A few times during the week 28% 24% 25% 26% 32% 22% 26% Once during the week 12% 16% 12% 15% 10% 16% 13% Never 56% 50% 60% 55% 50% 55% 54% Source: Student survey; Authors’ calculations Exhibit 43: Students Playing a Game in the Classroom around Alphabets District Frequency Atsaphone Nong Outhoumphone Phalanxai Phin Vilabouly Sepon Every day 8% 3% 8% 8% 9% 7% 11% A few times during the week 27% 17% 29% 40% 34% 19% 31% Once during the week 14% 13% 14% 15% 12% 7% 17% Never 51% 67% 50% 41% 46% 67% 41% Source: Student survey; Authors’ calculations IMPAQ International, LLC Page 74 Food For Education Baseline Report Exhibit 44: Access to Reading Materials by District Reading Materials Atsaphone Outhoumphone Phalanxai Phin Vilabouly Nong Sepon Textbooks 77% 94% 94% 94% 94% 88% 87% Magazines 7% 5% 10% 6% 5% 11% 8% Newspapers 11% 15% 16% 7% 11% 16% 10% Storybooks/COMICS 24% 24% 29% 21% 21% 25% 18% Coloring an drawing books 56% 50% 60% 52% 41% 60% 45% None of these 13% 3% 4% 4% 5% 8% 8% Source: Student survey; Authors’ calculations Exhibit 45: Number of Days Children Missed School due to Sickness Source: Student Survey; Authors’ calculations; N=1,927 Excluded were students who did not know the answer or refused to answer. 68% 9% 10% 6% 2% 4% 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% 0 1 2 3 4 5 # of Days IMPAQ International, LLC Page 75 Food For Education Baseline Report APPENDIX 5: OTHER SUBTESTS OF READING ASSESSMENT As mentioned in Section 5.3.1, we also tested students on other literacy skills including expressive vocabulary, phonemic awareness, word recognition, and object-to-picture matching. This appendix presents the outcomes of these subtests to shed more lights on children’s literacy outcomes. Expressive Vocabulary To assess children’s vocabulary skills in Lao, in two different questions students were asked to name as many animals and foods as they could in Lao. Enumerators were supposed to count the total number and mark down children’s responses from 0 to 10, for each question. Students were able to name on average 7 animals and 7 foods in Lao (on average 14 out of 20 animals and foods all together). Boys did slightly better than girls by naming one item more than their female peers. Exhibit 45 shows Lao and Phoutay speakers on average were able to slightly name more items in Lao than students who spoke other languages at home or did not know their main language. Exhibit 46: Expressive Vocabulary Skills by Language Source: Students’ survey; Authors’ Calculation; N=496 Note: Excluded students were those who did not know the answer or refused to answer Phonemic Awareness Enumerators were supposed to read a word to the child and then tell her/him the word’s first letter’s sound (e.g., Cats starts with /c/). Then, students were read three words aloud and asked to identify the one that starts with the same sound (for three different sets of words). Lao speakers, followed by Phoutay speakers paired words the most with their first-letter sounds compared to students who did not know their main language at home (Exhibit 46). 16 15 12 12 0 2 4 6 8 10 12 14 16 18 20 Lao Phoutay Miscellaneous Unknown Number of Items Named IMPAQ International, LLC Page 76 Food For Education Baseline Report Exhibit 47: Phonemic Awareness by Language Source: Students’ survey; Authors’ Calculation; N=496 Note: Excluded students were those who did not know the answer or refused to answer Word Recognition To assess children’s word recognition skill, students were given a chart of 20 words comprising the most frequently used words from learners’ language arts textbooks, selected based on frequency of their appearance. On average students were able to read 5 out of 20 of the most common words (25 percent). ນາ was the easiest word for students to read in which almost half of students read this word correctly, while ຕະຫຼາດ and ອະນຸຍາດ were the most difficult words that only 14 percent of the second graders could read them. Exhibit 47 shows the distribution of most used words scores by language. Students whose main language was Phoutay at home read the least number of words correctly compared to their peers. Similar findings in the LEARN report (2016), the distribution of scores follows a bell curve that is skewed to the left, showing most of the students were not able to read even the most common words from their textbooks correctly. 14% 14% 37% 32% 14% 16% 20% 26% 22% 27% 14% 21% 50% 43% 29% 21% 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% Lao Phoutay Other Don't know % of the Sample 0 pair words correct 1 pair word correct 2 pair words correct 3 pair words correct IMPAQ International, LLC Page 77 Food For Education Baseline Report Exhibit 48: Most Common Used Words Distribution by Main Language Source: Students’ survey; Authors’ Calculation; N=496 Note: Excluded students were those who did not know the answer or refused to answer Object to Picture Matching To measure children’s ability to understand symbolic representation, we showed students 5 words and asked them to match those objects with their corresponding pictures. This activity was repeated twice with two different sets of words and pictures to match (10 words in total). To clarify the instructions, enumerators used one of the words as an example, which was excluded from the analyses. On average, students were able to match four objects out of nine with their pictures (45 percent). The words “cup” (ຈອກ) and “book” (້ປຶ້້້່້້້ມ) were the most difficult to match for students in which only 36 percent got them right, while students found “snake” (ງູ) the easiest to match with its corresponding picture (63 percent). Students whose main language at home was Lao were able to match at least 75 percent of the objects (seven out of nine) at a higher rate than their peers (55 percent of Lao speakers compared to 19 percent of Phoutay speakers). 0% 10% 20% 30% 40% 50% 60% None Identified 1-5 words Identified 6-10 words Identified 11- 15 words Identified 16-20 words Lao Phoutay Other Didn't Know IMPAQ International, LLC Page 78 Food For Education Baseline Report APPENDIX 6: INTER-RATER RELIABILITY Reading Assessment To measure the reliability and level of homogeneity of enumerators’ scores on children’s literacy skills, 13 percent of the sample (66 out of 496) were assessed by two different enumerators simultaneously. Long one-way Analysis of Variance (ANOVA) techniques, which is used to determine whether the mean of a dependent variable is the same in two or more unrelated and independent groups, were used to calculate the intra-class correlation within pairs of assessors for a measure of inter-rater reliability. Adapted from Fleiss et al. (1973), we interpreted the intra-class correlations as it follows:  Less than .40 – Poor  Between .40 and .75 – Good or fair  Greater than .75 – Excellent Exhibit 48 shows the percent of agreement between the raters, as well as inter-rater reliability ratings. There was an excellent inter-rater reliability on every measure for literacy skills, showing high internal validity of the scores. Exhibit 49: Inter-rater Reliability by Literacy Skill Subtests Literacy Skill Sub-test Inter-rater Reliability Rating Expressive Vocabulary 99% Excellent Phonemic Awareness (word pairs) 98% Excellent Letter Knowledge 99% Excellent Word Recognition 99% Excellent Object to Picture Matching 99% Excellent Reader 100% Excellent Fluency 91% Excellent Accuracy 99% Excellent Comprehension (Reading and Listening) 93% Excellent Source: Student Survey; Authors’ calculations Classroom Observations For the classroom observation, we also assessed the interrater reliability of enumerators’ scores. In 17 schools (about 20 percent of the sample), two enumerators simultaneously conducted the observations for inter-rater reliability purposes. Enumerators had higher inter-rater reliability for teacher and student activities, at 91 percent, than for student attentiveness, at 69 percent.