1 USDA IDEA Guatemala Baseline Results Lead Authors: Lucrecia Méndez & Clara Pava Co-author: Local Consultant Guillermo Franco With special thanks to SCI Guatemala Country Office Staff and M&E Coordinator /José Giron and Program Operations Manager/Rolando Herrera; SC/US Home Office technical and enumerators. June 2015 © 2015 FY 2014 USDA McGovern-Dole IDEA Guatemala Project Baseline Results Program: McGovern Dole Agreement Number: FFE-520-2014/043-00 Funding Year: Fiscal Year 2014 Project Duration: 2019-2022 Implemented by: Save the Children Evaluation Authored by: Lucrecia Méndez Clara Pava Guillermo Franco DISCLAIMER: This evaluation was conducted by an independent third party firm. The views expressed in this report do not necessarily reflect the views of the United States Department of Agriculture or the United States Government. Accessibility Note: An accessible version of this report can be made available upon request by emailing FAS.monitoring.evaluation@usda.gov. 2 Table of Contents Executive Summary ........................................................................................................................................... 4 I. Introduction .................................................................................................................................................... 5 II. Context ........................................................................................................................................................... 6 III. Methods ........................................................................................................................................................ 7 Sampling ......................................................................................................................................................... 7 Measurement: School Survey and Learner Assessment ........................................................................... 9 Analysis .........................................................................................................................................................10 IV. Children’s Background Characteristics..................................................................................................11 V. Children’s Home Literacy Environment................................................................................................. 14 VI. Children’s Spanish Literacy Skills...........................................................................................................17 Letter Identification ....................................................................................................................................18 Word Reading: Most Used Words ............................................................................................................18 Decoding ......................................................................................................................................................19 Oral reading ..................................................................................................................................................19 Fluency and Accuracy ................................................................................................................................. 19 Reading Comprehension ............................................................................................................................20 VII. Children’s Mother Tongue Literacy Skills ...........................................................................................21 Letter Identification ....................................................................................................................................22 Word Reading: Most Used Words ............................................................................................................22 Fluency and Accuracy ................................................................................................................................. 22 Reading Comprehension ............................................................................................................................23 VIII. Learning Equity and Struggling Students ...........................................................................................24 Early Education ...........................................................................................................................................24 Repetition Status ..........................................................................................................................................25 IX. School survey results ................................................................................................................................27 Increased Access to Clean Water and Sanitation Services ....................................................................27 Increased Access to Preventive Health Interventions ...........................................................................27 Increased Access to Requisite Food Prep and Storage Tools and Equipment ..................................28 Increased engagement of local organizations and community groups ................................................28 Reading practices and resources ................................................................................................................29 Teaching staff ...............................................................................................................................................29 3 X. Other indicators of particular interest to USDA ...................................................................................30 Improved attentiveness ..............................................................................................................................30 Reduced Short-Term Hunger ....................................................................................................................32 Increased Access to Food (School Feeding) ...........................................................................................33 Improve the Attendance of Students .......................................................................................................33 Main Causes of School Absences .............................................................................................................34 Increased student enrollment ....................................................................................................................34 Increased Community Understanding of the Benefits of Education ..................................................35 Increased Knowledge of Safe Food Prep and Storage Practices .........................................................35 Increased Knowledge of Health and Nutrition Practices .....................................................................36 XI. Next steps in the evaluation ....................................................................................................................37 Sharing and incorporating baseline results ..............................................................................................37 Midline evaluation .......................................................................................................................................37 XI. Conclusions ................................................................................................................................................38 Appendix A. Inter-rater reliability .................................................................................................................40 Appendix B. Descriptive Statistics ................................................................................................................41 Table B1: Background Characteristics by Sample Group .....................................................................41 Table B2. Nutritional Status and Attention by Sample Group ............................................................42 Table B3. Home Literacy Environment by Sample Group ..................................................................43 Table B4. Spanish Literacy Skills by Sample Group ..............................................................................44 Tables B5. Detailed Spanish Results by Item. Letters, MUW and Decoding ....................................44 Tables B6. Detailed Ixil Results by Item. Letters and MUW................................................................47 Tables B7. Detailed K’iche´ Results by Item. Letters and MUW ........................................................49 Tables B8. School Characteristics by Sample Group .............................................................................51 Appendix C. Regression Results ....................................................................................................................55 Table C1. Multivariate Regression Analysis. Spanish Literacy Skills ...................................................55 Table C2. Multivariate Regression Analysis. Ixil Literacy Skills ...........................................................55 4 Executive Summary This report examines the results of a learner background survey, reading assessment, and school survey conducted between February and March 20151 . The survey and reading assessment covered 1108 learners throughout 70 schools in six municipalities of the Quiché Department in Guatemala. The 70 schools are split into 35 primary schools in the Ixil area designated to receive Literacy Boost and 35 comparison primary schools in the K’iche’ area receiving no intervention during the first year of the program. The results show that learners’ in comparison schools are more advantaged than their intervention peers, the low Spanish skills in the intervention group, and the even lower skills in mother tongue for both groups. Learners in the comparison group have access to more socioeconomic resources: animals, assets (fridge, bicycle, motorbike amongst others) and this translates into better basic and advanced Spanish literacy skills. This poses a challenge for the midline evaluation for which this report proposes Propensity Score Matching as the statistical technique to be used to assess the impact of the program next year. In terms of reading skills in the intervention group, on average a child is able to identify correctly 56% of the Spanish letters in the alphabet (15 out of 27 letters) and is able to read on average only 28% of the most used words in their language textbook (6 out of 20 words). In reading comprehension they are able to answer on average 1 out of 10 questions correctly. The ability in mother tongue is even lower where on average children in the intervention area are able to read less than 1 out of 20 simple words. This calls for the program to focus on basic reading skills of children, starting with letter identification through engaging materials and activities with the goal of having them read with comprehension by the end of the program. Additionally, the results show little engagement of families in reading activities with children and a low percentage of reading materials. Only 32% of children within the intervention group report having storybooks at home and on average only 27% of their family members read to them. The Literacy Boost program could work on making more Spanish and mother tongue materials available to families and community members and increasing the reading practices at home. Finally, the results show the students that are struggling and that Literacy Boost could help the most. Within the intervention group repeaters, students that did not attend an ECD center and those with a lower socioeconomic status have lower predicted literacy skills. The program should aim to target these students (repeaters, students that did not attend an ECD center and those more disadvantaged socioeconomically) so that they do not fall further behind in their literacy skills and they are able to make gains to catch up with their more advantaged peers. 1 Some of the results in sections IX and X come from a school census and community meetings conducted between October 2014 and January 2015 and are marked as such in the document. 5 I. Introduction This report examines the results of a learner background survey, reading assessment and school survey conducted between February and March 2015. The survey and reading assessment covered 1108 learners throughout 70 schools in six municipalities of the Quiché Department in Guatemala. The 70 schools are split into 35 primary schools designated to receive Literacy Boost and 35 comparison primary schools receiving no intervention during the first year of the IDEA (Inversión para el Desarrollo del Altiplano) program. The Literacy Boost methodology, a Save the Children toolkit, includes teacher training, community reading activities, and age-appropriate local language material creation to support literacy skills among early-grade children. These skills include concepts about print, letter awareness, single word reading of most used words, reading fluency, reading accuracy, and reading comprehension. As part of Literacy Boost, learners are periodically assessed in each of these skills through an adaptable assessment tool to inform programming and estimate program impact. The data gathered from these schools is analyzed to present a snapshot of the literacy skills of grade 2 learners in these schools and to inform the adaptation of SC’s Literacy Boost program to this context. The key questions to be explored in this report include: 1. How comparable are learners in Literacy Boost schools versus comparison schools in terms of reading skills, background characteristics, home literacy environment, and school environment? 2. What can the baseline tell us about learners' reading skills? What does this mean for Literacy Boost programming? 3. How do the learners' reading skills vary by student background, school environment, home literacy environment, and other dimensions of equity? What does this mean for targeting Literacy Boost's two strands of intervention? 4. Given the results, what does this mean for conducting a midline evaluation? To investigate these questions, this report will first describe the research methods used; including sampling, measurement, and analysis. Next, in order to see if groups are statistically similar, the comparability of Literacy Boost and comparison schools will be examined through t-tests both for the student background characteristics and their Spanish literacy skills. Then, the report will investigate any correlations of the literacy skills with student background, school environment, or home literacy practices & environment variables using multilevel regression analysis. Finally, the report will explore the main results from the school survey data and other indicators that are relevant to USDA and that were collected through the baseline survey, a school census, and community meetings. 6 II. Context Guatemala has a territory of 108,889 square kilometers and is divided into 22 departments and 338 municipalities. Its capital is Guatemala City. According to data from the Livelihoods National Survey – ENCOVI 2011- Guatemala’s population is of 14,636,487 with a total of 135 habitants per square kilometer. According to the National Statistics’ Institute -INE-, the country currently holds a total of 16 million people. A total of 48.5% of the country’s population lives in urban areas while 51.5% lives in rural areas and 40% of the population is indigenous. A total of 53.7% live in poverty and 13.3% in extreme poverty. These levels of poverty show the exclusion, marginalization and inequality that people live in. According to the ENCOVI survey from 2011, the departments with higher levels of poverty are: Alta Verapaz (78.2%), Sololá (77.5%), Totonicapán (73.3%), Quiché (71.9%) and Suchitepéquez (70.7%). The departments that show a lower “Development Index” in terms of life expectancy, health, education and income levels are: Alta Verapaz, Quiché, Totonicapán, Huehuetenango and San Marcos. Guatemala is a multicultural, multilingual and multiethnic country, with a total of 22 languages, a population growth rate of 2.4% and a fertility rate of 3.6% - the highest among Latin America. The average number of family members is 5. The population of both regions of Quiché, mostly Mayan and rural, are characterized by some of the worst economic and social indicators in Guatemala: 82% of Quiché’s population lives in poverty and 37% live in extreme poverty (less than $1 USD per day). This challenging socioeconomic situation is further exacerbated in Quiché’s Ixil Area including Nebaj, Chajul and Cotzal) and the Northern Area (Sacapulas, Cunen and Uspantan). Both regions are considered to be two of the most marginalized territories in the country where access to basic social services (included education), provided by either the government or NGO is extremely limited. They are among the 125 municipalities with the highest poverty in Guatemala, according to the General Secretariat of Planning - SEGEPLAN - (Spanish acronym). In relation to education, there are several gaps in the pre-school and primary levels – the coverage deficit is of 40% at the national level and among 60% or 70% for some departments, including El Quiché. According to information from the National Statistic’s Institute -INE 2011-, Guatemalans have an average schooling of 5.98 years. In urban areas, this rate is higher (8.35 years), while in urban areas the average schooling is of 3.84 years. Women in the rural areas have more disadvantages, since they have less opportunities and hence lower levels of schooling and education. Evidently, rural and indigenous communities, have less access to education and women are the most affected. The Ministry of Education has focused its efforts in increasing coverage and has promoted bilingual and intercultural education – trying to ensure that education is given to children in their modern tongue. However, the reality is very different. Only half of children in school age that attend primary school receive bilingual and intercultural education. For the half that does not have this privilege, education is a limitation to learning since it comes in a language different from their culture- a language that they do not understand and this then leads them to learning difficulties. Linked to this situation, most of the teachers do not have updated techniques and technologies to promote learning among students – as a result, most of the students to not pass the evaluations that the Ministry of Education holds. Also, an important percentage of students drop out of school due to poverty, demotivation, 7 child labor, and illness amongst others. The Ministry of Education has led many efforts to improve the educational system but there are many challenges to overcome in order to ensure access to quality of education of children for the rural and indigenous communities in Guatemala. Save the Children proposes the use of the “Literacy Boost” methodology to contribute to the policies and priorities to improve quality of education through reading and creative and innovative techniques that break the traditional teaching scheme. Proven experience and results in the implementation of the “Literacy Boost” methodology in other countries show that it improves the teaching process and the reading skills of children in school-age. One of the challenges for Guatemala is to improve the quality of education; the results of national level evaluations show that only 10% of the students are able to pass reading evaluations. School drop-out levels are higher in rural areas because of lack of economic opportunities, low quality of educational services, teachers’ skills, lack of educational infrastructure, lack of materials and a weak supervision and follow-up system. III. Methods Sampling The baseline theoretical sample was of 1,260 students from second grade (primary school) divided into 35 schools which will receive the Literacy Boost2 component during the first year (n=630 students) and 35 schools from the comparison group (630) students, which will receive the intervention during the second year. 70 schools where calculated as the minimum sample size needed to detect a project effect size of 0.3 standard deviations with 80% of probability and 95% confidence – assuming a 25% attrition rate between the baseline and the final evaluation, a correlation of 0.6 among the base line and the final evaluation and a 0.25 intra-cluster correlation amongst the results in different schools. In practice, the baseline sample was of 1108, 12.06% lower than the theoretical sample. The reasons for students’ absence were diverse: disease, a local festivity, help in household chores (collection of firewood to cook), and loss of a relative or neighbor – with sickness being the most prevalent. In both groups (intervention and comparison), where more than 20 second grade students were present, 20 students were chosen randomly (10 boys and 10 girls when possible). In the schools where 20 or less students were present in the second grade classroom at the time of the assessment, the total number of students were included in the study. If the school had more than one second grade section, one of the sections was randomly selected. The final sample contains 550 students from the intervention group and 558 from the comparison group. Table 1 shows the total of students disaggregated by gender per group and the composition of the sample. For the year 1 intervention, a total of 269 girls and 281 boys were part of this sample, while 295 boys and 263 girls were part of the comparison group. From the total number of students that participated in the baseline, 52% were boys and 48% were girls. 2 The Intervention Group (Literacy Boost) for year 1, appears as the Intervention Group No. 1 in this report. 8 Table 1. Number of students disaggregated by gender Students from both groups disaggregated by gender Gender Literacy Boost/Intervention Comparison Whole Sample Girls Boys Total Girls Boys Total Girls Boys Total Total 269 281 550 263 295 558 532 576 1108 Source: Baseline Database, IDEA, April 2015. The average number of students per school was 16 students among the Literacy Boost/Intervention group and 15 students among the comparison group. Out of the 70 schools in the sample, only 27 had more than 20 students in second grade. Table 2. Average number of students per school and schools with more than 20 students in Second grade Literacy Boost/Intervention Comparison Group Average number of students per school 16 15 No. Of schools with more than 20 students in Second grade 15 12 Source: Baseline Database, IDEA, April 2015. The schools were chosen according to the following criteria: acceptance and approval to carry out the baseline study; having more than 30 students enrolled in Second Grade, approval from community leaders and from representatives from the Ministry of Education at local level. After all these criteria were met, 35 schools from each intervention year were chosen randomly. 9 Table 3. Number of schools and students that participated in the study by municipality General characteristics from the Sample Literacy Boost Intervention Group Comparison Group/Comparison Whole Sample Municipality Chajul Cotzal Nebaj Total Sacapulas Cunen Uspantán Total Whole Sample Total Number of Schools 14 7 14 35 12 13 10 35 70 Student’s Sample total number 206 121 223 550 206 188 164 558 1108 Source: Baseline Database, IDEA, April 2015. Measurement: School Survey and Learner Assessment The school profile data was collected via direct observation and a school survey administered to head teachers in every school in the sample. This data includes information on enrolment & class size, school facilities and infrastructure, and teacher backgrounds. For the student assessment, all learners in the sample were asked about their background characteristics (age, household possessions, household building materials, etc.). Learners also were asked about their family members and reading habits in their home (who they had seen reading in the week prior to the assessment, who had read to them, etc.). After collecting this background data, all learners were also given an emergent literacy test in Spanish composed of six components administered through four sub-tests: letter identification, single word recognition (reading of most used words), invented word decoding, reading fluency & accuracy (words read correctly per minute and total percentage of a passage read correctly; both within the same sub￾test), and a set of comprehension questions linked to the reading passage. The set of reading comprehension questions were administered only to those learners who could read independently. All instructions were given in the child’s mother tongue. Additionally, depending on the mother tongue of the child and the school’s language of instruction, students were also administered a test in K’iche’ or Ixil and assessed in letter identification, most used words, reading fluency, reading accuracy, and reading comprehension. For a description of the instruments and their components please refer to Table 4. Additionally, as part of the data collection 12.5% of the learners were assessed by two enumerators for inter-rater reliability. Details on this indicator are provided in Appendix A. 10 Table 4. Literacy Boost Assessment Instruments Student background Examples General Sex, age, language spoken at home, work/chores School-related Repetition history, ECD attendance Socioeconomic status Type of home, household size, household amenities/possessions Nutritional status Breakfast Home Literacy Environment Access to print Materials present in home, types of materials Reading activities at home Presence and percentage of family members who children see read, and who engage in literacy activities with children Reading Outcome Description Spanish Ixil K´iche Alphabet knowledge Number of letters/sounds known of 27 27 37 Single word reading Number of single words read correctly of 20 20 20 Decoding skills Number of invented words read correctly of 20 -- -- Fluency Number of words in a short story read correctly in a minute Accuracy Percentage of words in a short story read correctly Comprehension Number of questions (out of 10 for each language assessed) related to a short story answered correctly. The questions were split in 1 summary question, 5 literal questions, and 4 advanced comprehension questions (2 inferential and 2 argumentative questions). Analysis The critical purpose of this analysis is to present a profile of children’s reading skills, as well as an in￾depth analysis of each skill. Summary statistics are used to analyze students’ performance in each of the reading sub-tests. Secondarily, this report tests whether the students in the Literacy Boost intervention schools and their comparison counterparts are equal in terms of background and skills. That is, at baseline do these students possess the same resources and capabilities? This question is important so that at end-line, we can know how much the Literacy Boost intervention has, or has not, contributed to students' accelerated learning. To test the comparability of learners in the Literacy Boost and comparison samples, this report uses comparison of means through t-tests assuming unequal variance between the two samples. Summary statistics, accompanied by t-tests, are used to analyze learners’ performance in each of the reading sub￾tests. Further, this report will look at multilevel regression models to explore relationships between literacy skills and student background characteristics, school environment, and home literacy environment. Finally, it outlines the next steps for the evaluation of the program given the baseline 11 results and shares some descriptive statistics obtained through the school survey and some indicators that are of interest to USDA. It is important to pay attention to the regional indicators that back-up the results obtained and that are a vital part of the analysis, as well as the sample characteristics; both groups are formed by neighboring municipalities and their historical development has been different – not only in relation to ethnical, cultural and traditions but also to the historical, economic, political and social conditions having a direct effect in their development. Quiché, the department where these municipalities are located, is the department that shows the highest rate of illiteracy in the country. According to official data3 , 31.6% of the population cannot read or write. Quiché also has the highest rate of grade repetition, with 15% from total students repeating at least a grade. In relation to the literacy rate among women4 , an important indicator to understand the data collected, 70.4% of women can read and write. This same indicator is also present amongst indigenous women where 37.5% cannot read or write. The results obtained from the fieldwork, are proof of this situation – worse for the year 1 Intervention group, whose social and economic situation has a direct impact among the students and is a limitation to their learning. IV. Children’s Background Characteristics On average, students in both groups (intervention and comparison) are 9 years old. All the students speak Spanish however, their mother tongues are different. Students from the intervention group schools also speak Ixil while the students from the comparison group schools speak K´iché. 29% of students have repeated at least one grade of which first grade was repeated the most. This data matches the national statistics from the Ministry of Education. 35% of students in intervention schools repeated first grade, while 36% of students in the comparison group did so, however the difference is not statistically significant. 3 Information from the National Statistics Institute from Guatemala, 2002 4 Information from the National Statistics Institute from Guatemala, 2001. 12 Graph 1. Percentage of students that repeated first grade p-value for differences between LB and Comparison: ~10%, *5%, **1%, ***0.1% Graph 2 shows the students that are currently repeating second grade. Compared to students in intervention schools, 7.2 percentage points fewer students in comparison schools are repeating second grade. The difference is significant at a 0.1% level. Graph 2. Percentage of students currently repeating second grade p-value for differences between LB and Comparison: ~10%, *5%, **1%, ***0.1% On average, household size consists of 6 members. In both groups, 70% of houses have two rooms – which might not be enough space for all family members. 69% of the houses have an iron sheet roof (a very common practice in rural areas in Guatemala) and 87% of the students’ families have a cell phone. Also, for the whole sample, 97% of students help in the house – doing different types of 35.3 34.7 35.9 - 10 20 30 40 50 60 70 80 90 100 Whole Sample LB Comparison 23.5 27.1 19.9 - 10 20 30 40 50 60 70 80 90 100 Whole Sample LB Comparison *** *** 13 chores- especially after school hours. 62% of students stated that their main chore is to collect firewood (boys mentioned this was their main activity at home although a minor percentage of girls also said that it was their responsibility). This activity is one of the reasons that students arrive late or skip school. When students arrive to school, they are tired and/or hungry, which can limit their ability to learn. Graph 3.Percentage of students that help with household chores p-value for differences between LB and Comparison: ~10%, *5%, **1%, ***0.1% The number of children that work outside the house is relevant as it affects children’s performance directly. 24% of girls and 25% of boys help other families in an economic activity or doing chores around the house after school hours. This compromises the children’s time to study and do homework. 96.6 97.1 96.1 - 10 20 30 40 50 60 70 80 90 100 Whole Sample LB Comparison 14 Graph 4. Percentage of students that work outside the household by gender p-value for differences between boys and girls: ~10%, *5%, **1%, ***0.1% There are statistical differences in the social and economic conditions among the comparison and the intervention group favoring the comparison group. Among the intervention group, 69.5% have electricity versus 80.5% from the comparison group, 38.2% versus 56.1% have a TV at home, and 82.4% of the houses have an iron sheets roof compared to 56.5% from the comparison group. As per the house walls, in the comparison group, 61% of children report to live in a house with adobe walls, 22% report to live in a house with block walls and 15% are from wood and in the intervention group, 72% of the houses are made from wood, 20% from block and 6% from adobe. A bigger percentage of children in the comparison group reported having domestic animals, including: pigs, chickens, and cows. Appendix B, Table B1 contains all the background characteristics and the result of the t-tests conducted to determine statistical differences. In the last two years, the schools located at the comparison group area have received support to improve the school’s infrastructure and to improve the reading and comprehension abilities of their students. This support has come from different organizations (non-government and international cooperation), such as SHARE Guatemala, UNICEF, Water for People, and Action against Hunger. However, their interventions will come to completion in 2015. The results obtained from the comparison group can be related to the support received by said organizations, although it would be interesting to carry out qualitative research on these aspects. V. Children’s Home Literacy Environment To determine the reading environment at home, children were also asked about the availability of reading materials and reading habits at home. The first section described the average prevalence of different types of printing materials – including textbooks, religious books, magazines, newspapers, storybooks and coloring books. The results on the availability of reading materials in the household 25.4 23.8 25.9 25.2 25.0 22.3 - 10 20 30 40 50 60 70 80 90 100 Whole Sample LB Comparison Boys Girls 15 show that the prevalence of printed and reading materials is higher for children in the comparison group for coloring books and newspapers, and higher in the LB group for storybooks and religious books. In the comparison group 32.8% children mentioned that they had newspapers, while in the intervention group, only 19.6% mentioned having newspapers; 18.5% of the children in the comparison group have storybooks, while at the intervention group, 32% had story books and 37.1% children in the comparison group had coloring books versus 27.5% from the intervention group. These differences are statistically significant at a 0.1% level. Meanwhile, there were not statistically significant differences in the presence of religious books and magazines. 71.8% reported to have textbooks and 1.1% to have magazines in their home. Graph 5. Home Literacy Environment. Percentage of children with educational and supplementary materials available at home in the intervention and comparison groups p-value for differences between LB and Comparison: ~10%, *5%, **1%, ***0.1% It is essential that the program focuses its efforts on improving and increasing the availability of storybooks for children in the intervention group (a 32% prevalence) and next year, for the comparison group (a prevalence of 18.5%). This is one of the variables where the intervention group presents a meaningful statistical difference compared to the comparison group; the availability of magazines needs to be improved also to promote reading habits among children from both groups (prevalence of 1.3 and 0.9 in the intervention and comparison groups). There are several magazines that promote reading. Prensa Libre, with the support of Santillana (Publishing House) has produced contextualized magazines for the rural area; additionally, the program “Literacy Boost” implemented by STC, with the technical and financial support from USAID, has elaborated magazines (supplementary reading materials), adapting these to the rural context of communities in El Quiché. 72.4 78.4 1.3 19.6 32.0 27.5 71.3 71.9 0.9 32.8 18.5 37.1 - 10 20 30 40 50 60 70 80 90 100 Textbooks Religious books Magazines Newspapers Storybooks Coloring books LB Comparison * *** *** *** 16 This implies the need to establish coordination mechanisms to access resources that already have the approval from the Ministry of Education. The second part of the Home Literacy Environment (HLE) survey describes the percentage of family members that support children in different reading activities. The results show a statistical significant difference amongst the percentage of family members that read to the child favoring those children from the comparison group. In terms of home reading habits, only three variables did not show a statistically meaningful difference among the intervention and comparison groups: the children reported that on average they have seen 47.3% of their family members reading during the last week; 69.5% of family members help them study and that 31.6% of their family members tells to them stories. Graph 6. Home Literacy Environment. Reading habits (Percentage of family members that ….) p-value for differences between LB and Comparison: ~10%, *5%, **1%, *** In terms of the Community Literacy Environment, when analyzing the results separately, 46.4% of children in the comparison group stated that in the community, someone reads for them, while at the intervention group, only 28.5% had this opportunity. 56.5% and 48.3% from the comparison group, expressed that they read with someone from the community and that they lend a book to/from someone outside their house – 38.5% and 35.5% where the numbers for the intervention group under the same category. These differences are statistically significant at a 0.1% level. 46.6 69.3 27.3 30.5 47.9 69.8 31.7 32.6 - 10 20 30 40 50 60 70 80 90 100 …The child has seen reading ...Help the child study ... Read to the child … Tell stories to the child LB Comparison ** 17 Graph 7. Community level Literacy Environment. Percentage of children reporting each activity p-value for differences between LB and Comparison: ~10%, *5%, **1%, ***0.1% VI. Children’s Spanish Literacy Skills The Spanish assessment evaluated 5 reading skills– letter identification, most used words reading, word decoding, oral passage reading and reading comprehension. Graph 8 shows the Spanish skills profile for the intervention and comparison students. Students in comparison schools show better results for the 5 skills assessed. The differences between them and intervention students are significant at a 0.1% level. 28.5 38.5 35.5 46.4 56.4 48.3 - 10 20 30 40 50 60 70 80 90 100 Someone in the community read to you Did you read to someone in the community Did you lend a storybook outside your home LB Comparison *** *** *** 18 Graph 8. Spanish literacy skills profile. LB and Comparison P-value for differences between LB and Comparison: ~10%, *5%, **1%, ***0.1% Letter Identification This sub-test evaluated letter identification through student cards that had all the letters from the Spanish alphabet (27 letters). For the assessor to count the item as correct, the student had to identify the letter or pronounce its sound. On average, the intervention group correctly identified 55.8% of the letters, while the comparison group identified 70.6%. The results show that students from the comparison group had more knowledge of the alphabet letters in comparison to the intervention group. Both average numbers are less than ideal for the age group, which commits the IDEA program to implement strategies aimed at improving the learning process of letters; the Literacy Boost methodology will facilitate these processes, with its creative and innovative approach. Word Reading: Most Used Words This skill was evaluated through student cards that included a set of 20 words most commonly found in the second grade language textbook. The students were asked to read the words in the order they were written in the student cards. The results favor once again comparison students. The average student in the intervention group recognized 28.3% of the words correctly, while in the intervention group the average student correctly recognized 47.4% of the words. All the words that were used are meaningful words and commonly used by children in both areas, according to their age. This subtest poses a challenge for the IDEA program. The implementation of the Literacy Boost methodology will help increase this result, as this methodology gives teachers practical, creative and innovative tools to promote the development of children’s skills to increase the children’s vocabulary, taking advantage of other in-classroom activities, such as reading buddies and community activities that include: reading camps, reading festivals, reading marathons among others. 55.8 28.3 23.4 22.9 5.0 20.2 10.5 70.6 47.4 39.1 37.8 9.7 34.9 19.1 - 10 20 30 40 50 60 70 80 90 100 % Letter ID % MUW % Decoding % of Children who read the passage Fluency(wcpm) Accuracy (% of words read correctly) % Reading Comprehension Literacy Boost Comparison *** *** *** *** *** *** *** 19 Mamá (mother) was the word that was most commonly identified correctly in both areas were 76% of students from the comparison group and 64% from the intervention group identified it correctly. Table 5 shows the four words that presented the greatest difficulty for intervention students. Table 5. The 4 words that most LB students struggled with difficult Word Percentage of children that read the word correctly comunidad 22.4 hoja 15.8 Güisquil 15.1 diccionario 13.1 Decoding The decoding subtest consisted of a student card with 20 invented (made-up) words that every student was asked to read. These 20 words were created by changing the order of the syllables, from the most common words found in the Communication and Language textbook of second grade students (for instance, the word “cara” (face) was modified into “raca”). The students had to read the words as they were written in the card. On average, students from the intervention group were able to decode 23.4% of invented words; while the students from the comparison group decoded 39.1% of the words. These results show a meaningful statistical difference among both groups, which determines that students from the comparison group have better reading skills to read made up words, in comparison to the intervention group. Oral reading Students were asked to read out loud, a 129 word Spanish story appropriate for their grade based in the most common words and phrases among communication and language books for second grade students. If the child was able to read at least 5 words in 30 seconds the assessor let him/her read till the end regardless of the time spent. Otherwise, the assessment would be stopped and the child would be considered a non-reader. 22.9% of the students from the intervention group and 37.8% of the comparison group read at least 5 words correctly in 30 seconds – and were classified as readers. Fluency and Accuracy Fluency (words read correctly per minute) and accuracy (percentage of the passage words read correctly) are presented together as they were measured using the same story that students read out loud. The number of words that students read correctly in the first minute evaluates fluency. As the student continues reading after the first minute, the total number of correctly read words from the whole passage, independent of the time the student takes, measures accuracy. 20 Taking both readers and non-readers into account, the average fluency for the intervention group was of 5.0 wcpm with 20.2% accuracy, while the comparison group presented 9.7 wcpm in fluency and 34.9% in accuracy. IDEA, with the implementation of the Literacy Boost methodology, will provide adequate tools and creative and innovative techniques to strengthen reading skills, by working in close partnership with teachers and involving parents in activities in-and-outside of the classroom. Reading Comprehension The final sub-test evaluated students that answered 10 comprehension questions related to the paragraph that was previously read. The results helped to understand the strengths and weaknesses of students that can read and understand what the paragraph was about. On average, taking both readers and nonreaders into account, the intervention group had a reading comprehension of 10.5% while the comparison group had 19.1%. The results show that there are statistical meaningful differences in the scores of the groups. A further measure of reading comprehension is to divide the students into reading comprehension tiers as shown in Graph 9. For the intervention/LB group only 5.8% can read and understand a second grade level reading text, while 77.1% could not read nor understand the paragraph. This represents a challenge for the program and through the implementation of the Literacy Boost methodology, this number will improve. Graph 9. Spanish. Reading with Comprehension Tiers LB students1 . 1/Nonreaders are defined as those students with fluency less than 5 wcpm and stop rule invoked; Beginners are defined as any not stopped, but answering less than 80% of the literal questions correctly; Readers with comprehension are defined as those that read the passage and answered 80% or more of the literal questions correctly 77.1 17.1 5.8 0 20 40 60 80 100 Non readers Beginners Readers with Comprehension 21 VII. Children’s Mother Tongue Literacy Skills Graphs 10 and 11 show the children’s literacy skills in their mother tongue; Ixil for intervention students and K’iche’ for comparison students. As it is shown and although not perfectly comparable as per the language differences, even in their corresponding mother tongue students in comparison schools have higher skills. Graph 10. Ixil literacy skills profile, average correct Graph 11. K’iche’ literacy skills profile, average correct 34.7 8.9 9.3 1.8 8.8 2.8 - 10 20 30 40 50 60 70 80 90 100 % Letter ID % MUW % of Children who read the passage Fluency(wcpm) Accuracy (% of words read correctly) % Reading Comprehension 67.5 24.2 20.6 4.1 17.1 3.7 - 10 20 30 40 50 60 70 80 90 100 % Letter ID % MUW % of Children who read the passage Fluency(wcpm) Accuracy (% of words read correctly) % Reading Comprehension 22 Letter Identification The average Ixil student could correctly identify 34.7% of the letters tested, showing that students are less familiar with Ixil alphabet as compared to the Spanish one. The letters “o, e, a, s, u”, were the ones that a greater percentage of children identified correctly; vowels, with the exception of the letter “i” where recognized correctly by more than 80% of the students. The letters “tx ,tch´, tz´, ch´, tx´” were identified by less than 2% of the students. The average student could identify 67.5% of the K’iche’ alphabet, again a lower result than the one showed when students were presented the Spanish alphabet. The letters “i, a, s, o, u”, were the easiest for children to read. On the other hand, the letters “q´, ´, tz, ch´, k´ were the ones that students could identify less frequently (less than 4% of the students identified them correctly). Word Reading: Most Used Words The average percentage of Ixil words that can be recognized by students is of 8.9%, a low result. Words that were easier to recognize were: “lee, chee, chuu, ich and pach” which were read correctly by at least 18% of the students. The less recognized words were “b’a’x, tx’i’, tx’umil, txay” with less than 2% of students reading them correctly. The results are shocking since most of these schools are working under the Bilingual/Intercultural Education Program. However, there is a huge gap among teachers – who do not teach classes in Ixil, despite the fact that students do speak Ixil. The average percentage of K’iche’ words that can be recognized by students is of 24.2%. Words that were easier to recognize were: “nan, alí, ja, kej, chak”, were at least 35% of the students recognized them correctly. The less recognized words were “,tz’i’, q ´ij, tz´il, ch´ek, me´s” with less than 7% of students being able to read them. Although the results of K’iche’ literacy skills are better in comparison to the Ixil ones, there is a clear difficulty in recognizing words in the children’s mother tongue. Fluency and Accuracy The average fluency of children when reading the Ixil passage was of 1.8 words correct per minute5 . Connecting letters to pronounce words is a big struggle for children. Accuracy on the other hand was of 8.8%. Many of the students were unable to finish reading the paragraph as they were not familiar with the words that they were asked to read and were unable to connect letter sounds to read the words. The average fluency of children when reading the K’iche’ passage was of 4.1 words correct per minute. Accuracy on the other hand was of 17.1%. Many of the students were unable to finish reading the paragraph as they were not familiar with the words that they were asked to read and were unable to connect letter sounds to read the words. 5 Fluency and Accuracy measures take into account readers and non-readers. 23 Reading Comprehension Only 0.6% of the students were able to read and comprehend the paragraph they read in Ixil. Some factors that influence this finding are that most students were unable to finish reading the passage and of those that read it most understood isolated words at most so they were unable to comprehend the reading passage as a whole. Graph 12. Ixil. Reading with Comprehension Tiers LB students1 . 1/Nonreaders are defined as those students with fluency less than 5 wcpm and stop rule invoked; Beginners are defined as any not stopped, but answering less than 80% of the literal questions correctly; Readers with comprehension are defined as those that read the passage and answered 80% or more of the literal questions correctly Only 1.2 % of the students were able to read and comprehend the paragraph they read in K’iche’ language. Some factors that influence this finding are that most students were unable to finish reading the passage and of those that read it the majority understood isolated words at most so they were unable to comprehend the reading passage as a whole. 90.7 8.7 0.6 0 20 40 60 80 100 Non readers Beginners Readers with Comprehension 24 Graph 13. K’iche’. Reading with Comprehension Tiers LB students1 . 1/Nonreaders are defined as those students with fluency less than 5 wcpm and stop rule invoked; Beginners are defined as any not stopped, but answering less than 80% of the literal questions correctly; Readers with comprehension are defined as those that read the passage and answered 80% or more of the literal questions correctly VIII. Learning Equity and Struggling Students Early Education Whether a child attended an early education program is a consistent predictor of their scores across basic and advanced literacy skills in Spanish (see Graph 14). Students that attended an early education program are predicted to score higher in letters (6pp), MUW (23pp), fluency (2pp), accuracy (10pp) and reading comprehension (7pp). This is valid for too for basic Ixil literacy skills where students that did not attend an ECD program are expected to score 6pp lower in letters and 3pp lower in MUW. Graph 14. Predicted Spanish Literacy Skills by ECD status (LB) p-value: ~10% *5% **1% ***0.1% 79.4 19.4 1.2 0 20 40 60 80 100 Non readers Beginners Readers with Comprehension 51.1 18.3 15.2 3.3 12.1 5.2 57.1 30.8 25.6 5.4 22.2 11.8 0 10 20 30 40 50 60 70 80 90 100 % Letters~ % MUW** % Decoding** Fluency(wcpm)~ % Accuracy* % Reading comprehension* No ECD ECD 25 Repetition Status Children that repeated 1st or 2nd grade have lower expected results across Ixil and Spanish literacy skills (see Graphs 15 and 16). Having repeated a grade is associated with a lower result in Spanish in letter recognition (-4pp), MUW (-9pp), decoding (-10pp), fluency (-3pp), accuracy (10pp), and reading comprehension (-5pp). Graph 15. Predicted Spanish Literacy Skills by Repetition Status (LB) p-value: ~10% *5% **1% ***0.1% These results are similar in Ixil literacy skills where students that have repeated a grade are predicted to have lower results in all subtests, the difference being 3pp in letters, 6pp in MUW, 2pp in fluency, 6pp in accuracy and 4pp in reading comprehension. 53.6 23.6 18.1 3.4 15.3 8.2 58.0 32.7 28.7 6.6 25.0 12.7 0 10 20 30 40 50 60 70 80 90 100 % Letters~ % MUW** % Decoding** Fluency(wcpm)* % Accuracy* % Reading comprehension~ Repeater Non-Repeater 26 Graph 16. Predicted Ixil Literacy Skills by Repetition Status (LB) p-value: ~10% *5% **1% ***0.1% Finally, we find that not surprisingly socioeconomic status also plays an important role when it comes to Spanish literacy skills. With the exception of fluency, students in the area of the first year of intervention have lower predicted scores in all assessed skills (see Graph 17). The difference is particularly interesting in advanced skills where students from the two lowest quantiles of socioeconomic status are expected to be able to answer on average ~1 reading comprehension question correct less than students pertaining to the other socioeconomic status quantiles. Graph 17. Predicted Spanish Literacy Skills by SES (LB) p-value: ~10% *5% **1% ***0.1% 33.2 6.0 0.9 5.5 1.0 36.3 11.8 2.7 11.9 4.7 0 10 20 30 40 50 60 70 80 90 100 % Letters~ % MUW** Fluency(wcpm)* % Accuracy* % Reading comprehension** Repeater Non-Repeater 24.0 19.2 4.0 16.3 7.2 32.6 27.7 6.0 24.0 13.8 0 10 20 30 40 50 60 70 80 90 100 % Letters~ % MUW~ % Decoding~ % Accuracy~ % Reading comprehension~ Bottom 2 quantiles of SES Top 3 quantiles of SES 27 IX. School survey results6 Increased Access to Clean Water and Sanitation Services The infrastructure assessment from all schools provided the following inputs:  16% of school infrastructure is in good condition in the intervention group and 35 % in the control group; of the infrastructure in bad condition in the intervention and control groups we can highlight: dirt floors, damaged latrines and stoves. In most schools in the northern vertex of Chajul and more distant schools in Nebaj and Cotzal they cook on the floor with open fires (Source School Census, October 2014).  100% of the schools have Access to water – it is not determined if the water is clean and safe (free of bacteria and chemical pollution and adequate to be consumed by children without damaging their health). 30% of the schools do not have water during the summer (February to April), which is a limitation to prepare food, the use of latrines, hand wash and water consumption. During those months, parents or local leaders carry water to school from their homes or rivers.25% of the schools have problems with the water infrastructures (tubes) that distribute water, since they are damaged and because of lack of preventive maintenance. There are also water leaks, wasting the limited water available. 60% of the schools do not have drains or if they do, these are in poor conditions.  Only 34% of the schools (80 in total) have improved water sources.  75% of the schools from the intervention group have sanitation facilities in poor conditions while 60% from the control group are in the same conditions. Aside from that, the amount is insufficient for the number of students and does not meet the established MoE standards (1 latrine for 25 girls; 1 latrine and 1 urinal for 50 boys). Some of the facilities are dry pit latrines, others are water￾pour latrines or toilets. The program needs to focus efforts to improve the sanitary facilities in both areas.  46.7% of the schools from the Ixil Area and 60% from the control group have electricity.  Most of the schools do not have a warehouse to store food. They have some wooden furniture to store the food they purchase for the School Meal Program from the MoE. The IDEA project will focus its efforts on rehabilitating and building adequate sanitation infrastructure, establishing coordination mechanisms with the private sector and local authorities. As part of the package, the WASH guidelines will be contextualized at the school level, to promote and ensure sanitary education among students, with the support of teachers. Increased Access to Preventive Health Interventions The Ministry of Education, in coordination with the Ministry of Health has promoted the implementation of the Health Schools’ Strategy to guarantee access to health preventive actions among school aged population. Most of the schools from the 6 municipalities where the IDEA project will be implemented, have not integrated the Health Schools Municipal Commissions – most activities have been focused in the integration of the Departmental Commissions for Healthy Schools. 6 The data from this section comes from the School Census conducted in October 2014 as well as the baseline assessment conducted in a random sample of schools in the intervention and comparison areas 28 The principals and teachers of the schools from the 6 municipalities recognize the importance of coordination with health providers to implement health preventive actions for school-aged children. None of the schools from the intervention area carry out deworming campaigns (to eliminate soil￾transmitted helminthes) or deliver micro-nutrients supplements (specifically iron). Only 15% of the schools have a first aid kit with basic drugs such as medication for fever or pain. . As stated above, 57% of the communities have no permanent health service. The oral health topic is included in the curricular content of the students but it is not possible to put it into practice since the schools do not have the budget to purchase toothpaste and tooth brushes for students. With the support of Colgate, some schools from Uspantan have obtained donations of toothpaste and tooth brushes. In general, there are no trained staff to deliver health services to students. In 11% of the communities, there is a health center but the staff prioritizes preventive and curative interventions for maternal and child health (children under 5 years of age, women in reproductive age, pregnant and lactating women). School health is not among their priorities. Increased Access to Requisite Food Prep and Storage Tools and Equipment The School Meals Program from the Ministry of Education is implemented in all schools from the project implementation area. 30% of the schools do not have a kitchen; the teachers have improvised some infrastructure or have placed wooden boards in order to cook food. Linked to this, schools do not have firewood saving stoves and 40% of the stoves from the schools are deteriorated. Only 25% of the schools have stoves in adequate conditions. 30% of the schools have no stoves at all, they cook on open fires or borrow a stove from a neighbor. In relation to warehouses to store food, only 10% of the schools have a warehouse: 70% of the schools use one of the classrooms as an improvised warehouse and 20% of the schools use the kitchen as a warehouse, organizing food in wooden furniture or wooden boxes. All schools do not have shelves to store food sacks, which will be delivered to prepare the school meal. 98% of the schools do not have kitchen utensils to prepare food for children. The only equipment they have are pitchers and small pots to prepare hot beverages or fried food. One challenge for the program is to ensure that schools have all necessary utensils to cook the food for breakfast/snack/lunch for school meals, including: pots, pans, cooking palette, knives and spoons to serve. Increased engagement of local organizations and community groups In both groups (intervention and comparison schools), there are Parent Teacher Associations – PTAs) who work closely with principals and teachers of schools. Most school PTAs are involved and support programs especially school repairs and supply of firewood to prepare food. The results show that PTAs are not involved in the maintenance of school infrastructure, especially the sanitary infrastructure which as mentioned above is in poor condition for a big proportion of the schools. This is a challenge for the program because it is necessary to promote the involvement of leaders, local authorities and PTAs in all interventions. 29 The Intervention group schools received support from different local NGOs, who have been working on the following projects: maternal and child health, food security and education; but schools in the comparison group received more support from Local NGOs and International NGOs. In the last two years, the schools located in the comparison group area have received support to improve the school’s infrastructure and to improve the reading and comprehension abilities of their students. This support has come from different organizations (non-government and international cooperation), such as SHARE Guatemala, UNICEF, Water for People, and Action against Hunger. However, their interventions will reach completion in 2015. The results obtained from the comparison group may relate to the support received by said organizations, although it would be interesting to carry out qualitative research to investigate this further. Reading practices and resources The Ministry of Education has promoted the practice of reading through the National Reading Program over the last three years; however, not all schools are implementing it. Although the Ministry has distributed books, reading practice at schools is not carried out. The results of the baseline show that only 47.1% of schools have school libraries and 74% have no book lending systems for children. Of those that have lending systems, they are designed for children to read inside the school. The reason behind it is that teachers have to care for and protect the books, because the library is rolling (going from grade to grade for a month), so many teachers prefer to keep the books at the school. In addition, schools do not have supplementary reading materials (Source: School Census, October 2014). Another finding of the baseline is that learning to read in their mother tongue is rarely promoted, and there is no supplementary material languages in Ixil and K’iche’. According to the results of the baseline, the teachers said that out of the 6 hours of classes, most of them only use 1 hour to teach in the mother tongue; very few teachers said they used four hours. Save the Children will focus efforts to promote and implement creative and innovative techniques to improve the reading practices at the school and community level through the Literacy Boost methodology. Teaching staff According to statistics from the Education Departmental Office of El Quiché, there are a total of 1505 schools, 62 in urban areas and 1,443 in rural areas. El Quiché is the fourth department that has more schools in the country and is the second with more public schools. The Ixil intervention group has a total of 188 registered schools and 165 were visited during the field work. The comparison group has a total of 190 registered schools. From the 987 teachers from the Ixil/intervention area, 480 teach first, second, and third grades. The comparison group has a total of 1,120 registered teachers out of which 620 teach first to third graders. From the field visit results it was found that a big percentage of those are concentrated in first grade. Additionally, the municipalities that have higher percentages of teachers are Nebaj, Uspantan and Cunen. 30 From the total number of children in both areas, 42% of teachers are women. On average they have been working as teachers for 7 years. 77% of teachers finalized their high-school studies and did not continue studying at the university level. Only 3% of teachers have attended the University. There is an absence of a training education program for teachers and although most of them have experience in the field, most of them express the need to learn more teaching techniques to improve their in-class abilities. X. Other indicators of particular interest to USDA Improved attentiveness Percent of students in target schools who indicate they are attentive or very attentive during class/instruction (student survey and teacher interviews as well). The results from the student’s surveys show that 94.5% of students from the sample, responded that they are aware or pay attention to the teacher’s instructions during class. When the results were analyzed individually, for the intervention group (Ixil area) 91.3% responded that they pay attention, while the comparison/comparison group showed 97.7%. The results show a meaningful statistical difference among the intervention/Literacy Boost group and the comparison group. Realizing that students have not had the opportunity to build a rapport with the evaluators, SC has questioned whether results reflect a reliable measure since. Students evaluated are between age 8 and 9 and are likely afraid of what the teacher might think as well as the consequences of responding that they do not pay attention in class. Students may be concerned that they will be punished if they reveal that they are not focusing on their lessons while in class. Graph 18. Pays attention all or most of the time while in class. Source: Students’ survey. IDEA Baseline. February-March 2015. 94.5 91.3 97.7 - 10 20 30 40 50 60 70 80 90 100 Whole Sample LB Control 31 During the first training of the “Literacy Boost” methodology, SC interviewed teachers about how students paid attention in class, according to their experience. Teachers expressed the following concerns to our technical team: “To have the student’s attention in class is a big challenge. Most of the students do not pay attention in class and if they do, it is only for a short period of time. Some students get distracted, others are tired or sleepy; there are very few who pay attention.” “To respond the question of why are they tired, sleepy or distracted, there are many reasons. Students come to the school without dinner or breakfast and they do not have the energy to stay in school or pay attention; some students usually wake up very early in the morning to help around house chores, such as chopping firewood, feed the animals or clean the house”. Teachers also mentioned that there are other factors that demotivate students and distract them including: outdated techniques, lack of materials to deliver classes, small environments, etc. The following table shows the results about attentiveness in class, according to teachers interviewed from first to third grade during the first training of the Literacy Boost methodology (April 2015). Graph 19. Percent of student´s attentiveness in class from teacher´s perspective. Source: Interviewed teachers from first to third grade, April 2015. In comparison to the answers given by the students, the difference is quite shocking. From the students’ perspective, 94.5% of the students pay attention in class, but from the teacher’s perspective, only 5% pay attention. Teachers expressed that 80% of students do not pay attention or are unable to maintain it during class. Through the Literacy Boost Methodology, the IDEA Program will focus its efforts in supporting teachers to develop their teaching abilities through creative and innovative techniques, to improve and sustain student’s attention during class through the use of adequate materials, games and other interesting activities for children. In addition, all students will receive breakfast as an incentive to 5 15 80 - 10 20 30 40 50 60 70 80 90 100 Children pay attention Children who try to pay attention Children do not pay attention 32 bring them to school – especially to reduce short-term hunger and to improve and maintain attention during class. Reduced Short-Term Hunger Number of students in target school who indicate that they are hungry or very hungry during the school day (collected through a student survey). The results of the baseline show that 49% of the students are not hungry and 51% responded that they are hungry (or very hungry, or always hungry) during the school day. According to the guide, this indicator requires a number response but, for the baseline, it is reported as a percentage since it was taken from a sample of the total of the students, as described in the methodology section. SC suggests reporting this indicator as a percentage. Graph 20. Percent of students in target school who indicate that they are hungry or very hungry during the school day. Source: Interviewed teachers from first to third grade, April 2015. The results of the interviews conducted with teachers, where the IDEA Project is implemented, show that most of the students go to school without having dinner the night before, and without breakfast the morning of the school day due to the difficult economic situations that families face. This is one of the reasons why parents prefer not to send their children to school. Several studies made by the Food Security Secretariat, report that 7 out of 10 students go to school without having breakfast, 51% 49% They aren´t hungry during the school day They are hungry or very hungry during the school day 33 which affects their attentiveness during school hours as well as their performance. The “Micro Nutrients National Survey during 2009-2010” reports that 45.5% of children in school are malnourished. SC, through the donation of food commodities, will provide a nutritional supplement for students through school breakfast/snack/lunch, as a mechanism to promote children’s attendance in school, to decrease short-term hunger, to improve attentiveness in class, and to maintain children’s attentiveness to improve their performance at school. There are no international standards for measuring hunger and even defining the term is fraught with difficulty7 . For SC, hunger is much more than the lack of food, it is this complex interaction between food consumption, access to food, and how people cope with the lack of it and their livelihoods. Hunger: Not having enough to eat to meet energy requirements. Hunger can lead to malnutrition, but absence of hunger does not imply absence of malnutrition8 . Increased Access to Food (School Feeding) Despite the fact that the Ministry of Education (MINEDUC) has created the School Meals Program and that it is implemented through the “Dirección General de Fortalecimiento de la Comunidad Educativa – DIGEFOCE –“ every year, the program start date is delayed, since the funds transfer is is often deferred due to administrative processes in the public sector. The MoE allocates $0.27 per child/per day for a school meal. The funds are managed by the PTAs who are also part of the school board to implement the program. The results of the School Census showed that the food products that are purchased the most for school meals, are: rice, wheat flour, oil, cereal (to prepare atol- a cereal based hot beverage), milk, incaparina/vitacereal (a soy based drink), sugar and salt. Save the Children is coordinating and negotiating with MINEDUC through the DIGEFOCE, for their approval so that the school meal committees can purchase products locally, to compliment the breakfast/snack/lunch that will be provided with USDA’s support. The School Meals’ program began in March 2015 within 50% of the schools from the 6 municipalities where the IDEA project is being implemented. The first disbursement was received by the PTAs in the last week of February. Improve the Attendance of Students A challenge for Guatemala’s Education System is improvement of student attendance, for 180 days a year. However, despite the many efforts, an important percentage of students do not attend classes regularly for various reasons, such as those explained in the following paragraph. 7 https://wfp.org/content/how-do-you-measure-hunger 8 WFP Glossary at https://www.wfp.org/hunger/glossary 34 Main Causes of School Absences According to interviews carried out with teachers and through review of the daily attendance registry that each teacher holds, it was determined that 20% of students are absent from school every year for reasons related to diseases that include diarrhea, respiratory infections, and conjunctivitis. The season in which children are most absent each year, is winter. 20% of the students are absent from school up to 5 days, depending on the severity of the health problem they face (diarrhea or respiratory infections) and when their health status decreases and children are absent for more than 8 days, parents prefer to take their children out of school. Teachers said that 70% of families solve their children’s health problems in their homes and that only 30% of the families go to the Health Care Center – the main reason they attend these services, is because the health problem of the child got increasingly complicated. Teachers from all schools said that they do not have a strategy to inform and train parents of the children who are most absent to promote best practices and decrease common diseases. The IDEA project is focused in promoting health practices and deceasing respiratory infections and diarrhea mobility among school aged children. Besides sickness and diseases, children are also absent from school because they help their parents in collecting coffee, cotton and sugar cane during the harvest season; when mothers are sick, the girls are usually the ones that absorb the home burden and chores and they therefore stop going to school. Increased student enrollment The major findings from the School Census were:  24,335 students enrolled in 2013 while 23,487 enrolled in 2014 in the intervention group. The enrollment of students decreased by 3.48% in 2014 compared to 2013. In the comparison group 26,760 students enrolled in 2013 while 24,655 enrolled in 2014. The enrollment of students decreased by 7.87 % in 2014 compared to 2013.  Of the 24,335 enrolled in 2013, 23,807 completed the school year; representing a 2.17% school dropout in the intervention group. In the comparison group of the 24,655 enrolled in 2014, 22,963 completed the school year, representing a 6.86% school dropout.  Of the total number of students enrolled in 2014, 49% were female in both groups (intervention and comparison). The results represent a challenge for project IDEA; the reasons why parents do not enroll their children must be studied further in order to define strategies that motivate them to recognize the benefits of education. 35 Increased Community Understanding of the Benefits of Education Number of parents in target communities who can name at least three benefits of primary education (Collected by a survey). Most of the parents don’t understand the benefits of education because they prefer children to help by working: boys in farm work and girls in the preparation of food and household chores. 100% of the parents cannot identify a minimum of three benefits of education. This poses a challenge, and the project needs to determine the strategy to increase the parents understanding about the benefits of education. Increased Knowledge of Safe Food Prep and Storage Practices The results show that out of all teachers that teach first, second and third grades of primary school 85% know at least three Safe Food Preparation and Storage Practices; 15% of teachers identified more than three practices – however it is relevant to note that only 20% of those who know more than three practices, actually apply them. The three practices that teachers know are: to wash their hands before preparing food; wash and disinfect food; make sure the food is in good conditions and prepare it in a clean environment. Unfortunately, most of the teachers do not apply these Safe Food Preparation and Storage Practices (see graph). Graph 21. Knowledge of Save Food Prep and Storage Practices of Teachers Source: Interviewed teachers from first to third grade, April 2015. It was possible to observe if teachers applied the practices they mentioned through a food preparation activity. It was evident that all teachers do not wash their hands before preparing the food and 90% 85% 15% Identifies less than 3 Identifies more than 3 36 were not aware of storage practices. Most of them expressed that they are not responsible for preparing or storing food since volunteer mothers are the ones that prepare and store the food products that they purchase with funds provided by MINEDUC to implement the School Meals Program. Among activities included in the IDEA project, are delivery of trainings about safe food preparation and storage practices for Volunteer Mothers, PTA’s and teachers. Increased Knowledge of Health and Nutrition Practices Despite the fact that the National Base Curriculum (NBC), approved by the Ministry of Education, includes the “development of competencies about good health conservation and nutrition improvements” in the Social and Natural Subject, not all teachers develop the subject content to promote health and nutrition practices. The results of the baseline show that 10% of schools develop health and nutrition topics according to the community’s epidemiological profile which include regular coordination activities with health service providers. 90% of the schools are neither integrating nor developing health nor education topics. The Ministry of Education, along with the Ministry of Health and the Food Security Secretariat have promoted the implementation of the Healthy Schools Program, this has been focused at national and departmental levels. One of the challenges for SC is to promote the implementation of the Healthy Schools Program in all the schools that are part of the IDEA Project. 75% of the interviewed teachers said that the main health practices they promote among students are handwashing, tooth brushing and daily bathing. Students have expressed that although they are aware of these practices; they are unable to meet them since they do not have water in their homes and if they do have access to water, it is not clean or safe. 100% of teachers expressed that they have trained the members of the PTAs on topics including i) avoiding the purchase of snacks (junk food) for the school meal; ii) not selling junk food in the school store, but selling fruits and vegetables instead; iii) purchasing nutritional food to prepare school meals; and iv) taking children to health services to monitor their health. Out of the total number of schools, 60% do not include junk food, such as oil-fried candies for the School Meals Program, while 40% of schools do. The results show that there is very limited knowledge regarding the nutritional value of food which is a challenge to overcome at the IDEA program. Within 56% of the communities, there are no health services available. These communities receive monthly visits by the “Extension Coverage Program” from the Ministry of Health (one doctor and one nurse). In 27% of the communities, there is are very basic health post infrastructures staffed with a community health worker and/or a midwife. 11% of communities have a Health Center (with one nurse and one health technician) and only 7% of the communities have a regional hospital (one in Nebaj and another one in Uspantan). 37 XI. Next steps in the evaluation Sharing and incorporating baseline results  Review, analyze, and discuss the main results and needed adjustments to program goals as per Attachment E, with the Country Office Technical Advisor Team and the donor.  Discuss and analyze main findings with the MoE at the National, Regional and local level, in order to define mechanisms that ensure an adequate response for quality implementation of the program.  Define together with the MoE the socialization of the main findings and commitments and responsibilities during the life of the program at the National, Regional and local level.  Share the main findings with the key stakeholders with the purpose of discussing, defining and prioritizing the addressed findings and recommendations to improve quality of education. The key stakeholders are USDA Guatemala Staff; MoH at the National, Regional, Departmental and Local Level; Ministry of Health, USAID office; local NGOs; international donors working in education projects and the community leaders, principals and teachers. Midline evaluation Baseline results show unbalanced characteristics between treatment and comparison schools beyond what is expected by chance. As a way to mitigate this challenge, Propensity Score Matching is suggested as a statistical technique to be used at midline to assess the impact of the program. The reasons behind the unbalance are various: the non-random choice between treatment and comparison schools given logistical challenges for food delivery, the non-inclusiveness of a representative sample of schools to be part of the program in each of the areas, and potential biases caused by other NGOs working in past years in the comparison area. Given that the program has a school feeding portion, the decision was made to do a phase-in approach where one area was to be targeted in the first year and the second starting in the second year to facilitate the logistics behind the food delivery amongst others. The resulting division led to two areas: the Ixil area which will be targeted this year, and the K’iche area were the program will start a year from now. Although the areas as a whole have similar social indicators, the difference in the predominant language in each could lead to a bias in the literacy skills of children. Furthermore, the K’iche area has schools that require a 5-7 hour walk to be reached, for which they were left out of the intervention schools possibly biasing further the sample for the area. Finally, the K’iche’ area schools have received technical support from the Education Program from UNICEF and SHARE – a program focused on improving reading capacities among primary students. Propensity Score Matching, where treatment and comparison observations are matched according to their observable characteristics, is the technique we are proposing to use at midline to mitigate this challenge. Some initial analysis done using the baseline data suggests its viability. Matching treatment and comparison students based on background characteristics (including sex, repeater status, age, whether the student attended an ECD center, a socioeconomic status index, and a home literacy environment index) and school characteristics (including whether the school has a board and a water point which were specific characteristics looked for in the schools to choose them for intervention) 38 the balancing property is satisfied when splitting the data into 5 blocks. The resulting propensity score histogram by treatment status is shown below.9 Graph 23 Resulting propensity score XI. Conclusions The baseline assessment results showed the differences between comparison and intervention groups, and allowed the development of an initial proposal for the midline assessment to evaluate the program’s impact. In addition it laid the groundwork for the skills and support areas that the program should focus on. The results show significant differences in socioeconomic conditions in both groups. Children in the intervention group (Ixil area), have a lower socioeconomic level than those in the comparison group (K’iche’ area). The lower socioeconomic conditions get translated into lower Spanish skills: for the five skills assessed, students in the Ixil area have lower results and differences in results are statistically significant at a 0.1% level. Given these differences, we propose that the midline assessment is analyzed through Propensity Score Matching. Initial analyses with the baseline data collected suggests its feasibility and as long as we have a low attrition the situation should not change once we collect the midline data. The assessment results show that children possess stronger literacy skills in Spanish than in their mother tongue (K´iche and Ixil respectively) but Spanish skills are not strong even for the basic skills 9 Further information on the do-file for this statistical analysis and the log file with the results is available upon request. 0 .2 .4 .6 .8 Propensity Score Untreated Treated 39 in the intervention group. In Spanish, students in the intervention area could use some support in letters (on average they are able to identify somewhat over half of the letters correctly), and moving from there build up their word reading skills until they are able to read and comprehend a passage. The low K’iche and Ixil skills on the other hand are likely related to the fact that Spanish is the dominant language in all schools, even for those schools classified as bilingual. Furthermore, K´iche /Ixil reading and educational materials at home are scarce. In letter identification, Literacy Boost activities should target children’s understanding of sound-symbol correspondence for the mother tongue as well as the letters that are different from Spanish, as these were the ones in which children struggled the most. Another area that warrants special attention in the Literacy Boost intervention is reading comprehension strategies. Even when children were able to read the passages on their own, only 5.8%, 1.2% and 0.6% were able to answer 8 or more reading comprehension questions correctly in Spanish, K´iche or Ixil respectively. Thus, the Literacy Boost program should also focus on ensuring that instruction in literacy skills is always accompanied by activities which help children construct meaning from what they have read. Literacy Boost can design activities which help children understand how to use their own background knowledge to improve comprehension. Once they are able to make connections from what they read to their own lives, reading may become a more relevant and enjoyable experience for these children. Furthermore, teaching children how to monitor their own comprehension may help them become more independent readers. This is a big challenge throughout the life of the program. Additionally, Literacy Boost should work in the Home Literacy Environment as the presence of child friendly books in the home and the percentage of family members who engage with them in reading is low. Thus, if the Literacy Boost program succeeds in making more Ixil and K´iche materials available to families and community members and increase Ixil and K´iche reading practices in the home, the next Literacy Boost Assessment can determine whether these efforts do lead to improvements in Ixil and K´iche reading abilities. Finally, the results identify students that are struggling and could benefit from Literacy Boost the most. Within the intervention group repeaters, students that did not attend an ECD center and those with a lower socioeconomic status have lower predicted literacy skills. The program should aim to target these students so that they are not left behind their most advantaged counterparts. 40 Appendix A. Inter-rater reliability Literacy skill sub-test Inter-rater Reliability Rating Spanish letter knowledge 0.96 Excellent K’iche’ letter knowledge 0.99 Excellent Ixil letter knowledge 0.92 Excellent Spanish Most Used Words 0.98 Excellent K’iche’ Most Used Words 0.98 Excellent Ixil Most Used Words 0.94 Excellent Spanish word decoding 0.99 Excellent Spanish fluency 0.98 Excellent K’iche’ fluency 0.98 Excellent Ixil fluency 0.99 Excellent Spanish accuracy 0.90 Excellent K’iche’ accuracy 0.90 Excellent Ixil accuracy 0.84 Excellent Spanish reading comprehension 0.99 Excellent K’iche’ reading comprehension 1.00 Excellent Ixil reading comprehension 0.96 Excellent To test inter-rater reliability, 12.5% of learners (139 out of 1108) were assessed by two enumerators simultaneously. Long one-way ANOVA techniques were used to calculate the intra-class correlation within pairs of assessors for a measure of reliability. Using Fleiss’ benchmarks for excellent (ICC>0.75), good or fair (0.75>=ICCA>0.4), and poor (0.4>=ICC); all of the literacy outcome variables exhibited excellent inter-rater reliability. As inter-rater reliability was high, we can be confident that the team of assessors measured children’s reading skills consistently. 41 Appendix B. Descriptive Statistics Table B1: Background Characteristics by Sample Group Background characteristics Whole Sample LB Comparison Statistical Sig. General characteristics Age 8.8 8.9 8.8 ~ Attended ECD (%) 76.6 79.4 73.9 * Female (%) 48.0 48.9 47.1 Repeated 1st grade (%) 35.3 34.7 35.9 Repeated 2nd grade (%) 23.5 27.1 19.9 ** Home language-Ixil (%) 37.4 75.3 - *** Home language-K’iche’ (%) 40.0 11.3 68.3 *** Home language-Spanish (%) 12.9 6.9 18.8 *** Home language-Spanish and Ixil (%) 2.1 4.0 0.2 *** Home language-Spanish and K’iche’ (%) 6.9 2.6 11.1 *** Home language-Other (%) 0.8 - 1.6 ** Household members 5.9 5.7 6.1 *** SES (%) House has iron roof 69.3 82.4 56.5 *** House has wood walls 43.1 72.0 14.7 *** Has TV at home 47.2 38.2 56.1 *** Has radio at home 73.4 74.9 71.9 Has mobile in household 87.3 87.3 87.3 Has electricity home 75.0 69.5 80.5 *** Has a fridge home 12.3 6.4 18.1 *** Has a bicycle home 27.2 21.1 33.2 *** Has a motorbike home 12.6 10.5 14.7 * Has a car home 7.9 3.1 12.5 *** Has cows 21.4 19.1 23.7 ~ Has pigs 62.0 52.9 71.0 *** 42 Has chickens 83.4 79.3 87.5 *** Has donkeys 4.3 3.5 5.2 Has horses 31.6 38.2 25.1 *** Has a separate kitchen home 87.3 85.6 88.9 Has 3 or 4 rooms at baseline 30.5 25.6 35.3 *** Work status Does chores at home (%) 96.6 97.1 96.1 Works outside home (%) 24.7 25.6 23.8 Total types of chores 3.4 3.3 3.6 ** Spends a lot of time in chores (%) 24.7 25.1 24.4 Observations 1108 550 558 p-value: ~10%, *5%, **1%, ***0.1% Table B2. Nutritional Status and Attention by Sample Group Nutritional status and attention (%) Whole Sample LB Comparison Statistical Sig. Had breakfast this morning 93.5 90.5 96.4 *** Breakfast was enough 94.8 94.0 95.5 School provides meals 48.7 46.9 50.5 Uses a latrine at home 76.4 76.6 76.3 Feels hungry at various times or constantly during the school day 15.6 22.0 9.4 *** Pays attention all or most of the time while in class 94.5 91.3 97.7 *** Observations 1107 550 557 p-value: ~10%, *5%, **1%, ***0.1% 43 Table B3. Home Literacy Environment by Sample Group Home and Community Literacy Environment (%) Whole Sample LB Compariso n Statistical Sig. Has textbooks 71.8 72.4 71.3 Has religious books 75.1 78.4 71.9 * Has magazines 1.1 1.3 0.9 Has newspapers 26.3 19.6 32.8 *** Has storybooks 25.2 32.0 18.5 *** Has coloring books 32.3 27.5 37.1 *** % of Family members seen reading 47.3 46.6 47.9 % of Family members that help the child study 69.5 69.3 69.8 % of Family members that read to the child 29.6 27.3 31.7 ** % of Family members that tell stories to the child 31.6 30.5 32.6 Someone in the community read to you 37.5 28.5 46.4 *** Did you read to someone in the community 47.5 38.5 56.4 *** Did you lend a storybook outside your home 42.0 35.5 48.3 *** Observations 1108 550 558 P-value: ~10%, *5%, **1%, ***0.1% 44 Table B4. Spanish Literacy Skills by Sample Group Spanish Literacy Skills Whole Sample Literacy Boost Comparison Statistical Sig. % Spanish letters correct 63.2 55.8 70.6 *** % Spanish MUW correct 37.9 28.3 47.4 *** % Spanish decoding correct 31.3 23.4 39.1 *** % of Children who read Spanish passage 30.4 22.9 37.8 *** Fluency(wcpm)-Spanish 7.4 5.0 9.7 *** Accuracy (% of words read correctly) -Spanish 27.6 20.2 34.9 *** % Spanish reading comprehension correct 14.8 10.5 19.1 *** Observations 1108 550 558 p-value: ~10%, *5%, **1%, ***0.1% Tables B5. Detailed Spanish Results by Item. Letters, MUW and Decoding Letters correct (%) LB Comparison e 88.2 95.7 O 86.7 95.3 i 86.2 95.9 u 84.4 88.9 A 83.5 87.3 S 83.3 92.3 m 71.5 88.5 r 70.9 84.1 P 64.4 79.0 t 63.6 80.3 x 60.5 68.6 45 L 56.0 77.6 Z 52.7 64.9 v 48.9 72.2 b 46.2 59.5 N 45.8 64.7 J 42.7 68.3 Y 42.5 55.6 f 42.0 67.9 c 41.8 62.7 D 40.4 59.3 Ñ 37.6 56.5 K 36.5 49.8 w 35.8 48.4 G 34.9 51.4 Q 32.9 44.1 h 26.9 46.8 MUW correct (%) LB Comparison mamá 64.2 76 sol 45.3 66.3 casa 38.0 61.8 luna 37.1 64.0 dedo 33.3 57.9 libro 27.3 48.4 lectura 27.1 46.1 papel 26.7 49.3 día 26.5 50.2 palabra 25.8 41.9 sobre 25.1 44.3 46 lugar 24.9 50.7 grupo 24.9 42.8 nombre 24.5 47.8 personas 24.4 41.8 compañero 24 39.4 comunidad 22.4 39.6 hoja 15.8 36 Güisquil 15.1 24.4 diccionario 13.1 18.5 Decoding correct (%) LB Comparison tuso 38.4 52.0 raca 31.1 46.8 pada 30.5 46.4 beta 28.2 46.6 serta 27.6 45.3 infas 25.6 45.2 vugar 24.7 43.7 fuande 24.2 43.4 tobre 23.3 40.0 greto 22.4 42.8 bersones 22.2 36.7 salabra 22.0 35.8 camaludad 20.5 32.4 lombro 20.4 36.0 vecto 20.0 40.0 rompeñero 19.5 32.3 saciones 18.5 23.3 47 ascrapa 18.0 38.2 rexta 16.0 27.6 tatá 15.1 28.0 Tables B6. Detailed Ixil Results by Item. Letters and MUW Letters correct (%) LB o 88.3 e 87.3 a 85.9 S 78.4 u 77.8 M 61.3 X 57.9 t 56.5 ee 55.8 P 55.8 I 54.8 L 54.8 R 52.2 OO 47.2 N 44.6 v 44.0 aa 42.9 UU 39.9 J 39.5 y 38.1 K 32.9 48 Q 31.3 q 15.1 ch 14.3 XH 4.2 TCH 2.6 tz 2.4 K’ 2.2 Q’ 2.2 T’ 2.2 b’ 1.8 TX 1.8 (’) 1.6 CH’ 1.6 TZ’ 1.6 tch’ 1.6 tx’ 1.4 MUW correct (%) LB lee 26.0 chee 19.6 chuu 19.0 ich 18.7 pach 18.1 kut 16.7 o’ 9.5 49 Te’k 7.5 t’el 6.8 Ku’ 6.0 ko’m 5.8 ab’ 4.8 i’b’oy 4.6 txikon 4.0 tz’ib’ 3.0 q’ii 2.4 txay 2.0 tx’umil 2.0 tx’i’ 1.4 b’a’x 1.2 Tables B7. Detailed K’iche´ Results by Item. Letters and MUW Letters correct (%) Comparison i 94.0 a 92.1 s 91.8 O 90.8 u 88.8 E 84.1 r 79.2 L 76.4 n 73.9 M 73.0 P 69.7 50 j 69.2 X 62.5 T 61.0 K 49.4 Y 45.9 w 41.4 Ch 41.2 Q 35.2 t’ 10.2 Tz 9.7 B’ 6.0 k’ 5.2 ch’ 4.7 (‘) 4.2 tz’ 4.0 q’ 2.5 MUW correct (%) Comparison nan 50.4 ali 50.4 ja 48.1 kej 35.5 chak 34.5 chakach 31.5 wuj 30.5 wachalal 27.0 winaq 24.3 51 B’alam 23.1 pix 22.1 tijob’al 21.8 meseb’al 19.6 che’ 15.6 juyub’ 14.9 me’s 11.7 ch’ek 7.2 tz’il 6.2 q’ij 5.2 tz’i’ 4.0 Tables B8. School Characteristics by Sample Group School Infrastructure and access Whole Sample Literacy Boost Comparison Statistical Sig. Distance to urban center (km) 8.4 1.7 15.1 *** Distance to pave road (km) 17.4 27.1 7.6 *** Electricity (%) 74.3 62.9 85.7 * Piped water (%) 90.0 94.3 85.7 Restroom facilities (%) 97.1 94.3 100.0 Letrine for girls (%) 84.3 91.4 77.1 School preparation facilities (%) 85.7 88.6 82.9 Observations 70 35 35 p-value: ~10%, *5%, **1%, ***0.1% Reading Teaching Practices and Resources (%) Whole Sample LB Comparison Statistical Sig. Library 47.1 57.1 37.1 ~ Mother tongue teaching hours 0 1.6 - 3.7 Mother tongue teaching hours 1-4 58.1 60.0 55.6 Mother tongue teaching hours 4-8 24.2 31.4 14.8 Mother tongue teaching hours 8-12 11.3 8.6 14.8 52 Mother tongue teaching hours >12 4.8 - 11.1 * Reading camps participation 12.9 22.9 2.9 Half an hour of reading 95.7 94.3 97.1 Book loans 25.7 40.0 11.4 ** Literary contests 7.1 11.4 2.9 Reading festivals 7.1 8.6 5.7 Other reading promotion activities 4.3 8.6 - ~ Observations 70 35 35 p-value: ~10%, *5%, **1%, ***0.1% Teaching staff characteristics Whole Sample LB Comparison Satistical Sig. % of Female teachers 41.7 37.7 45.6 Average years teaching 6.9 7.3 6.4 * % of Teachers with a diversificado education level 76.7 81.0 72.4 % of Teachers with a profesorado education level 20.4 15.2 25.6 % of Teachers with a licenciatura education level 2.9 3.8 2.0 Observations 70 35 35 p-value: ~10%, *5%, **1%, ***0.1% Children present and enrollment Whole Sample LB Comparison Satistical Sig. Total enrollment 160 182 139 % Boys present out of enrolled grade 1 85.9 93.8 77.9 * % Boys present out of enrolled grade 2 86.8 90.1 83.4 % Boys present out of enrolled grade 3 89.0 93.2 84.9 % Boys present out of enrolled grade 4 93.7 93.9 93.5 % Boys present out of enrolled grade 5 91.9 95.9 87.8 % Boys present out of enrolled grade 6 91.6 96.1 87.2 % Boys present out of enrolled 89.2 93.8 84.6 % Girls present out of enrolled grade 1 85.4 94.8 76.1 ** % Girls present out of enrolled grade 2 90.7 94.7 86.8 53 % Girls present out of enrolled grade 3 88.1 92.0 84.1 % Girls present out of enrolled grade 4 93.7 93.6 93.8 % Girls present out of enrolled grade 5 91.6 95.1 88.1 % Girls present out of enrolled grade 6 91.4 97.6 85.5 * % Girls present out of enrolled 89.9 94.9 85.0 *** Observations 70 35 35 p-value: ~10%, *5%, **1%, ***0.1% Community and NGO Involvement (%) Whole Sample LB Comparison Statistical Sig. School board 100.0 100.0 100.0 Other NGOS working at the school 18.6 25.7 11.4 Observations 70 35 35 p-value: ~10%, *5%, **1%, ***0.1% 54 Variables % Letters % MUW % Decoding Fluency(wcpm) Accuracy % Reading comp. Age 0.003 -0.004 -0.011 -0.062 -0.003 -0.004 Attended ECD 0.060~ 0.125** 0.104** 2.095~ 0.101* 0.066* Female 0.014 0.022 -0.003 0.672 0.010 0.000 Repeated 1st or 2nd grade -0.044~ -0.091** -0.106** -3.169* -0.097* -0.045~ Home language￾Spanish 0.060 0.099 0.109 3.341 0.078 0.148~ Home language￾Spanish and Ixil 0.184** 0.270*** 0.173** 5.450 0.258* 0.147~ Bottom two quintiles of the SES index -0.059~ -0.086~ -0.085~ -1.964 -0.077~ -0.067~ Bottom two quintiles of the HLE index 0.005 0.036 0.024 0.635 0.027 0.019 Top two quintiles of work index -0.013 -0.014 -0.015 -0.513 -0.004 -0.029 Had breakfast this morning -0.011 -0.016 -0.043 -1.159 -0.032 -0.007 Feels hungry at various times or constantly during the school day -0.053* -0.058~ -0.048 -1.185 -0.040 -0.039 Pays attention all or most of the time while in class 0.057 0.052 0.031 2.278 0.077 0.069* Constant 0.487*** 0.249 0.345* 4.773 0.173 0.082 R-squared 0.071 0.110 0.100 0.076 0.084 0.100 Observations 541 541 541 541 541 541 ~0.10 *0.05 **0.01 ***0.001 55 Appendix C. Regression Results Table C1. Multivariate Regression Analysis. Spanish Literacy Skills Table C2. Multivariate Regression Analysis. Ixil Literacy Skills Variables % Letters % MUW % Decoding Fluency(wcpm) Accuracy % Reading comp. Age -0.005 -0.004 -0.094 -0.012 0.000 Attended ECD 0.057* 0.033~ 0.679 0.021 0.014 Female 0.026 0.001 0.307 -0.016 0.002 Repeated 1st or 2nd grade -0.036~ -0.061** -1.898* -0.066* -0.037** Home language-Ixil 0.119 0.081*** 1.456~ 0.064** 0.040** Home language-Spanish and Ixil 0.171* 0.140** 3.862~ 0.169* 0.104** Bottom two quintiles of the SES index -0.058** -0.047~ -1.158 -0.048 -0.021 Bottom two quintiles of the HLE index 0.004 0.040* 0.253 0.022 0.007 Top two quintiles of work index -0.035* -0.029~ -0.296 -0.006 -0.009 Had breakfast this morning -0.021 -0.009 0.805 0.035 0.028* Feels hungry at various times or constantly during the school day -0.017 -0.004 -0.742 -0.040* -0.008 Pays attention all or most of the time while in class 0.016 0.027 1.166 0.060 0.033** Constant 0.289*** 0.054 0.493 0.095 -0.046 R-squared 0.117 0.113 0.060 0.079 0.072 Observations 495 495 495 495 495 ~0.10 *0.05 **0.01 ***0.001