Baseline Evaluation Report Learning for Life Project Implemented by CRS and its partners with USDA assistance 1 Acknowledgements Staff of CRS Country Representative Paul Townsend Head of Programs John Briggs Deputy Head of Programs Courtney Plummer MEAL Regional Technical Advisor Matt Peckarsky Youth and Childhood Education Program Manager Brenda Urizar Chief of Party Emily Drummer Field Coordinator Juan Carlos Lau MEAL Coordinator Liza Castillo PRODESSA’s Staff Co-director Federico Roncal Co-director Edgar García Tax Departmental Coordinator Efraín Tzaquitzal Pastoral Social Caritas Arquidiócesis de Los Altos’s Staff Director EImer Vásquez Departmental Coordinator Elmer Palala Juárez & Associates, Inc.’s Staff Senior Project Director Fernando Rubio Evaluation Specialist Leslie Rosales Staff of Dirección General de Evaluación e Investigación Educativa - DIGEDUCA General Director Luisa Müller Durán Deputy Director of Tools Development María José del Valle Deputy Director of Execution Leslly de León Trujillo Bilingual Tools Development Coordinator Eva Ramírez Jiménez Specialist in K’iche’ Ruberto Carlos García Aguilar Staff of Dirección Departamental de Educación de Totonicapán Departmental Director María Dominga Pú Chief of Technical-Pedagogical Department Juan Basilio Tahay Chief of Educational Delivery Carlos García School Education Coordinator Claudia Canastuj Primary Level Coordinator Arnulfo Vásquez School Infrastructure Coordinator Michael Menchú Chief of Administration of Support Programs Julio Tzul Chief of Department of Educational Quality Regino Baltazar Velasquez Strengthening Staff of External Evaluator - ADOC Director Friné Paz Data Base and Statistics Specialist Erick Calderón Field Coordinator Edgar Ruano Field Coordinator María André Solares Education Analyst Telma Pérez Education Analyst Sayra Cardona This publication was made possible by financial support of the United States Department of Agriculture, under cooperative agreement No. FFE-520-2016/010-00. Any opinions, findings, conclusions, or recommendations expressed in this publication are those of the author(s) and do not necessarily reflect the views of the United States Department of Agriculture or the United States Government. An accessible version of this report can be made available upon request by emailing FAS.monitoring.evaluation@usda.gov. Guatemala, August 2018 2 Table of Contents I. ACRONYMS ......................................................................................................................... 6 II. EXECUTIVE SUMMARY...................................................................................................... 7 III. BACKGROUND .................................................................................................................. 11 IV. CONTEXT........................................................................................................................... 12 4.1. Illiteracy ..............................................................................................................................................12 4.2. School Enrollment..............................................................................................................................12 4.3. Primary school completion and dropout rates ................................................................................13 4.4. Repetition rates .................................................................................................................................13 4.5. Pass rates ...........................................................................................................................................13 4.6. Reading attainment in Totonicapán .................................................................................................14 V. RESEARCH PLAN.............................................................................................................. 14 5.1. Objectives of the baseline study.......................................................................................................14 5.2. Key research questions......................................................................................................................15 5.3. Baseline evaluation methodology.....................................................................................................15 5.4. Planned and achieved quantitative data sample .............................................................................17 5.5. Sample of other key stakeholders ....................................................................................................19 5.6. Methodology for the Collection and Analysis of Quantitative Data ...............................................20 5.7. Methodology for the Collection and Analysis of Qualitative Data..................................................20 5.8. Management of Confidentiality ........................................................................................................21 VI. ACADEMIC AND SCHOOL DEVELOPMENT INDICATORS............................................ 24 6.1. (Indicator 1) Percent of students who, by the end of first grade of primary school, demonstrate that they can read and understand the meaning of grade-level text. Global Indicator: 1.7%.........................24 6.2. (Indicator 2) Percent of students (girls and boys) who, by the end of 3rd and 6th grade, demonstrate grade level reading comprehension as defined by national standards at USDA supported schools. Global Indicator: 8.29%..........................................................................................................................29 6.3. (Indicator 22) Number of students enrolled in schools receiving USDA assistance. Global Indicator: 48,664 ..................................................................................................................................................31 6.4. (Indicator 20) Number of students regularly (80%) attending USDA supported classrooms/schools. Global Indicator: 42,727 ....................................................................................................31 6.5. (Indicator 9) Number of teachers/educators/teaching assistants in target schools who demonstrate use of new and quality teaching techniques or tools as a result of USDA assistance. Global Indicator: 24.5% Number: 499 teachers..............................................................................................................33 6.6. (Indicator 8) Number of textbooks and other teaching and learning materials provided as a result of USDA assistance ....................................................................................................................................38 6.7. (Indicator 12) Number of school administrators and officials in targeted schools who demonstrate use of new techniques or tools as a result of USDA assistance. Global Indicator: 40.4%; Number: 134 school principals.............................................................................................................................45 3 6.8. (Indicator 24) Percentage of municipal education supervisors from the Totonicapán Department of Education that have increased their capacity to fulfill their roles and responsibilities. Global Indicator: 0% 46 Conclusions: Academic and School Development...............................................................................................48 Recommendations: Academic and school development....................................................................................50 VII. THE HUNGER SCALE AND STUDENT ATTENTION SPAN............................................ 52 7.1. (Indicator 14) Percentage of students in participating primary schools who indicate not being hungry during the school day Global Indicator: Low Level 54.4%......................................................................52 7.2. (Indicator 13) Percentage of students in the classrooms defined as "very attentive" using a scale that defines established criteria. Global Indicator: 67.1% .................................................................................54 Conclusions: Hunger scale and attention span ..................................................................................................57 Recommendations: Hunger scale and attention span.......................................................................................57 VIII. HEALTH, NUTRITION, AND HYGIENE-RELATED INDICATORS ................................... 59 8.1. (Indicator 5) Number of individuals who demonstrate use of new child health and nutrition practices as a result of USDA assistance. Global Indicator 25.9%; Number: 4,380 volunteers ........................59 8.2. (Indicator 7) Percent of schools reaching the tier/level 1 of healthy school initiative tiers Global Indicator: 95.3%....................................................................................................................................................64 8.3. (Indicator 6) Number of individuals who demonstrate use of new safe food preparation and storage practices as a result of USDA assistance Global Indicator: 67.6%; Number: 11,433 volunteers.........68 Conclusions: Health, Nutrition and Hygiene-related indicators..........................................................................70 Recommendations: Health, Nutrition and Hygiene-related Indicators ............................................................72 IX. GENERAL RECOMMENDED ACTIONS ........................................................................... 73 X. REFERENCES ................................................................................................................... 74 XI. ANNEXES........................................................................................................................... 75 11.1. Learning for Life Baseline indicator data, disaggregated ................................................................75 11.2. Learning for Life Baseline Data Confidence Intervals ......................................................................79 11.3. Learning for Life Baseline Indicator data: treatment and control groups......................................84 11.4. Learning for Life Baseline Indicator data: global indicator and indicator’s representation as a whole number (out of entire sample for given indicator) .................................................................................89 11.5. Learning for Life Baseline Indicators, characteristics of the sample...............................................92 11.6. Learning for Life Baseline data: statistical significance....................................................................94 11.7. Proposal for constructing Learning for Life healthy schools indicator .........................................100 4 List of Tables Table 1. Primary school enrollment in the Department of Totonicapán ................................................ 13 Table 2. Total No. of schools evaluated, CRS Learning for Life Baseline study, by group................ 17 Table 3. Total No. of schools evaluated, CRS Learning for Life Baseline study, by municipality ..... 18 Table 4. Total No. of students that participated in the study, by treatment and control groups ........ 18 Table 5. Total No. of parents that participated in the study, by treatment and control group, disaggregated by sex .................................................................................................................................... 18 Table 6. Total No. of first-grade parents that participated in the study, by treatment and control group, disaggregated by sex........................................................................................................................ 19 Table 7. Total No. of school principals that participated in the study, by treatment and control group, disaggregated by sex .................................................................................................................................... 19 Table 8. Sample of other key stakeholders included in Baseline study ................................................ 19 Table 9. Students with Reading Comprehension ELGI (Spnish)........................................................... 24 Table 10. ELGI Reading Levels .................................................................................................................. 24 Table 11. Abilities or Areas Evaluated in ELGI......................................................................................... 25 Table 12. Number of students evaluated in ESWUJ (K´iche´) ............................................................... 27 Table 13. Abilities or Areas Evaluated in ESWUJ.................................................................................... 27 Table 14. Number of Learning for Life project beneficiaries ................................................................... 31 Table 15. Teacher perception on classroom attendance ........................................................................ 31 Table 16. Percentage of teachers that demonstrate use of teaching techniques and tools .............. 34 Table 17. Percentage of materials and tools promoted by Learning for Life........................................ 38 Table 18. Percentage teachers that indicate the use of components in the reading material ........... 42 Table 19. Percentage of students that have breakfast at home............................................................. 52 Table 20. Type of food students receive at school................................................................................... 52 Table 21. Sensations of first, third and sixth-grade students.................................................................. 53 Table 22. Percentage of students indicating a certain attention level in class..................................... 54 Table 23. Students’ levels of attention in class according to teachers.................................................. 54 Table 24. Students’ behavior in the classroom during the study compared to the control group ..... 55 Table 25. Class participation of girls and boys ......................................................................................... 55 Table 26. Definition of the hunger and attention scale levels ................................................................. 56 Table 27. Hunger levels by grade, in percentages................................................................................... 56 Table 28. Percentage of parents that mention steps to treat diarrhea.................................................. 60 Table 29. Percentage of students that carry out hygiene practices....................................................... 61 Table 30. Percentage of personal hygiene practices in the household ................................................ 61 Table 31. Percentage of FMV that identify household hygiene practices............................................. 62 Table 32. Health care treatments available to students in schools, in percentages ........................... 62 Table 33. Dietary Diversity identified in schools and homes that participated in the study................ 63 Table 34. Foods with the highest consumption in households............................................................... 63 Table 35. Foods with the highest consumption in schools...................................................................... 64 Table 36. Condition of the warehouses assessed in the baseline study .............................................. 65 Table 37. Percentage of schools that meet baseline study kitchen conditions.................................... 66 Table 38. Percentage of schools where good health practices were observed in food preparation 68 Table 39. Methods used in schools to purify water, by percentages..................................................... 68 Table 40. Percentage of schools with clean environments..................................................................... 69 5 List of Graphs Graph 1. Third-grade student reading performance, according to sex and treatment group ........... 30 Graph 2. Sixth-grade student reading performance, according to sex and treatment group............ 30 Graph 3. Percentage of teachers with Spanish language skills........................................................ 33 Graph 4. Percentage of teachers with K’iche’ language skills .......................................................... 33 Graph 5. Techniques used by teachers in their reading class, by percentage ................................. 34 Graph 6. Tools used by teachers in their reading class, by percentage ........................................... 35 Graph 7. Reasons given by teachers and school principals for using/not using Learning for Life methodologies, in percentages ......................................................................................................... 35 Graph 8. Percentage of teachers that have received training in Learning for Life............................ 36 Graph 9. Percentage of teachers that know Learning for Life methodologies .................................. 37 Graph 10. How teachers became familiar with the Learning for Life methodologies (percentages) 38 Graph 11. Percentage of teachers that know about the steps to prepare their reading classes ...... 40 Graph 12. Percentage of teachers that indicate the resources available to them for teaching how to read in Spanish & K’iche’ .................................................................................................................. 41 Graph 13. Percentage first, third, and sixth-grade teachers that indicate availability of teaching material in K’iche’ and Spanish for each Learning for Life Methodology .......................................... 41 Graph 14. Percentage of teachers that perform activities after reading ........................................... 43 Graph 15. Percentage of student participation in reading activities by sex, according to teachers.. 44 Graph 16. Percentage of schools with school improvement plans ................................................... 45 Graph 17. Percentage of schools that implement planning and coordination activities.................... 45 Graph 18. Periodicity with which schools monitor their improvement plan, in percentages ............. 46 Graph 19. Percentage of FMV with knowledge on the treatment of diarrhea................................... 59 Graph 20. Percentage of schools with resources available for personal hygiene ............................ 60 Graph 21. Types of stoves found in schools, in percentages ........................................................... 67 Graph 22. Percentage of schools that implemented a vegetable garden the previous year and have space for a vegetable garden............................................................................................................ 69 List of Figures Figure 1. Quasi-experimental design: random control trial..................................................................... 16 Figure 2. Qualitative data collection and analysis.................................................................................... 21 Figure 3. Reasons why teachers apply or do not apply the techniques ............................................... 36 Figure 4. Comments regarding teaching resources ................................................................................ 39 Figure 5. Phenomena affecting bilingual education................................................................................. 42 Figure 6. Reasons why girls and boys do not participate in reading activities .................................... 44 Figure 7. Challenges identified by the DCs............................................................................................... 47 Figure 8. Aspects of food handling in warehouses, referred by parents .............................................. 66 6 I. ACRONYMS ADOC Association for Community Organizational Development (Asociación de Desarrollo Organizacional Comunitario, ADOC) AIR American Institutes for Research APV Learning for Life (Aprendizaje para la Vida, APV) BL Baseline CG 1-A Control Group 1-A CRS Catholic Relief Services DC District Coordinator (Ministry of Education, formerly Technical Administrative Coordinator) DDS Dietary Diversity Score DIDEDUC Departmental Directorate of Education (Dirección Departamental de Educación, DIDEDUC) DIGEBI General Directorate of Intercultural Bilingual Education (Dirección General de Educación Bilingüe Intercultural, DIGEBI) DIGEDUCA General Directorate of Educational Research and Evaluation (Dirección General de Evaluación e Investigación Educativa DIGEDUCA) DIGEFOCE General Directorate for Strengthening Educational Community (Dirección General de Fortalecimiento de la Comunidad Educativa, DIGEFOCE) DIGEPSA General Directorate of Community Participation and Support Services (Dirección General de Participación Comunitaria y Servicios de Apoyo, DIGEPSA) ELGI Early Grade Reading Assessment in Spanish (Evaluación de Lectura en Grados Iniciales en español, ELGI) ENCOVI Survey on National Living Conditions (Encuesta Nacional de Condiciones de Vida, ENCOVI) ESWUJ Early Grade Reading Assessment in K’iche’ (Evaluación de Lectura en Grados Iniciales en k’iche’, ESWUJ) FMV Fathers, Mothers and Volunteers GDP Gross Domestic Product HDDS Household Dietary Diversity Score INE National Institute of Statistics (Instituto Nacional de Estadística, INE) IPC Integrated Phase Classification JdL Jardin de Letras (Garden of Letters) KC Kemom Ch’ab’al Methodology MGD McGovern-Dole Food for Education MINEDUC Ministry of Education NGO Non-governmental organization PMP Performance Monitoring Plan pp Percentage points PRODESSA Santiago Development Project (Proyecto de Desarrollo Santiago, PRODESSA) PTA Parent-teacher association RENAS National Registry for Sexual Offenders (Registro Nacional de Agresores Sexuales, RENAS) SEGEPLAN Presidential Secretariat for Planning and Programming (Secretaría de Planificación y Programación de la Presidencia, SEGEPLAN) SINAE National System of Educational Accompaniment (Sistema Nacional de Acompañamiento Educativo, SINAE) TAC Technical Administrative Coordinator (Ministry of Education) USCCB United States Conference of Catholic Bishops USDA United States Department of Agriculture 7 II. EXECUTIVE SUMMARY Catholic Relief Services-United States Conference of Catholic Bishops (CRS-USCCB) will implement Phase II of the Learning for Life project1 from 2016 to September 2021, in six municipalities located in Totonicapán, Guatemala. This area is characterized by high poverty and extreme poverty indices (77.5% and 8.9%, respectively); and little investment in education, resulting in a 28.6% illiteracy rate, the majority of which are women. 2 Totonicapán also mirrors a worrying national trend: school enrollment has decreased, grade repetition is at 9.9%, and dropout rates are at 2.3%. General assessments in Reading and Mathematics show that students continue to fall behind in attaining expected knowledge and skills. The second regional comparative and explanatory study, published by UNESCO in 2008, placed Guatemala as one of the last Latin American and Caribbean countries in the ranking of Mathematics and Reading test performance.3 In Totonicapán, sixth grade reading achievement has remained below the national average in every single evaluation year. In 2014, Totonicapán was ranked 16 out of 22 departments, according to its percentage of reading proficiency. That same year, third grade reading proficiency in 50% of the municipalities in Totonicapán was below departmental achievement levels. In response to this situation, Phase II of Learning for Life has the following strategic objectives: 1) Improve the skills of school-age girls and boys in Reading; and 2) Increase the use of dietary and health practices. These objectives are based on the general theory of change defined in the McGovern-Dole project results framework, which stipulates that children’s reading ability will improve if they consume daily food with adequate nutritional values to ensure their attention in the classroom; if school attendance is more constant; and if teachers improve their skills in teaching reading. The baseline study (BL) was conducted to determine the initial conditions of the treatment and control groups. Results will be compared with data from the midterm and final evaluations to help determine the effect and impact of the interventions. This will also enable the collection of information to monitor indicators and reorient or adjust strategies based on the starting point in and within the six municipalities where the McGovern-Dole Learning for Life project will be implemented. The BL used a mixed methodology that combined quantitative and qualitative instruments (focus groups and interviews). The results are shown by group (treatment and control). In addition, some results are disaggregated by sex, socioeconomic status, parent education and preschool attendance. The differences identified between the groups indicate a statistical significance. The BL was initially designed using a sample of 179 schools, with a 95% confidence level and 80% power. In the end, 175 schools participated in the study, as school principals and teachers of four schools that were part of the initial sample did not permit the study team to raise information. The sample schools were distributed across the 6 targeted municipalities and were chosen at random by the American Institutes for Research (AIR). The following children were evaluated with the standardized reading tests: 1,786 girls and boys in first grade; 2,171 girls and boys in third grade; and 1,965 girls and boys in sixth grade. The study also included consultation with school principals, teachers, and supervisors, as well as focus groups with fathers and mothers. The study used official reading tests from the Ministry of Education (MINEDUC) that were complemented with observation guides, interviews, focus groups and data collection guides for local and municipal registration. 1 Aprendizaje para la Vida (APV) in Spanish 2 National Institute of Statistics (INE) 2013 3 United Nations Educational, Scientific and Cultural Organization 2008 8 Prior to the collection of baseline data, field staff were trained in research ethics on the application of physical and digital tools to gather information on the evaluation indicators and on how to complete the official MINEDUC reading tests. The official MINEDUC tests included: i. Early-grade reading assessment in Spanish (ELGI) and K’iche’ (ESWUJ) for first-grade students. ii. Reading comprehension assessments for third and sixth-grade students. The BL survey team included a field supervisor, three first-grade data collectors, one third and sixth grade reading comprehension data collector, and one data collector for the parents. The quasi-experimental design compared the different groups to find any statistically significant differences. To examine the impact of the program’s various components, a series of ordinary least squares regressions were carried out to compare results between and within groups. All regressions used standard deviation for grouped data at the school level. The Atlas.ti software was used for the analysis of qualitative data. BL study results are described in specific chapters in order of indicators. The main results by indicator are the following: • The 1.7% of first grade students could read and comprehend a first-grade Spanish language text. • The results of the first-grade reading comprehension tests for the Spanish language are similar across most of the treatment and control groups. • In the reading tests, 9.3% of third grade students scored “satisfactory” or “excellent” when reading a grade-level text, while 7.2% of sixth grade students scored “satisfactory” or “excellent” when reading a grade-level text. Third-grade students have higher results than those shown by sixth￾grade students. The highest satisfactory achievement in third-grade students can be found in group 2-A (12.7%), while the highest satisfactory achievement for sixth-grade students is in Group 2-B (10.1%). • In the Spanish comprehension assessment, over half the students showed an adequate level of comprehension of oral instructions. • The Kemom Ch’ab’al (KC) reading program is recognized by most (over 80%) of the teachers in the treatment group. • Teachers, school principals, and supervisors positively rate the tools and material provided by the Learning for Life Project, but they need reinforcement on its use, especially for the material in the K’iche’ language. • Strategies for preparing the reading class (the three steps for class preparation) and teaching in the classrooms, are not mastered by teachers. In general, less than 50% of the teachers in all groups apply the class preparation steps. • In all groups, teachers in general use less than 60% of the different reading techniques. Though teachers are familiar with the different steps for preparing for a reading class, they do not use the techniques and tools due to their lack of skills and interest, and to the inadequacy of the materials. • The material to teach reading in the mother tongue does not respond to all the variants according to context and is the most difficult to access, due to the fact that there is no leveled or generalized reading material in K'iche that can be used for the different variants of the language that are spoken in the communities. • Teachers talk, read and comprehend better in Spanish than in K’iche’. • Reading comprehension in K’iche’ is lower than that in Spanish. • The evaluator perceived that families have lost interest in speaking their native tongue, which may make bilingual education more difficult. 9 The BL study included an analysis of school performance indicators, such as number of students that regularly attend classrooms/schools supported by the United States Department of Agriculture (USDA); and attendance in schools that receive USDA support, disaggregated by sex. • Over 80% of the teachers in all groups indicated that classroom attendance is regular: between 80 and 100% of students are reported to attend. • In all groups, teachers in general consider that there is no significant difference between girls and boys with regards to their participation in reading activities, their participation in the classroom, and their learning of reading in general. • Parents indicate that the Learning for Life project methodology has helped girls and boys become more interested in learning how to read and has increased their participation in reading activities. The BL related the students’ nutritional situation during the school period with their level of classroom attention and participation. • More than 77% of the children indicated that they eat breakfast in their homes. Children in third grade eat breakfast less often. Over 91% of first-grade students report that they do eat breakfast. • It was observed in the schools that children receive a meal in the first hours of the school day. • According to the mothers’ focus groups, meals that are served in the schools, such as beans and coffee, are easily accepted by the children. Some mothers expressed that it is difficult for them to prepare certain foods, and this is the reason why children do not like them. • First, third and sixth grade students were asked if they felt hungry, sleepy, ill, sad or fatigued during the school day, assuming these could be hunger-related symptoms. Although children were able to identify that all of these sensations are attributed to hunger, a low percentage indicated feeling them. • When comparing the results of classroom attention levels with feelings hunger, sleepiness, fatigue, illness or sadness, there is no statistically significant correlation. It should be considered that, at the time of data collection, the Learning for Life project had already begun delivering food during school hours and children in the evaluated schools had been receiving food for more than a month. • The data that were obtained via observation or provided by the teachers and students coincide; there is a high level of student attention in the classrooms. • It was observed that class participation between girls and boys was equal. • Some parents consider that the food given to the children affects their attention levels, while others do not associate the lack of attention to hunger, but to other causes such as lack of teacher supervision and problems in the home. • The hunger scale built for the BL study showed that 3% of first-grade students are the least vulnerable group, followed by 6% of sixth-grade students. Third grade students are the most vulnerable, as they are at the high end of the scale with a percentage close to 15%. Increasing and improving feeding and nutrition practices is one of the strategic objectives of the Learning for Life project. The following are the BL results for health, nutrition and hygiene-related indicators: • The students receive deworming, vitamins and vaccine services in their schools from the Ministry of Health. • In general, parents indicated having a high percentage of knowledge regarding the treatment of diarrhea for their children. However, when asked about the three steps for treating diarrhea, most were able to correctly mention only one or two steps. • Regarding nutrition, the reported dietary diversity score (DDS) is acceptable, although this only shows access to food and not the nutritional quality of the food consumed by the students. The DDS is between 5 and 6 food groups in schools, and 4 food groups in the households. Families in Group 2-B showed the least DDS (3 food groups). 10 • The communities in Momostenango, San Andrés Xecul, San Bartolo Aguas Calientes, and Santa Lucía La Reforma were categorized as having high levels of nutritional vulnerability in the 2015 national census for measuring the height of school children. Those communities showed a low DDS (3 food groups). • Regarding hygiene, results show that less than 71% of the schools have the resources to promote personal hygiene practices amongst students. The hygiene practice that was most reported was handwashing after eating, while the least reported practice was brushing their teeth after eating. • Daily bathing is the hygiene practice most frequently mentioned at the household level. Also mentioned were use of latrines, putting animals in corrals, using soap for handwashing, using bins or bags for trash, and cleaning the house on a daily basis. • 98% of homes have a specific place allocated for cooking. • The highest percentage of families report consumption of boiled water. Other hygiene-related indicators that were evaluated in the schools were the conditions of warehouses, kitchens, and cleanliness of different areas in the schools. • All schools have warehouses for food storage, although not all are used exclusively for program food items and not all warehouses are in good conditions, nor equipped for the measuring of rations. Project signs were found in warehouses that had been built by the project. • Evidence of pests was observed in 50% of the schools. The majority of schools are in rural, open areas where pests are often an issue. This means it is important to include specific pest control actions within training content and plans for food management to reduce contamination risk. • Although the committees recognized that they have been trained in the management of the warehouses, it is necessary to clean and organize them. • Most school kitchens have an exclusive space allocated for cooking, cement doors and floors, and improved stoves with a fireplace and use of wood. • In schools that have not received improved kitchen and stove interventions, ventilation is not adequate; there is an excess of smoke accumulation. • Mothers reported that, though they have utensils to prepare food, these are not sufficient for the large population of school children. • 3% of school in the control group do not have toilets or latrines, while all the schools in the treatment group report having latrines and toilets. • Toilets are not always separated for boys and girls. • In all the groups, over 90% of the schools have piped water. While data on times the water was accessible during school hours were not collected, the researchers noticed that most schools do not have access to water during the school hours. • In all groups, few schools have spaces allocated for vegetable gardens. Parents value these gardens and indicated that the main obstacles to implementing them are lack of access to water and lack of space in the school. Based on the evaluation’s main findings, the following recommendations have been prioritized with specific actions and practices that should be analyzed by the project: • Focus and strengthen the methodological process of teaching how to read in K’iche’, taking into account the linguistic variants of the language in order to respond to the context and the priority established by the MINEDUC: bilingual and intercultural education. This will facilitate teacher appropriation of the methodology and will allow the majority of teachers to not only know the techniques, but also to apply them, and will be reflected in the students’ achievement in reading comprehension. • At the central and departmental levels, define jointly with the MINEDUC, through the General Directorate of Intercultural Bilingual Education4 and the General Directorate of Quality Monitoring 4 Dirección General de Educación Bilingüe Intercultural (DIGEBI) in Spanish 11 and Verification, 5 a classroom pedagogical accompaniment plan to monitor teacher application of the methodology and the techniques to improve the reading skills of students, and provide technical assistance according to their needs. The plan should be framed within the National System of Educational Accompaniment6 that the MINEDUC recently launched, which is "aimed at making the practices in the classroom culturally and linguistically relevant, visualizing the pedagogical advice as permanent support for the improvement of the pedagogical practice in the classroom.” The contributions that stem from the Learning for Life program are key and will allow the institutionalization and sustainability of the interventions and methodology. CRS can involve its partner, the Santiago Development Project, 7 who facilitates the process of teacher training and accompaniment in the intervention schools. • Design a strategy to promote and facilitate hygiene practices at the household level, involving community leaders, model mothers, and representatives of school governments to conduct hygiene days or marathons, transmitting key messages to promote behaviors and healthy practices. A key partner is CARITAS who has direct contact with parents and community leaders. • Design a strategy that includes a toolkit with simple and contextualized tools to strengthen the management capacity of directors, teachers and community leaders. This will allow them to negotiate support from the municipalities to improve the infrastructure of school warehouses (municipalities have funds that can and should be allocated to improve school infrastructure, especially the provision of piped water and school warehouses to safeguard food). Community and COCODE leaders, as well as other stakeholders, must be involved in presenting proposals to the COMUDES that specify school needs, such as: rehabilitation and/or construction of a school warehouse, describing if the school has land, contributions from the community, and/or contributions from the MINEDUC through the Free Fund. This will allow the Municipalities to commit themselves, give their contribution, recognizing that children need decent and healthy environments for learning. It should be made visible that the Learning for Life program is holistic and is contributing to improve educational quality. • Prepare and socialize a practical guide for the maintenance and cleaning of the school warehouses, ensuring that they are free of rodents and that they comply with the minimum standards to safeguard food. This document broadly describes the results for each indicator. Each section includes the main conclusions and recommendations that highlight and relate the most significant findings. In the final section, the document outlines a series of general recommendations made by the evaluation team to optimize the project. III. BACKGROUND CRS-USCCB has worked in Guatemala since 1963, implementing a variety of development programs to serve the poorest and most vulnerable populations. CRS interventions in Guatemala focus on education, food security, nutrition and health, sustainable agriculture, migration, disaster risk reduction and emergency response. CRS-USCCB will implement Phase II of the Learning for Life project from October 2016 to September 2021, in six municipalities located in the department of Totonicapán: Momostenango, San Andrés Xecul, Santa Lucía La Reforma, San Bartolo Aguas Calientes, Santa María Chiquimula and the rural area of Totonicapán (known as the highland). The project’s strategic objectives are: 1) To improve the reading skills of school-aged girls and boys; and 2) To increase the use of health and dietary practices. 5 Dirección General de Monitoreo y Verificación de la Calidad (DIGEMOCA) in Spanish 6 Sistema Nacional de Acompañamiento Educativo (SINAE) in Spanish 7 Proyecto de Desarrollo Santiago (PRODESSA) in Spanish 12 The general theory of change claims that reading abilities in children will improve with a daily consumption of foods that have the adequate nutritional values to ensure classroom attention, if school attendance is more constant, and if teachers improve their skills in teaching reading. This theory of change is reflected in the McGovern-Dole project results framework, which shows a relational cause-effect chain, based on activities carried out to achieve immediate and intermediate results, which in turn pursue a strategic objective of improved reading in school-aged children. IV. CONTEXT The department of Totonicapán is located in Guatemala’s Western Highlands, with a territorial extension of 1,061 km² and an altitude of 2,495 meters above sea level. It borders the department of Huehuetenango to the North; the department of Sololá to the South; the department of Quiché to the East; and the department of Quetzaltenango to the West.8 The departmental capital is located 203 km from the country’s capital city. It has a population of 537,584 inhabitants, out of which 97% are of Mayan origin and 52.4% live in rural areas.9 Totonicapán is one of the departments in Guatemala with the highest rates of poverty and extreme poverty. According to the 2014 National Survey on Living Conditions, 10 Totonicapán has a 77.5% poverty rate; this is only surpassed by Alta Verapaz and Sololá, whose poverty rates are above 80%. Around 8.9% of the population in Totonicapán lives in extreme poverty.11 The educational context in Totonicapán is described as follows: Totonicapán reflects the low levels of investment that are assigned to education in Guatemala. Only 2.9% of the Gross Domestic Product is invested nationally on education. Out of the assigned budget, 70% is used for teachers’ salaries; 29% for administrative personnel salaries and 1% on infrastructure and projects to improve the quality of education.12 4.1. Illiteracy At the beginning of the 21st century, nearly a third of the population in Guatemala over the age of 15 was unable to read and write. Between 2002 and 2011, the population over the age of 15 that was illiterate decreased from 29.3% to 17.5%. However, women’s literacy levels remain a challenge to this day. In 2011, 3 in 10 women over the age of 15, 1 in 10 women between the ages of 15 and 24, and 1 in 2 indigenous women over the age of 14, could not read or write.13 Additionally, according to the National Committee on Literacy, in 2011, the illiteracy rate in Totonicapán was 28.6%.14 In 2014, the literacy rate of men in Totonicapán was 82.2, compared to that of 62.5 for women.15 4.2. School Enrollment Totonicapán has 531 schools, out of which 450 are official primary schools with a student intake of 69,431 in 2016. Out of these, 33,620 were girls and 35,811 were boys.16 8 Ministry of Economy 2015 9 National Institute of Statistics 2013 10 Encuesta Nacional de Condiciones de Vida (ENCOVI) in Spanish 11 Ministry of Economy 2015 12 Ministry of Education n.d. 13 National Council for Urban and Rural Development (CONADUR), Presidential Secretariat for Planning and Programming (SEGEPLAN) 2014 14 National Institute of Statistics (INE) 2013 15 United Nations Development Programme (UNDP) 2014 16 Ministry of Education 2016 13 Totonicapán mirrors a worrying national trend of decreased school enrollment. In 2009, the universalization of primary school was 98.7%17 at the national level; however, it started to decrease to the point where the net enrollment rate in 2015 was 81%. This decrease in enrollment rates is also reflected in Totonicapán. Some reasons that parents gave in the 2014 ENCOVI for not sending their 7-year-old children to school included: lack of money (32%), age (24%), lack of interest (19%) and illness or disability (12%).18 Table 1. Primary school enrollment in the Department of Totonicapán YEAR Initial primary school enrollment in the department of Totonicapán | both sexes Initial primary school enrollment at the national level (both sexes) 2005 77,118 2,345,301 2006 78,313 2,405,041 2007 80,015 2,448,976 2008 81,156 2,500,575 2009 85,339 2,659,776 2010 85,914 2,653,483 2011 85,272 2,644,683 2012 81,306 2,556,314 2013 78,447 2,476,379 2014 75,539 2,417,429 2015 72,831 2,381,916 2016 69,468 2,323,954 Source: Ministry of Education. 2016. Statistical educational yearbook. 4.3. Primary school completion and dropout rates Primary school completion rates measure the percentage of students who complete primary school in the stipulated time (6 years). The opposite of school completion is school dropout. At the national level, only 60% of all children that start school in the first grade, finish the 6 th grade. An average of 7.7% of students drop out of primary school each year. The dropout rate increases from the second grade of primary school. In 2016, Totonicapán had a 2.3% dropout rate at the primary school level. 4.4. Repetition rates In 2013, INE’s Statistical Compendium of Education stated that the repetition rate in primary education was 11.2% for men and 9.2% for women at the national level. In contrast, the 2016 Statistical Yearbook indicated that in the department of Totonicapán, there is a repetition rate of 9.9%, out of which 43.1% of the students are girls and 56.9% are boys. 4.5. Pass rates The primary school pass rate refers to the percentage of students that complete and pass this level of schooling. In 2009, the pass rate was 86.4%, and in 2013, it was 86.6%. In 2016, Totonicapán had a 90% pass rate. Out of those, 51% were male students and 49% were female students. 17 Presidential Secretariat for Planning and Programming (SEGEPLAN) 2010 18 Center for National Economic Research (CIEN) 2015 14 4.6. Reading attainment in Totonicapán Assessments demonstrate that students continue to fall behind in attaining important knowledge and skills in Reading and Mathematics. In 2016 and 2017, reading attainment of students who completed the diversified cycle was 32.3%.19 The second regional comparative and explanatory study, 20 published by UNESCO in 2009, placed Guatemala as one of the last Latin American and Caribbean countries in the ranking for Mathematics and Reading test performance.21 In every single evaluation year, third and sixth grade reading achievement in Totonicapán has remained below national achievement levels. In 2014 it was 39.3% for third grade and 36.3% for sixth grade, compared to 40.4% at the national level.22 When ranking the Guatemalan departments from highest to lowest in terms of third-grade reading achievement, Totonicapán was ranked 18 out of 22 in 2014, and 16 of 22 for the ranking of sixth-grade reading achievement.23 At the municipal level, in 2014, the municipalities of San Cristóbal Totonicapán (66.9%), San Francisco el Alto (44.9%) and Totonicapán (42%) had reading attainment levels that were above the national (40.4%) and departmental (36.3%) attainment levels. The rest of the municipalities (62%) of the municipalities in Totonicapán, the reading attainment level is below that of the national level. When comparing the results obtained in 2013, there seems to be a tendency towards the improvement in the percentage of attainment.24 In the case of third-grade students, the reading attainment levels in the municipalities of San Cristóbal Totonicapán (76.8%) and San Francisco el Alto (65.1%) were above the national attainment level (49.9%). The attainment levels in both San Cristóbal Totonicapán and San Francisco el Alto, in addition to Totonicapán (46.8%) and San Bartolo Aguas Calientes (39.7%), are above the departmental attainment level (39.3%). The rest of the municipalities (50%) are below the departmental attainment level. When comparing the results obtained in 2013, it is observed that in several municipalities, the tendency is toward a slight improvement.25 V. RESEARCH PLAN 5.1. Objectives of the baseline study 1. To establish the initial conditions of the treatment and control groups and provide the basis against which the project will compare mid-term and final evaluation data. 2. To collect information to develop each indicator and refine the objectives, supporting the project team to adjust implementation, if necessary. 3. To establish the starting point within and between the six municipalities in which the McGovern￾Dole "Learning for Life" project will be implemented until September 30, 2021, to allow the team to better understand how to work with a view towards the sustainability of the project after its completion. 19 General Directorate of Educational Research and Evaluation (DIGEDUCA) 2017 20 Segundo Estudio Regional Comparativo y Explicativo (SERCE) in Spanish 21 UNESCO 2008 22 General Directorate of Educational Research and Evaluation (DIGEDUCA) 2014 23 Ibid. 24 Ibid. 25 Ibid. 15 5.2. Key research questions The main research question that guided the study on the project’s impact at the highest level was, “What is the effect of using different packages of tools and methodologies to teach Reading, on the reading comprehension level from first to sixth grade,” specifically: • What is the effect of the bilingual early reading and writing Jardín de Letras methodology on the results of the official and contextualized versions of the early-grade reading assessments in Guatemala for Spanish (ELGI) and K’iche’ (ESWUJ)? • What is the different between the results of the ELGI/ESWUJ reading tests of students enrolled in schools that implement the Jardín de Letras methodology, and the results of those schools that do not implement it? • What is the difference between the results of the official MINEDUC reading tests of third and sixth-grade students in schools that use the bilingual methodology Kemom Ch'ab'al (KC) and in those that do not use it? • What effect do other variables, such as teacher training and pedagogical accompaniment visits, have on reading comprehension skills? • How do the reading results of first-grade students vary after applying the bilingual early reading and writing methodology Jardín de Letras for two years? • What are the effects on reading comprehension and fluency of the Jardín de Letras methodology after 5 years of project implementation? • What additional effects will the use of a toolkit designed to help first-grade teachers have on the reading levels of first-grade students? These questions will be explored through a quasi-experimental design. 5.3. Baseline evaluation methodology During the 2017 school year, the 331 schools that participated in Learning for Life were grouped into 3 categories: 1. New schools where Phase I of the Project was not implemented, and that will not receive meals until the second year of Phase II, 2018 (Control Group 1-A, n = 106) 2. Phase I schools that continue to receive meals and the KC methodology (Group 2-A, n= 160). 3. Phase I schools that continue to receive meals and the KC methodology, and that have also been randomly chosen to receive the Jardín de Letras methodology (Group 2-B, n = 65). In 2018, the 331 schools have been grouped into 5 categories: 1. New schools that started receiving meals (Group 1-A, n = 53, reduced control group) 2. New schools that are receiving the complete package of intervention after mid-term evaluations; that is, meals, KC, Jardín de Letras and teacher toolkit (Group 1-B, n = 53). 3. Schools that continue receiving meals and KC (Group 2_A, n = 160). 4. Schools that continue receiving meals, KC and Jardín de Letras methodology (Group 2-B, n= 33) 5. Schools that continue receiving meals, KC, and Jardín de Letras methodology, and that will also be randomly chosen to receive the teacher toolkit intervention. 16 Figure 1. Quasi-experimental design: random control trial Source: Evaluation Plan. ‘Learning for Life’ Baseline measurement. In December 2016, AIR randomly assigned schools to the 3 categories used in 2017, based on the following criteria: 1. Schools that have not been exposed to the KC bilingual reading methodology or to the meals provided by the project, but that follow the Basic National Curriculum (106 schools). 2. Schools that have previously and continuously been exposed to meals and the KC methodology (160 schools) 3. Schools that have previously and continuously been exposed to meals and the KC methodology, in addition to the Jardín de Letras methodology (65 schools). At the start of the 2018 school year, 2 out of the 3 existing groups were randomly subdivided by AIR in the following way: • Group 1 was divided in two: Group 1-A continued as a control group but received school meals (53 schools) as per the Evaluation Plan and Group 1-B began receiving the complete intervention package. • Group 2-B was also divided into two: Group 2-B continued receiving the same interventions as in 2017 and Group 2-C began receiving the complete intervention package. Group 2-A continued under the same modality as in 2017. AIR established a sample size of n = 179 schools, using cluster sampling methods with a 95% confidence level and 80% power. Schools that make up the sample were chosen randomly by AIR. The sample of students for the official reading tests was made up of students selected according to grades as described as follows: • First grade: Spanish and K’iche’ assessment tests administered to 15 students chosen at random (if there are less than 15 students enrolled in a school, all must be tested). • Third grade: 20 students chosen at random (if there are less than 20 students enrolled in a school, all must be tested). 17 • Sixth grade: 20 students chosen at random (if there are less than 20 students enrolled in a school, all must be tested). The sample for the surveys and interviews was selected in the following way: 1. Surveys on hunger sensation (short survey) to 5 students chosen at random from each school, from first to sixth grade. 2. Quantitative and qualitative interviews to all teachers of first, third and sixth grades of the sample schools, to collect data related to the reading tests, the application of KC and Jardín de Letras reading methodologies, and the toolkit designed for teachers. 3. Surveys and direct classroom observation to 3 teachers randomly chosen from each school (where possible) from first to sixth grade. 4. Observation of all school principals of the sample schools (n = 179). 5. MINEDUC District Coordinators (DCs) and Technical Administrative Coordinators (TACs) of all of the schools participating in the project (n = 12).26 6. In each school, 4 parents or caregivers were selected at random, including those who are or are not volunteers within the project or members of PTA’s. 7. 4 volunteers were randomly selected from each school within the different components of the project (food preparation, school infrastructure works, Spaces to Grow, or others). 5.4. Planned and achieved quantitative data sample This section presents data on the achievement of samples according to the baseline study design of the Learning for Life project. The sample includes the following key informants: • Schools, as observation units • Students of first, third and sixth grades • Teachers and school principals • First-grade teachers • Fathers, mothers and volunteers (FMV) • PMV of first-grade students The sample selected a total of 179 schools, out of which 175 (97.7%) were evaluated. School principals and teachers of four schools that were part of the initial sample, did not permit the study team to raise information in those schools. The achieved quantitative data sample has a 95% confidence level (1 – α), 80% power (1 – β), and a minimal detectable effect size (MDES) of 0.299 standard deviations. Tables 2 and 3 show the total number of schools that were evaluated by the study team and their respective municipalities. Table 2. Total No. of schools evaluated, CRS Learning for Life Baseline study, by group Group Total no. of sample schools Total no. evaluated in Baseline CG 1-A 40 38 1-B 40 39 2-A 34 34 2-B 33 32 2-C 32 32 Total 179 175 Source: Database, 2018 Learning for Life Baseline Study 26 Recently the figure of TAC was changed to DC, with similar and additional functions. 18 Table 3. Total No. of schools evaluated, CRS Learning for Life Baseline study, by municipality Municipality Total number of schools in sample Total number of schools that were evaluated Momostenango 61 60 San Andrés Xecul 9 9 San Bartolo 14 14 Santa Lucía La Reforma 15 15 Santa María Chiquimula 49 46 Totonicapán 31 31 Total 179 175 Source: Database, 2018 Learning for Life Baseline Study The next table shows that there was a difference between the number of assessed students and the intended sample size. The difference was 889 first-grade students, 1,409 third-grade students and 1,615 sixth grade students. As mentioned previously, the evaluation team could not evaluate 4 of the sample schools because the teachers did not agree to the study. In addition, some of the teachers did not allow first-grade students to finish their assessments due to cultural and sport activities that were taking place, shortening of the school day due to cold weather and administrative activities in the school. In the case of third and sixth grades, the majority was not evaluated because of a lack of enrollment for these grades. For evaluation purposes, the complete tests were taken in ELGI or ESWUJ; actions were carried out to return to the field and collect pending information. Table 4. Total No. of students that participated in the study (pre-standardized testing), by treatment and control groups Group First-grade students Third-grade students Sixth-grade students Girls Boys Total Total sample Girls Boys Total Total sample Girls Boys Total Total sample CG 1-A 174 194 368 600 180 187 367 800 171 176 347 800 1-B 197 235 432 600 272 259 531 800 192 252 444 800 2-A 165 172 337 510 205 219 424 680 203 203 406 680 2-B 162 146 308 495 180 211 391 660 175 172 347 660 2-C 167 184 351 480 242 216 458 640 198 223 421 640 Total 865 931 1,796 2,685 1,079 1,092 2,171 3,580 939 1,026 1,965 3,580 Source: Database, 2018 Learning for Life Baseline Study Table 5 shows the number of survey participants. Of the total number of individuals surveyed, 79% were women and 21% were men. The difference between the sample size and the total number of individuals surveyed was 50 (n = 716, 4 individuals per school). Table 6 shows that 80% of those surveyed were mothers and 20% were fathers with children in the first-grade. The difference between the total sample size (n = 716, 4 individuals per school) and the actual sample was of 218 fathers and mothers that did participate because they were not at the school on the day the survey took place. Table 5. Total No. of parents that participated in the study, by treatment and control group, disaggregated by sex Group Women Men Total CG 1-A 102 37 139 1-B 135 31 166 2-A 109 22 131 2-B 88 21 109 2-C 91 30 121 19 Total 525 141 666 Source: Database, 2018 Learning for Life Baseline Study Table 6. Total No. of first-grade parents that participated in the study, by treatment and control group, disaggregated by sex. Group Women Men Total GC 1-A 89 25 114 1-B 84 24 108 2-A 71 13 84 2-B 84 13 97 2-C 73 22 95 Total 401 97 498 Source: Database, 2018 Learning for Life Baseline Study Table 7 shows the number of school principals that were interviewed. Of the total number of school principals of the sample schools (n = 179), 28 did not participate; some refused to participate, others were not at the school on the day of the interview. Table 7. Total No. of school principals that participated in the study, by treatment and control group, disaggregated by sex Group Women Men Total CG 1-A 12 17 29 1-B 12 25 37 2-A 20 11 31 2-B 10 17 27 2-C 10 17 27 Total 64 87 151 Source: Database, 2018 Learning for Life Baseline Study 5.5. Sample of other key stakeholders Table 8. Sample of other key stakeholders included in Baseline study Key informant Method Quantity MINEDUC District Coordinator Interview 7 Focus Groups Focus groups that included fathers, mothers or other caregivers; volunteers, members of PTA’s, teachers, principals, and key members of the community. 10 focus groups (two groups representing each treatment group and one for CG 1-A) Data from teacher observation Guide for observing teaching skills and the use of teaching methodology in the classroom 294 classrooms27 Data from school observation Guide for observing healthy environments, Access to drinking water and sanitation services, handling and storage of food, preparation and use of menus, hygiene practices 170 schools28 27 Some observations to teachers and schools could not be carried out due to union instructions. 28 Idem. 20 5.6. Methodology for the Collection and Analysis of Quantitative Data Baseline data were stratified according to: type of school, sex, economic situation, grade repetition and other factors that were considered to influence the analysis of key predictors that can impact reading performance. To maintain the comparability of the data between the different measurements, the official MINEDUC tests were used and supplemented with observation guides, interviews, focus groups and data collection guides for local and municipal registration, to obtain all the additional data needed and related to each performance indicator. The data collectors were selected after a series of interviews and technical tests to verify their knowledge of the K'iche 'language. Of the total number of applicants, those who met the following criteria were selected to participate in the training: they were from the region, had experience in evaluation, showed mastery of the K'iche 'language and had the RENAS certification. Prior to the collection of baseline data, field staff were trained in research ethics on the application of physical and digital tools to gather information on the evaluation indicators and in how to complete the official MINEDUC reading tests. The training process complied with the standardized guidelines in the MINEDUC manuals. For the reading tests, the instruments were explained, and nearby schools were selected to perform practice surveys outside the coverage area. In addition to the practice during the trainings, the validation of the other quantitative instruments used was also carried out. The methodology, language and logic of the questions were validated to later make adjustments. The official MINEDUC tests were collected as follows: i. Early Grade Reading Assessments in Spanish and K’iche’ (ELGI/ ESWUJ29) for first-grade students. ii. Reading comprehension tests for students in third and sixth grades. The survey team included a field supervisor, three first-grade data collectors, one third and sixth grade reading comprehension data collector, and one data collector for the parents. The supervisor ensured the quality of the survey process, as well as the safeguarding of MINEDUC Reading instruments, the revision of consent forms, random sampling of first-grade students, and assignment of students. The quasi-experimental design will compare the groups to find any differences that may be statistically significant. The data collected in the baseline will be compared with data from the mid￾term and final evaluations (to be collected in September 2018, September 2019 and September 2021). The data will be collected longitudinally during the baseline, midterm and final evaluations, complemented by qualitative data from focus groups, interviews with key actors, and special studies. To examine the impact of the program’s various components, a series of ordinary least squares regressions were carried out to compare results between and within groups. All regressions used standard deviation for grouped data at the school level. 5.7. Methodology for the Collection and Analysis of Qualitative Data Interviews were conducted with school principals and first, third and sixth-grade teachers from 175 schools. Two focus groups were integrated by group according to treatment (1-A, 1-B, 2-A, 2-B, 2-C) to find an explanation of the information that was found through the collection of quantitative data. 29 Tests developed by Guatemala’s General Directorate of Educational Research and Evaluation (DIGEDUCA) in Spanish and K’iche’, respectively. 21 The focus group participation criterion was for participants to be parents and/or volunteers within each school. The facilitators who collected the qualitative information received training. Before applying the guides, a pilot was carried out to validate and adjust them to the local context and language and include variables that had not been previously considered. Figure 2 shows the data collection process. Figure 2. Qualitative data collection and analysis Facilitators held meeting with participants to provide all the necessary information to obtain their participation. The Atlas.ti software, which permits the codification and organization of data by subjects and subthemes, was used, which facilitated the management and analysis of the information that was collected. The rigor and quality criteria used to order the data were: dependence, credibility, transfer and confirmation. This allowed the following codification: • Open or first-level, comparing units to create categories and codes • Axial or second-level, to compare categories that give rise to data interpretation and development of themes or patterns. • Selective, to codify the development of theories, narratives, and explanations. 5.8. Management of Confidentiality The data collection process was accompanied and supervised by parents, volunteers, teachers, school principals, and community leaders who also participated in the study, such as Local Community Council members. Prior to collecting data, the team shared the study’s objective, scope and methodology; solved any doubts and delivered a telephone number that was enabled during and two weeks after the study to the principal of each school, so that they could call and request more information or report situations that they considered improper. No such call requesting information or Facilitator training • Revision of methodology, objectives, subjects, language and rationale of the guides. Group organization • Criteria verification • Request for consent Execution of the Focus Group, interview, observation, field notebook. Facilitator training to review material • Transcription of field interviews • Discussion of preliminary results Results report • Coding • Results analysis 22 reporting any improper action was recorded. In both the focus groups and interviews, the team requested authorization and voluntary participation from the interviewees and participants, as well as their authorization to record the sessions. The Ministry of Education, through the General Directorate of Educational Research and Evaluation (DIGEDUCA, in Spanish) granted authorization to apply the official tests. During the process, DIGEDUCA and Juárez & Associates personnel provided training and accompaniment to the baseline study team. Application of the official tests required a confidentiality agreement from all people who had access to them (typography, data collectors, data entry and supervisors). All information was handled under confidential terms. A file manager was responsible for handling the MINEDUC reading tests. These were treated confidentially by all personnel involved in the study. At the end of the study, the Association for Community Organizational Development30 delivered the tests and participant databases to CRS. Additionally, they erased all information from their physical and electronic files. 30 Asociación de Desarrollo Organizacional Comunitario (ADOC) in Spanish 23 Academic and school development indicators 24 VI. ACADEMIC AND SCHOOL DEVELOPMENT INDICATORS This section describes the BL study results for academic and school development indicators related to the teaching and learning of reading. Both the skills in the teaching of the Spanish and K’iche’ language, as well as the use of new, quality teaching techniques and tools for reading and writing of both languages, are analyzed. At the same time, the section describes the level of student participation in reading and analyzes the accompaniment provided by school principals and DCs. 6.1. (Indicator 131) Percent of students who, by the end of first grade of primary school, demonstrate that they can read and understand the meaning of grade-level text. Global Indicator: 1.7% During the baseline, 862 girls (48.3%) and 924 boys (51.7%) were evaluated with the national first grade standardized tests (ELGI and ESWUJ). The first-grade indicator is measured using the DIGEDUCA’s early grade reading model, which consists of 5 linear stages of reading (see Table 10). The sum of the percentage of students that successfully reached stages 4 and 5, “Initial automated” and “Fluent” - the highest early grade reading abilities – were used to calculate the global indicator. Table 9. Students in the “Initial Automated” and “Fluent” levels of the early grade reading model: ELGI (Spanish) 1-A 1-B 2-A 2-B 2-C Global Indicator 2.8% 0.9% 3.0% 1.1% 0.9% 1.7% Source: Student ELGI test, Learning for Life 2018 Baseline Study The highest percentage of students evaluated in Spanish (with ELGI) only reached the “Emergent” and “Initial” levels (stages 1 and 2) when the test was applied (beginning of the school year). 43.8% to 58.3% of first-grade students are in the emergent level, while 36.8% to 47.4% are in the “Initial” level, which demonstrates more knowledge of letters and their formation into words, but not enough to have major decoding, fluency, or adequate reading comprehension. Table 10. ELGI Reading Levels Level Stage 1-A 1-B 2-A 2-B 2-C 1 Emergent 45.9% 55.8% 43.8% 52.3% 58.3% 2 Initial 47.4% 39.8% 45.5% 42.7% 36.8% 3 Decoder 3.9% 3.4% 7.7% 3.9% 4.0% 4 Initial Automated 2.1% 0.7% 1.2% 0.7% 0.3% 5 Fluent 0.8% 0.2% 1.8% 0.4% 0.6% Source: Student ELGI test, Learning for Life 2018 Baseline Study The ELGI test assesses eight abilities or areas, each containing subtests, (see Table 11). 31 The indicators that are in parentheses are taken from the Performance Monitoring Plan (PMP). 25 Table 11. Abilities or Areas Evaluated in ELGI No. Abilities or Areas Evaluated in ELGI 1-A 1-B 2-A 2-B 2-C Unit 1 Oral Language Proper understanding of oral instructions 72.0 74.6 83.0 82.1 82.8 Percentage of students per group with proper understanding level Understanding a passage 0.3 1.0 1.2 3.0 0.3 Percentage of students, per group, who answered all questions correctly after listening a passage 2 Knowledge of the Alphabet Knowledge of letter names 10.30 8.24 12.34 9.23 8.83 Mean of number of letter names that students know (from a total of 30 letters) Knowledge of letter sounds 14.96 12.17 19.36 15.71 14.42 Mean of number of letter sounds that students know (from a total of 30 letters) 3 Decoding Reading short words 15.5 17.3 20.5 20.8 14.9 Percentage of students, per group, who have read all the short words correctly (10 short words presented) Accuracy and speed to read meaningless words 13.8 9.7 13.0 13.6 11.3 Mean of number of meaningless words read per minute 4 Phonological Awareness Phoneme knowledge 33.2 30.0 41.2 33.8 37.1 Percentage of students, per group, with advanced level of phoneme knowledge 26 No. Abilities or Areas Evaluated in ELGI 1-A 1-B 2-A 2-B 2-C Unit Phoneme separation 4.7 2.5 8.4 6.1 4.4 Percentage of students, per group, with advanced level of phoneme separation 5 Automatic Reading Speed Accuracy and speed to name letters 17.3 16.5 18.1 16.9 17.4 Mean of number of letters students named correctly per minute Accuracy and speed to name the sounds of the letters 22.3 23.1 26.7 24.1 23.5 Mean of number of letters students named correctly by sound, per minute 6 Fluency Accuracy and speed to read familiar words 12.5 10.6 12.4 14.6 12.6 Mean of number of familiar words students read correctly per minute Accuracy and fluency when reading a passage 16.8 15.0 18.5 18.1 19.3 Mean of number of words per minute students read correctly when reading a passage 7 Reading Comprehension Reading and comprehension of a passage of text 2.76 1.16 5.38 2.09 2.01 Mean of percentage of correct comprehension answers after students read a passage 8 Writing Writing a dictated sentence 1.6 1.8 2.7 2.5 2.6 Percentage of students who wrote correctly the whole dictated sentence Source: Student ELGI test, Learning for Life 2018 Baseline Study The data suggest that in Spanish, students have basic vocabulary, know the sequence of letters in written word, and visually recognize letters or sounds. Students recognize 9.8 letters by name on 27 average (out of 30 presented), and they read 40.8 letters per minute, of which 17.3 letters are correct, and they manage to recognize the sound of 15.1 average letters (of 30 letters presented). Prior to reaching first grade, only 19.8% of the students sampled in this study attended 2 years of preschool, while 73.5% attended 1 year of preschool. A ‘concept of print’ test was also applied to first grade students. Students were asked to identify the front and back covers of the book, as well as the direction for reading the text (right to left or left to right), identification of a word or a letter. First￾graders scored low on the concept of print test (average score of 46%), which demonstrates the poor preparedness for beginning print instruction. The evaluation (standardized and ‘concept of print’ tests) was done at the beginning of the school year, so the data reflects that students are not generally exposed to print media or early grade reading skills leading up to first grade. Table 12. Number of students evaluated in ESWUJ (K´iche´) and ELGI (Spanish) Description Total Students evaluated in ESWUJ and ELGI 1786 Girls 862 Boys 924 Source: Student ESWUJ test, Learning for Life 2018 Baseline Study According to the data in the following table (Table 13), students generally scored slightly lower in K’iche’ oral language skills (ESWUJ) than in Spanish (ELGI). Table 13. Abilities or Areas Evaluated in ESWUJ No. Abilities or Areas Evaluated in ESWUJ 1-A 1-B 2-A 2-B 2-C Unit 1 Oral Language Proper understanding of oral instructions 67.6 69.7 55.4 66.7 57.5 Percentage of students per group with proper understanding level Understanding a passage 1.0 1.2 0.6 1.4 0.3 Percentage of students, per group, who answered all questions correctly after listening a passage 2 Knowledge fo the Alphabet Knowledge of letter names 8.9 7.4 11.5 7.7 8.2 Mean of number of letter names that students know (from a total of 27 letters) Knowledge of letter sounds 11.1 9.6 14.0 10.3 11.1 Mean of number of letter sounds that students 28 No. Abilities or Areas Evaluated in ESWUJ 1-A 1-B 2-A 2-B 2-C Unit know (from a total of 27 letters) 3 Decoding Reading short words 0.0 0.0 0.0 0.0 0.0 Percentage of students, per group, who have read all the short words correctly (10 short words presented) Accuracy and speed to read meaningless words 22.3 13.0 18.0 22.0 13.6 Mean of number of meaningless words read per minute 4 Phonological Awareness Phoneme knowledge 19.6 23.6 23.4 21.3 25.4 Percentage of students, per group, with advanced level of phoneme knowledge Phoneme separation 2.7 2.6 4.7 4.5 3.8 Percentage of students, per group, with advanced level of phoneme separation 5 Automatic Reading Speed Accuracy and speed to name the sounds of the letters 12.5 13.9 11.02 13.4 11.3 Mean of number of letters students named correctly by sound, per minute 6 Fluency Accuracy and speed to read familiar words 19.8 11.5 17.1 21.0 11.4 Mean of number of familiar words students read correctly per minute Accuracy and fluency when reading a passage 20.5 13.6 17.4 23.7 18.0 Mean of number of words per minute students read correctly when reading a passage 29 No. Abilities or Areas Evaluated in ESWUJ 1-A 1-B 2-A 2-B 2-C Unit 7 Reading Comprehension Reading and comprehension of a passage of text 0.3 0.2 0.7 0.4 0.5 Mean of percentage of correct comprehension answers after students read a passage 8 Writing Writing dictated words 0.0 0.0 0.0 0.0 0.0 Percentage of students who wrote correctly all the dictated words Source: Student ESWUJ test, Learning for Life 2018 Baseline Study 6.2. (Indicator 2) Percent of students (girls and boys) who, by the end of 3rd and 6th grade, demonstrate grade level reading comprehension as defined by national standards at USDA supported schools. Global Indicator: 8.29% This indicator is measured using the sum of levels 3 and 4 (“Satisfactory” and “Excellent”) of the Guatemalan Ministry of Education’s scoring matrix. In third grade, group 2-A shows the highest levels of reading comprehension (12.7%), while group 1-B (4.3% -which just started receiving literacy interventions after baseline) has the lowest reading comprehension scores. It should be noted that the third and sixth grade tests were only applied in Spanish. In sixth grade, the group with the highest percentage of “Satisfactory” or “Excellent” levels of reading comprehension was the group 2-B (10.1%), followed closely by groups 2-A (9.9%) and 2-C (9.7%). The lowest group, similar to third grade, was group 1-B (1.6%). Groups 2-A, 2-B, and 2-C had received the Kemom Ch’ab’al literacy intervention for four years prior to the baseline application, which may explain why these groups scored higher in reading comprehension than groups 1-A (control) and 1-B (began receiving Kemom Ch’ab’al after baseline). 30 Graph 1. Third-grade student reading performance, according to sex and treatment group Source: Student Tests, Learning for Life 2018 Baseline Study Graph 2. Sixth-grade student reading performance, according to sex and treatment group 38% 34% 37% 37% 29% 28% 33% 28% 31% 40% 52% 55% 58% 60% 58% 59% 57% 65% 58% 48% 9% 9% 5% 3% 11% 10% 9% 7% 10% 11% 2% 2% 0% 0% 1% 2% 0% 0% 1% 1% 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% Male Female Male Female Male Female Male Female Male Female 1-A 1-B 2-A 2-B 2-C Unsatisfactory Must improve Satisfactory Excellent 41% 40% 35% 45% 29% 31% 33% 28% 26% 23% 53% 55% 65% 53% 60% 61% 58% 62% 64% 67% 5% 5% 1% 2% 9% 7% 8% 9% 7% 9% 1% 0% 0% 1% 2% 1% 2% 2% 3% 2% 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% Male Female Male Female Male Female Male Female Male Female 1-A 1-B 2-A 2-B 2-C Unsatisfactory Must improve Satisfactory Excellent 31 Source: Student tests, Learning for Life 2018 Baseline Study 6.3. (Indicator 22) Number of students enrolled in schools receiving USDA assistance. Global Indicator: 48,664 The following table shows beneficiary students that receive support from USDA. The total number of students in Phase II includes those in Phase I. Table 14. Number of Learning for Life project beneficiaries No. of Learning for Life student beneficiaries Women Men Total First Phase, 2016 school year 16,394 16,790 33,184 Second Phase, 2017 school year 23,863 24,801 48,664 Source: Learning for Life Project records, Phase 2 Database 6.4. (Indicator 20) Number of students regularly (80%) attending USDA supported classrooms/schools. Global Indicator: 42,727 The baseline included the analysis of academic performance indicators such as: number of students that regularly attend schools/classrooms supported by USDA, and students’ attendance in schools supported by USDA, disaggregated by sex. 6.4.1. Attendance according to the Learning for Life phase I final evaluation According to the data collected in final evaluation of Learning for Life phase 1, conducted in 2017, 52.6% of the teachers kept records of male student attendance, and 50.8% kept records of female student attendance. The percentage of attendance according to the available records in the final Learning for Life evaluation was: • Third-grade boys = 56.4% • Sixth-grade Boys = 50.5% • Third-grade girls = 84.7% • Sixth-grade girls = 85% 6.4.2. Attendance according to teacher’s perception, Learning for Life phase II baseline Table 15. Teacher perception on classroom attendance32 Perception of teacher CG1-A 1-B 2-A 2-B 2-C Regular 80-100% 87.9 80.6 84.1 93.2 94.4 Usual 60-79% 12.1 16.4 11.1 4.5 3.7 No answer 0 3.0 4.8 2.3 1.9 Total 100 100 100 100 100 Source: Teachers’ surveys, Learning for Life 2018 Baseline Study 32 This indicator was not disaggregated by sex because it was taken from the teacher's perception, as they did not have attendance records. 32 One of the challenges of the education system at the national level is the registration and monitoring of students' regular school attendance. The Ministry of Education (MINEDUC) has established 180 days of classes per year; however, it is difficult to verify how many actual days children attend school due to the lack of attendance records and the constant suspension of classes due to holidays, extracurricular activities of children and teachers, teacher strikes, and cultural factors. Furthermore, only MINEDUC has the authority in Guatemala to demand regular student attendance records. The study compared teachers' perception of students’ school attendance. The results show that there is greater school attendance of children in groups 2-B and 2-C, compared with CG 1-A. However, after running the Chi-square tests for each of the groups, it was observed that these differences are not statistically significant. Group compared against CG 1-A Significance value 1-B 0.435 2-A 0.937 2-B 0.187 2-C 0.103 Source: Teachers’ survey, Learning for Life 2018 Baseline Study 33 6.5. (Indicator 9) Number of teachers/educators/teaching assistants in target schools who demonstrate use of new and quality teaching techniques or tools as a result of USDA assistance. Global Indicator: 24.5% Number: 499 teachers 6.5.1. Teachers’ language skills in Spanish and K’iche’ In all groups, 100% of the teachers had sufficient command of reading in the Spanish language. However, in group 2-C, 1.9% of the teachers did not have oral command of Spanish. For the reading comprehension in Spanish, teachers in group 2-A did not reach the 100% mark (see Graph 3). No group has 100% command of the K’iche’ language, and it is the variable with the lowest score for writing. The group containing the most teachers with written command of K’iche’ is 2-B with 77.3%; and group 2-C is the one with the lowest command of writing, with 61.1%. In the case of reading in K’iche’, Group 2-A is the one with the most teachers than can read in K’iche’ (87.3%), and group 2-C is the group with the least amount of teachers that read the language (81.5%). No group has 100% oral understanding of K’iche’; the group with the highest oral understanding was group 2-C with 96.3%, while group 1-B had the lowest percentage at 92.4% (see Graph 4, Percentage of teachers with K’iche’ language skills). Based on the results, the Spanish language is spoken, read, and understood better than K’iche’. Graph 3. Percentage of teachers with Spanish language skills Source: Teachers’ survey, Learning for Life 2018 Baseline Study Graph 4. Percentage of teachers with K’iche’ language skills Source: Teachers’ survey, Learning for Life 2018 Baseline Study 0.0% 20.0% 40.0% 60.0% 80.0% 100.0% CG 1-A 1-B 2-A 2-B 2-C Reads Spanish Speaks Spanish Writes Spanish Understands Spanish 82% 85% 87% 82% 82% 85% 90% 91% 91% 76% 70% 75% 73% 77% 61% 92% 93% 94% 93% 96% CG 1-A 1-B 2-A 2-B 2-C Reads K'iche' Speaks K'iche' Writes K'iche' Understands K'iche' 34 6.5.2. Teachers demonstrate use of new and quality teaching techniques and tools Group 2-A reports the highest percentage of use of the following teaching tools: modeling and exploring the reading material; reading and modelling of vocabulary; performing and reflecting on the exercises. However, it is CG 1-A that shows the highest percentage for using the fourth tool – implementing group reading. Within groups 1-B, 2-B, and 2-C, group 2-B models and explores, reads and models vocabulary, performs and reflects on exercises at a lesser percentage. Table 16. Percentage of teachers that demonstrate use of teaching techniques and tools Variable CG 1-A 1-B 2-A 2-B 2-C Teachers that model and explore reading materials 59.1 79.1 90.5 70.5 85.2 Teachers that read and model vocabulary 47.0 73.1 81.0 56.8 72.2 Teachers that perform and reflect on the exercises 53.0 70.1 77.8 63.6 74.1 Teachers that implement group reading 59.3 40.9 44.4 44.8 45.5 Source: Teachers’ survey, Learning for Life 2018 Baseline Study According to the data, teachers in all groups use less than 60% of the different reading techniques. Though teachers seem to know the steps towards preparing a reading class, most of them do not use the techniques and tools. More than 70% have materials in both Spanish and K’iche’. Graphs 5 and 6 show the results. Graph 5. Techniques used by teachers in their reading class, by percentage Source: Teachers’ survey, Learning for Life 2018 Baseline Study 30 37 30 39 44 53 52 51 52 52 46 45 44 41 59 0 20 40 60 CG 1-A 1-B 2-A 2-B 2-C Group discussion One-on-one reading support Group reading support 35 Graph 6. Tools used by teachers in their reading class, by percentage Source: Teachers’ survey, Learning for Life 2018 Baseline Study Up to 59.3% of the teachers and principals in all groups indicated that the tools are useful and provide them with resources for teaching students how to read. Group 2-C shows the highest percentage of teachers and principals in favor of the methodologies. The results are shown in the following graph. Less than 11% of teachers and principals indicated that the reasons for not using the tools are: ‘they are hard to implement in the reading class’ (1-B, 1-A, 2-C), ‘they do not adapt to the students’ context’, and ‘I have the materials but do not know how to use them.’ Graph 7. Reasons given by teachers and school principals for using/not using Learning for Life methodologies, in percentages Source: Teachers’ survey, Learning for Life 2018 Baseline Study 0 10 20 30 40 50 60 Use of stories for reading Use of posters for learning to read Use of cards and games for learning to read Use of a student workbook Use of a reading textbook CG 1-A 1-B 2-A 2-B 2-C 0 20 40 60 80 100 They are useful in teaching how to read They provide me with resources for teaching how to read They are hard to implement in my reading class They do not adapt to my students' context I have Learning for Life materials but don't know how to use them I do not like to use Learning for Life materials CG 1-A 1-B 2-A 2-B 2-C 36 Focus group information that complements the analysis The treatment group teachers that participated in the focus groups provided possible reasons why their peers do not implement techniques to promote reading. These are summarized in Figure 3. Figure 3. Reasons why teachers apply or do not apply the techniques Source: Teacher Focus Group Discussions, Learning for Life 2018 Baseline Study 6.5.3. Teachers trained by CRS on the Learning for Life Methodologies Graph 8 shows that groups 2-B, 2-A, and 2-C report the most training in Learning for Life. These groups started in Phase 1 of the program. In the case of Group 1-B (74.6%) began in 2018. CG 1-A is the one with the fewest teachers that have been trained (12.1%). The percentage of training reported by the Control Group may reflect social desirability (answering what is thought to be correct). Graph 8. Percentage of teachers that have received training in Learning for Life Source: Teachers’ survey, Learning for Life 2018 Baseline Study Teachers' interest •"Applying reading techniques depends more on the interest of teachers than on the resources available or the training received." Teachers' abilities •"Many teachers apply these techniques and other actions to improve the performance of children who have learning difficulties." Material adequacy •"Workshops sometimes interrupt the curriculum because they are time￾consuming." 12.1 74.6 88.9 90.9 87 0 20 40 60 80 100 CG 1-A 1-B 2-A 2-B 2-C 37 6.5.4. Learning for Life methodologies known by teachers According to the results in Graph 9, the Kemom Ch’ab’al reading methodology is the one with which teachers in all the groups are most familiar with. Control Group 1-A has the least knowledge of this methodology; teachers mentioned that they are familiar with it, though they have not been trained in its implementation. Data shows a statistically significant correlation of 0.667 between knowing and implementing the methodology. Graph 9. Percentage of teachers that know Learning for Life methodologies Source: Teachers’ survey, Learning for Life 2018 Baseline Study Knowledge of the Jardín de Letras methodology for groups 1-B, 2-B and 2-C is very similar. Although the recognition is not very high (less than 48.8%) these groups receive the Jardín de Letras program. In contrast, group 2-A, which has not received training on the Jardín de Letras program, reports a knowledge level of 8.6%. In the case of the formative evaluation kit for first grade teachers, the percentage of knowledge is lower than that of other methodologies. This is a program that started in 2018 and that seems to have been more socialized in group 2-C. (By 2018, the schools in group 1-B and 2-C are the only ones that receive the treatment of the kit, totaling 85 schools between both groups. When the baseline data was collected, the Learning for Life project had not yet completed the trainings for use of the kit). Groups that do not participate in the interventions have some knowledge of them. When asking the teachers about the Learning for Life methodologies they were familiar with, the options were not read. It can be inferred that the teachers of GC 1-A are familiar with the methodologies through other sources such as other teachers, principals, and/or MINEDUC DCs. In the focus groups, they also mentioned transfer of knowledge among peers. Another factor is high teacher turnover, which may place teachers who have already participated in phase I of the project, in phase II schools. 35 35 12 83 46 4 91 9 19 88 49 12 92 39 31 0 20 40 60 80 100 Kemom Ch’ab’l Letter Garden Teacher's kit CG 1-A 1-B 2-A 2-B 2-C 38 6.5.5. How teachers became familiar with the Learning for Life methodologies Teachers report in different groups that they have learned about the Learning for Life methodologies mainly through CRS and its implementing partners. Some report in lower percentages that they have learned about the methodologies through a friend or another teacher. CG 1-A is where there is less knowledge of the program’s methodologies (see the following graph). Graph 10. How teachers became familiar with the Learning for Life methodologies (percentages) Source: Teachers’ survey, Learning for Life 2018 Baseline Study 6.6. (Indicator 8) Number of textbooks and other teaching and learning materials provided as a result of USDA assistance 6.6.1. Spanish and K’iche’ bilingual teaching practices In the classroom, the baseline study team observed the use of materials and tools promoted by the Learning for Life programs in Spanish and K’iche’, such as lettered classrooms, tools in Spanish and K’iche’, a hygiene corner33 and attendance record notebook. Table 17 shows that the tools in Spanish are the most used in the classroom, while hygiene corner tools are the least used. Table 17. Percentage of materials and tools promoted by Learning for Life Materials and tools CG 1-A 1-B 2-A 2-B 2-C Lettered classroom 69 64 76 77 71 Tools in Spanish 67 77 82 71 87 Tools in K’iche’ 53 77 79 74 90 Hygiene corner 50 51 52 52 52 Attendance notebook 81 87 91 84 81 Source: Classroom observation. Learning for Life 2018 Baseline Study The previous graph also shows that there are no significant differences between groups regarding the use of materials and tools. Groups 2-C and 2-A show the greatest use of materials and tools 33 Hygiene Corner is a space within the classroom where there are resources that facilitate the students’ personal hygiene, such as toothbrushes, soap, towels, etc. 46 76 89 91 91 5 15 8 9 3 4 0 0 0 0 11 15 3 0 0 15 0 0 0 19 0 20 40 60 80 100 CG 1-A 1-B 2-A 2-B 2-C CRS MINEDUC Another teacher Director Friend 39 overall. This could be explained by the fact that they participated in Phase 1 of the program. In the case of CG 1-A, the materials observed in the classroom are not from Learning for Life. Finally, Group 2-A had the highest percentage of classrooms with their attendance record notebook (91%) on the day of the study, while CG 1-A had the lowest (81%). Focus group information that complements the analysis Male and female teachers from the Xetená, Cipreces, Chicorral San Antonio schools were consulted in focus groups. All agree that “yes, they use the program’s tools” and “yes, they have been trained in their use.” Figure 4 shows the comments grouped by categories. Figure 4. Comments regarding teaching resources Source: Teacher Focus Group Discussions, Learning for Life 2018 Baseline Study In the case of CG 1-A, teachers from the Chimente and La Concordia schools were consulted in focus groups. Teachers agree that they have received some training on teaching methodologies (from MINEDUC and other organizations) and consider important that they receive additional training from the Learning for Life project. They said, “We would use the methodology if we were trained.” 6.6.2. Teachers that know the steps to prepare their reading classes A large percentage of teachers from Groups 2-A and 2-C, which are two of the three groups of schools that participated in Phase I, mentioned that they know the key moments of preparation for the reading class. According to how the information was collected, it is not possible to determine whether the teachers know about the preparation steps through the MINEDUC or through the Learning for Life project. There are some noticeable differences between groups in terms of the “before” stage and knowledge of the steps. For example, group 1-B, which started in the year 2018, shows greater knowledge than group 2-B, which started in 2017. This may be attributable to the context of the communities in this group, since Santa María Chiquimula is supported by other organizations with a focus on education. In the case of CG 1-A, less than 60% of the teachers know the key moments in the planning of a reading class (see Graph 11). Content •"This is good material, these are practical tools and it is a guide, but we need reinforcement." •"The guides are good because they provide answers." •"It is necessary to contextualize the books' content." •"The Learning for Life material helps us to improve our teaching abilities." Training •"Training has helped teachers to interpret the tools." •"It is important to learn about the new teaching tecniques/materials that can be applied to improve reading." •"The first training sessions were better; the second ones were repetitive." Support in the K'iche' language •“We need a workshop in K'iche', to better know the words, because sometimes we do not understand." •"We generally don't have material for teaching in K'iche'. The project has provided us with the material so that we can teach the language." Follow-up and accompaniment •"There is a lack of monitoring on the use of the tools." •"The time to implement the methodology did not adapt to the class schedule, various periods are needed." 40 Graph 11. Percentage of teachers that know about the steps to prepare their reading classes Source: Teachers’ survey, Learning for Life 2018 Baseline Study Focus group information that complements the analysis Teachers were consulted through focus groups. Some interesting comments they gave at the Chimente and Patulup schools were: "In class, we help children read books and correct them when they get confused; these strategies have been quite successful. Now there are children who are excellent at reading." In CG 1-A, teachers mentioned that they implement strategies to improve their skills in teaching reading, though they do not have a defined methodology. They indicated that, if they were to receive training, they would implement the Learning for Life methodology. They also mentioned that currently, teachers design their own strategies to achieve better student reading results. 6.6.3. Available resources in Spanish and K’iche’ for teaching reading Graph 12 shows that CG 1-A has less availability of bilingual resources than the treatment groups. The different treatment groups have reading workbooks in Spanish and K’iche’. Group 2-C has the most of these resources (98.1%) and CG 1-A has the least (43.9%). In the case of the Spanish workbooks and teaching guides, CG 1-A has more resources than the rest of the groups. Similarly, the percentage of teaching guides in Spanish and K’iche’ is significantly greater in the treatment groups than in the control group. With regards to the teaching guides that are written only in K’iche’, all the groups are below 6%. Most of the treatment groups have material from the KC methodology; this may be associated with the fact that groups 2-A, 2-B, and 2-C participated in Phase I of Learning for Life and have kept their material. In comparison, group 1-B started in 2018, and CG 1-A does not have this methodology (see Graph 12). During data collection, it was found that the project was delivering material to the different schools in the treatment groups. 59 79 91 71 85 47 73 81 57 72 53 70 78 64 74 8 13 24 23 22 0 20 40 60 80 100 CG 1-A 1-B 2-A 2-B 2-C Before (modelling the strategy and exploring the reading) During (reading and defining vocabulary) After (doing and revising exercises, reflection) Other 41 Graph 12. Percentage of teachers that indicate the resources available to them for teaching how to read in Spanish & K’iche’ Source: Teachers’ survey, Learning for Life 2018 Baseline Study In the case of the Jardín de Letras methodology, the following graph shows that, in group 2-A, only 6% of the teachers declare to have the material. This is because this group does not have this methodology included in its treatment, as in CG 1-A. The other treatment groups show 30% availability of this methodology. The teachers were asked specifically about the project material. It can be inferred that teachers in the control group have the material for the following reasons: teaching personnel turn-over, acquisition by another teacher, school principal or Departmental Coordinator. The formative evaluation kit for first grade teachers is a program that began in 2018. For this reason, teachers seem to have little access to the material. In both CG 1-A and group 2-A, which do not have this treatment, some teachers indicated that they had access to it (possibly because they were confused with the kit provided by MINEDUC, also for first grade). The teachers were specifically asked about the methodologies of the Learning for Life project. It should be considered that in the context where the project is developed people is very well organized in the community level, so the transfer of resources and knowledge is possible. Graph 13. Percentage first, third, and sixth-grade teachers that indicate availability of teaching material in K’iche’ and Spanish for each Learning for Life Methodology Source: Teachers’ survey, Learning for Life 2018 Baseline Study 0 10 20 30 40 50 60 70 80 90 100 Books in K'iche' Guides in K'iche' Books in Spanish Guides in Spanish Books in Spanish and K'iche' Guides in Spanish and K'iche' 2-C 2-B 2-A 1-B CG 1-A 14 72 91 82 89 9 39 6 39 30 5 8 16 25 26 0 20 40 60 80 100 CG 1-A 1-B 2-A 2-B 2-C Kemom Ch'ab'al Letter Garden Teacher's kit 42 Table 18 shows that groups 1-B and 2-C indicated that they implemented in a higher proportion the components of a culture of peace and human rights, gender equality, logical thinking, and bilingualism and interculturality, in the reading learning material. The culture of peace and human rights and gender equality components are implemented the most. These components are related to the promotion of children’s rights at the school level. The components of logical thinking and bilingualism and interculturality are the least included in the reading material. Group 1-B stands out: though it started in Phase II of the project, it already implements 58% of logical thinking material in the reading. In the case of the bilingualism and interculturality component, all treatment groups are in similar conditions, with CG 1-A being below the average of the other groups by more than 7 pp. Table 18. Percentage teachers that indicate the use of components in the reading material Component CG 1-A 1-B 2-A 2-B 2-C Culture of peace and human rights 32 58 52 50 63 Gender equality 27 69 52 48 63 Logical thinking 24 58 27 18 41 Bilingualism and interculturality 14 33 30 21 32 Source: Teachers’ survey, Learning for Life 2018 Baseline Study Focus group information that complements the analysis The teachers that participated in the focus groups believe that one of the phenomena facing bilingual education is the loss of interest in preserving and learning the native language, K’iche’. Teachers from the schools in Chimente, La Concordia and Xetená agree that 50% of the children in the classrooms speaks K’iche’, while the other 50% speaks Spanish. In the case of children who speak K’iche’, the effort to read in this language is greater. Children speak and understand the language very well, but they cannot read or write it (see Figure 5. Phenomena affecting bilingual education). Teachers in CG 1-A say it takes more effort for students to read in K’iche’. Figure 5. Phenomena affecting bilingual education Source: Teachers’ surveys and focus groups, Learning for Life 2018 Baseline Study 6.6.4. Activities that teachers carry out after reading Teachers were asked about the main activities that they carried out in their class after reading, considering what the methodology of the Learning for Life project promotes. According to the data, figure 14 shows the percentage of activities that teachers perform after reading. These are: group reading comprehension, over 61.1% for treatment groups except Group 1-B, which is at 59.7% and CG 1-A, 56.1%. Slightly above 60% perform individual reading comprehension, which that is the activity most implemented in CG 1-A (68.2%). Loss of interest in preserving and learning the native language. “There is a phenomenon occuring in the communities. The mother tongue is now Spanish." “Parents demand the students speak more Spanish, [but] the biggest challenge is to return to studying and teaching K'iche'." “Bilingual teaching has been hard; when we ask questions, they do not answer." 43 The reading reinforcement activities that are least implemented are: Asking questions to verify reading comprehension (performed by less than 44% and, in the case of group 1-B, only 11.9%). Less than 30% reported assigning homework to reinforce reading. 50% of the teachers practice filling out exercises in the notebook as part of the work their students must do in the classroom, but when compared to the practice of asking questions, a lower percentage is observed, which may indicate the lack of feedback that students receive on their learning. Graph 14. Percentage of teachers that perform activities after reading Source: Teachers’ surveys, Learning for Life 2018 Baseline Study Focus group information that complements the analysis In the focus groups, teachers of the Chumumús school indicated that they carry out follow-up activities for reading, and offered the following opinion: “Since Learning for Life trained us on the Kemom Ch’ab’al and Jardín de Letras literacy methodologies, we have used the strategies of these methods. However, when there are children with reading difficulties, in addition to these strategies, we give them individual support, we read with them, and we also assign a child leader with developed reading skills to read with them and provide accompaniment. We also talk to their parents so that, at home, they motivate them to read. Sometimes we lend them the Kemom Ch’ab’al books because it stimulates them to truly read.” When consulted, teachers in CG 1-A said, “the only activity we do is correcting the child when she/he gets confused.” Another interesting opinion from this group is “Unlike other schools, in the community we do not have the reading spaces promoted by PRODESSA. We have been told that facilitators paid by PRODESSA are the ones who attend to these centers, the children go there every afternoon to do the homework they are given at school and to continue reading. They tell us that these centers are filled with children, although it seems that they can only fit twenty. We hope that in the future they will also be implemented here in our school, it would be a great support for us.” This comment is probably referring to the extracurricular program Spaces to Grow34 implemented each year by the project in a limited number of schools. Currently, the program is carried out only in Groups 2-A, 2-B, and 2-C schools; not in CG 1-A. 34 Espacios para Crecer, in Spanish 0 20 40 60 80 Group exercises Individual exercises Filling in the workbook Homework Teacher asking questions Finishing the class Does not do anything CG 1-A 1-B 2-A 2-B 2-C 44 6.6.5. Student participation in the reading activities. As shown in the following graph, most of the teachers in all groups believe that there is no difference between girls and Boys with regards to their participation in reading activities. Graph 15. Percentage of student participation in reading activities by sex, according to teachers Source: Teachers’ surveys, Learning for Life 2018 Baseline Study Focus group information that complements the analysis Teachers consulted in focus groups confirm that both boys and girls participate equally in reading activities. The main reasons why many children do not participate in reading activities, according to the teachers, appear in figure 6. Figure 6. Reasons why girls and boys do not participate in reading activities Source: Teacher Focus Group Discussions, Learning for Life 2018 Baseline Study Teachers also indicated that the Learning for Life methodology has helped students to become more interested in learning to read and participating more in reading activities: • “Children like to read. When reading time is over, they want to keep reading, more so after we finish the comprehension exercises.” • “Children have fun with reading, they like to identify the characters in the story and what they were doing.” • “It is gratifying for us as teachers to see how the children come out reading fluently at the end of each school year, not to mention the other grades, because their reading is also strengthened.” 0 20 40 60 80 100 Girls Boys The same for girls and boys CG 1-A 1-B 2-A 2-B 2-C Parents do not collaborate because of family disintegration or alcohol problems. Students' personalities and bullying in the classroom. “We have met children who do not want to participate in reading activities, because sometimes other children laugh at them." 45 6.7. (Indicator 12) Number of school administrators and officials in targeted schools who demonstrate use of new techniques or tools as a result of USDA assistance. Global Indicator: 40.4%; Number: 134 school principals To calculate the indicator score, three dimensions were used and, within them, the following variables weighted: • Knowledge of reading techniques: a. key preparation moments; b. teaching techniques used; and c. what do you do after a reading activity? • Nutrition knowledge: knowledge of food groups. • Management: a. frequency of planned activities; b. occurrence of school plan; and c. Frequency of school monitoring. The percentage of school principals and officers that demonstrated the use of new teaching techniques and tools is 40.4%. 6.7.1. Schools with improvement plans Over 81.5% of principals indicated that their schools have a school improvement plan. There is no significant percentage difference between the groups. Graph 16. Percentage of schools with school improvement plans Source: Principals’ surveys, Learning for Life 2018 Baseline Study According to the following graph, over 96.3% of principals implement planning and coordination activities in their schools. In Group 2-C and CG 1-A, 100% indicated they have implemented these activities. Graph 17. Percentage of schools that implement planning and coordination activities. Source: Principals’ surveys, Learning for Life 2018 Baseline Study 8 93.1 89.2 93.5 81.5 96.3 0 20 40 60 80 100 CG 1-A 1-B 2-A 2-B 2-C 100 97.3 96.8 96.3 100 0 20 40 60 80 100 CG 1-A 1-B 2-A 2-B 2-C 46 6.7.2. Monitoring of school improvement plans Between 60.8% and 62.9% of school principals indicated they monitor their school improvement plans every trimester or less. Those in the treatment groups who indicate they monitor in periods of 4 and 12 months are between 39.4% and 54.5%. CG 1-A is at 33.3%. Annual monitoring occurs in the range of 18.2% to 30.8%. Data shows that schools have a school improvement plan, but do not perform timely monitoring to be able to make decisions or to improve management. Graph 18. Periodicity with which schools monitor their improvement plan, in percentages Source: Principals’ surveys, Learning for Life 2018 Baseline Study 6.8. (Indicator 24) Percentage of municipal education supervisors from the Totonicapán Department of Education that have increased their capacity to fulfill their roles and responsibilities. Global Indicator: 0% The figure of Technical Administrative Coordinator (TAC) was changed to District Coordinator (DC), with both similar and expanded functions. As part of the study, seven supervisory DC representing the six project municipalities were interviewed. All the respondents indicated they implement methodologies and tools for the teaching of reading. One of them referred to an action they are implementing to provide accompaniment: “In Momostenango, we have organized a commission called the Learning Community, formed by some principals and teachers. We meet at four in the afternoon in the office of the TAC (currently DC) every day. We talk about the progress and existing problems in each district, and this is the way we prioritize the schools’ accompaniment, because more than supervision, we provide accompaniment.” Additionally, 100% of the DC indicated that their functions are more administrative in nature. The principals are the ones responsible for providing technical and pedagogical accompaniment. However, despite their administrative functions, the DC indicated that one of their priorities is the supervision of first-grade teachers (see the following quotes). They informed that they do not have an established number of visits to the schools. Some monitor via phone calls, as they are absorbed by their administrative responsibilities. According to the DC, it is priority for them to accompany the schools according to the urgent needs and requirements of the MINEDUC. 0 10 20 30 40 Yearly Every 6 months Every 4 months Every 3 months Every month Weekly CG 1-A 1-B 2-A 2-B 2-C 47 • “As TAC (DC), I agree that the criteria for visiting schools are determined according to the needs and specific requirements of the principals, or based on some pilot projects that are being implemented and that are of utmost interest to the MINEDUC…” • “The TACs (DCs) of the six municipalities agree that we have a strong administrative burden that limits our ability to carry out our assessment or accompaniment activities to principals and teachers; our visits to the schools are not very frequent. Some teachers are happy that we do not arrive, other expect our visit. Sometimes, we may go to a school and we must cancel the visit due to urgent administrative requirements, so we have no other option but to provide advice via telephone. Our priority is to supervise first-grade teachers, so that they maximize their skills and abilities. They are given technical/pedagogical recommendations and the principal is responsible for following up on them.” • “Because first grade is our priority, we strive to focus on them and provide technical, pedagogical and methodological support. We advise because of their needs and according to their needs; it is not an audit, as some teachers see it.” • According to the DCs, the challenges to bilingual education are lack of materials, the dialectal variants of the K'iche’ language and staff rotation. Therefore, they have been supported by Learning for Life technical team to cover those needs. The following figure describes the challenges identified by the DCs mention about their bilingual context. Figure 7. Challenges identified by the DCs Source: DC interviews, Learning for Life 2018 Baseline Study Below are some quotes that support the challenges mentioned in the previous figure: • Implementation of the methodology o "One of the main challenges to the achievement of bilingual reading quality is the correct application of the Kemom Ch'ab'al and Jardín de Letras methodologies. There are manuals, but that is not enough, it is also important for this to be complemented with constant teacher trainings; Both they and we need constant reinforcement in both methodologies that place so much emphasis on K’iche’ as well as on Spanish.” • Access to the material o “Another no less important challenge is to replace all materials that are worn-out, and to ensure that there is enough material so that each child has a set of both methodologies for personalized use.” o “The books that are donated by MINEDUC are too few in relation to the school population; the Kemon Ch’ab’al books are urgent needed.” o “It is urgent that CRS/Santiago Development Project (PRODESSA, in Spanish) send specialists to train teachers in K’iche’ to strengthen their skills in this language.” Implementation of the methodology •Correct implementation •Teacher training Access to the material •Replace worn-out material •Have enough material •Have more books Loss of the native language •Teachers do not master it •Children do not speak it •Parents ask for Spanish 48 • Loss of the native language o Both the DC group as well as other groups coincide that teachers have lost their K’iche’ language, which hinders the students in developing reading skills in this language. o “This also makes it difficult for teachers to apply the Learning for Life methodologies due to the different variants according to context.” • Accompaniment o Although the DC see accompaniment of the project technical team as very positive, they also identify a poor coordination with them, primarily because of lack of time. They also indicated that they do not receive Monitoring and Evaluation reports. o “There is scarce coordination with Learning for Life; they regularly communicate with the principals, but they have very little time to communicate with us. Sometimes our availabilities do not match. For two years we haven’t received any monitoring reports from the PRODESSA/CARITAS technical team, though they say schools are doing a good job.” The DCs were consulted in focus groups about the contribution that the Learning for Life project has had in the improvement of the schools and the performance of the students. Below, some illustrative comments are transcribed. "Before the Learning for Life program reached schools, the repetition rate was maintained at 15%, this indicator has been modified thanks to the interventions of the program. It is important to mention the food program promoted by CARITAS, which is a determining factor because it not only benefits the children but also involves the parents as being co-responsible for the education of the children. No less important has been the provision of bilingual materials and the accompaniment of pedagogical technicians of CRS / PRODESSA. Learning for Life helps to improve the quality of life and contributes to improving the quality of education.” "District No. 6-26 (highlands of municipality of Totonicapán, mostly rural area) had been marginalized for many years because it is in a remote part of Totonicapán, Learning for Life has been supporting us not only in the pedagogical part, but also in school infrastructure, and this motivates us to continue. CRS through CARITAS / PRODESSA is supporting us a lot. PRODESSA is focused on three communities according to their level of need (Pachoc, Coxom and San Ixnam) and has three technicians assigned to provide accompaniment to students for the tasks of school reinforcement, which includes 30 children per school that are selected to be served by a teacher. In Spaces for Growth (Espacios para Crecer) they provide reinforcement, with the slogan ‘Education with cultural and linguistic relevance’". Conclusions: Academic and School Development 1. Percent of students who, by the end of first grade of primary school, demonstrate that they can read and understand the meaning of grade-level text. In general, all of the groups studied in first grade present similar levels of achievement using the Ministry of Education’s early grade reading model (global indicator: 1.7%), which is to be expected as the tests are designed to be applied at the end of the school year rather than the beginning of the school year. The bulk of the students were still working on “Emergent” and “Initial” reading skills when the baseline was applied. Students perform slightly better in Spanish early grade reading abilities than in K´iche’, though the cause has not yet been determined. 49 2. Percent of students (girls and boys) who, by the end of 3rd and 6th grade, demonstrate grade level reading comprehension as defined by national standards at USDA supported schools. Overall, the percentage of students that demonstrated they can read and understand a grade-level text according to the standards established by the Ministry of Education, was higher in third grade than in sixth grade, suggesting that something may be happening between third and sixth grade of primary school in regards to student attentiveness, attendance, teacher performance, or other possible factors. This will be explored more in depth after September’s data analysis to see if there are correlations between student reading abilities and associated factors. In third grade, Group 2-A showed the highest levels of reading comprehension, while group 1-B (which just started receiving literacy interventions after baseline) had the lowest reading comprehension scores (see Graph 1). In sixth grade, the group with the highest percentage of “Satisfactory” or “Excellent” levels of reading comprehension was the group 2-B (10.1%), followed closely by groups 2-A (9.9%) and 2-C (9.7%). The lowest group, similar to third grade, was group 1- B (1.6%). Groups 2-A, 2-B, and 2-C had received the Kemom Ch’ab’al literacy intervention for four years prior to the baseline application, which may explain why these groups scored higher in reading comprehension than groups 1-A (control) and 1-B (began receiving Kemom Ch’ab’al after baseline). 3. Number of teachers/educators/teaching assistants in target schools who demonstrate use of new and quality teaching techniques or tools as a result of USDA assistance, and number of school administrators and officials in target schools who demonstrate use of new techniques or tools as a result of USDA assistance. Results showed that 100% of teachers can speak and read in Spanish; however, no group had teachers that were able to speak, read or write fluently in K’iche’. Despite the schools being categorized as bilingual, there is a predominance of teaching in the Spanish language. Most teachers recognize that Kemom Ch'ab'al methodology implemented by the project is effective and helps improve literacy; the teachers of the treatment groups who have not been trained also mentioned the methodology. Regarding the Jardín de Letras methodology, one third of the teachers of CG 1-A were familiar with the methodology, but used it infrequently. In summary, despite the cross-over that could occur between groups, the highest percentages are in line with the research design. Teachers positively qualify the tools and material that the Learning for Life project has given them; however, they expressed that they need reinforcement on their use, especially for all the material in the K'iche' language. The challenges that teachers identify to teach the K'iche 'language are that students cannot read or write it, coupled with teachers not mastering the K'iche' language. In the classroom, 50% of children speak and understand K'iche 'and the other 50% speak only Spanish. Regarding barriers with the methodology, teachers expressed that the material in K’iche’ does not meet all the linguistical variants in each municipality and are not easily adapted to the context, which limits its use and application. Furthermore, there are teachers who come from other municipalities, which limits their ability to teach in communities outside of their place of origin. In general, less than 50% of the teachers in all the groups use the different teaching techniques. Additionally, not all trained principles use the methodologies and tools for the teaching of reading. The methodology that they are most familiar with and use is the Kemom Ch’ab’al methodology. The main reason why they implement it is because they receive resources to teach how to read. Over 81.5% of the schools have an improvement plan, and over 96% carry out planning and coordination activities. There are even two groups that report 100% in this action, and their respective plans are monitored every three months in most cases. 50 100% of the DCs that were interviewed indicated that they prioritize supervising first-grade teachers, and that they know how to implement the methodologies and tools for teaching reading. One relevant aspect is that they attribute the reduction of the repetition rate to the Learning for Life project, among other important achievements. They also value the accompaniment on the part of the project technicians in the schools, but they identify a poor coordination with them and mention that they do not receive technical monitoring reports. From the DCs’ perspective, the main challenges in bilingual education are: lack of materials, the linguistic variants of K'iche’, and the rotation of the personnel, besides promoting and ensuring the correct application of the Kemom Ch’ab’al’ and Jardín de Letras methodologies; These challenges are being addressed through the actions of the Learning for Life program as recorded in their monitoring. All the groups agree that teachers have lost their skills in K’iche’, which limits the students from attaining reading skills in this language. Recommendations: Academic and school development • Design a strategy together with MINEDUC’s General Directorate of Intercultural Bilingual Education (DIGEBI, in Spanish) to strengthen and intensify the training and reinforcement of teachers on Learning for Life’s reading methods, mainly in K'iche’, while considering the existing language variations. This strategy can be linked to the planning carried out by most school principals, so that it is part of daily school life. • Establish cooperation agreements with the Academy of Mayan Languages and the EBI Program to promote and ensure the process of training teachers in K’iche’. • Strengthen the communication of Learning for Life project technicians with DCs. The technicians can establish channels of communication to inform DCs about the progress of the project and seek support so that the reading teaching techniques are intensified. • Raise awareness amongst the school and the community about the importance of the K'iche’ language. Success stories of famous people in the community could be presented in Mayan languages. A “Language Day” could also be implemented where geographical variations are explained. These and any other proposed activities must be agreed upon together with the communities and local authorities. 51 The hunger scale and student attention span 52 VII. THE HUNGER SCALE AND STUDENT ATTENTION SPAN The baseline relates the students’ hunger levels during the school day with their attention span and level of participation in the classroom. It measured the time in which schoolchildren received food, and whether they had breakfast at home, coupled with the effects of hunger in their classroom attention span from the perspective of the students, the teachers and the data collectors. 7.1. (Indicator 14) Percentage of students in participating primary schools who indicate not being hungry during the school day Global Indicator: Low Level 54.4% The indicator was measured through the following questions: breakfast at home before arriving at school; food consumed during the school day (program support) and at what time school meals are served. 7.1.1. Students that have breakfast at home First, third, and sixth-grade children were asked if they had breakfast at home. As shown in the following table, over 91% of first-grade children in all groups, said they ate breakfast. For third-grade students, the positive response is between 77% and 81%; the lowest of all three groups. Between 82% to 92% of sixth-grade students indicated they had breakfast. For all grades, children in CG 1-A indicated they had breakfast the least often. Table 19. Percentage of students that have breakfast at home First-grade students Third-grade students Sixth-grade students % of students CG 1-A 1- B 2- A 2- B 2- C CG 1-A 1- B 2- A 2- B 2- C CG 1-A 1- B 2- A 2- B 2- C 92 95 96 96 96 78 80 81 77 78 82 92 90 89 90 Source: Students’ surveys, Learning for Life 2018 Baseline Study 7.1.2. Type of food students receive at school The children responded that they receive a school meal in the early hours of the morning (8:00 - 10:00 a.m.), showing that most schools in all groups do serve food at the beginning of the school day. School meals at the end of the morning (11:00 - 12:00 a.m.) are less frequent, and only two groups of schools report serving lunch to the children. Table 20. Type of food students receive at school Type of food Study groups 1-B 2-A 2-B 2-C CG 1-A Breakfast 30.8% 18.2% 22.6% 12.9% 44.4% Food at the beginning of the morning 61.5% 66.7% 71.0% 80.6% 47.2% Food at the end of the morning 7.7% 9.1% 3.2% 0.0% 8.3% Lunch 0.0% 0.0% 3.2% 6.5% 0.0% Source: Principals’ surveys, Learning for Life 2018 Baseline Study 53 Focus group information that complements the analysis According to the mothers’ focus groups, students accept most of the foods that are provided to them in schools. Some mothers expressed they face difficulties in preparing some meals and therefore children do not like them. According to mothers from the school in Chimente, “Most foods that children receive in school is good, we just need to change the brown beans for black beans. They are more accepted when they are well prepared but when the tamale or tortilla is drenched, smoked, poorly cooked, or the atoll lacks sugar or is in poor condition, the children don’t eat it.” 7.1.3. Hunger-related effects in students First, third and sixth-grade students were asked about five physical states they felt during the school day: hunger, sleepiness, illness, sadness or fatigue, if these effects can be attributed to hunger. Table 21 shows the results. Although children could identify all the sensations attributed to hunger, a low percentage responded to having any of them. Yet the feeling of hunger is present in three groups, and it is most frequent amongst the oldest children. For first-grade students, it is 12.6%; for third-grade students, 12.5% and for sixth-grade students, 17.2%. Fatigue is a sensation that is not very visible, especially in children of third and sixth grades. This can be attributed to older children’s greater resilience to hunger. Among the first-grade students, the least frequent sensation is sleepiness. Table 21. Sensations of first, third and sixth-grade students Sensation First-grade students CG 1-A 1-B 2-A 2-B 2-C I feel hungry 8.3% 12.6% 7.4% 11.5% 7.4% I feel sleepy 1.7% 1.5% 1.7% 2.0% 3.0% I feel ill 3.9% 5.7% 6.8% 7.5% 5.4% I feel sad 3.9% 4.6% 5.1% 7.5% 4.9% I feel fatigued 3.9% 5.0% 5.1% 5.2% 9.0% Total 21.7% 29.40 26.1% 33.7% 29.7% Sensation Third-grade students CG 1-A 1-B 2-A 2-B 2-C I feel hungry 8.5% 7.5% 12.3% 12.5% 10.2% I feel sleepy 3.7% 4.1% 4.7% 4.0% 3.5% I feel ill 0.9% 1.9% 2.0% 1.3% 2.8% I feel sad 0.9% 1.7% 1.2% 1.3% 1.4% I feel fatigued 0% 0.% 0% 0% 0% Total 14.0% 15.2% 20.4% 19.1% 17.9% Sensation Sixth-grade students CG 1-A 1-B 2-A 2-B 2-C I feel hungry 4.3% 4.3% 3.3% 1.2% 4.1% I feel sleepy 1.2% 0.9% 2.0% 0.6% 1.7% I feel ill 0.6% 2.3% 1.0% 0.9% 1.0% I feel sad 1.7% 0.9% 1.5% 3.2% 1.2% I feel fatigued 0% 0% 0% 0% 0% Total 7.8% 8.4% 7.80% 5.90% 8.00% Source: Students’ survey, Learning for Life 2018 Baseline Evaluation 54 7.2. (Indicator 13) Percentage of students in the classrooms defined as "very attentive" using a scale that defines established criteria. Global Indicator: 67.1% 7.2.1. Student attention during the school hours The baseline survey measured the percentage of students who are "very attentive" in the classrooms by using a scale with a set of criteria. The level of attention was evaluated through different informants: a. Students’ own perception about their levels of attention in the classroom b. Teachers’ perceptions about the students’ level of attention in the classroom c. Baseline data collectors’ observations about the students’ level of attention during the study The students themselves most frequently indicated they paid attention ‘all the time’ and ‘most of the time’. Although “does not know” was also a frequent answer. CG 1-A showed the highest level of attention (all the time and most of the time). In third and sixth grades, group 2-A is the one that reported the highest level of attention. When comparing the results of the levels of attention with the sensations felt by the students (hunger, sleepiness, fatigue, illness or sadness), there is no statistically significant correlation. Table 22. Percentage of students indicating a certain attention level in class Perception of attention First-grade students Third-grade students Sixth-grade students CG 1-A 1-B 2-A 2-B 2-C CG 1-A 1-B 2-A 2-B 2-C CG 1-A 1-B 2-A 2-B 2-C Pays attention all the time 51 39 48 44 42 44 43 49 43 37 49 46 52 50 40 Pays attention most times 21 25 22 22 21 15 12 18 14 16 17 15 28 22 23 Is easily distracted 7 7 7 9 11 14 12 11 14 17 9 17 10 14 14 Is always distracted 3 9 9 8 8 7 7 7 6 10 8 6 5 7 10 Does not know 18 20 14 17 18 20 26 15 20 20 17 16 5 7 13 Source: Students’ survey, Learning for Life 2018 Baseline Evaluation 7.2.2. Perception of teachers regarding the students’ level of attention in class First, third, and sixth-grade teachers were asked about their perception regarding the students’ level of attention in the classroom. The teachers were asked about the following student behaviors: they are active, they ask questions, they participate, they hand in homework and they follow instructions (see Table 23). Table 23. Students’ levels of attention in class according to teachers CRITERIA CG 1-A 1-B 2-A 2-B 2-C Students are active 88% 87% 88% 81% 87% Students ask questions in class 54% 60% 54% 51% 62% Students participate in class 69% 74% 76% 79% 80% Students hand in homework 76% 79% 80% 83% 84% Students follow instructions 75% 76% 74% 80% 84% Source: Teachers’ survey, Learning for Life 2018 Baseline Evaluation 55 According to teachers, most children (between 83 and 88%) are active during class. The next criterion that received the highest qualification from the teachers is the handing in of homework (75 to 84% of the students). The criterion with the lowest percentage was to ask questions in class. Group 2-C received the highest percentage in this criterion (62%). In general terms and according to the teachers, the children pay attention during class. 7.2.3. Students’ level of attention observed in the classroom on the day of the study The baseline data collectors observed the classrooms of the sample schools. During the observation, they characterized the students according to the following behaviors: being alert and following instructions; participating promptly; looking tired or drowsy; being restless or out of place; and not following instructions. Table 24. Students’ behavior in the classroom during the study compared to the control group Student level of attention observed in the classrooms on the day of the study CG 1-A 1-B 2-A 2-B 2-C Students are alert and follow instructions 80% 78% 77% 86% 81% Students participate promptly 70% 58% 67% 69% 67% Students look tired or drowsy 11% 17% 15% 9% 11% Students are restless or out of place 14% 12% 12% 12% 10% Students are not following the teachers’ instructions 21% 23% 28% 21% 18% Source: Classroom observation. Learning for Life 2018 Baseline Study Baseline data collectors frequently observed that, in all groups, students seemed alert, followed instructions, and participated promptly. Group 2-B was the most attentive. The behavior of non￾attention was the least observed. The group with the highest percentage in this criterion was 2-A, with 28%. Both observation data as well as that provided by teachers and students coincide. There is a high percentage of alert children in the classrooms. 7.2.4. Participation of students in class or in group work, disaggregated by sex The classroom observation tool compared the class participation of girls and boys through two aspects: whether boys and girls responded to the teachers’ questions in equal proportions, and whether there were both girls and boys in the groups when working in group activities (there were no marked groups with only boys or only girls). The results of the comparison show that there is equal participation of girls and boys in the classroom. Table 25. Class participation of girls and boys Equal participation by sex CG 1-A 1-B 2-A 2-B 2-C There is equal participation by both sexes 93.60% 97.60% 89.20% 90.60% 96.90% There is not equal participate by both sexes 6.40% 2.40% 10.80% 9.40% 3.10% Source: Classroom observation. Learning for Life 2018 Baseline Study 56 7.2.5. Hunger scale and attention span To evaluate hunger and its relation with attention in the classroom, a scale was constructed in which the independent variable is access to some food at the beginning of the day (breakfast); and the dependent variable, the effects related to hunger as indicated by the students, and their results in attention. Four levels were built to categorize students and relate literacy performance results. Each category has a weighting where the independent variable has a score greater than 40 and the two dependent variables each had a score of 30. The following ranges were established per level: Level 1 (Low): 0-25; Level 2 (Regular): 26-50; Level 3 (Medium): 51-75; and Level 4 (High): 76-100. Table 26. Definition of the hunger and attention scale levels Category Level 1 Level 2 Level 3 Level 4 Independent variable Access to breakfast Score: 40 Had breakfast at home and at school Had breakfast at home, but not at school Had breakfast at school, but not at home Did not have breakfast at school, nor at home Dependent variable Effects of hunger Score: 30 1 effect 2 effects 3 effects 4 or 5 effects Student attention results Score: 30 I pay attention all the time I pay attention most of the time I get easily distracted I am always distracted Level Low Score 0-25 Regular Score 26-50 Medium Score 51-75 High Score 76-100 The results of the hunger scale in students are presented in the following table. Table 27. Hunger levels by grade, in percentages Hunger level First-grade students Third-grade students Sixth-grade students CG 1-A 1-B 2-A 2-B 2-C CG 1-A 1-B 2-A 2-B 2-C CG 1-A 1-B 2-A 2-B 2-C High 3 2.1 1.5 2.3 2 12.8 14.3 9.7 13.8 14.2 6.6 3.4 2.7 6.9 6.7 Medium 3 4.2 2.1 2.3 1.7 6.8 9.6 8.5 8.2 8.1 8.4 4.3 3.7 3.2 3.8 Regular 29.3 34.7 29.1 34.4 35.9 31.6 29.2 24.3 31.5 32.8 29.7 38.1 22.4 22.5 33.5 Low 64.7 59 67.4 61 60.4 48.8 46.9 57.5 46.5 45 55.3 54.3 71.2 67.4 56.1 Source: Database, Learning for Life 2018 Baseline Study In general, in all three grades, the majority of children are at the low and middle levels of the hunger scale, so it can be inferred that the food provided by the project is helping to reduce hunger in the short term. Focus group information that complements the analysis Parents consider that food influences the children’s attention, in addition to the support they receive from parents and teachers. Some opinions of the parents of Los Cipreses school are: • "Children who are not hungry in school are attentive to their duties and to the activities they do, while hungry children do not pay attention to the teacher and do not do their homework.” 57 • "The food our children receive helps them to pay attention and do their homework.” If children are fed well, they have enough strength and their brain is powered so that they learn more.” Other parents do not associate lack of attention to hunger, but rather to other causes such as lack of accompaniment of teachers and household problems: • "The support and good treatment that teachers give to children help them to pay attention and have enough materials so that children are focused and pay attention." • “When there is abuse at home, children suffer, and it affects their studies.” Conclusions: Hunger scale and attention span Through the McGovern-Dole FFE Program, Learning for Life is contributing to reducing hunger in the short term, complementing the feeding of school-age children in the schools covered by the project. Most schools are providing a meal in the early hours of the school day, before 10:30 am. Most students (over 77%) reported that they had breakfast at home, before going to school. Regarding classroom attention levels, observation and survey data match: students are very attentive, active and participate in the class all or most of the time. There is also equal participation of girls and boys. In the students’ hunger scale, it was observed that the group of third-grade students is the most vulnerable – 21.2% are in the high and medium levels. Students who reported that they did not eat breakfast mentioned having 4 or 5 effects of hunger and indicated that they are always distracted. Recommendations: Hunger scale and attention span • In trainings given to parents, include explanations about the importance of providing breakfast for their children, and how to recognize the effects of hunger, especially on school performance. • Reinforce with the volunteer cooks, as well as with teachers, the importance of students receiving a nutritious meal in the early hours of the school day (while it is possible that they have consumed breakfast at home, in the rural area this tends to be composed of tortillas or food with little nutritional value). According to the Food and Agriculture Organization (FAO), school-age children need a proper diet to grow, to develop, to be protected against diseases and have the energy to study, learn and be physically active. For this reason, meal and nutrition programs are key for children to exercise their rights to food, education and health. This is best achieved if the food is distributed in the first hours of the school day, considering that according to the context, the breakfast of most of the children consists of a corn tortilla and coffee. • In line with the above, the food should be balanced and have all the necessary nutrients for the correct feeding of children. For this reason, it is recommended that Learning for Life promote amongst parents a contribution of food that complements the school diet. 58 Health, Nutrition, and Hygiene￾related Indicators 59 VIII. HEALTH, NUTRITION, AND HYGIENE-RELATED INDICATORS One of the strategic objectives of the Learning for Life project is to increase health and nutrition practices. The following section presents the results of health, nutrition and hygiene-related indicators at the school and household levels in treatment and control groups. 8.1. (Indicator 5) Number of individuals who demonstrate use of new child health and nutrition practices as a result of USDA assistance. Global Indicator 25.9%; Number: 4,380 volunteers35 8.1.1. Knowledge of fathers, mothers and volunteers -FMV- on the treatment of diarrhea Graph 19 shows that FMV of groups 2-A, 2-B and 2-C (schools which will continue receiving the program’s activities) have the most knowledge on the treatment of diarrhea; they are at 97.7%, 95.4% and 97.7% respectively. On the other hand, the FMV of group 1-B (new schools) and of CG 1-A have 92.8% knowledge on the treatment of diarrhea. Graph 19. Percentage of FMV with knowledge on the treatment of diarrhea Source: Father, mother, volunteer surveys, Learning for Life 2018 Baseline Study 8.1.2. Steps for treating diarrhea It is expected that FMV mention at least three of the following steps for treating children with diarrhea: 36 • Give the child more fluids than usual to prevent dehydration • Give the child oral serum • Continue giving food to the child to prevent malnutrition • Seek medical help • Wash hands after cleaning the child • Wash cloths and hands in contact with vomit and stool • Administer a zinc supplement 35 An approximate number of individuals has been extrapolated with a 95% confidence level based on the total number of sampled vs. registered beneficiaries and their response for the indicators with number as their unit of measure. 36 The definition of the steps for the care and / or treatment of children with diarrhea is based on studies that ADOC has carried out to measure this issue. 92.8 92.8 97.7 95.4 97.7 7.2 7.2 2.3 4.6 8.3 0 20 40 60 80 100 CG 1-A 1-B 2-A 2-B 2-C With knowledge Without knowledge 60 Table 28 shows that 50% of group 2-B was able to identify at least three steps as established by the indicator; while the rest of the treatment groups obtained answers below 22%. In contrast, in the control group, 23.3% of parents, mothers and volunteers mentioned at least 3 steps. In most of the treatment groups (except group 2-C) and in the control group, they mentioned only 1 step. Table 28. Percentage of parents that mention steps to treat diarrhea CG 1-A 1-B 2-A 2-B 2-C Does not mention any step 2 2 4 1 0 One step 41 42 38 14 37 Two steps 34 40 41 30 39 Three steps 23 16 15 50 22 More than three steps 0 1 2 6 3 Source: Father, mother, and volunteer surveys, Learning for Life 2018 Baseline Study In addition, observation instruments, such as checklists, were used. Through them, the Baseline study team observed and verified classroom hygiene practices that students have at school. Graph 20 shows that for the treatment groups 2-A, 2-B, and 2-C, 60.6%, 71%, and 61.3% of the respective students have a hygiene corner in the first, third and six-grade classrooms. In the CG 1-A, only 50% of the classrooms use the hygiene corner. Finally, in group 1-B, 59% have use hygiene corner. 8.1.3. Resources available in schools for personal hygiene Another observation instrument was used to verify the resources available to students in schools for their personal hygiene. According to the data, groups 2-B and 2-A have similar percentages above 61.3%. In the case of groups 2-C and 2-B, they have percentages below 61.3%. CG 1-A has resources below 55.6% Graph 20. Percentage of schools with resources available for personal hygiene Source: Father, mother, and volunteer surveys, Learning for Life 2018 Baseline Study With respect to toilet/latrines facilities, 60% of schools in CG 1-A and 1-B have padlocks, while it is over 75% for other treatment groups. Groups that had the least number of toilet facilities separated by sex were CG 1-A, with 58.3% and Group 2-C with 61.3%. In the other groups it was observed that the number of schools with toilet facilities separated by sex was above 75%. 0 20 40 60 80 CG 1-A 1-B 2-A 2-B 2-C Schools with towels or other Schools with soap Schools with a personal hygiene corner 61 8.1.4. Personal hygiene practices carried out by students in schools Handwashing before eating is the practice that was observed in most groups (between 56% to 70%). Between 52% and 67% of students wash their hands after using the toilet. Group 1-B had the lowest percentage in this practice (49%). The practice that was least observed in all groups was students brushing their teeth after eating. The range for all groups is within 33% and 64%, with CG 1-A and Group 1-B having the lowest percentage at 33%. When comparing this information with the data on personal hygiene resources, one can observe that these groups indicating having the least amount of resources from all groups. One possible explanation for this is that the project has not yet implemented health and hygiene activities in the groups of schools that are part of Phase II. Table 29. Percentage of students that carry out hygiene practices Personal hygiene practices in schools CG 1-A 1-B 2-A 2-B 2-C Handwashing before eating 56 59 70 68 58 Handwashing after using the toilet 53 49 67 55 52 Brushing their teeth after eating 33 33 64 55 45 Source: Observation in schools, Learning for Life 2018 Baseline Study 8.1.5. Personal hygiene practices in the household In the interview with FMV, data collectors explored the personal hygiene practices performed in the households and identified the four practices most mentioned by the interviewees in each group. The following table shows FMV of CG 1-A mentioned in greater percentage the following household hygiene practices: handwashing (64%); bathing daily (61%), handwashing before cooking (56%) and handwashing after using the toilet (42%). In the treatment groups, the personal hygiene practice most mentioned by FMV was bathing daily, followed by handwashing before cooking, handwashing before eating, and handwashing after using the toilet. Table 30. Percentage of personal hygiene practices in the household Personal hygiene practices in the household CG 1-A 1-B 2-A 2-B 2-C Handwashing before eating 64 49 45 42 42 Bathing daily 61 54 54 53 60 Handwashing before cooking 56 51 47 51 42 Handwashing after using the toilet 42 40 31 40 38 Source: Father, mother, and volunteer surveys, Learning for Life 2018 Baseline Study In the FMV surveys where the household hygiene practices were identified, every group mentioned five hygiene practices (see Table 31 for the results.). The FMV in CG 1-A mentioned the first four household hygiene practices in greater percentages. The results of the treatment group are similar to those of the control group. The practice of using the latrine was most mentioned in Group 2-C. 62 Table 31. Percentage of FMV that identify household hygiene practices General hygiene practices in the household CG 1-A 1-B 2-A 2-B 2-C Daily cleaning of the house 94 87 93 84 93 Use of bins or bags for trash 53 51 37 45 49 Use of soap for handwashing 37 33 29 32 26 Keeping animals in pens 35 23 15 17 21 Use of latrines 17 19 16 18 23 Source: Father, mother, and volunteer surveys, Learning for Life 2018 Baseline Study 8.1.6. Health care services available to students in schools Through surveys with school principals, data were obtained regarding health care services provided by the Ministry of Health to students at schools. According to the data, over 88% of students are dewormed. This was the most frequently-reported treatment. The following table shows results for Vitamin A, vaccination, delousing, vitamin supplements and day-long medical missions. Table 32. Health care treatments available to students in schools, in percentages Treatment CG 1-A 1-B 2-A 2-B 2-C Vitamin A 8 10 36 22 20 Deworming 100 93 88 87 92 Vaccination 23 10 24 13 20 Delousing 8 10 12 17 16 Vitamin supplements 8 3 12 13 4 Medical missions 8 7 8 22 12 Source: Principals’ surveys, Learning for Life 2018 Baseline Study. Regarding vaccination, between 10% and 24% indicated they received vaccines, with group 2-B having the lowest percentage. The vaccines are usually provided by local health posts and the parents must keep record; some seasonal vaccines, such as the flu vaccine, are distributed at the school, sometimes by an epidemic alert or by the school’s initiative. Vitamin supplements are the least-provided treatment. They reported percentages between 3% and 13%. CG 1-A and Group 1-B are the ones that least implement this treatment. Group 2-A reported that 36% of schools delivered Vitamin A, 14 pp above the highest percentage of the other groups. Deworming is delivered as part of the Ministry of Health’s national strategy, which is why this treatment has a high percentage in all groups. The distribution of other supplements, such as Vitamin A and vitamins in general, depends on each school request. 8.1.7.Dietary diversity of food provided in schools In the absence of a Dietary Diversity Score (DDS) classification, the information in this study is analyzed according to context. Four food groups were considered as an acceptable cut-off point, as established in the Food Security Integrated Phase Classification (IPC) reference tables. Results show that, as per the cutoff point established according to context, dietary diversity is at an acceptable level in the different treatment groups; its average is above 5. However, it is below the average of 12 food groups established by the FAO, where the smaller the number, the less dietary diversity. Dietary diversity should not be interpreted as an indicator of nutrition that reflects the quality of the diet, but rather as an indicator of access to food. 63 The following food groups are measured in the DDS: 1. Cereals 2. White tubers and roots 3. Vegetables 4. Fruits 5. Meats 6. Eggs 7. Fish and other seafood 8. Legumes, nuts and seeds 9. Milk and milk products 10. Oils and fats 11. Sweets 12. Spices, condiments and beverages The DDS is similar for all the groups, both treatment and control. All groups (including the control group) receive food and nutritional training from the project, in addition to the disbursement from MINEDUC for school meals. Additionally, it was observed that the household dietary diversity score (HDDS) is either equal or lower than the cut-off according to the context (4). Group 2-B has the lowest average HDDS. The communities in this group are from the municipalities of Momostenango, San Andres Xecul, San Bartolo Aguas Calientes and Santa Lucia La Reforma, which were characterized as having high nutritional vulnerability by the 2015 National Census that measures the height of school children. Table 33. Dietary Diversity identified in schools and homes that participated in the study Dietary Diversity CG 1-A 1-B 2-A 2-B 2-C DDS of school food menus planned for the day of the study 5 5 6 5 6 DDS of school food menus prepared on the day of the study 5 6 6 6 6 HDDS according to FMV survey 4 4 4 3 4 Source: Father, mother, and volunteer surveys and Observation in schools, Learning for Life 2018 Baseline Study The foods that are most consumed in the households and schools are presented in the following tables: Table 34. Foods with the highest consumption in households Total in households No. Percentage Vegetables 395 59.3% Eggs 392 58.9% Fruits 317 47.6% Sweets (sugar, honey) 245 36.8% Cereals 217 32.6% Oil or cream 209 31.4% Meats and charcuterie 191 28.7% Legumes and seeds 166 24.9% Milk and milk products 153 23.0% Varied foods 134 20.1% Roots and tubers 122 18.3% Fish or seafood 36 5.4% Source: Father, mother, and volunteer surveys, Learning for Life 2018 Baseline Study 64 Table 35. Foods with the highest consumption in schools Total in school No. Percentage Sweets (sugar, honey) 150 88.2% Milk and milk products 135 79.4% Oil or cream 116 68.2% Cereals 106 62.4% Varied foods 89 52.4% Fruits 76 44.7% Vegetables 75 44.1% Legumes and seeds 63 37.1% Eggs 58 34.1% Meats and charcuterie 34 20.0% Roots and tubers 22 12.9% Fish or seafood 0 0.0% Source: School observation, Learning for Life 2018 Baseline Study The table shows that the foods that are most consumed in schools are sugars, dairy products, fats, and cereals, while in the households, it is vegetables, eggs, fruits and sugars. The school diet includes a high intake of carbohydrates and fats, in addition to sugars; other food groups are included at less than 50%. 8.2. (Indicator 7) Percent of schools reaching the tier/level 1 of healthy school initiative tiers Global Indicator: 95.3% At the time of the study, the healthy schools initiative had been modified, so the project decided to evaluate the infrastructure because of some characteristics considered in the Learning for File project strategy, since in the 2018 school year, MINEDUC no longer uses levels in the schools. (See Annex 11.7: Proposal to build the healthy schools indicator for Learning for Life) The baseline study included the analysis of the state of the school infrastructure, both in general and specifically, for the management of the project’s food. In the case of the warehouses, they were evaluated using a 12-point checklist: food exclusivity, order and cleanliness, moisture-free walls, foods placed on shelves or furniture, free of rodents, an up-to￾date Kardex, having equipment to measure rations, sealed food that hadn’t been spilled, having transport guides, having a food portion estimation table, having an identification sign, and being free of ceiling leakages and having closed windows. 8.2.1. Warehouse conditions All groups report having schools with warehouses exclusively for food, though Group 1-B is the one reporting the fewest schools that meet this condition (51%). The schools in group 2-C have the highest percentage of schools with warehouses exclusively for food. In CG 1-A, more than 55% of schools meet this condition (although schools sometimes use food cellars to store other things, such as school supplies or musical instruments). In general, most (not all) warehouses are moisture-free. The schools of Group 2-A present the best conditions (94% of the warehouses are moisture-free). Also, this group has better conditions of cleanliness and order in the warehouses (94%). In contrast, Group 1-B has the lowest conditions 65 (69%), possibly because in these schools, which were incorporated in Phase II in 2018, there had been no training in food handling and storage. Storage in shelves or furniture does not occur in 100% of the warehouses. The warehouses of group 1-B present the best conditions (87%). Only 75% of the warehouses in CG 1-A fulfill this condition. Regarding the equipment to measure rations, group 2-C is the one that is best equipped, with 74% of the schools. In comparison, the warehouses of CG 1-A are only equipped at 47%. Additionally, 56% or less of the schools in all groups use tables to estimate the quantity of food according to school population. It was observed that schools do take care to avoid spillages, since warehouses in all groups scored 94% or higher in these criteria. Regarding the controls on the input and output of food using the Kardex, it was found that they are updated in only 52% or less of the warehouses in all groups (the project has planned to train volunteer mothers during Phase II on how to use Kardex so that they are kept up to date). Warehouses show evidence of pests such as rodents, cockroaches or other insects, in 39% or more of the schools across all groups. Group 1-B has the most warehouses with evidence of pests at 56%, while the warehouses Group 2-C are those that show the least evidence of pests (39%). The data reveal that the physical infrastructure of the warehouses is adequate. However, their management is not appropriate since many of the conditions in most groups are at 50% or lower. The following table shows the results of the school observations for all groups with regards to the warehouse conditions. Table 36. Condition of the warehouses assessed in the baseline study Condition of warehouse CG 1-A 1-B 2-A 2-B 2-C Exclusively for food 56 51 70 65 84 Clean and organized 83 69 94 81 74 Moisture-free walls 86 82 94 87 90 Food kept in shelves or inside furniture 75 87 82 77 77 Evidence of rodents or bugs 56 56 55 55 39 Kardex up to date 42 44 49 42 52 Equipment for measuring rations 47 49 58 71 74 Food sealed without spillage 94 97 94 97 94 Transportation guides 36 59 49 58 42 Food portion estimation table 33 56 52 45 42 Identification sign 28 3 46 45 26 Leak-free roof 89 92 94 87 90 Closed windows 83 90 97 97 90 Source: School observation, Learning for Life 2018 Baseline Study Focus group information that complements the analysis Fathers and mothers of schoolchildren were consulted in focus groups on various aspects of food handling in warehouses (see Figure 9). 66 Figure 8. Aspects of food handling in warehouses, referred by parents 8.2.2. Conditions of the school kitchens As part of the baseline study, the conditions of the school kitchens were evaluated. The following criteria were evaluated: having a space exclusively for the kitchen, having a door, having cement floor, having adequate ventilation, being organized, being clean, having a trash can and having a wash basin. The data shows the school kitchens of group 2-C are in the best condition, since they have an exclusive space, a door and a cement floor. The criterion of having a trash can was the least prevalent of all groups. The ventilation in the kitchens is not adequate. Group 2-A had the most kitchens with the best ventilation conditions (85%), while in Group 1-B, only 69% of the kitchens had adequate ventilation. The results for all criteria are shown in the following table. Table 37. Percentage of schools that meet baseline study kitchen conditions Condition CG 1-A 1-B 2-A 2-B 2-C Has a space exclusively for the kitchen 92 90 97 97 100 Has a door 94 100 94 94 100 Has a cement floor 92 85 100 100 100 Has adequate ventilation 72 69 85 81 84 Is in order 86 82 97 81 94 Is clean 89 87 94 97 94 Has a trash can 64 51 67 74 68 Has a wash basin 72 82 82 77 94 Source: School observation, Learning for Life 2018 Baseline Study The management of food in the cellars is a task usually carried out by parents, but in some schools the mothers and teachers are responsible. Training opportunities on the handling of food are different. •Three out of ten groups indicated that only PTA's members have participated in the food storage training. •Other volunteers indicate they have received training on this practice. Food storage practices are useful so that food is not contaminated and children do not get sick. •"The trainings have been very useful and we have put them into practice both in the school and in the houses. We know that food must be stored in a suitable place and well covered, free from contamination, because it will be to feed the children and if not well protected, they can get sick and we do not have enough resources to cure them. " •"Thank God, at school we have enough plastic cans with lids to store food properly. In our homes we sometimes need them but we try to make sure that the most delicate foods are well covered. In the community we use pots a lot to store food." (Parents and mothers of the Chomumus school) 67 8.2.3. Types of stoves found in school kitchens In the school kitchens, the following types of stoves were observed: improved stove, wood-saving stove, stove with a chimney, stove with an open wood fire, and wood-only stove. It was found that the combination of improved, fireplace and wood-only stove was the most common in all the groups. The use of stoves with an open fireplace is the least frequent cooking option in all groups. Some schools have more than one stove due to the amount of food they prepare. They can have a wood-burning and open fireplace stove or other combination. The following graph shows the type of stoves that were found in the schools. Graph 21. Types of stoves found in schools, in percentages Source: School observation, Learning for Life 2018 Baseline Study The baseline also checked if the utensils available for cooking were in good condition and if they were sufficient for the school population bring served. Group 2-C has 100% school kitchens with utensils in good condition, while Group 1-B has 74% school kitchens with utensils in good condition, but with enough utensils for only 46% of the school population. Group 2-A showed sufficient utensils in good condition in 73% of the schools. (The project plans on delivering kitchen utensils for CG 1-A and Group 1-B during the 2018 school year) Focus group information that complements the analysis Parents consulted in focus groups helped to understand the situation of food preparation in schools. They reported that, although they have utensils to prepare food, these are not enough for the large school population. They lack necessary appliances such as blenders or gas stoves, which hinders their ability to prepare food. One group points out the need for a guide to calculate the amount of food that needs to be prepared according to the school population, so as not to waste food. The difference between the schools that reach level 1 of the healthy schools initiative was compared and, upon knowing the results, it can be observed that groups 2-A, 2-B and 2-C show a lower percentage (94.7%) than groups 1-A and 1-B (96%). However, after running the Chi-square tests for each of the groups, it was observed that these differences are not significant at 95% confidence level. 0 20 40 60 80 100 Improved stove Wood-saving stove Stove with a chimney Stove with an open wood fire Wood-only stove CG 1-A 1-B 2-A 2-B 2-C 68 8.3. (Indicator 6) Number of individuals who demonstrate use of new safe food preparation and storage practices as a result of USDA assistance Global Indicator: 67.6%; Number: 11,433 volunteers37 8.3.1. Food preparation in schools The baseline analyzed the following important criteria in school kitchens: garbage-free kitchen, animal-free kitchen, washing hands before cooking; wearing hairnets; wearing an apron, washing utensils before using them to cook, and washing food before it is cooked. Group 2-B is the one with the highest percentage of schools that use good practices in the preparation of school meals. In all groups, the practice least found in the schools was that of using a hair net (with CG 1-A having the lowest percentage). In fact, CG 1-A had the least percentages of all the schools regarding the safe preparation of food. The following table shows the results. Table 38. Percentage of schools where good health practices were observed in food preparation Practice CG 1-A 1-B 2-A 2-B 2-C Garbage-free kitchen 86 92 97 100 95 Animal-free kitchen 89 95 97 100 97 Washing hands before cooking 83 95 97 100 97 Wearing hairnets 39 41 73 45 58 Wearing aprons 61 80 94 94 90 Washing utensils before using them to cook 92 92 97 100 97 Washing food before it is cooked 92 97 100 100 97 Source: School observation, Learning for Life 2018 Baseline Study 8.3.2. Type of water consumed in schools The criteria that were evaluated are related to the type of water consumed in schools: whether it is chlorinated, boiled, filtered or disinfected using another method. Most schools boil their water, which makes it safe for drinking. However, this practice requires other resources such as wood, and takes up the mothers’ time. Chlorination is the least-used method: only 3% of schools in Group 1-B report doing this. Table 39. Methods used in schools to purify water, by percentages Method CG 1-A 1-B 2-A 2-B 2-C Boiled 97 95 91 90 87 Chlorinated 0 3 0 0 0 Filtered 0 0 6 10 13 Other 3 3 3 0 0 Source: School observation, Learning for Life 2018 Baseline Study 37 An approximate number of individuals has been extrapolated with a 95% confidence level based on the total number of sampled vs. registered beneficiaries and their response for the indicators with number as their unit of measure. 69 8.3.3. Schools with clean environments To define the state of cleanliness in the schools, key areas were observed, such as classrooms, sanitary services and courtyards. Group 2-B presents the cleanest conditions. Of the spaces observed, the classrooms were the least clean. The following table shows the percentage of schools that met each criterion for cleanliness. Table 40. Percentage of schools with clean environments Environments CG 1-A 1-B 2-A 2-B 2-C Trash cans in the classrooms 78 87 97 94 84 Clean toilets/latrines 83 69 82 94 84 No accumulated trash 78 90 91 94 94 No trash in the classrooms 83 90 100 94 100 Clean classrooms 89 82 94 94 97 Classrooms with hygiene corner 50 59 61 71 61 Source: School observation, Learning for Life 2018 Baseline Study 8.3.4. School vegetable gardens The school vegetable garden is a Learning for Life strategy that seeks to teach science and promote values such as leadership, teamwork and peer relationships. Schools were asked whether they have spaces for vegetable gardens and if during the previous year, they planted one. It was found that few schools within all the groups have space for a vegetable garden. Group 2-A reported the most schools that have a space for a vegetable garden and planted it the previous year: 48.5% for both criteria. Schools in groups 1-B and 2-C report having a space for a vegetable garden, but not necessarily having implemented the gardens in the previous year. Schools in Groups 1-B and 2-C report the least percentage of vegetable gardens the previous year; however, they do have space to implement the gardens. Graph 22. Percentage of schools that implemented a vegetable garden the previous year and have space for a vegetable garden Source: School observation, Learning for Life 2018 Baseline Study Focus group information that complements the analysis Parents consulted through focus groups consistently mentioned how school vegetable gardens were beneficial to the students’ health: “The children learn to till and cultivate the land, we make gardens in the house, this contributes to the family economy, it also helps the children to be healthy, some 0 20 40 60 Implemented a vegetable garden the previous year Has space for a vegetable garden CG 1-A 1-B 2-A 2-B 2-C 70 children do like to work in the gardens while others don’t.” They pointed out that the main difficulties in implementing them are lack of water and lack of space. The difference between individuals that demonstrate the use of new practices for safe food preparation and storage was compared. Results show that there is greater use of these practices (73.7%) in groups 2-A, 2-B and 2-C, compared with 60% in groups 1-A and 1-B. However, after running the Chi-square tests it is observed that these differences are not significant at 95% confidence level. Conclusions: Health, Nutrition and Hygiene-related indicators The project’s comprehensive approach incorporates actions that promote good health and hygiene practices. Through the Ministry of Public Health and Social Assistance, students of all groups are dewormed, and receive vitamin supplements and vaccines. In addition, campaigns are carried out to delouse the children. Medical missions were the least mentioned of all activities. Parents generally expressed a high percentage of knowledge on how to manage diarrhea in their children. Regarding nutrition, in the different treatment groups, dietary diversity is acceptable. Results show an average diversity above five (with four being an acceptable score in this context), although this only shows access to food and not necessarily the quality of the students’ nutrition. The communities of Momostenango, San Andres Xecul, San Bartolo Aguas Calientes and Santa Lucia La Reforma, which were characterized as having a high nutritional vulnerability by the 2015 National Census that measures the height of students, the dietary diversity scores were low. There is less dietary diversity in households than in schools, as households show that they consume one or two food groups less than in schools. This could be attributed to the program’s food contribution, family food availability and access to food. In the schools, there is a high consumption of sugars, fats and carbohydrates; less than 50% of the other food groups are included in the students’ diets. Regarding hygiene measures and practices, it is necessary for students to have access to supplies such as towels, soap, hygiene corners, toothbrush and toothpaste. Results show that no more than 71% of the schools have resources to ensure the students’ personal hygiene. Handwashing before eating was the most-reported practice, while brushing teeth after eating was the least-reported practice. The implementation of the healthy schools strategy is expected to increase handwashing in schools. This practice is fundamental, since it is directly related to the health and nutrition of the students. In the households, daily bathing is the most mentioned personal hygiene practice. Also mentioned were the use of latrines, keeping animals in pens, using soap for washing hands, using bins or bags for garbage, and cleaning the house daily. The majority of households (98%) have a place designated for cooking. Most families report boiling water for drinking, rather than using other methods such as filtering, chlorinating or not treating the water. Data reveal that handwashing and teeth-brushing are practices more frequently in schools than in the households. This may be related to the fact that, while these practices are promoted in schools, they do not seem to be transferred to the households. Other hygiene-related indicators that were evaluated in the schools were the conditions of storage facilities, the conditions in the kitchens, and cleanliness in different areas of the school. Regarding storage facilities, all schools have spaces designated for the storage of food; half the schools in groups CG 1-A and 1-B have exclusive storage facilities for food, and in the treatment groups between 65% (2-B) and 84% (2-C). 71 Between 82% (1-B) and 94% (2-A) of the schools have moisture-free spaces. The windows were closed in over 90% of the treatment groups, while it was 83% for CG 1-A. In CG 1-A, over 75% of schools stored their food in shelves and shelves. The highest percentage for this indicator was found in Group 1-B (87%). On average, there was evidence of plagues in 50% of the schools, though group 2-C had the least amount of warehouses with evidence of plagues (39%). Between 42% (CG 1-A, Group 2-B) and 49% of the schools use Kardex as a control mechanism. Between 47% (CG 1-A) and 58% (Group 2-A) used the equipment to measure rations, while groups 2-B (71%) and 2-C (74%) are the ones where more schools have the measuring equipment. Even though the committees recognize that they have been trained in the management of the warehouses, it is necessary to clean and order them. Adequate handling of food to avoid spillage is observed; during the study, no food spills were observed in any of the warehouses. Regarding the conditions in the school kitchens, over 90% have a space exclusively for the kitchens; doors and cement floors; and improved stoves with a chimney. The wood stove is the most used. Some schools have more than one stove because of the amount of food they prepare, so they can have an improved stove with an open chimney, or any other combination. Ventilation in groups 1-B (69%) and CG 1-A (72%) is not adequate; smoke accumulation was observed. In some schools, mothers reported that, although they have the utensils to prepare food, those are not enough for the large student population, and they lack appliances such as blenders or gas stoves. Some schools have opted for mothers to cook in their homes and bring the food once it is prepared. Mothers indicated that they need a guide to calculate the amount of food to be prepared, according to the school population, so as not to waste or prepare more food than necessary. In most schools, it was observed that they prepared the same menu that they had planned, indicating that there is a control of the diet. However, in the focus groups, the mothers indicated that they need to be strengthened in the preparation of certain foods. When food is not prepared properly, it is wasted or not prepared and accumulated in the storage facilities. Regarding cleanliness, 3% of the schools in the control group do not have a toilet or latrine; while all schools in the treatment groups report having latrines or toilets. In CG 1-A and Group 2-C, between 58.3% and 61.3% of the schools do not have toilets/latrines segregated by sex. Less than 75% of the schools in CG 1-A and group 1-B have a padlock for the toilets doors. Regarding hand washing, schools have more than one option; between 26% and 37% of the toilet stations have soap or a towel. It was found that in all the groups, over 90% of the schools have pipelining for water. Through their observation, data collectors found that some of the schools do not have supply of water during school day. Most of the school kitchens in the treatment groups comply with aspects of cleanliness in the preparation of food. The practice most neglected was the use of hairnets or hair tied up. It was reported that boiled water is the most used option in all schools. In conclusion, during the baseline study, most schools presented clean classrooms, toilets/latrines and courtyards. The aspect that needs to be improved is the presence of hygiene corners in the classrooms. Another issue related to health, which was referred to during the baseline study, were the school vegetable gardens. Few schools in all groups have space for garden; between 25% (CG 1-A) and 48.5% (Group 2-A). Parents appreciate the school gardens and pointed out that the main difficulties in implemented them are the lack of water and space. 72 Recommendations: Health, Nutrition and Hygiene-related Indicators One recommendation is to increase parents’ knowledge on health care and hygiene in the home, using andragogic methods. The results aim to reinforce strategies to promote better practices or healthy behaviors in households. Another training topic that is relevant to nutrition is dietary diversity. It is recommended that school diets be complemented with any resources that parents can contribute, to diversify the student diet. It is important to educate parents on the importance of consuming different food groups and reducing sugar intake. In the focus groups, it was identified that mothers who are trained are not always the same as those who cook in schools. The project must reorient the strategy and methodology used to train mothers, considering that they rotate in groups to prepare food. It is important to provide norms and standards for handling food to mothers who prepare food, including hygienic practices and quantities to prepare according ration size. The training should be done periodically, with accompaniment provided by the program technical team. It is important to strengthen training and monitoring of food storage facilities so that they comply with good practices in the handling of food. Increase monitoring and promotion of good personal hygiene practices in schools and homes through training for children and parents. The hygiene conditions of hand washing stations must be supervised to ensure they comply with the requirements for a proper hand-washing process. In addition to the trainings, it is also important to promote that schools acquire the necessary supplies (towels, soap, among others). It is recommended that the separation of toilets for boys and girls be promoted in schools. Request MINEDUC authorities that facilities comply with hygiene and safety standards, especially for girls. Other studies carried out by ADOC have noted that this aspect influences school attendance in girls since during their menstrual period, they tend to absent themselves to classes because they do not have adequate services. 38 This is an important aspect that promotes the care and respect for the privacy of students, mainly girls, and also reduces the absence of girls during their period. The Learning for Life project should increase coordination and advocacy among departmental, municipal and community authorities of both local governments and MINEDUC authorities. To achieve the transfer of actions in schools, it is recommended that the project and its local executors work with MINEDUC to reach agreements with the teachers' union, since during the study it was observed that this group strongly influences decisions within the schools. Regarding indicator-related recommendations, during the baseline study, there was no indicator to measure the time that water is available in schools. It is recommended to determine what actions can be carried out to store water and ensure its availability during the school day. Finally, it would be important to consider the relationship of school vegetable gardens that was mentioned by parents during the focus groups. Implementing vegetable gardens in schools does not necessarily mean finding space within the schools themselves; it could be possible to request community spaces or coordinate with families with available space, whose children attend the school. 38 UNESCO (2014) indicates that an inadequate school toilet facility contributes to absenteeism in girls. According to an interview with the Director of the DIGEFOCE conducted by ADOC in 2018, girls are absent mostly in the fourth grade due to this same situation. 73 IX. GENERAL RECOMMENDED ACTIONS The process evaluation is carried out in the investigations with quasi-experimental or experimental designs that have randomized trials. The main objective of this evaluation is to ensure the fidelity of the implementation of the intervention. Considering that the control group in some indicators showed higher percentages than the intervention groups, together with a high percentage of institutions that support the control group, 39 it is essential to install a process evaluation that explains in detail the effects of the intervention that will be carried out over the course of the next years. In order to increase the response rate for the midline and endline evaluations, the evaluator recommends to redesign the sampling methodology for midline and endline evaluations to exclude the smallest schools from the sampling list and randomly sample the remaining schools (increasing the chances of reaching the expected student sampling sizes). Some components of the process evaluation could be: scope (the percentage of members of the target audience that were exposed to any component of the intervention), administered dose (the number of units administered by the intervention agents), received dose (degree in which the target audience participates actively and receives intervention activities) and fidelity (how well the components of the intervention were delivered according to the operations manual). In addition, it could be supplemented with measures such as the teacher's performance and the operability and acceptability of the intervention. An international model called RE-AIM (by its acronym in English) and proposed by Russell Glasgow, evaluates the implementation of the intervention, and not only its effectiveness. This model considers five factors: scope, effectiveness, acceptability, implementation and sustainability. The results in the factors related to the organization could provide feedback to the public-private partnerships that are generated to implement projects in education. 39 When evaluating the first post-raised data results in the field, teachers, school principals and DCs from the area of Santa María Chiquimula, where 65% of the sample is located, were interviewed. They mentioned that there are organizations that provide schools with support (Fe y Alegría, AECID, private initiative supports and faith-based organizations) 74 X. REFERENCES Centro de Investigaciones Económicas Nacionales (CIEN). (2015) Educación Preprimaria. Disponible en http://www.mejoremosguate.org/cms/content/files/biblioteca/propuesta/Educacion_Pre_primaria. pdf Consejo Nacional de Desarrollo Urbano y Rural (CONADUR), Secretaría de Planificación y Programación de la Presidencia (SEGEPLAN). (2014) Plan Nacional de Desarrollo K’atun: nuestra Guatemala 2032. Disponible en: http://www.undp.org/content/dam/guatemala/docs/publications/undp_gt_PND_Katun2032.pdf Dirección General de Evaluación e Investigación Educativa (DIGEDUCA). (2014). Evaluación de primaria 2014. Informe de resultados para el departamento de Totonicapán. Disponible en: http://www.mineduc.gob.gt/digeduca/documents/Resultados_Primaria_2014/Totonicapan.pdf Dirección General de Evaluación e Investigación Educativa (DIGEDUCA). (2017). Resultados Evaluación Graduandos 2017. Disponible en: http://www.mineduc.gob.gt/digeduca/documents/resultados/Trifoliar_Digeduca_Graduandos_20 17.pdf Instituto Nacional de Estadística (INE). (2013) Caracterización departamental Totonicapán 2013. Disponible en: https://www.ine.gob.gt/sistema/uploads/2015/07/20/EfsWFqUtoEkcXfE2PB1sVbSpfVPHbJVY.p df Ministerio de Economía de Guatemala (MINECO). (2015). Información Socioeconómica de Guatemala. Departamento de: Totonicapán. Disponible en: http://dae.mineco.gob.gt/mapainteractivo/index.php?controller=crm&action=detalles&id=21#Gru pos%20%C3%89tnicos Ministerio de Educación (MINEDUC). (s.f.). Situación Educativa Departamento de Totonicapán. Disponible en: http://siteresources.worldbank.org/GUATEMALAINSPANISHEXT/Resources/500796- 1166830633691/9.PresentacionTotonicapan.pdf Ministerio de Educación (MINEDUC). (2016). Anuario estadístico de la educación en Guatemala, C.A. Estadísticas Educativas Cifras e Indicadores. Disponible en: http://estadistica.mineduc.gob.gt/anuario/2016/main.html Organización de las Naciones Unidas para la Educación, la Ciencia y la Cultura (UNESCO). (2009). Reporte Técnico SERCE: Segundo Estudio Regional Comparativo y Explicativo. Los aprendizajes de los estudiantes de América Latina y el Caribe. Disponible en: http://unesdoc.unesco.org/images/0019/001902/190297s.pdf Programa de las Naciones Unidas para el Desarrollo (PNUD). (2014). Tasa de alfabetismo por sexo según departamento. Disponible en: http://desarrollohumano.org.gt/estadisticas/estadisticas-genero/tasa-de-alfabetismo-por-sexo￾segun-departamento/ Secretaría de Planificación y Programación de la Presidencia (SEGEPLAN). (2010) Tercer informe de avances en el cumplimiento de los objetivos de desarrollo del milenio. Disponible en: http://www.gt.undp.org/content/dam/guatemala/docs/publications/UNDP_gt_ODM_III%20Inform e%20ODM.pdf 75 XI. ANNEXES40 11.1. Learning for Life Baseline indicator data, disaggregated # Indicator Baseline data All measurements are percentages, unless otherwise indicated with “#” 1 Percent of students who, by the end of first grade of primary school, demonstrate that they can read and understand the meaning of grade￾level text Standard Indicator: # 26 Variable CG 1-A 1-B 2-A 2-B 2-C Global Level 1- Emergent 45.9 55.8 43.8 52.3 58.3 1.741 Girls= 1.9 Boys= 1.6 Level 2 – Initial 47.4 39.8 45.5 42.7 36.8 Level 3 - Decoder 3.9 3.4 7.7 3.9 4 Level 4 - Initial Automated 2.1 0.7 1.2 0.7 0.3 Level 5 - Fluent 0.8 0.2 1.8 0.4 0.6 2 Percent of students (girls and boys) who, by the end of 3rd and 6th grade, demonstrate grade level reading comprehension as defined by national standards at USDA supported schools Third grade CG 1-A 1-B 2-A 2-B 2-C Global Unsatisfactory 36 36.9 28.8 30.7 35.6 8.29 Girls= 8.1 Boys= 8.5 Third grade= 9.3 Girls= 8.9 Boys= 9.7 Sixth grade= 7.2 Girls= 7.2 Boys= 7.1 Must improve 53.4 58.8 58.5 60.9 52.8 Satisfactory 8.7 4.1 10.8 8.2 10.7 Excellent 1.9 0.2 1.9 0.3 0.9 Sixth grade CG 1-A 1-B 2-A 2-B 2-C Unsatisfactory 40.9 39.0 30.0 30.3 24.7 Must improve 53.9 59.5 60.1 59.7 65.6 Satisfactory 4.6 1.4 7.9 8.4 7.6 Excellent 0.6 0.2 2.0 1.7 2.1 5 Number of individuals who demonstrate use of new child health and nutrition practices as a result of USDA assistance Variable GC 1-A 1-B 2-A 2-B 2-C Global Knowledge on the treatment of diarrhea 92.8 92.8 97.7 95.4 97.7 25.9 # 4,380 volunteers W= 4,205 Deworming 100 93.1 88 87 92 Dietary diversity in schools 5.3 5.8 5.6 5.7 5.5 Handwashing after using the toilet 52.8 48.7 66.7 54.8 51.6 40 For the indicators with number as their unit of measure, an approximate number of individuals has been extrapolated with a 95% confidence level based on the total number of sampled vs. registered beneficiaries and their response. 41 Using for the indicator level 4 and 5 76 Standard Indicator: # 19 Handwashing before eating 55.6 59 69.7 67.7 58.1 M= 175 Brushing teeth after eating 33.3 33.3 63.6 54.8 45.2 Schools that distribute breakfast 30.8 18.2 22.6 12.9 44.4 Schools that distribute mid￾morning meal 61.5 66.7 71.0 80.6 47.2 Schools that implemented the school garden in previous year 22.2 23.1 48.5 45.2 38.7 6 Number of individuals who demonstrate use of new safe food preparation and storage practices as a result of USDA assistance Standard Indicator: # 21 Variable CG 1-A 1-B 2-A 2-B 2-C Global Warehouses exclusively for food 55.6 51.3 69.7 64.5 83.9 67.6 # 11,433 volunteers W=10,747 M=686 Food stored in shelves or inside furniture 75 87.2 81.8 77.4 77.4 Evidence of rodents or plagues 55.6 56.4 54.5 54.8 38.7 Kardex up to date 41.7 43.6 48.5 41.9 51.6 Transportation guides 36.1 59 48.5 58.1 41.9 Food portions estimation table 33.3 56.4 51.5 45.2 41.9 Before cooking, mothers wash their hands 83.3 94.9 97 100 96.8 Washing food before it is cooked 91.7 92.3 97 100 96.8 Use of soap for handwashing at home 36.7 33.1 29 32.1 25.6 Daily cleaning of the house 93.5 86.9 93.1 83.5 92.6 Handwashing after using the toilet at home 42.4 40.4 30.5 40.4 38 Handwashing before cooking at home 56.1 50.6 46.6 50.5 42.1 Handwashing before eating at home 64 49.4 45 42.2 41.9 7 Percent of schools reaching the tier/level 1 of healthy school initiative tiers Variable CG 1-A 1-B 2-A 2-B 2-C Global Classrooms with hygiene corners 50 59 60.6 71 61.3 95.3 Schools without accumulated trash 77.8 89.7 90.9 93.5 93.5 Schools with clean bathrooms 83.3 69.2 81.8 93.5 83.9 Schools that have piped water 94.4 97.4 93.9 93.5 93.5 77 Schools that consume boiled water 97.2 94.9 90.9 90.3 87.1 Kitchens that are trash -free 86.1 92.3 97 100 95.3 Clean classrooms 88.9 82.1 93.9 93.5 96.8 Schools that have a washing basin 69.4 92.3 87.9 80.6 80.6 Schools with soap for handwashing 55.6 51.3 66.7 67.7 54.8 9 Number of teachers/educators /teaching assistants in target schools who demonstrate use of new and quality teaching techniques or tools as a result of USDA assistance Standard Indicator: # 5 Variable CG 1 - A 1 - B 2 - A 2 - B 2 - C Global Teachers that model the strategy and explore the reading material 59.1 79.1 90.5 70.5 85.2 24.5 # 499 W=294 M=205 Momos - tenango=223 Santa Lucia La Reforma=34 San Bartolo Aguas Calientes=30 San Andres Xecul=56 Santa Maria Chiquimula=97 Totonicapan=59 Teachers who read model vocabulary 47 73.1 81 56.8 72.2 Teachers that perform and check the reflection exercises 53 70.1 77.8 63.6 74.1 Teachers that implement group reading 59.3 40.9 44.4 44.8 45.5 One -on -on reading support 51.9 52.3 50.8 52.2 53 Use of posters for learning 25.9 31.8 20.6 25.4 25.8 Use of reading textbook for student 44.4 36.4 42.9 46.3 54.5 Use of workbook for student 37 25 22.2 38.8 25.8 Teachers who have reading workbooks in K'iche' 3 1.5 0 0 0 Teachers who have reading workbooks in Spanish 56.1 13.4 7.9 18.2 1.9 Teachers who have reading workbooks in K'iche' and Spanish 43.9 85.1 92.1 84.1 98.1 12 Number of school administrators and officials in targeted schools who demonstrate use of new techniques or tools as a result of USDA assistance Standard Indicator: # 3 Variable CG 1 - A 1 - B 2 - A 2 - B 2 - C Global Indicator composed of the following variables: a. Knowledge of the reading techniques b. Knowledge of nutrition c. Management 34.5 29.7 54.8 37 48.1 40.4 # 134 W=55 M=79 78 13 Percentage of students in the classrooms defined as "very attentive" using a scale that defines established criteria Variable CG 1 - A 1 - B 2 - A 2 - B 2 - C Global First -grade students who indicate that they pay attention all the time 50.8 39.4 47.8 43.5 42.2 67. 1 Third -grade students who indicate that they pay attention all the time 43.9 42.7 49.3 43 37.1 Sixth -grade students who indicate that they pay attention all the time 49 45.7 52 50.4 39.9 Teachers who indicate that their students participate in the classroom 69.1 73.9 76.2 79.4 79.8 Classrooms where students seemed attentive and followed instructions, according to data collector observation during the study 80.4 78.1 76.7 86.4 81.4 14 Percentage of students in participating primary schools who indicate not being hungry during the school day Variable CG 1 - A 1 - B 2 - A 2 - B 2 - C Global Low level in the hunger scale, first grade 64.7 59 67.4 61 60.4 Lowest level 54.4 Low level in the hunger scale, third grade 48.8 46.9 57.9 46.5 45 Low level in the hunger scale, sixth grade 55.3 54.3 71.2 67.4 56.1 20 Number of students regularly (80%) attending USDA supported classrooms/ schools Standard Indicator: # 1 Variable CG 1 - A 1 - B 2 - A 2 - B 2 - C Global Classroom assistance collected in the final evaluation of Learning for Life phase I Boys 81.7 Girls 87.5 87.8 #42,727 Girls =20,95 2 Boys =21,77 5 Teacher perception a. Usual (60 -79%) regular (8 -100%) Usual: 12.1 Regular: 87.9 16.4 80.6 11.1 84.1 4.5 93.2 3.7 94.4 Classroom assistance observed on the day of the study 85.6 86.3 91.2 89.3 89.4 22 Number of students enrolled in schools receiving USDA assistance Standard Indicator: # 8 Variable CG 1 - A 1 - B 2 - A 2 - B 2 - C Global First phase 2016 33,184 48,664 Girls =23,8 6 3 Second phase 2017 48,664 79 Boys=24,80 1 23 Percentage of parents in participating schools that help their children with their school work Variable CG 1-A 1-B 2-A 2-B 2-C Global Parent who indicates that she/he helps her/his child with their school homework 78.4 74.1 79.4 85.5 83.5 80.1 24 Percentage of CTAs from Totonicapán that have increased their capacity to fulfill their roles and responsibilities As this indicator measures change after project intervention, the midline and endline evaluations will measure this. 0 11.2. Learning for Life Baseline Data Confidence Intervals 11.2.1. Number of individuals who demonstrate use of new child health and nutrition practices as a result of USDA assistance INTERVENTION GROUP 2018 Difference in means 95% confidence interval for the difference Inferior Superior CG 1-A Do you know what to do when your child has diarrhea? 192.81 188.46 197.16 1-B Do you know what to do when your child has diarrhea? 192.77 188.79 196.75 2-A Do you know what to do when your child has diarrhea? 197.71 195.11 200.31 2-B Do you know what to do when your child has diarrhea? 195.41 191.42 199.40 2-C Do you know what to do when your child has diarrhea? 191.74 186.76 196.71 INTERVENTION GROUP 2018 Difference in means 95% confidence interval for the difference Inferior Superior 1-B Received deworming 93.10 83.29 102.91 2-A Received deworming 88.00 74.31 101.69 2-B Received deworming 86.96 72.07 101.85 2-C Received deworming 92.00 80.57 103.43 INTERVENTION GROUP 2018 Difference in means 95% confidence interval for the difference Inferior Superior 1-A HDDS Dietary diversity score of the menu prepared on the day of the study 536.11 441.04 631.18 80 1-B HDDS Dietary diversity score of the menu prepared on the day of the study 507.69 433.18 582.20 2-A HDDS Dietary diversity score of the menu prepared on the day of the study 560.61 454.62 666.59 2-B HDDS Dietary diversity score of the menu prepared on the day of the study 570.97 477.12 664.82 2-C HDDS Dietary diversity score of the menu prepared on the day of the study 551.61 438.41 664.82 INTERVENTION GROUP 2018 Difference in means 95% confidence interval for the difference Inferior Superior 1-A Wash their hands with soap after using the toilet 52.78 35.65 69.91 Wash their hands with soap before eating 55.56 38.50 72.61 Brush their teeth after eating 33.33 17.16 49.51 1-B Wash their hands with soap after using the toilet 48.72 32.30 65.13 Wash their hands with soap before eating 58.97 42.82 75.13 Brush their teeth after eating 33.33 17.85 48.81 2-A Wash their hands with soap after using the toilet 66.67 49.69 83.64 Wash their hands with soap before eating 69.70 53.15 86.25 Brush their teeth after eating 63.64 46.31 80.96 2-B Wash their hands with soap after using the toilet 54.84 36.28 73.39 Wash their hands with soap before eating 67.74 50.31 85.17 Brush their teeth after eating 54.84 36.28 73.39 2-C Wash their hands with soap after using the toilet 51.61 32.98 70.25 Wash their hands with soap before eating 58.06 39.67 76.46 Brush their teeth after eating 45.16 26.61 63.72 INTERVENTION GROUP 2018 Difference in means 95% confidence interval for the difference Inferior Superior 1-A Schools that distribute breakfast 44.44 27.39 61.50 Schools that distribute a mid morning meal 47.22 30.09 64.35 1-B Schools that distribute breakfast 30.77 15.61 45.93 Schools that distribute a mid morning meal 61.54 45.56 77.52 2-A Schools that distribute breakfast 18.18 4.29 32.07 Schools that distribute a mid morning meal 66.67 49.69 83.64 2-B Schools that distribute breakfast 22.58 6.99 38.17 Schools that distribute a mid morning meal 70.97 54.04 87.89 2-C Schools that distribute breakfast 12.90 .40 25.40 Schools that distribute a mid morning meal 80.65 65.91 95.38 81 11.2.2. Number of individuals who demonstrate use of new safe food preparation and storage practices as a result of USDA assistance INTERVENTION GROUP 2018 Value 95% confidence interval for the difference Inferior Superior 1-A Storage exclusively used for food 55.56 38.50 72.61 Food is kept in shelves or inside a clean and secure unit 75.00 60.14 89.86 There is evidence of rodents or cockroaches (other insect or animal) 55.56 38.50 72.61 Transportation guides are in place 36.11 19.63 52.59 Food portions estimation table is in use 33.33 17.16 49.51 Mothers wash their hands before preparing food 83.33 70.54 96.12 Food is washed before cooking 91.67 82.18 101.15 1-B Storage exclusively used for food 51.28 34.87 67.70 Food is kept in shelves or inside a clean and secure unit 87.18 76.20 98.16 There is evidence of rodents or cockroaches (other insect or animal) 56.41 40.13 72.69 Transportation guides are in place 58.97 42.82 75.13 Food portions estimation table is in use 56.41 40.13 72.69 Mothers wash their hands before preparing food 94.87 87.63 102.12 Food is washed before cooking 97.44 92.25 102.63 2-A Storage exclusively used for food 69.70 53.15 86.25 Food is kept in shelves or inside a clean and secure unit 81.82 67.93 95.71 There is evidence of rodents or cockroaches (other insect or animal) 54.55 36.62 72.48 Transportation guides are in place 48.48 30.49 66.48 Food portions estimation table is in use 51.52 33.52 69.51 Mothers wash their hands before preparing food 96.97 90.80 103.14 2-B Storage exclusively used for food 64.52 46.68 82.36 Food is kept in shelves or inside a clean and secure unit 77.42 61.83 93.01 There is evidence of rodents or cockroaches (other insect or animal) 54.84 36.28 73.39 Transportation guides are in place 58.06 39.67 76.46 Food portions estimation table is in use 45.16 26.61 63.72 2-C Storage exclusively used for food 83.87 70.16 97.58 Food is kept in shelves or inside a clean and secure unit 77.42 61.83 93.01 There is evidence of rodents or cockroaches (other insect or animal) 38.71 20.55 56.87 Transportation guides are in place 41.94 23.54 60.33 Food portions estimation table is in use 41.94 23.54 60.33 Mothers wash their hands before preparing food 96.77 90.19 103.36 Food is washed before cooking 96.77 90.19 103.36 INTERVENTION GROUP 2018 Difference in means 95% confidence interval for the difference Inferior Superior 1-A 5. Use of soap for handwashing 36.69 28.58 44.80 2. Cleaning the house daily 93.53 89.38 97.67 3. Handwashing after using the toilet 42.45 34.13 50.77 5. Handwashing before preparing food 47.48 39.08 55.89 2. Handwashing before eating 64.03 55.95 72.11 1-B 5. Use of soap for handwashing 33.13 25.90 40.37 2. Cleaning the house daily 86.14 80.83 91.45 3. Handwashing after using the toilet 40.36 32.82 47.90 5. Handwashing before preparing food 45.78 38.12 53.44 2. Handwashing before eating 49.40 41.71 57.08 82 INTERVENTION GROUP 2018 Difference in means 95% confidence interval for the difference Inferior Superior 2-A 5. Use of soap for handwashing 29.01 21.13 36.88 2. Cleaning the house daily 93.13 88.74 97.52 3. Handwashing after using the toilet 30.53 22.54 38.53 5. Handwashing before preparing food 31.30 23.25 39.34 2. Handwashing before eating 45.04 36.41 53.67 2-B 5. Use of soap for handwashing 32.11 23.20 41.02 2. Cleaning the house daily 83.49 76.40 90.57 3. Handwashing after using the toilet 40.37 31.01 49.73 5. Handwashing before preparing food 39.45 30.13 48.77 2. Handwashing before eating 42.20 32.78 51.62 2-C 5. Use of soap for handwashing 25.62 17.73 33.51 2. Cleaning the house daily 92.56 87.82 97.30 3. Handwashing after using the toilet 38.02 29.24 46.79 5. Handwashing before preparing food 43.80 34.83 52.77 2. Handwashing before eating 47.93 38.90 56.96 11.2.3. Number of teachers/educators/teaching assistants in target schools who demonstrate use of new and quality teaching techniques or tools as a result of USDA assistance. INTERVENTION GROUP 2018 Difference in means 95% confidence interval for the difference Inferior Superior 1-A 3. Have you been trained by the Learning for Life program in the methodologies or in the reading programs? 20.69 5.01 36.37 7. Do you implement CRS’s Learning for Life methodologies? 24.14 7.57 40.70 1-B 3. Have you been trained by the Learning for Life program in the methodologies or in the reading programs? 75.68 61.17 90.18 7. Do you implement CRS’s Learning for Life methodologies? 70.27 54.82 85.72 32. Do you carry out educational planning and coordination activities in your school? 97.30 91.82 102.78 83 INTERVENTION GROUP 2018 Difference in means 95% confidence interval for the difference Inferior Superior 2-A 3. Have you been trained by the Learning for Life program in the methodologies or in the reading programs? 90.32 79.30 101.35 7. Do you implement CRS’s Learning for Life methodologies? 90.32 79.30 101.35 32. Do you carry out educational planning and coordination activities in your school? 96.77 90.19 103.36 2-B 3. Have you been trained by the Learning for Life program in the methodologies or in the reading programs? 96.30 88.68 103.91 7. Do you implement CRS’s Learning for Life methodologies? 85.19 70.86 99.51 32. Do you carry out educational planning and coordination activities in your school? 96.30 88.68 103.91 2-C 3. Have you been trained by the Learning for Life program in the methodologies or in the reading programs? 88.89 76.22 101.56 7. Do you implement CRS’s Learning for Life methodologies? 85.19 70.86 99.51 84 11.3. Learning for Life Baseline Indicator data: treatment and control groups # Indicator Variable CG 1-A Intervention * 42 1 Percent of students who, by the end of first grade of primary school, demonstrate that they can read and understand the meaning of grade-level text. First-grade attainment 2.85% 1.4% 2 Percent of students (girls and boys) who, by the end of 3rd and 6th grade, demonstrate grade level reading comprehension as defined by national standards at USDA supported schools Third-grade attainment 10.63% 9.04% Sixth-grade attainment 5.2% 7.6% 5 Number of individuals who demonstrate use of new child health and nutrition practices as a result of USDA assistance Knowledge on the treatment of diarrhea 92.8% 94.3% Deworming 100.0% 90.2% Dietary diversity at school 5.36 5.46 Washing hands after using the toilet 52.8% 55.2% Washing hands before eating 55.6% 63.4% Brushing teeth 33.3% 48.5% Schools that distribute breakfast 44.4% 21.6% Schools that distribute a mid morning meal in the morning 47.2% 69.4% Schools that implemented a vegetable garden in the previous year 22.2% 38.1% 6 Number of individuals who demonstrate use of new safe food preparation and Schools with spaces meant exclusively for food storage 55.6% 66.4% 42 Note: In this report, the “intervention” category was calculated with groups 1-B, 2-A, 2-B, and 2-C even though group 1-B had not yet received any literacy interventions at the time of data collection. Group 2-B’s literacy indicator for 3rd and 6th grade is notably lower, which likely reflects that this group had not had any literacy interventions prior to baseline, unlike groups 2-A, 2-B and 2-C. 85 # Indicator Variable CG 1-A Intervention * 42 storage practices as a result of USDA assistance Food in shelves or stored inside furniture 75.0% 81.3% Evidence of rodents or other plagues 55.6% 51.5% Kardex up to date 41.7% 46.3% Transportation guides 36.1% 52.2% Food portions estimation table 33.3% 49.3% Handwashing before cooking 83.3% 97.0% Washing food before cooking 91.7% 98.5% Use of soap for handwashing at home 36.7% 30.2% Houses are cleaned daily 93.5% 88.8% Handwashing after using the toilet at home 42.4% 37.4% Handwashing before cooking at home 56.1% 47.6% Handwashing before eating at home 64.0% 46.5% 7 Percent of schools reaching the tier/level 1 of healthy school initiative tiers Classrooms with hygiene corners 50.0% 62.7% Schools without accumulated trash 77.8% 91.8% Schools with clean bathrooms 83.3% 81.3% Schools that have piped water 94.4% 94.8% Schools that consume boiled water 97.2% 91.0% Kitchens that are trash-free 86.1% 95.5% Clean classrooms 88.9% 91.0% 86 # Indicator Variable CG 1 - A Intervention * 42 Schools that have a washing basin 69.4% 85.8% Schools with soap for handwashing 55.6% 59.7% 9 Number of teachers/educators/teaching assistants in target schools who demonstrate use of new and quality teaching techniques or tools as a result of USDA assistance Teachers that model the strategy and explore the reading material 59.1% 82.0% Teachers who read model vocabulary 47.0% 71.9% Teachers that perform and check the reflection exercises 53.0% 71.9% Teachers that implement group reading 56.1% 67.5% One -on -on reading support 53.0% 51.8% Use of posters for learning 25.8% 25.4% Use of reading textbook for student s 54.5% 43.0% Use of student workbook 25.8% 31.1% Teachers who have reading workbooks in K'iche' 3.0% 3.1% Teachers who have reading workbooks in Spanish 62.1% 16.2% Teachers who have reading workbooks in K'iche' and Spanish 37.9% 81.1% 12 Number of school administrators and officials in targeted schools who demonstrate use of new techniques or tools as a result of USDA assistance School principals trained by Learning for Life 20.7% 86.9% School principals that implement Learning for 24.1% 82.0% 87 # Indicator Variable CG 1 - A Intervention * 42 Life methodologies Principals whose schools have a school improvement plan 93.1% 90.2% 13 Percentage of students in the classrooms defined as "very attentive" using a scale that defines established criteria First -grade students who indicate that they pay attention all the time 50.8% 42.9% Third -grade students who indicate that they pay attention all the time 43.9% 42.9% Sixth -grade students who indicate that they pay attention all the time 49.0% 46.8% Teachers who indicate that their students participate in the classroom 69.15 77.07 Classrooms where students seemed attentive and followed instructions, according to data collector observation during the study 80.42 80.39 14 Percentage of students in participating primary schools who indicate not being hungry during the school day Low level in the hunger scale, first grade 64.7% 61.8% Low level in the hunger scale, third grade 48.8% 48.8% Low level in the hunger scale, sixth grade 55.3% 61.8% 20 Number of students regularly (80%) attending Classroom assistance collected in the 87.9% 87.3% 88 # Indicator Variable CG 1-A Intervention * 42 USDA supported classrooms/schools final evaluation of Learning for Life phase I 89 11.4. Learning for Life Baseline Indicator data: global indicator and indicator’s representation as a whole number (out of entire sample for given indicator) # Indicator Variable Quantity Indicator 1 Percent of students who, by the end of first grade of primary school, demonstrate that they can read and understand the meaning of grade-level text First-grade attainment 31 1.7 2 Percent of students (girls and boys) who, by the end of 3rd and 6th grade, demonstrate grade level reading comprehension as defined by national standards at USDA supported schools Third-grade attainment 202 9.3 Sixth-grade attainment 141 7.2 5 Number of individuals who demonstrate use of new child health and nutrition practices as a result of USDA assistance Knowledge on the treatment of diarrhea 626 93.9 Deworming 118 92.1 Dietary diversity at school 36 5.4 Washing hands after using the toilet 93 54.7 Washing hands before eating 105 61.8 Brushing teeth 77 45.3 Schools that distribute breakfast 45 26.5 Schools that distribute a mid morning meal in the morning 110 64.7 Schools that implemented a vegetable garden in the previous year 59 34.7 6 Number of individuals who demonstrate use of new safe food preparation and storage practices as a result of USDA assistance Schools with spaces meant exclusively for food storage 109 64.1 Food in shelves or stored inside furniture 136 80.0 Evidence of rodents or other plagues 89 52.3 Kardex up to date 77 45.2 Transportation guides 83 48.8 Food portions estimation table 78 45.8 Handwashing before cooking 160 94.1 90 # Indicator Variable Quantity Indicator Washing food before cooking 165 97.0 Use of soap for handwashing at home 210 31.5 Houses are cleaned daily 598 89.7 Handwashing after using the toilet at home 256 38.4 Handwashing before cooking at home 329 49.4 Handwashing before eating at home 334 50.1 7 Percent of schools reaching the tier/level 1 of healthy school initiative tiers Classrooms with hygiene corners 102 60.0 Schools without accumulated trash 151 88.8 Schools with clean bathrooms 139 81.7 Schools that have piped water 161 94.7 Schools that consume boiled water 157 92.3 Kitchens that are trash￾free 159 93.5 Clean classrooms 154 90.5 Schools that have a washing basin 140 82.3 Schools with soap for handwashing 100 58.8 9 Number of teachers/educators/teaching assistants in target schools who demonstrate use of new and quality teaching techniques or tools as a result of USDA assistance Teachers that model the strategy and explore the reading material 226 76.8 Teachers who read model vocabulary 195 66.3 Teachers that perform and check the reflection exercises 199 67.6 Teachers that implement group reading 191 64.9 One-on-on reading support 153 52.0 Use of posters for learning 75 25.5 Use of reading textbook for students 134 45.5 Use of student workbook 88 29.9 91 # Indicator Variable Quantity Indicator Teachers who have reading workbooks in K'iche' 9 3.0 Teachers who have reading workbooks in Spanish 78 26.5 Teachers who have reading workbooks in K'iche' and Spanish 210 71.4 12 Number of school administrators and officials in targeted schools who demonstrate use of new techniques or tools as a result of USDA assistance School principals trained by Learning for Life 112 74.1 School principals that implement Learning for Life methodologies 107 70.8 Principals whose schools have a school improvement plan 137 90.7 13 Percentage of students in the classrooms defined as "very attentive" using a scale that defines established criteria First-grade students who indicate that they pay attention all the time 800 44.5 Third-grade students who indicate that they pay attention all the time 935 43.0 Sixth-grade students who indicate that they pay attention all the time 927 47.1 Teachers who indicate that their students participate in the classroom 1478 75.2 Classrooms where students seemed attentive and followed instructions, according to data collector observation during the study 1301 66.1 14 Percentage of students in participating primary schools who indicate not being hungry during the school day Low level in the hunger scale, first grade 1120 62.3 Low level in the hunger scale, third grade 1060 48.8 Low level in the hunger scale, sixth grade 1192 60.6 20 Number of students regularly (80%) attending USDA supported classrooms/schools Classroom assistance collected in the final evaluation of Learning for Life phase I 257 87.8 92 11.5. Learning for Life Baseline Indicators, characteristics of the sample Average age of parents and mothers that participated in the study disaggregated by sex Group Female Male CG 1-A 36.7 43.4 1-B 37.4 44.2 2-A 36.8 44.1 2-B 36.7 40.1 2-C 36.5 41.1 Mean age of parents of first-grade students that participated in the study disaggregated by sex Group Female Male CG 1-A 34.1 37.5 1-B 33.6 40.6 2-A 34.5 36.4 2-B 33.9 39.6 2-C 34.5 39.8 Total number of mothers and fathers of first-grade students that participated in the study, by treatment and control group, disaggregated by sex Group Total no. of women in the sample Total no. of women surveyed Total no. of men surveyed Total CG 1-A 160 89 25 114 1-B 160 84 24 108 2-A 136 71 13 84 2-B 132 84 13 97 2-C 128 73 22 95 Total 716 401 97 498 Difference 218 Total number of teachers that participated in the study by treatment and control group disaggregated by sex Group Female Male Total CG 1-A 36 30 66 1-B 40 27 67 2-A 36 27 63 2-B 28 16 44 2-C 34 20 54 Total 174 120 294 Total number of school principals that participated in the study by treatment and control group, disaggregated by sex Group Female Male Total CG 1-A 12 17 29 1-B 12 25 37 2-A 20 11 31 2-B 10 17 27 93 2-C 10 17 27 Total 64 87 151 Total number of first-grade teachers that participated in the study by treatment and control groups, disaggregated by sex Group Female Male Total CG 1-A 21 8 29 1-B 18 11 29 2-A 17 6 23 2-B 16 6 22 2-C 14 5 19 Total 86 36 122 Language spoken by the teachers that participated in the study by treatment and control groups Group K´iche´ Spanish Spanish and K’iche’ CG 1-A 0 11 55 1-B 1 6 60 2-A 3 10 54 2-B 3 4 38 2-C 0 12 42 Language spoken by the principals that participated in the study by treatment and control groups Group K´iche´ Spanish Spanish and K’iche’ CG 1-A 0 1 28 1-B 1 5 31 2-A 0 5 26 2-B 0 7 20 2-C 0 2 25 Average age of teachers and school principals that participated in the study disaggregated by sex Group Female Male CG 1-A 36.8 36.2 1-B 36.0 37.1 2-A 36.7 38.0 2-B 35.7 37.4 2-C 36.9 35.9 94 11.6. Learning for Life Baseline data: statistical significance 11.6.1. Statistical significance: Student assistance according to teacher’s perception Chi-square tests between CG 1-A and Group 1-B Value df Asymptotic significance Exact significance (2-sided) Exact significance (1-sided) Pearson's Chi￾square .609a 1 .435 Continuity correction b .283 1 .595 Likelihood ratio .611 1 .434 Fisher’s Exact test .467 .298 Linear-by-linear association .604 1 .437 No. of Valid cases 131 a. 0 cells (0.0%) have expected count less than 5. The minimum expected count is 9.43 b. Computed only for a 2x2 table Chi-square tests between CG 1-A and Group 2-A Value df Asymptotic significance Exact significance (2-sided) Exact significance (1-sided) Pearson's Chi￾square .006a 1 .937 Continuity correction b 0.000 1 1.000 Likelihood ratio .006 1 .937 Fisher’s Exact test 1.000 .579 Linear-by-linear association .006 1 .938 No. of Valid cases 126 a. 0 cells (0.0%) have expected count less than 5. The minimum expected count is 7.14. b. Computed only for a 2x2 table 95 Chi-square tests between CG 1-A and Group 2-B Value df Asymptotic significance Exact significance (2-sided) Exact significance (1-sided) Pearson's Chi￾square 1.744a 1 .187 Continuity correction b .962 1 .327 Likelihood ratio 1.899 1 .168 Fisher’s Exact test .310 .164 Linear-by-linear association 1.728 1 .189 No. of Valid cases 109 a. 1 cell (25.0%) has expected count less than 5. The minimum expected count is 3.94. b. Computed only for a 2x2 table Chi-square tests between GC 1-A and 2-C Value df Asymptotic significance Exact significance (2-sided) Exact significance (1-sided) Pearson's Chi￾square 2.661a 1 .103 Continuity correction b 1.687 1 .194 Likelihood ratio 2.882 1 .090 Fisher’s Exact test .182 .095 Linear-by-linear association 2.639 1 .104 No. of Valid cases 119 a. 1 cell (25.0%) has expected count less than 5. The minimum expected count is 4.45. b. Computed only for a 2x2 table 96 11.6.2. Statistical significance: Parents’ level of education Fathers with complete-incomplete primary schooling Chi-squared tests Value df Asymptotic significance Exact significance (2-sided) Exact significance (1-sided) Pearson's Chi￾square 3.633a 1 .057 Continuity correction b 3.101 1 .078 Likelihood ratio 3.860 1 .049 Fisher’s Exact test .060 .036 Linear-by-linear association 3.623 1 .057 No. of Valid cases 361 a. 0 cells (0.0%) have expected count less than 5. The minimum expected count is 20.57. b. Computed only for a 2x2 table Mothers with complete-incomplete primary schooling Chi-squared tests Value df Asymptotic significance Exact significance (2-sided) Exact significance (1-sided) Pearson's Chi￾square .476a 1 .490 Continuity correction b .267 1 .605 Likelihood ratio .491 1 .483 Fisher’s Exact test .604 .308 Linear-by-linear association .475 1 .491 No. of Valid cases 330 a. 0 cells (0.0%) have expected count less than 5. The minimum expected count is 12.99. b. Computed only for a 2x2 table 97 Fathers know how to read-do not know how to read Chi-squared tests Value df Asymptotic significance Exact significance (2-sided) Exact significance (1-sided) Pearson's Chi￾square 4.163a 1 .041 Continuity correction b 3.410 1 .065 Likelihood ratio 3.951 1 .047 Fisher’s Exact test .053 .035 Linear-by-linear association 4.142 1 .042 No. of Valid cases 195 a. 0 cells (0.0%) have expected count less than 5. The minimum expected count is 11.73. b. Computed only for a 2x2 table Mothers know how to read-do not know how to read Chi-squared tests Value df Asymptotic significance Exact significance (2-sided) Exact significance (1-sided) Pearson's Chi￾square .467a 1 .494 Continuity correction b .215 1 .643 Likelihood ratio .490 1 .484 Fisher’s Exact test .661 .331 Linear-by-linear association .466 1 .495 No. of Valid cases 297 a. 0 cells (0.0%) have expected count less than 5. The minimum expected count is 7.56. b. Computed only for a 2x2 table 98 11.6.3. Correlation: parents’ level of schooling with the help they provide their child Mother’s level of schooling Mother’s level of schooling 1. Do you help your son / daughter with their homework? Mark the answer Mother’s level of schooling Pearson’s correlation 1 -.152** Sig. (2 tailed) .006 N 330 330 1. Do you help your son / daughter with their homework? Mark the answer Pearson’s correlation -.152** 1 Sig. (2-tailed) .006 N 330 330 **. Correlation is significant at the 0.01 level (2-tailed) Father’s level of schooling Father’s level of schooling 1. Do you help your son / daughter with their homework? Mark the answer Father’s level of schooling Pearson’s correlation 1 -.040 Sig. (2-tailed) .445 N 361 361 1. Do you help your son / daughter with their homework? Mark the answer Pearson’s correlation -.040 1 Sig. (2-tailed) .445 N 361 361 11.6.4. Correlation: knowledge of the Learning for Life methodologies or reading programs￾implementation of the methodologies 1. Are you familiar with the Learning for Life methodologies or reading programs? 7. Do you implement CRS’s Learning for Life methodologies? 1. Are you familiar with the Learning for Life methodologies or reading programs? Pearson’s correlation 1 .667** Sig. (2- tailed) .000 N 291 285 7. Do you implement CRS’s Learning for Life methodologies? Pearson’s correlation .667** 1 Sig. (2- tailed) .000 N 285 287 **. Correlation is significant at the 0.01 level (2-tailed) 99 11.6.5. Teacher is familiar with the Learning for Life methodologies or reading programs-it is observed that the teacher handles the tools in L1 and L2. Chi-squared tests In Spanish L1 Value df Asymptotic significance Exact significance (2-sided) Exact significance (1-sided) Pearson's Chi￾square .929a 1 .335 Continuity correction b .688 1 .407 Likelihood ratio .926 1 .336 Fisher’s Exact test .349 .203 Linear-by-linear association .926 1 .336 No. of Valid cases 294 a. 0 cells (0.0%) have expected count less than 5. The minimum expected count is 33.42. b. Computed only for a 2x2 table In K’iche’ L2 Value df Asymptotic significance Exact significance (2-sided) Exact significance (1-sided) Pearson's Chi￾square .584a 1 .445 Continuity correction b .396 1 .529 Likelihood ratio .582 1 .445 Fisher’s Exact test .501 .264 Linear-by-linear association .582 1 .446 No. of Valid cases 294 a. 0 cells (0.0%) have expected count less than 5. The minimum expected count is 33.16. b. Computed only for a 2x2 table 100 11.7. Proposal for constructing Learning for Life healthy schools indicator Percent of schools reaching the tier/level 1 of healthy school initiative tiers for the Learning for Life Project School classrooms with hygiene corners + Schools without accumulated trash + Schools with clean bathrooms + Schools that have piped water + Schools that consume boiled water + Trash-free kitchens + Clean classrooms + Schools that have a washing basin + Schools with soap for handwashing