BASELINE REPORT EVALUATION OF THE SAVE THE CHILDREN’ LENS PROGRAM DATE: OCTOBER 31, 2019 PRESENTED TO: SAVE THE CHILDREN FEDERATION, INC. ATTN: CLARA PAVA 501 KINGS HIGHWAY EAST SUITE 400 FAIRFIELD, CT 06825 PRESENTED BY: NORC AT THE UNIVERSITY OF CHICAGO ATTN: ALEJANDRO OME 4350 EAST-WEST HIGHWAY SUITE 800 BETHESDA, MD 20184 Baseline Evaluation of the Save the Children’ FY 2018 McGovern-Dole LENS Program in Guatemala Program: McGovern-Dole International Food for Education and Child Nutrition Agreement Number: FFE-520-2018/012-00 Funding Year: Fiscal Year 2018 Project Duration: 2018 - 2023 Implemented by: Save the Children Evaluation Authored by: NORC and the University of Chicago DISCLAIMER: This publication was produced at the request of the United States Department of Agriculture. It was prepared by an independent third-party evaluation firm. The author’s views expressed in this publication do not necessarily reflect the views of the United States Department of Agriculture or the United States Government. Accessibility Note: An accessible version of this document can be made available by contacting fas.monitoring.evaluation@usda.gov NORC | LENS PROGRAM BASELINE REPORT | i Table of Contents Table of Contents......................................................................................................................... i Table of Tables............................................................................................................................ii Table of Figures .........................................................................................................................iv Acronyms ...................................................................................................................................vi Executive Summary.................................................................................................................... 1 Program Background............................................................................................................ 1 Evaluation Purpose and Evaluation Questions..................................................................... 1 Impact Evaluation Design and Findings ............................................................................... 1 Design 1 Findings....................................................................................................................... 2 Program Evaluation Design and Findings ............................................................................ 2 Design 2 Findings....................................................................................................................... 3 Programmatic recommendations.......................................................................................... 6 Evaluation Limitations and Recommendations..................................................................... 7 1. Introduction......................................................................................................................... 9 2. Program Background......................................................................................................... 9 3. Objectives and Research Questions ..............................................................................11 4. Data Collection..................................................................................................................14 4.1 Quantitative Data..........................................................................................................14 4.2 Qualitative Data ............................................................................................................16 4.3 Training.........................................................................................................................16 4.4 Fieldwork and quality control.....................................................................................17 5. Impact evaluation .............................................................................................................18 5.1 Research design overview............................................................................................18 5.2 Sample and Data Collection .........................................................................................19 5.3 Baseline results ............................................................................................................24 NORC | LENS PROGRAM BASELINE REPORT | ii 5.4 Balance.........................................................................................................................29 5.5 Conclusions ..................................................................................................................31 6. Program evaluation ..........................................................................................................32 6.1 Research design overview............................................................................................32 6.2 Sample and data collection...........................................................................................32 6.3 Baseline results ............................................................................................................34 Student background and reading skills .....................................................................34 Teachers’ surveys and interviews .............................................................................42 School Observations ................................................................................................. 64 Local producers .........................................................................................................73 6.4 Conclusions ..................................................................................................................75 7. Programmatic recommendations....................................................................................78 8. Limitations and recommendations for the evaluation ..................................................79 Annex I. Instruments ................................................................................................................81 Annex II. Results for all ELI subtasks .....................................................................................89 Annex III. Average reading fluency and reading comprehension, by language most spoken at home.......................................................................................................................130 Table of Tables Table 1. ELI Assessment Overview............................................................................................................ 15 Table 2. Parent FGD Gender Breakdown ................................................................................................... 17 Table 3. Target and surveyed treatment schools by municipality............................................................... 24 Table 4. Target and surveyed comparison schools by municipality ........................................................... 24 Table 5. Sociodemographic characteristics – IE ......................................................................................... 25 Table 6. Nutrition – IE ................................................................................................................................ 26 Table 7. PSM results - Balanced characteristics at baseline ....................................................................... 30 Table 8. Number of samples schools by municipality ................................................................................ 33 Table 9. Program Evaluation Data Collection ............................................................................................ 33 Table 10. Sociodemographic characteristics by students’ sex – PE ........................................................... 34 NORC | LENS PROGRAM BASELINE REPORT | iii Table 11. Nutrition by students’ sex – PE .................................................................................................. 35 Table 12. Sociodemographic characteristics by municipality – PE............................................................ 38 Table 13. Nutrition by municipality – PE................................................................................................... 38 Table 14. Correlations between reading skills and sociodemographic characteristics ............................... 42 Table 15. Teachers’ characteristics............................................................................................................. 43 Table 16. School kitchen characteristics (N = 259).................................................................................... 65 Table 17. Food preparation practices.......................................................................................................... 65 Table 18. Kitchen staff interview (N=259)................................................................................................. 66 Table 19. Food storage characteristics (N=251)......................................................................................... 67 Table 20. Toilets/latrines characteristics (N=258)...................................................................................... 69 Table 21. Other school characteristics (N=262) ......................................................................................... 72 Table 22. Harvest after own consumption, selected products (a) ............................................................... 74 NORC | LENS PROGRAM BASELINE REPORT | iv Table of Figures Figure 1. Impact evaluation municipalities................................................................................................. 20 Figure 2. Poverty levels in 2011 by impact evaluation municipalities (%) ................................................ 21 Figure 3. Total students and students per teacher in primary schools in 2017, by impact evaluation municipalities.......................................................................................................................... 22 Figure 4. Reading skills in 3rd grade 2014 national assessments, by impact evaluation municipalities.... 23 Figure 5. Reading skills – IE (%correct / CWPM) ..................................................................................... 27 Figure 6. Reading skills in Spanish and K’iche’ households – IE (%correct / CWPM)............................. 28 Figure 6. Propensity score by treatment status ........................................................................................... 31 Figure 7. Reading skills by students’ sex – PE........................................................................................... 37 Figure 8. Reading skills by municipality – PE............................................................................................ 40 Figure 9. Average reading fluency by number of reading comprehension questions correctly answered. . 41 Figure 10. Language used in class, by local language ................................................................................ 44 Figure 11. La Taza Escolar ......................................................................................................................... 44 Figure 12. Where teachers saw the image for the first time........................................................................ 45 Figure 13. Teachers’ selection of best school lunch ................................................................................... 46 Figure 14. Teachers’ belief that a typical carbohydrate-based diet is nutritious......................................... 47 Figure 15. Foods that help children grow according to teachers ................................................................ 48 Figure 16. Foods that give children energy, according to teachers............................................................. 49 Figure 17. Foods that protect children from illnesses, according to teachers............................................. 50 Figure 18. Moments when students should wash their hands..................................................................... 51 Figure 19. Moments when students washed their hands the day before data collection (as reported by the teacher).................................................................................................................................... 52 Figure 20. Awareness of child anemia (%)................................................................................................. 53 Figure 21. Percentage of teachers’ who identified foods to prevent anemia .............................................. 54 Figure 22. Teachers’ knowledge of the importance of iodized salt ............................................................ 55 Figure 23. Percentage of teachers’ who agreed with the statement that Incaparina is rich in calcium....... 56 Figure 24. Percentage of teachers’ who agreed with the statement that “to pay attention in class, children need to drink enough water.” .................................................................................................. 57 NORC | LENS PROGRAM BASELINE REPORT | v Figure 25. Percentage of teachers’ who agreed that students need to learn about nutrition to prevent obesity. .................................................................................................................................... 58 Figure 26. Negative effects of sugar........................................................................................................... 59 Figure 27. Teaching the risks of too much sugar and teachers’ awareness that drinking water instead of soda helps to prevent diabetes................................................................................................. 60 Figure 28. Percentage of teachers who agree with nutrition-related statements......................................... 61 Figure 29. Teachers’ perception of their own capacity, responsibility, and time availability for nutrition education ................................................................................................................................. 62 Figure 30. Teaching hygiene and nutrition practices.................................................................................. 63 Figure 31. Teachers reporting that students worked in the garden and that students consumed garden produce (only schools with gardens)....................................................................................... 64 Figure 32. Food storage space..................................................................................................................... 66 Figure 33. Iodized salt................................................................................................................................. 67 Figure 34. Sanitary services........................................................................................................................ 68 Figure 35. Characteristics of handwashing facilities (fraction, multiple responses allowed) (N=245)...... 70 Figure 36. Water sources (%) (N=173)....................................................................................................... 71 Figure 37. Water treatment methods (%) (N=132)..................................................................................... 72 NORC | LENS PROGRAM BASELINE REPORT | vi Acronyms DID Difference-In-Difference DIGEDUCA National Directorate For Educational Evaluation And Research EDSSs Electronic Data Storage Systems EGRA Early Grade Reading Assessment ELI Evaluación De Lectura Inicial FGD Focus Group Discussion FtF Feed The Future HFC High Frequency Checks ICT Information And Communication Technology IDB Inter-American Development Bank IE Impact Evaluation IRB Institutional Review Board J&A Juárez And Associates KAP Knowledge, Attitudes, And Perceptions KII Key Informant Interview LASP Local Agriculture Service Provider LENS Literacy Education And Nutrition For Sustainability MAGA Ministry Of Agriculture MCHN Maternal, Child Health, And Nutrition MGD McGovern-Dole MINEDUC Ministry Of Education MOH Ministry Of Health OSY Out-School-Youth PCI Project Concern International PE Performance Evaluation PSM Propensity Score Matching PTA Parent-Teacher Association RCT Randomized Control Trial RFP Request For Proposals SC Save The Children SCUS Save The Children United States SFTP Secure File Transfer Protocol SHN School Health And Nutrition SHRP School Health And Reading Project SO Strategic Objective NORC | LENS PROGRAM BASELINE REPORT | vii TFET Techniques For Effective Teaching UNICEF United Nationals International Children's Emergency Fund USAID United States Agency For International Development USDA United States Department Of Agriculture ViM Victory Against Malnutrition NORC | LENS PROGRAM BASELINE REPORT | 1 Executive Summary This report presents the results of the baseline program and impact evaluations of Save the Children’s Literacy Education and Nutrition for Sustainability (LENS) program, an intervention aimed at improving the quality of literacy instruction, children’s attentiveness and children’s attendance, by decreasing short￾term hunger and increasing the use of health and dietary practices. Program Background LENS is funded by the United States Department of Agriculture’s (USDA) McGovern Dole International Food for Education and Child Nutrition Program (MGD). The primary goals of LENS are to improve the quality of literacy instruction, children’s attentiveness and children’s attendance, by decreasing short-term hunger and increasing the use of health and dietary practices. To achieve these goals SC, the program implementer, has partnered with the Ministry of Education (MINEDUC) and the Ministry of Agriculture, Livestock, and Nutrition (Ministerio de Agricultura Ganadería y Alimentación - MAGA). The LENS program will be implemented over a period of five years, and will target pre-primary and primary students in 274 schools across six program municipalities of the department of Quiché.1 The overall goals of the program are broken out into two overarching Strategic Objectives which include (SO1) improving literacy of school-age children by improving the quality of literacy instruction, student attentiveness, and student attendance, and (SO2) reduce health-related absences by increased use of health and dietary practices. Through the LENS program, Save the Children will incorporate commodities from the MGD program into school feeding activities. Evaluation Purpose and Evaluation Questions The LENS evaluation seeks to provide quantitative and qualitative evidence on the impact of school feeding programs on child education outcomes. The evaluation also aims to address key research questions from the McGovern-Dole Learning Agenda, including: (1) in what ways do the combination of school meal interventions and educational interventions improve education and literacy levels? And (2) what kinds of partnerships with the private sector and/or host country governments are the most effective at ensuring program sustainability? The purpose of the baseline evaluation is to establish conditions in the program area prior to the start of program intervention and thus capture improvements when comparing the baseline to midline and endline data, as well as to control for any difference between treatment and control groups observed at baseline. Impact Evaluation Design and Findings Design The impact evaluation will use a matched difference-in-difference (DiD) model, which combines DiD with matching methods to reduce potential sources of bias and provide a compelling approach to inferring causality. The key assumption behind this model is that, in the absence of treatment, students in treatment 1 Originally there were 283 schools that were supposed to be part of the program. Before data collection started 9 schools decided not to be part of the program. SC is working to get these 9 schools back in the program. NORC | LENS PROGRAM BASELINE REPORT | 2 schools would observe the same change in the outcome of interest than students in the comparison schools. Findings Below we summarize the main baseline findings for the impact evaluation: • In terms of socioeconomic indicators, there are no major differences between treatment and comparison groups. • There are notable differences between treatment and comparison groups in terms of language spoken at home, as there are no students that speak Ixil at home in the comparison group. Although this compromises how comparable are the two impact evaluation groups, we will be able to offset baseline differences using the baseline data, in the context of the DID model. • Household environment seems more suitable for learning how to read for the treatment group than for the comparison group. • Most students ate something before coming to school. Similarly, most students ate something at school the day before data collection. • Across all five reading subtasks, students in the comparison have higher scores than their counterparts in the treatment group, and all differences are statistically significant. • Students read 51 cwpm in the treatment group and 60.7 cwpm in the comparison group. However, reading comprehension results are relatively low, as the treatment group students answered correctly only 37 percent of the items, while the figure for the comparison group is 50 percent. Program Evaluation Design and Findings Design The program evaluation will use a mixed methods approach to address questions on the program’s relevance, effectiveness, efficiency, sustainability and impact. Quantitatively, we will analyze data from the student literacy assessments, student attendance, and teacher responses to the health and nutrition knowledge, attitudes and practices (KAP) survey to study how these outcomes change over waves of data collection. Alongside the quantitative analysis, we will conduct FGDs with parents and KIIs with teachers, school principals, PTA members, LENS staff, local producers and government officials to add depth and nuance in explaining changes we see in the quantitative outcome measures.2 Protocols for FGDs and KIIs for each group will follow the program evaluation questions. Each protocol will be structured such that qualitative and quantitative data can be triangulated to provide insights to answer the different research questions. 2 For the baseline program evaluation, qualitative data was only collected from teachers, parents, and local producers. LENS staff and government officials will be included in future rounds of data collection. NORC | LENS PROGRAM BASELINE REPORT | 3 Findings For the program evaluation the main findings are summarized as follows: Students’ background and reading skills • Students are 9.6 years old, on average. About four fifths of the sample attended preschool, and half the students have repeated or are repeating a year. • Roughly 90 percent of students ate something at school the day before data collection. During focus group discussions parents reported they aren’t sure whether their child needs to eat before school or not because schools sometimes offer breakfast and sometimes a snack (refacción). This suggests that parents should be better informed on what meals children are going to receive during the school day. • Parents suggested menu diversification as a way to improve school meals. As opposed to the student responses, the majority of the focus groups, eight of ten, participants stated that their children are not satisfied with the school meals. The two main reasons for this perception were (1) repetition of meals and (2) quality of the cooked food. • Students in Ixil municipalities come from more disadvantaged backgrounds than students in K’iche’ municipalities. In effect, students in Ixil municipalities are more likely to have repeated a year than students in K’iche’ municipalities, and mothers in K’iche’ municipalities are more likely to be literate than in Ixil municipalities. Moreover, households in K’iche’ municipalities are more likely to have electricity and have more household assets, than households in the Ixil municipalities. • There are major differences in reading skills between municipalities. For example, while students in Cotzal read, one average, 60.5 cwpm, in Chajul this figure is only 38 cwpm. • The data show that students that answer 9-11 questions correctly read, on average, 77 words per minute. • Bivariate regression analyses show that speaking Spanish at home, mother’s literacy and number of household assets, are positively correlated with reading skills. By contrast, age and having repeated a grade are negatively correlated with reading skills. Teacher surveys and interviews • Third grade teachers have relatively small classrooms, with an average of 18 students per classroom. • Teachers’ absenteeism rate is 18 percent, according to teachers’ own reports. • While most teachers use Spanish in class, local languages are also prevalent. In the three K’iche’ municipalities (Cunén, Sacapulas and Uspantán) 94 percent of the teachers use Spanish, 52 percent use K’iche’, 6 percent use Uspanteco and 16 percent use Sacapulteco. In Ixil municipalities (Chajul, Cotzal and Nebaj), 93 percent of the teachers use Spanish, 73 percent use Ixil and 5 percent use K’iche’. NORC | LENS PROGRAM BASELINE REPORT | 4 • Related to teachers’ experience with nutrition education, the majority of teachers (70 percent) reported having seen the Taza Escolar (an image of food groups developed for the Ministry of Education) prior to the survey. • There are important gaps in teachers’ knowledge on what types of foods help children grow and give them energy. 58 percent incorrectly selected leafy greens, vegetables and fruit as types of food that help children grow. Also, only 18 percent of teachers correctly selected cereals, grains and tubers, as sources of energy. • The majority of teachers seem knowledgeable about which types of food protect children from diseases. 64 percent correctly selected leafy greens, vegetables and fruit as foods that protect children from diseases, and the most common incorrect response was milk, meat, eggs and other animal source foods, selected by 23 percent. • Regarding school hygiene, a high percentage of teachers (87 percent) mentioned the importance of washing hands before eating, but only half mentioned the need to wash hands after using the bathroom. Similarly, nearly all teachers (94 percent) said their students washed their hands before eating on the previous day, but less than half mentioned that hand washing occurred after going to the bathroom, indicating a gap area in optimal practices. About a third of teachers mentioned that their students brushed their teeth at school the day before data collection. • In terms of awareness of micronutrients, 94 percent of teachers recognize anemia’s symptoms, but only a quarter could identify two or more foods that prevent anemia. Very few teachers (9 percent) knew why consuming iodized salt is important. Most teachers (89 percent) recognized the fortified blending flour Incaparina as a source of calcium, and three-quarters agreed that staying hydrated by drinking water can improve students’ attention in class. • Although almost all teachers said they teach students about the negative effects of too much sugar, there are gaps in teachers’ understanding about the negative impacts sugar consumption can have. While 70 percent see the association with diabetes, only a third recognize the relationship with obesity, and about half (52 percent) recognize sugar as a cause of cavities. • A high percentage of teachers agreed that students need nutrition education to prevent obesity (94 percent), and 84 percent recognize the importance of teaching children about the effects of junk food. Almost all teachers (96 percent) recognize they have a role as communicators of nutrition￾related messages in their communities, 82 percent feel trained to teacher about nutrition, and 69 percent say they have adequate time in class to teach about nutrition. • Only 28 percent of teachers reported that their schools had a garden. However, among schools with a garden, 80 percent of teachers said their students had worked in the garden in the last two weeks, and about a third of teachers (35 percent) said students consumed produce from the garden during the previous month. School observations • Most schools have a kitchen. 92 percent of the kitchens have a stove; 68 percent have a sink and 72 percent had either running or stored water in the kitchen at the time of the observation. Only 43 percent of kitchens have soap for kitchen staff to wash their hands. 39 percent have a trash can, and 46 percent have the weekly menu on display. NORC | LENS PROGRAM BASELINE REPORT | 5 • In the majority of kitchens the stored water observed (if any) is visibly clean, and utensils and tables are visibly clean. In terms of staff attire, about half do not wear an apron and only 26 percent wear a hairnet or handkerchief over their hair. • In 96 percent of the 262 observed schools there is a food storage space. Food is kept off of the ground in 80 percent of the storage spaces. 91 percent have measurement equipment, 50 percent have a daily stock control sheet, 35 percent have a portioning table on display, 43 percent visibly display the Save the Children logo, and 17 percent have a checklist for visits by Save the Children technical staff. • Enumerators were able to observe the salt package in only 64 percent of the schools. Among these schools where the salt package was seen by enumerators, 94 percent of the salt was confirmed to be iodized. • 88 percent of schools have improved sanitation facilities. These include flush toilets, manual pour-flush toilets, pit latrines with slab and compost latrines. Among the schools with unimproved latrines, the majority have pit latrines without slab, and the rest have other types of latrines. • In 245 schools (94 percent) students have handwashing facilities. Only in 35 percent of schools with handwashing facilities, or 32 percent of all 262 observed schools, are there both water and soap for students to wash their hands. • Water for student consumption is available in only 173 schools (66 percent). 93 percent of these schools, or 61 percent of all 262 observed schools, have an improved water source, such as piped water, rainwater, and protected well or spring. • In 97 percent of schools, school administrators reported that there is a school meals program, funded in most cases by MINEDUC. The majority of schools provide a small meal and a drink (67 percent), based example of a tamale and a hot drink for that option. Less than half of school administrators reported that their school currently provides a full meal (39 percent). • 70 percent of schools have a library or reading room. Local producers • Producers identified possible economic benefits of having secure buyers for their products, including reduced transportation and storage costs. In addition, their products are commonly found as ingredients in the official school menus. This suggests that producers are in a good position to provide food to schools’ feeding programs. • Producers also mentioned that the program are well-positioned to improve students’ nutrition. • One of the most discussed challenges faced by local producers is access to water. This includes not having access to water itself, and not having access to irrigation systems to pump water when it is available. Other challenges mentioned include storage, transportation, sale prices, availability of resources, and lack of technical knowledge. • Three producers reported being registered with the Superintendent of Tax Administration (Superintendencia de Administración Tributaria – SAT). These producers did not mention that it NORC | LENS PROGRAM BASELINE REPORT | 6 was particularly difficult to register. However, two of the remaining producers mentioned they were not registered with the SAT because of the associated fees. • None of the ten interviewed producers, including those who have previously sold food to schools, reported having the new MAGA accreditation that will be required to sell food to schools. • It was challenging to recruit producers for the interviews. If this reflects an overall scarcity of local producers in the area, then LENS may find it more challenging to recruit local producers. Programmatic recommendations The baseline findings revealed the following areas of opportunity for the LENS project: 1. Promote student feedback on menus and preparation. During FGD parents stated that their children are not satisfied with the school meals, primarily due to (1) repetition of meals and (2) inadequate quality of the cooked food (e.g. undercooking). The project can promote additional local foods to assure a variety of dishes. LENS can also help schools collect and use feedback from children to improve recipes and/or meal preparation. 2. Support school communication on school meals. Parents reported they aren’t sure whether their child needs to eat before school or not because schools sometimes offer breakfast and sometimes a snack (refacción). This suggests that parents should be better informed about which meals children are going to receive during the school day. 3. Build on teachers’ interest in nutrition education and knowledge gaps. Findings from the teacher surveys indicate that nearly all teachers recognize the value of providing nutrition education to their students and view themselves, as well as parents, as having the responsibility to teach children about nutrition to improve their health. LENS should find opportunities to capitalize on teachers’ base of interest, providing training on priority topics that teachers and students consider most relevant and which pose the greatest threats to student health. 4. Facilitate active learning about nutrition and health. Given that many teachers already mention engaging in some nutrition-related games with their students, LENS can help teachers maximize the potential of these games. 5. Promote realistic, local solutions to nutrition challenges in Quiché. It is essential that LENS promote locally-available, lower-cost nutrient-rich foods that are feasible for families, given documented income and access constraints faced by families in Quiché. 6. Focus on gaps in school meals systems. Although most schools have kitchens and food storage spaces, there are multiple opportunities for LENS to improve the use of existing infrastructure, address gaps or strengthen processes. 7. Examine and address priority water, sanitation and hygiene constraints. Given that only a third of schools had both soap and water present for students’ handwashing, there is a need for LENS to address the factors that impede availability of water and (especially) soap, and help schools generate feasible, long-term solutions (including improved student monitoring). Regarding hygiene practices, handwashing before eating was better understood and practiced in schools, suggesting a need for LENS to emphasize handwashing after using the bathroom (such as process for students to mark whether water and soap were present at the handwashing place, each time they visit the bathroom). 8. Support optimal use of gardens. Only about a third of of schools have a garden, but schools with gardens report high use of students’ participation in gardening, although a lower number have recently consumed garden produce. This is an opportunity for LENS to support establishment of additional gardens and use of existing gardens for practical nutrition education. NORC | LENS PROGRAM BASELINE REPORT | 7 9. Provide information and support to local producers to mitigate constraints. It was challenging to recruit local producers for the baseline interviews. LENS’ recruiting activities should elaborate on the benefits of program participation, as well as the potential risks and how they could be addressed. In addition, very few of the interviewed producers were registered with the SAT, and most did not even know about the accreditation MAGA requires to participate in schools feeding program. Efforts should be made by LENS and/or the Government of Guatemala to advertise the permits needed to be a school meal provider. Evaluation Limitations and Recommendations All LENS schools were part of Save the Children’s previous school feeding program (Investment for Educational Development in the Highlands Project [IDEA]) so we know that treatment and comparison schools are different in fundamental ways. Not only does this affect comparability of treatment and comparison schools, but it can also affect some of the results found in this baseline study. Regarding comparability, the fact that we are using DID models will allow us to isolate any differences at baseline between treatment and comparison schools. In other words, we do not need to assume that schools are observationally equivalent at baseline for the impact evaluation, only that the treatment schools would observe the same change in the outcomes of interest than the comparison schools, if the treatment schools did not receive treatment. Program substitution could affect the validity of the impact evaluation results. If other organizations are offering services similar to LENS in comparison schools, it would be ideal to collect data to at least control for this substitution. To do this we would need data on what other programs are being implemented in the comparison schools. In comparison schools only students are being surveyed, so we could ask them questions about feeding and literacy programs that are being implemented in their schools; however, it would probably be preferable to survey teachers to collect this type of data. At midline and endline, SC should consider having the evaluation team survey teachers in comparison and treatment schools, so data on alternative school feeding programs and early reading interventions can be recorded. Student attendance is one of the key outcomes of the evaluation. The survey collected attendance data for the surveyed children, and the results indicate that attendance is quite high. Being results for the students attending the day of the survey, these results could be an overestimation of the true attendance rates. As SC will start collecting attendance data more systematically as part of their monitoring activities, the evaluation team will be able to incorporate these data in the analysis.3 The school observations were programmed to be conducted at baseline only. However, it would valuable for the evaluation team to also conduct school observations at midline and endline, to record and analyze progress, at least on the 72 schools that will be visited for the other activities. While key informant interviews (KII) with teachers provided valuable insights on teachers’ views and practices regarding nutrition and hygiene, with fewer interviews the same information could probably be acquired. Previous research shows that 12 interviews are enough to discover between 80 and 90 percent of the themes. Along these lines, Save the Children should consider, for the midline and endline, reducing the number of KIIs not only with teachers, but with principals, PTA members, and government officials. 3 In addition, once the Ministry of Education produces enrollment data for 2019, we will be able to calculate attendance rates for each of the classes where s, as the survey recorded the number of students present the day of data collection. NORC | LENS PROGRAM BASELINE REPORT | 8 Savings from these reductions in KIIs could help cover the costs of surveying teachers in impact evaluation schools and/or conducting school observations at midline and endline, as proposed above. Finally, it was challenging to recruit local producers for the interviews, so the documented results may not replicate if other local producers are interviewed. LENS’s efforts to recruit local producers for the program will intensify over the next two years, in preparation for the training activities to be implemented in years four and five of the program. For the midline data collection, it would be ideal for the evaluation team to have access to the pool of local producers recruited for the program, so a more representative sample of local producers can be interviewed. NORC | LENS PROGRAM BASELINE REPORT | 9 1. Introduction Save the Children (SC) contracted NORC at the University of Chicago to conduct the evaluation of the Literacy Education and Nutrition for Sustainability (LENS) program, an intervention aimed at improving the quality of literacy instruction, children’s attentiveness and children’s attendance, by decreasing short￾term hunger and increasing the use of health and dietary practices, in 274 schools in the department of Quiché, Guatemala. The evaluation will comprise three waves of data collection, namely baseline, midline and endline. NORC will undertake two types of evaluations: (1) A program evaluation (PE) using multiple sources of quantitative and qualitative data collected in six municipalities of the Quiché department, namely Cunén, Chajul, Cotzal, Uspantán, Nebaj and Sacapulas. This will include student assessments, student attendance data, program monitoring data, focus group discussions (FGDs) with parents, and key informant interviews (KIIs) with teachers, school principals, PTA members, LENS staff, local producers and government officials. (2) An impact evaluation (IE), using data collected in the same six municipalities, and in three control municipalities, also located in the Quiché department. NORC will conduct a quasi￾experimental design evaluation and analysis to estimate the impact of school feeding, School Health and Nutrition (SHN) activities, and literacy activities on reading skills and students’ attendance, among other outcomes of interest. For the program evaluation, NORC will randomly sample 72 schools from the 274 selected by LENS for treatment activities. These 274 will receive different combinations of treatment activities. After an assessment of needs, LENS selected 87 of the 274 schools to receive the full set of program activities.4 Of these 87 schools, NORC randomly sampled 50 schools for the impact evaluation. The 50 comparison schools for the impact evaluation were drawn from schools in Quiché outside the six municipalities where LENS is being fielded. Baseline data were collected between May 27 and July 4, 2019. This report presents findings from the collected data. 2. Program Background LENS is funded by the United States Department of Agriculture’s (USDA) McGovern Dole International Food for Education and Child Nutrition Program (MGD). 5 The primary goals of LENS are to improve the quality of literacy instruction, children’s attentiveness and children’s attendance, by decreasing short-term hunger and increasing the use of health and dietary practices. To achieve these goals, SC, the program implementer, has partnered with the Ministry of Education (MINEDUC) and the Ministry of Agriculture, Livestock, and Nutrition (Ministerio de Agricultura Ganadería y Alimentación - MAGA). The LENS program will be implemented over a period of five years, and will target pre-primary and primary students in 274 schools across six program municipalities of the department of Quiché. The overall goals of the program are broken out into two overarching Strategic Objectives which include 4 Originally there were 283 schools that were supposed to be part of the program, 91 of which were going to receive all treatment components. Before data collection started 9 schools decided not to be part of the program, including 4 of the ones that were going to receive all treatment components. SC is working to get these 9 schools back in the program. 5 Cooperative agreement number FFE-520-2018/012-00. NORC | LENS PROGRAM BASELINE REPORT | 10 (SO1) improving literacy of school-age children by improving the quality of literacy instruction, student attentiveness, and student attendance, and (SO2) reduce health-related absences by increased use of health and dietary practices. Through the LENS program, Save the Children will incorporate commodities from the MGD program into school feeding activities. LENS’s literacy strategy will target activities that impact the “5 Ts” (Texts, Tongue, Testing, Teachers, and Time). Below are specific activities that LENS will carry out as part of its literacy strategy. - LENS will address the needs of struggling learners by helping teachers to adopt more inclusive teaching practices, such as through an adaptation of the Student Needs Action Pack (SNAP), which provides teachers with easy-to-implement inclusion strategies for the classroom. - LENS will help teachers adopt multilingual instruction strategies through the introduction of simple strategies into teacher peer learning circles, such as using illustrations or gestures, activating background knowledge, using sentence frames, etc. - LENS will implement reading camp activities in 166 schools. Save the Children will work with key teachers in each school to recruit and train youth volunteers who will serve as camp facilitators, and encourage students with reading difficulties to attend and participate in the camps. - LENS will ensure that children have access to reading materials in their schools and communities through book banks. Book banks are collections of reading materials used by students and community members that are stored in trunks, chests or other mobile storage options. - Throughout LENS, Save the Children will establish literacy roundtables to: i) Plan, guide, and troubleshoot the handover of school feeding, early grade reading activities, and school health and nutrition activities supported by Save the Children and other implementing partners; ii) Coordinate development work across projects so actors are not competing for teacher or school administrator time or taking teachers away from their classroom duties unnecessarily; and iii) Advise MINEDUC on budget allocations in order to make best use of available resources in light of development partner investments, such as USDA and USAID. As part of the LENS Nutrition Strategy: - LENS will provide school meals of iron-fortified rice, black beans, and vegetable oil for an estimated 43,850 students, their teachers and school cooks over the 180-day school term. The selection of the foods and portion size build on experiences under the previous IDEA program, meeting 27 percent of students’ daily caloric requirement, 34 percent of the daily protein requirement, and 39 percent of his/her daily iron requirement. - LENS will support the long-term availability of local foods, such as fruit, vegetables, spices and animal protein. LENS will foster organizational linkages between PTAs and local producers so that MINEDUC’s school meals subsidy can be partially invested in local production, as required by the new school feeding law. For this, LENS will select 50 school communities for a pilot project. In these schools LENS will work with new or existing groups of local food producers, to facilitate connections between them and Parent-Teacher Associations (PTAs). LENS will select a single local agriculture service provider (LASP) to train and work with local producers and PTA members to build a supply chain for sustainable school meal programs. NORC | LENS PROGRAM BASELINE REPORT | 11 - LENS will continue actions started under IDEA to establish school gardens and/or maximize their use, through training and provision of tools, seeds and other supplies. - LENS will produce, in collaboration with the National School Feeding Coordination Group and School Feeding Steering Committee, a pictorial leaflet to be used for the training and certification of school cooks in best practices for safe and hygienic food preparation. - LENS will provide a take-home ration of iron-fortified rice and black beans to pregnant women, lactating mothers with children 0 to 23 months, and children aged 6 to 23 months. Distribution days will be paired with nutrition and health education, peer support and referrals for health services. To complement these activities, LENS will develop radio messages on relevant health, nutrition, early childhood stimulation and parenting topics, for broadcasting on municipal radio stations, and sharing at national and sub-national levels. Note that this program component will not be considered in this evaluation. Finally, LENS will promote safe water, sanitation and hygiene in schools, through the following activities: - LENS will support teachers and school governments to conduct child-to-child activities focused on washing hands and brushing teeth, filling water filters, use of a hygiene traffic light checklist to monitor the cleanliness of school grounds, and report/address WASH deficits with school leadership. - LENS will partner with the Ministry of Health and Public Assistance (MSPAS) to assure that children are dewormed. - LENS will support government capacity, regulatory frameworks and laws with respect to school nutrition, health and hygiene. - LENS will help PTAs to conduct maintenance of school water, sanitation and hygiene infrastructure, and implement guidelines for healthy schools according to the School Feeding Program. - LENS will also support COCODES and PTAs to advocate for improvements in school infrastructure and work with community mayors, an activity started under IDEA, to advocate for funds in the community. Additionally, LENS will sponsor annual celebrations of International School Meals Day to raise awareness of the contribution of school meals to addressing hunger and poverty. Project activities started after baseline data was collected. 3. Objectives and Research Questions Previous research shows that children with nutritional deficiencies are susceptible to metabolic changes that impact cognitive ability and performance of the brain.6 Despite this, studies that have evaluated the impacts of school feeding find that links between school feeding and learner performance are largely 6 What is the relationship between child nutrition and school outcomes?, https://webarchive.nationalarchives.gov.uk/20130320135909/https://www.education.gov.uk/publications/eOrderingDownload/W BL18.pdf NORC | LENS PROGRAM BASELINE REPORT | 12 inconclusive.7 These studies cite data reliability, short implementation timelines, and the complex relationship between child nutrition and schooling outcomes as the primary constraints for detecting impacts of school feeding programs on educational outcomes.8,9 A recent International Initiative for Impact Evaluation (3ie) review of programs in low and middle income countries shows a link between improved nutrition and health and an improved ability to learn, but more rigorous research is needed to fully explore the relationships.10 The LENS evaluation, therefore, offers an excellent opportunity to resolve some of these methodological challenges in order to contribute to the global body of research on the relationship between school feeding and educational outcomes. While the link between school feeding on educational outcomes is unclear, the literature does clearly point to increased school attendance as a result of school feeding programs in low and middle income countries.11 A 2002 evaluation of a school feeding intervention in Bangladesh found that the intervention increased enrollment rates and reduced dropout rates in Bangladeshi schools, particularly for students in food insecure areas.12 The LENS evaluation seeks to provide quantitative and qualitative evidence on the impact of school feeding programs on child education outcomes. The evaluation also aims to address key research questions from the McGovern-Dole Learning Agenda, including: (1) in what ways do the combination of school meal interventions and educational interventions improve education and literacy levels? And (2) what kinds of partnerships with the private sector and/or host country governments are the most effective at ensuring program sustainability? For this evaluation three rounds of data collection will be implemented: baseline, midline and endline. Baseline data collection serves three purposes: 1) to measure pre-implementation values for performance indicators, 2) to provide inputs for setting indicator targets, and 3) to establish baseline values for the impact evaluation and assess the comparability between treatment and comparison students. Below we present the midline evaluation questions. Some questions refer specifically to the pilot program and are denoted with an asterisk. Relevance • Do program stakeholders (students, teachers, PTAs, parents, local agriculture producers and local officials) feel the LENS program is meeting their needs? Why or why not? • How culturally appropriate are the in-school meals provided? • In what ways is the support to local producers appropriate in the program context?* Effectiveness • To what extent has the program achieved its output and outcome targets? 7 Does school breakfast make a difference? An evaluation of an in-school breakfast programme in South Africa, https://www.sciencedirect.com/science/article/pii/S0738059316301808 8 Ibid 9 The Impact of Child Health and Nutrition on Education In Less Developed Countries*, http://emiguel.econ.berkeley.edu/assets/miguel_research/31/_Chapter__Impact_of_Child_Health_and_Nutrition_on_Education.p df 10 The impact of education programmes on learning and school participation in low- and middle-income countries, http://pubdocs.worldbank.org/en/429341481774674870/The-Impact-of-Education-Programmes-on-Learning-and-School￾Participation.pdf 11 Realist review to understand the efficacy of school feeding programmes, https://www.ncbi.nlm.nih.gov/pmc/articles/PMC2043412/ 12 Impact of Feeding Children In School: Evidence From Bangladesh, https://www.researchgate.net/publication/253422918_Impact_of_feeding_children_in_school_evidence_from_Bangladesh NORC | LENS PROGRAM BASELINE REPORT | 13 • What factors have inhibited or facilitated achievement of goals, objectives, and expected results? Efficiency • How timely is the delivery of different intervention components? • Is the program delivering commodities on time and in a way that minimizes waste? • How often and in what ways are schools utilizing produce from local producers to supplement and/or substitute donated food?* Sustainability • According to stakeholders, how has the program influenced the amount of food for school meals purchased from local producers? Who are the local producers that have been most successful in satisfying that demand and what factors have influenced this result?* • How have activities of the LASP with local producers affected sustainability?* • What are the current barriers to achieving sustainability according to the different stakeholders?* Impact • How do literacy outcomes compare across the treatment and comparison groups? Is there evidence of a positive impact of LENS on literacy outcomes? • Have literacy skills of school-age children generally improved in the LENS program area? • Have health KAP outcomes of teachers changed in the LENS program area? • Has LENS contributed to increased enrollment and regular attendance of school-age children? • Have nutrition, dietary, and food safety practices in schools improved in the LENS program area? • Have there been any positive or negative impacts in the target areas, besides the realization of the strategic objective-level results? For the endline, the evaluation questions are: Relevance • Did stakeholders feel that their voices were heard and their needs considered throughout the program? • Has the program integrated activities to support literacy and improved nutrition in culturally appropriate ways in the target communities? Effectiveness • To what extent has the program achieved its output and outcome targets? • What factors have inhibited or facilitated achievement of goals, objectives, and expected results? • How effective are the contracts and the quality monitoring between local producers and PTAs?* Efficiency • Did the program deliver intervention components within their planned timeline? • Which commodity management strategies were most efficient for quick delivery and waste reduction? Sustainability • Under which conditions are local producers able to satisfy the food demand of schools for the feeding program? What are the pathways that lead to a successful sale and delivery of food by local producers to schools?* • Do schools have the necessary infrastructure, food management plans, and systems in place to continue feeding and literacy activities after the program concludes?* NORC | LENS PROGRAM BASELINE REPORT | 14 • What are the necessary components for successful school handover of activities, as modeled by this program? What were the lessons learned?* Impact • How do literacy outcomes compare across the treatment and comparison groups? Is there evidence of a positive impact of LENS on literacy outcomes? • Has LENS improved access to and availability of early grade multilingual reading materials? • Have literacy skills of school-age children improved in the LENS program area? • Have health and nutrition KAP outcomes of teachers improved in the LENS program area? • Has LENS contributed to increased enrollment and regular attendance of school-age children? • Have nutrition, dietary, and food safety practices in schools improved in the LENS program area? • How have PTAs contributed to the school feeding program? • Have there been any positive or negative impacts in the target areas, besides the realization of the strategic objective-level results? 4. Data Collection In this section we describe data collection for both the program and impact evaluations, as data collection for the two evaluations occurred in parallel. Baseline data collection took place May 22 – July 4, 2019. NORC’s data collection partner, Juarez & Associates (J&A), conducted all data collection activities, working closely with NORC. While the program will benefit pre-primary and primary students in target schools, to assess the impact of the program on reading skills the evaluation will focus on 3rd grade students.13 Data collection activities include literacy assessments (ELI), teacher Knowledge, Attitude and Practice (KAP) surveys, school observation checklists, Key Informant Interviews (KIIs) with teachers and local producers, and Focus Group Discussions (FGDs) with 3rd grade parents. These instruments were fielded in the six municipalities where there are LENS schools, namely Cunén, Chajul, Cotzal, Uspantán, Nebaj and Sacapulas. In addition, the ELI and student survey were fielded in comparison schools, located in three Quiché municipalities, including Chicamán, San Pedro Jocopilas and San Andrés Sajcabajá. Below is a description of each of the instrument fielded during the baseline data collection: 4.1 Quantitative Data a) Student literacy assessment: NORC fielded the Evaluation of Initial Reading (Evaluación de Lectura Inicial in Spanish - ELI) to ten third graders (five boys and five girls) in both treatment and control schools. The assessment was developed and provided by the Ministry of Education (MINEDUC), who modeled the instrument after the widely used Early Grade Reading Assessment (EGRA). The assessment was fielded only in Spanish, regardless of each student’s maternal language, as designed by MINEDUC. In addition to the assessment questions, NORC included additional questions to capture information about students’ sociodemographic background, satisfaction with in-school meals, nutritional history, and home literacy environment. The items developed by NORC are available in Annex I (forthcoming). The assessment includes subtasks on letter naming, letter sounds, initial phoneme identification, phoneme separation, short word reading, familiar word reading, non-word reading, listening comprehension, oral reading fluency, reading comprehension, and dictation. Table 13 The original plan was to survey 2nd graders, after discussions J&A, NORC and SC, SC decided to assess 3rd graders instead. NORC | LENS PROGRAM BASELINE REPORT | 15 1 below details the name, description, duration, and number of items included in each literacy subtasks included as part of the ELI. Table 1. ELI Assessment Overview Sub-Task Name Sub-Task Description Duration Number of Items Letter Naming - Task 1 Child reads a list of 30 letters, once with the lower-case version of the letters and once with the upper-case version of the letters, for a total of 60 letters, and gives the name for each letter. 240 seconds 60 Letter Naming - Task 2 Child reads a mixed list of 100 lower- and upper-case letters, and gives the name for each letter. 60 seconds 100 Letter Sounds - Task 1 Child gives the sound for each letter on a list of 30 letters, once with the lower-case version of the letters and once with the upper-case version of the letters, for a total of 60 letters, 240 seconds 60 Letter Sounds - Task 2 Child gives the sound for each letter on a mixed list of 100 lower- and upper-case letters. 60 seconds 100 Initial Phoneme Identification Child gives the first sound for each word on a list of ten words. Untimed 10 Phoneme Separation Child separates all letter sounds for each word on a list of ten words. Untimed 10 Short Word Reading Child reads a list of 10 short, one-syllable words. 120 seconds 10 Familiar Word Reading Child reads a list of 70 common, grade￾appropriate words. 60 seconds 70 Non-Word Reading Child reads a list of 70 invented words with no meaning. 60 seconds 70 Listening Comprehension Child answers ten comprehension questions about a text read by the interviewer. Untimed 10 Oral Reading Fluency Child reads a 100-word story. 60 seconds 100 Reading Comprehension Child answers eleven comprehension questions on the same 100-word passage from the Oral Reading Fluency sub-task. The number of questions varies by child depending on the amount of the text they are able to read. A zero value is automatically assigned to students who are unable to read anything. Untimed 11 Dictation Enumerator reads a six-word sentence and asks the child to write the sentence. Untimed 6 b) Teacher Knowledge Attitude and Practice Survey: The teacher KAP instrument was designed to collect information on teachers’ knowledge, attitudes, and practices around health and nutrition. The instrument includes specific questions on teachers’ exposure to nutrition education, their knowledge about nutrition, sanitation and hygiene topics, their attitude about the importance of teaching about these topics and their experiences addressing these issues with their students. In NORC | LENS PROGRAM BASELINE REPORT | 16 addition to KAP questions, the instrument includes questions to capture information on teachers’ backgrounds, and other information. c) School Observation Checklist: The School Observation Checklist was designed to document school infrastructure and practices related to food storage and manipulation, water treatment and maintenance, school gardens and reading resources. 4.2 Qualitative Data NORC developed KII and FGD scripts that address the LENS evaluation questions that could be fully or partially answered with qualitative data. We drafted the interview guides in English and shared them with Save the Children for feedback. Once the instruments were finalized, the instruments were translated to Spanish, the language they were ultimately fielded in. As baseline data collection occurred prior to implementation, the interviews focus primarily on respondents’ experience with previous projects and their expectations of the LENS program. Below, we describe the content of each instrument: a) Key Informant Interviews with 3rd Grade Teachers in treatment schools recorded teachers’ views on different program aspects, including expectations of the program, perceived opportunities and challenges, current and past activities related to nutrition and hygiene, perceived opportunities for local foods to promote dietary diversity, and adequacy of books and other reading materials. b) Focus Group Discussions with 3rd Grade Parents sought to elicit information from parents about their expectations of the program, appropriateness of past program meals, perceived opportunities to integrate local foods into school meals, and overall satisfaction with similar programs. c) Key Informant Interviews with Local Producers recorded their expectation of the program, prior experience with similar activities, perceived future opportunities or challenges associated with the supply of food to schools, and possible relationships between local producers and school stakeholders. These interviews also captured the food production profile (quantities, variety, and seasonal availability) of each producer. 4.3 Training Prior to launching data collection, NORC and J&A conducted qualitative and quantitative data collection training between May 17 and 25, 2019 in Santa Cruz de Quiché, Guatemala.14 J&A recruited trainees from a pool of assessors who had experience with school-based data collection and, in most cases, had worked with J&A on previous implementation of the ELI. The training included 24 quantitative enumerators, six quantitative supervisors, three school observers, three qualitative recruiters, and four qualitative moderators. The team was specifically recruited to ensure that enumerators spoke the local languages of the municipalities they would be working in. The training started with a two-day supervisor training that introduced the program background, instruments, and field plans to the selected team supervisors. Following the supervisor training, NORC and J&A trained the full data collection team, and conducted a 1-day field exercise, where quantitative teams practiced applying the ELI, KAP and school observation instruments. The field teams, including the qualitative teams working with parents and local producers, practiced introducing themselves and explaining the evaluation in both K’iche’ and Ixil as part of the training. Additionally, the qualitative team practiced administering the local producer and parent instruments in the local languages before leaving for the field. 14 Baseline training and data collection dates overlap because dedicated classroom observers were trained within the first two days of full-team training and commenced their fieldwork on May 22 in order to meet data collection deadlines. NORC | LENS PROGRAM BASELINE REPORT | 17 Throughout the training, NORC conducted multiple inter-rater reliability (IRR) tests to ensure enumerators administering the ELI assessment were correctly marking responses. J&A based final field team selections on each enumerator’s performance during the field exercise and IRR tests. 4.4 Fieldwork and quality control Quantitative and qualitative data collection took place between May 22 – July 4, 2019. The dedicated classroom observers launched their data collection on May 22, while the rest of the field teams started data collection on May 27, 2019. As part of the quality control measures, a NORC team member observed the first two days of quantitative data collection to isolate any issues with the implementation of data collection and worked with J&A to resolve them during the first week. One of the most important observations from the first week of data collection was that students had difficulty answering NORC’s survey questions in Spanish at the end of the ELI. NORC and J&A jointly decided to allow enumerators to provide translations of these questions into local languages. However, we emphasized to enumerators that MINEDUC’s instrument, including the literacy assessment, was not to be translated. Throughout data collection J&A continued observing and recording any issues the data collection teams faced. J&A was denied access to eight LENS schools - five in Uspantán, two in Cunén and one in Cotzal. The schools in Uspantán were replaced with schools from Chajul. Although there were more LENS schools in Uspantán we could potentially have used as replacements, refusals were driven by the municipal teachers’ union, in other words this was a municipality-level refusal, rather than a school￾specific one, so there was no point in trying to survey other schools in the same municipality. Refusals in Cunén and Cotzal, by contrast, were replaced with schools from the same municipalities. In addition, 16 schools in the comparison communities also denied access, most of them in Chicamán. These schools were replaced with other schools from the other two comparison municipalities, namely San Pedro Jocopilas and San Andres Sajcabaja. The main reason why comparison schools denied access was that, as they were not receiving program support, they were not interested in having their students assessed. Also, as was the case in Uspantán, some schools requested that J&A should get approval from the teachers’ union. The qualitative data collection also launched on May 27, 2019 and was conducted by two sets of data collection teams. The Teacher KIIs were conducted by the quantitative supervisors as they traveled from treatment school to treatment school, while the Parent FGDs and Producer KIIs were conducted by dedicated qualitative data collection teams that traveled independently of the quantitative teams. The teams carried out a total of 28 Teacher KIIs, ten Local Producer KIIs, and 10 Parents FGDs. For each Parent FGD, a centrally located school was selected as the reunion point where the FGD was conducted. Once the school was selected, the teams worked with nearby schools to recruit parents in order to ensure that parents from more than ten schools participated in the interviews. The teams also attempted to get as close to a gender balance as possible for each FGD, however the reality is that in most cases female parents were more willing and able to participate in the FGDs. Table 2 below presents a gender breakdown of the participants for each FGD. Table 2. Parent FGD Gender Breakdown FGD Males Females Total K’iche’ 1 – Uspantán 1 9 10 K’iche’ 2 – Cunén 10 9 19 K’iche’ 3 – Uspantán 0 5 5 NORC | LENS PROGRAM BASELINE REPORT | 18 FGD Males Females Total K’iche’ 4 – Sacapulas 1 9 10 K’iche’ 5 – Cunén 0 13 13 Ixil 1 – Nebaj 3 7 10 Ixil 2 – Nebaj 3 3 6 Ixil 3 – Nebaj 2 11 13 Ixil 4 – Chajul 2 13 15 Ixil 5 – Cotzal 0 10 10 NORC originally planned to conduct only 8 parent FGDs, however due to low participant turnout in two of the FGDS, two more were added to increase the total number of FGD participants. The low turnout in these two FGDs was caused by two issues: (1) some schools in Uspantán did not allow parents to participate in the FGDs, similar to them refusing access to students for the quantitative data collection, and (2) some parents from Nebaj entered the wrong meeting, which was taking place at the same school and the same time as the LENS Parents FGD. Finally, enumerators interviewed ten local producers to better understand their current position as producers, as well as their capacity to provide food for school feeding programs in the future. To recruit these local producers, two approaches were followed. First, we used a list of local producers who would potentially be interested in participating in LENS, given to NORC by Save the Children. During fieldwork we found that contacting these local producers was very challenging, and eventually only two producers from this list were interviewed. Most of these producers were not interested in participating in the survey. The second approach was to recruit producers when advanced teams were sent to the schools to recruit parents for the focus groups discussions. This method proved to be more effective to recruit local producers. 5. Impact evaluation 5.1 Research design overview Impact evaluations (IEs) allow us to attribute changes in an outcome to a specific intervention or set of interventions by answering the counterfactual question “What would have happened to program participants in the absence of the intervention?” The ideal comparison group stems from the use of experimental methods in which eligible participants are randomly assigned to receive the intervention or not. The process by which schools were selected for LENS treatment was not randomized. In fact, there are two selection processes that determined school selection for the impact evaluation. First, LENS schools were part of another Save the Children literacy and school feeding program between 2014 and 2018 called IDEA. Second, the 87 schools that will receive the full LENS treatment (i.e. full package of interventions) were selected based on a needs assessment conducted by SC in early 2019. These two characteristics of the treatment schools suggest they are not directly comparable to non-LENS schools in Quiché, from which the comparison schools will be drawn. Because some characteristics between treatment and comparison schools are presumably different, comparing their outcomes at midline or endline will most likely lead to biased estimates of the treatment effect because the difference in their outcomes will reflect not only the impact of LENS, but the fact that schools in the two groups would be different even in the absence of the program. To address this challenge, we will use a matched difference-in-difference (DiD) model for the evaluation. Matched DiD, which is widely used in IEs and peer reviewed studies, combines DiD with matching methods to reduce potential sources of bias and provide a compelling approach to inferring causality. DiD NORC | LENS PROGRAM BASELINE REPORT | 19 involves comparing the changes in the outcome of interest between baseline and endline in treatment schools to changes between baseline and endline in comparison schools. The key assumption behind this model is that, in the absence of treatment, students in treatment schools would observe the same change in the outcome of interest than students in the comparison schools. Formally, to estimate the impact at midline of LENS we will estimate: 𝐴𝐴𝑖𝑖𝑖𝑖 = 𝛼𝛼 + 𝛽𝛽𝐷𝐷𝑖𝑖 + 𝛾𝛾𝑀𝑀 + 𝛿𝛿𝐷𝐷𝑖𝑖𝑀𝑀 + 𝐱𝐱𝐢𝐢𝐢𝐢 ′ 𝛉𝛉 + 𝑢𝑢𝑖𝑖𝑖𝑖 (1) Where 𝐴𝐴𝑖𝑖𝑖𝑖 measures the outcome of interest (e.g. students’ test scores) for student i in period t; 𝐷𝐷𝑖𝑖 is a dummy variable for treatment status; 𝑀𝑀 is a dummy variable for the midline, xit is a vector of characteristics at the student level (e.g. student age and gender), 𝑢𝑢𝑖𝑖𝑖𝑖 is an error term and α, 𝛽𝛽, 𝛾𝛾, 𝛿𝛿 and θ are parameters to be estimated. The parameter of interest is 𝛿𝛿, which captures the effect of being in the program at midline. The model in Equation 1 can be adapted to incorporate the endline analysis, so only observations at baseline and endline are included. To analyze differential impacts between boys and girls, we will conduct this analysis stratifying by students’ gender. Standard errors will be clustered at the school level to control for autocorrelation within schools. 5.2 Sample and Data Collection We surveyed 50 treatment and 52 comparisons schools in the department of Quiché, for a total of 102 schools. The 50 treatment schools were randomly selected from the pool of 87 schools selected for full treatment by SC/Guatemala (based on the aforementioned needs assessment), stratifying by municipality.15 To construct the comparison group, we selected 50 schools from three municipalities with no LENS activities, but that neighbor the six LENS municipalities, namely Chicamán, San Pedro Jocopilas and San Andrés Sajcabajá. Figure 1 shows the treatment and control municipalities. 15 This design will allow estimation of a minimum detectable effect size of 0.27 percent of a standard deviation, assuming school￾intracluster correlation of 0.15, power of 80 percent, alpha of five percent, and that covariates explain 20 percent of the variance in the outcomes of interest. NORC | LENS PROGRAM BASELINE REPORT | 20 Figure 1. Impact evaluation municipalities These control municipalities are similar to the LENS municipalities across poverty levels. Figure 2 shows the fraction of the population living below the poverty line and in extreme poverty, according to census data collected between 2008 and 2011. Within the treatment group, Cunén is the municipality with the lowest poverty incidence; only 9.2 percent of the population live in extreme poverty, and 55 percent live below the poverty line. These levels are similar to those observed by San Pedro Jocopilas, the control municipality with the lowest levels of poverty. The treatment municipalities Chajul, Sacapulas, Uspantán y Nebaj have levels of total poverty between 68 and 83 percent, and of extreme poverty between 12 and 27 percent, which are relatively similar to the indicators for the control municipalities Chicamán and San Andrés Sajcabajá. Finally, San Juan de Cotzal is an extreme case with 97.8 percent of the population living below the poverty line, and of these, roughly two thirds living under extreme poverty. Chajul Chicaman Cunén Nebaj Sacapulas San Andrés Sajcabajá San Juan Cotzal San Pedro Jocopilas Uspantán Treatment Control NORC | LENS PROGRAM BASELINE REPORT | 21 Figure 2. Poverty levels in 2011 by impact evaluation municipalities (%) Note: Extreme poverty is defined as the fraction of people living with less income than the cost of purchasing a minimum of recommended calories. Total poverty includes extreme poverty and those not below the extreme poverty line, but that do not have enough income to purchase a minimum of goods and services other than food. Source: Instituto Nacional de Estadística, 2013 Comparison municipalities are also similar to the LENS municipalities in terms of their school characteristics. Figure 3 shows total students and students per teacher by municipality. While schools in San Juan de Cotzal and Chajul are larger than schools in the control group, on average, school sizes in the rest of the LENS municipalities are similar to the sizes observed in the control municipalities. Moreover, the student/teacher ratio is remarkably similar between treatment and control municipalities. 77.5 77.7 55.4 97.8 83.4 75.8 76.0 68.3 55.0 21.5 21.4 9.6 67.3 26.8 22.7 19.3 12.1 9.2 Chicamán San Andrés Sajcabaja San Pedro Jocopilas San Juan Cotzal Chajul Sacapulas Uspantán Nebaj Cunén Comparison Treatment Extreme poverty Total Poverty NORC | LENS PROGRAM BASELINE REPORT | 22 Figure 3. Total students and students per teacher in primary schools in 2017, by impact evaluation municipalities Source: Ministry of Education. Finally, control and treatment municipalities are also similar in terms of students’ reading skills. Figure 4 shows results of a national reading assessment of 3rd graders conducted by the Ministry of Education in 2014 (Ministerio de Educación, 2015). The results are representative at the municipality level. The scores are standardized such that the national mean and standard deviation are zero and one, respectively, which is why scores are negative for most impact evaluation municipalities. In fact, only two municipalities, Sacapulas in the treatment group, and San Pedro Jocopilas in the control group, have positive scores. The rest of the municipalities in the evaluation have negative scores, which implies that, on average, students in these municipalities scored below the national mean. With the exception of Cunén, reading scores are relatively similar between treatment and comparison groups. In effect, Cunén constitutes an extreme case. In this municipality students scored, on average, 1.38 standard deviations lower than the national mean. Perhaps paradoxically, Cunén is also the municipality with the lowest poverty indexes in 2011, as shown in Figure 2. There could be a few explanations for this result. First, data on poverty was collected between 2008 and 2011, while the reading assessments were conducted in 2014; hence, it is possible that these indicators may have changed between the two data collections points. Another explanation could be that, with Cunén being relatively wealthier than the rest of the target municipalities, there are fewer out of school students. If there are fewer out of school children in Cunén than in other evaluation municipalities, 16.9 19.4 23.6 21.7 19.1 19.7 16.9 22.2 21.6 58.4 82.1 98.6 79.9 88.1 103.6 104.2 135.3 158.8 San Andres Sajcabaja San Pedro Jocopilas Chicaman San Miguel Uspantan Sacapulas Cunen Nebaj Chajul San Juan Cotzal Comparison Treatment Total students (grades 1-6) Students per teacher NORC | LENS PROGRAM BASELINE REPORT | 23 then in Cunén schools should have more low ability students than schools in other municipalities, because fewer students have dropped out in Cunén. If schools in Cunén have more low ability students than other municipalities, this could explain why they have lower average test scores. Figure 4. Reading skills in 3rd grade 2014 national assessments, by impact evaluation municipalities Source: Ministry of Education (2015) In sum, comparison municipalities are similar to LENS municipalities in terms of poverty indicators, school characteristics and students’ reading skills. However, there are a few extreme cases in the treatment group for which, at the municipality level, no clear counterfactual can be found in the comparison group. For example, the fraction of people living in extreme poverty in San Juan Cotzal is 67.3 percent, while the equivalent figures in the control municipalities are much lower. However, it is important to highlight that the comparison municipalities do not need to be observationally equivalent to the LENS municipalities. As we will use matching methods for the impact evaluation, we will be able to balance treatment and control groups at the student level, even if there are some imbalances at municipality level. Moreover, the DID estimator will allow us to control for baseline differences. The 50 treatment schools were selected in proportion to the number of schools selected in each municipality for full treatment by SC. Table 3 shows the target number of schools and the number of schools surveyed, by municipality. As mentioned before, five schools in Uspantán refused participation, one of which was part of the treatment group of the impact evaluation (the other four are part of the program evaluation). This was replaced with a school from Chajul. In addition, one school in Cunén and one in Cotzal, that also refused participation, were replaced with schools from the corresponding municipalities. -0.54 -0.24 0.29 -1.38 -0.51 -0.45 -0.43 -0.40 0.23 San Andrés Sajcabaja Chicaman San Pedro Jocopilas Cunen San Miguel Uspantan Nebaj Chajul San Juan Cotzal Sacapulas Comparison Treatment NORC | LENS PROGRAM BASELINE REPORT | 24 Table 3. Target and surveyed treatment schools by municipality Municipality Target Surveyed Chajul 3 4 Cotzal 7 7 Cunén 13 13 Nebaj 9 9 Sacapulas 9 9 Uspantán 9 8 Total 50 50 Within the three control municipalities, we selected 50 schools with similar characteristics to the 50 treatment schools in terms of number of teachers and number of students16, thereby balancing the treatment and control groups across these two dimensions. As Table 4 shows, the objective was to survey 16 schools in Chicamán, 18 in San Andres Sajcabajá and 16 in San Pedro Jocopilas, for a total of 50 schools. We surveyed only six schools in Chicamán, 20 schools in San Andres Sajcabajá and 26 in San Pedro Socopilas, for a total of 52 schools. The reason why the number of schools surveyed in Chicamán is lower than the target, is that after surveying a few schools in Chicamán enumerators were denied access to the schools in this municipality, so the evaluation team decided to survey more schools in San Andres Sajcabajá and San Pedro Jocopilas to compensate. Note that, relative to the original plan, two more comparison schools were surveyed. We keep these additional schools in the analysis, so we can have replacements in case other schools refuse participation at midline or endline. Table 4. Target and surveyed comparison schools by municipality Municipality Target Surveyed Chicamán 16 6 San Andrés Sajcabajá 18 20 San Pedro Jocopilas 16 26 Total 50 52 In schools with more than one third grade classroom, enumerators selected one class randomly. The target was to survey ten 3rd grade students (5 girls and 5 boys) in each sampled school. However, a number of schools were too small so fewer than 10 students were surveyed. To palliate sample deterioration, in some large schools more than ten students were surveyed. In total 930 3rd graders - 466 in treatment schools and 464 in comparison schools - were surveyed, for a completion rate of 93 percent. Each surveyed child answered the Evaluación de Lectura Inicial (ELI), as well as a short student survey, to record students’ age, gender, parental literacy, and other characteristics. 5.3 Baseline results Table 5 shows descriptive characteristics by treatment status. Students are ten years old on average, and students in the comparison group are slightly older than students in the treatment group. All students are 16 More specifically, we calculated quintals of the number of students and quartiles of the number of teachers, and used these variables to divide the sample in a grid of 20 cells (some of these cells were empty). We divided the schools in the three control municipalities using the same grid, and randomly sample, per cell, the same number of schools found in the treatment group. NORC | LENS PROGRAM BASELINE REPORT | 25 in 3rd grade. A little over half the students are male, both in the treatment and control groups. Differences in the languages spoken at home are worth highlighting. In the comparison group, about half the students speak Spanish and the other half K’iche’. In the treatment group, 30 percent speak Spanish, 36 percent speak K’iche’, and 28 percent speak Ixil. Note that there are no students that speak Ixil at home in the comparison group. The three municipalities where Ixil is prevalent in the department of Quiché are Chajul, Cotzal and Nebaj; because no schools were surveyed as controls in these municipalities, there are no schools in the comparison group where this language is spoken.17 This hinders the comparability between treatment and comparison groups. Regarding preschool attendance, 77 percent of students in the treatment group attended preschool, while the fraction for the comparison group was 61 percent. Roughly half the students have repeated at least one year, both in the treatment and comparison groups, including students that repeated 1st or 2nd grade, or are repeating 3rd grade.18 Interviewers recorded student attendance during the previous week by consulting teachers’ attendance records. The attendance rate during the previous week was roughly 95 percent for both groups Note that this may constitute a biased estimated of the true attendance rate, as we are missing all the students that did not attend the day of the survey. The survey also included items on reading environment at home. Household environment in the treatment group seems more conducive to learning how to read than in the comparison group. For example, households in the treatment group are more likely to have books, parents and other household members are more likely to read to children, and mothers are more likely to be literate. Some of these results could be explained by the fact that these communities were part of the IDEA program. The differences across these three dimensions are statistically significant. In terms of socioeconomic indicators, we do not find major differences between treatment and comparison groups. A little over 80 percent of the households have electricity in both groups and, in most households, someone owns a cell phone. To approximate household wealth we constructed an asset index for the number of times students affirmatively answered questions on whether their household owns, in good condition, a chair, a bed, a clock, a radio, a television, a computer, a bicycle, a motorcycle, a car/truck/boat, a refrigerator, and a stove. Table 5 shows that students have roughly eight of these items, and there are no major differences between the treatment and comparison groups. Table 5. Sociodemographic characteristics – IE Treatment Comparison Difference Age (years) 9.7 10.2 -0.44*** Male 0.54 0.55 -0.01 Language spoken at home Spanish 0.30 0.46 -0.15*** K’iche’ 0.36 0.59 -0.23*** Ixil 0.28 0.00 0.27*** Other 0.02 0.02 -0.00 NR 0.10 0.04 0.06** Attended preschool 0.77 0.61 0.16*** 17 During the design phase NORC suggested to survey schools in Chajul, Cotzal and Nebaj that were not part of the program for the comparison group, but this idea was rejected due to potential contamination of the comparison group. 18 These results are consistent with previous research conducted for the IDEA evaluation. For the IDEA baseline data show that the repetition rate for 1srt grade was 35 percent, and for 2nd grade was 23 percent. The equivalent figures for the LENS baseline for the impact evaluation schools are 35 percent for 1st grade and 22 percent for 2nd grade. NORC | LENS PROGRAM BASELINE REPORT | 26 Treatment Comparison Difference Repeated or repeating a year 0.50 0.49 0.01 Attendance last week (fraction)(a) 0.94 0.95 -0.01 Household reading environment There are books at home 0.73 0.59 0.14*** Household members read with child 0.54 0.44 0.11*** Mother's literacy 0.63 0.53 0.10** Father's literacy 0.91 0.87 0.03 Socioeconomic indicators Household has electricity 0.83 0.82 0.01 Household owns a cellphone 0.93 0.94 -0.01 Asset index (number of items) 8.14 7.93 0.22 N 466 464 Note: For the differences tests, standard errors are clustered at the school level. Sample sizes are a few observations smaller for some items due to item-specific missing data. (a) 10 comparison schools in San Andres Sajcabajá were closed the week before data collection, so no attendance rate can be computed for these schools. * p<.1 ** p<.05 *** p<.01 Nutrition-related indicators, displayed in Table 6, show that most students ate something before coming to school. Students in the treatment group are statically significantly more likely to have received something to eat at school than in the comparison group, albeit the great majority of students in both groups are receiving food at school. Notably, among students receiving food, those in the comparison group seem to be receiving more food than in the treatment group. In effect, while students in the treatment are more likely to receive a small meal and a drink than those in the comparison group, this is compensated with students in the comparison group being more likely to receive a full meal. It is important to highlight that, being a baseline survey, SC had not started program activities when the data was collected. This explains why the treatment group seems to be receiving less food than students in the comparison schools (within the students that reported having received any). It is also possible that students in the treatment schools are used to a larger meal, thanks to their participation in the IDEA program between 2015 and 2018, and their current meals are perceived as smaller than they really are, compared to what students in comparison schools are receiving. 19 Notably, almost all students in both treatment and comparison groups, like the school meals. Table 6. Nutrition – IE Treatment Comparison Difference Did you eat something before coming to school today? 0.90 0.90 -0.00 Did you receive food at school yesterday? 0.93 0.86 0.07** What did you eat? (For students that ate something at school) A drink only (e.g. atol) 0.08 0.04 0.04 A small meal and a drink 0.46 0.37 0.09 A small meal 0.19 0.13 0.06 Full meal 0.19 0.37 -0.19*** 19 While IDEA started in 2014, food delivery started in 2015. NORC | LENS PROGRAM BASELINE REPORT | 27 Treatment Comparison Difference NR 0.08 0.09 -0.01 Do you like school meals? 0.97 0.96 0.01 N 466 464 Note: For the differences tests, standard errors are clustered at the school level. Sample sizes are a few observations smaller for some items due to item-specific missing data. * p<.1 ** p<.05 *** p<.01 There are also differences in reading skills between treatment and comparison schools. Figure 5 shows ELI scores by treatment status for selected subtasks, namely familiar word reading, non￾word reading, listening comprehension, oral reading fluency, and reading comprehension. 20 Across all five subtasks, students in the comparison group have higher scores than their counterparts in the treatment group, and all differences are statistically significant. This highlights the importance of collecting baseline data, so we can control for these historical differences when the midline and endline evaluations are conducted. Figure 5. Reading skills – IE (%correct / CWPM) Note: Sample sizes are 455 for the treatment group and 463 for the comparison, although the number of observations may be smaller for some items due to item-specific missing data. The differences with respect to the sample sizes for the sociodemographic variables correspond to students that did not pass the initial instructions to answer the assessment. For the differences tests, standard errors are clustered at the school level. * p<.1 ** p<.05 *** p<.01 On average, reading performance is fairly low for 3rd graders. For familiar word reading students are able to read correctly roughly half the words, and the number of non-words per minute read is relatively low for both groups (as shown in Table 1, the maximum number of non-words students could read was 70). While students read a fairly high number of words per minute (students in the treatment group read 51 cwpm and in the control group they read 60.7 cwpm, out of a maximum of 100 words), 20 Annex II shows results for all 12 ELI’s subtasks, for the main body of the report we focus on these five subtasks because these are the most similar to EGRA. 57.3 30.4 46.3 60.7 50.3 48.8 26.2 36.0 51.0 36.4 8.4** 4.1** 10.3** 9.6** 13.9*** Familiar Word Reading - % Correct Non-Word Reading - CWPM Listening Comprehension - % Correct Oral Reading Fluency - CWPM Reading Comprehension - % Correct Comparison Treatment Difference NORC | LENS PROGRAM BASELINE REPORT | 28 their reading comprehension results are relatively low, for the treatment group students answered correctly only 37 percent of the items, while the figure for the comparison group is 50 percent. In sum, the students survey shows that there are major differences between treatment and comparison schools. One of the most apparent is that there are no students who speak Ixil at home in the comparison group. There are also differences in reading skills, with students in the comparison group clearly outperforming those in the treatment group. Although this is in part because there are more households that speak Spanish in the comparison group than in the treatment group, even for Spanish speaker households, comparison students outperform treatment students, and the same goes for K’iche’ speaking households, as Figure 6 shows. Figure 6. Reading skills in Spanish and K’iche’ households – IE (%correct / CWPM) 55.2 29.0 49.2 60.2 50.8 59.6 31.5 56.3 65.0 59.2 Familiar Word Reading - % Correct Non-Word Reading - CWPM Listening Comprehension - % Correct Oral Reading Fluency - CWPM Reading Comprehension - % Correct A. Households where Spanish is the most spoken language Treatment Comparison NORC | LENS PROGRAM BASELINE REPORT | 29 The fact that students in comparison schools outperform students in the treatment group is perhaps surprising given there are no major differences in socioeconomic indicators (e.g. number of household assets, electricity); moreover, the reading environment in treatment households seems more favorable than in comparison households; for example, treatment households are more likely to have books, and their members more likely to read to children, than in the comparison group). This suggest there are unobservable differences between the treatment and comparisons groups that affect reading skills. We will use difference-in-differences estimators and baseline data to isolate and control for these baseline differences in the midline and endline analyses. 5.4 Balance In this section we use propensity score matching methods to show how baseline characteristics between treatment and comparison groups can be balanced. More specifically, we estimate the probability of program participation for both treatment and comparison students, and match students in the treatment group with students in the comparison group with similar participation probabilities. We model program participation as a function of students’ age and gender, home reading environment indicators (whether there are books in the household and mother’s literacy), whether the student attended preschool, whether the student has repeated a year, whether they speak Spanish at home, mother’s literacy, household socioeconomic indicators (electricity, cell phone ownership and the asset index), and household size. To control for historical differences in reading performance between municipalities, we also include the municipal mean score of the 2014 national reading assessments discussed in subsection 5.2. 48.2 25.7 28.6 49.3 28.7 56.4 30.1 37.6 58.5 43.1 Familiar Word Reading - % Correct Non-Word Reading - CWPM Listening Comprehension - % Correct Oral Reading Fluency - CWPM Reading Comprehension - % Correct B. Households where K'iche is the most spoken language Treatment Comparison NORC | LENS PROGRAM BASELINE REPORT | 30 We use the estimated probabilities to construct matching weights. The result is a weighted comparison group that approximates the characteristics of the treatment group, on average. Table 7 shows how this weighting procedure reduces the differences between treatment and comparison schools for selected characteristics. In the first column the differences between the treatment and the “raw” comparison groups are displayed, in the second column the differences between the treatment and the weighted comparison group are shown. Note that the model is successful in balancing most of the selected variables. One important exception is language spoken at home. This is because there are no communities in the comparison municipalities where Ixil is spoken, so the model cannot really balance local languages. Table 7. PSM results - Balanced characteristics at baseline Unbalanced Difference Matching Age (years) -0.44*** -0.13 Language spoken at home Spanish -0.15*** 0.01 K’iche’ -0.23*** 0.13 Ixil 0.27*** 0.28*** Other -0.00 -0.21* NR 0.06** -0.16** Attended preschool 0.16*** -0.02 Repeated or repeating a year 0.01 -0.11 Household reading environment There are books at home 0.14*** -0.01 Household members read with child 0.11*** -0.01 Mother's literacy 0.10** 0.01 Note: Standard errors are clustered at the school level. Matching uses kernel weights (Epanechnikov). * p<.1 ** p<.05 *** p<.01 Figure 7 shows the propensity score distributions by treatment status. It is apparent that there are major differences in the distributions of the propensity to participate in the program between treatment and comparison groups. However, it is also worth highlighting that the distribution of the comparison group almost completely overlaps the treatment’s, indicating that we are able to find counterfactuals for most of the students in the treatment group. NORC | LENS PROGRAM BASELINE REPORT | 31 Figure 7. Propensity score by treatment status 5.5 Conclusions Below we summarize the main baseline findings for the impact evaluation: • In terms of socioeconomic indicators, there are no major differences between treatment and control groups. • There are notable differences between treatment and comparison groups in terms of language spoken at home, as there are no students that speak Ixil at home in the comparison group. • Household environment seems more suitable for learning how to read for the treatment group than for the comparison group. • Most students ate something before coming to school. Similarly, most students ate something at school the day before data collection. • Across all five reading subtasks, students in the comparison have higher scores than their counterparts in the treatment group, and all differences are statistically significant. Although this is in part because there are more households that speak Spanish in the comparison group than in the treatment group, even within Spanish speaking households, comparison students outperform treatment students, and the same happens within K’iche’ speaking households. 0 1 2 3 4 0 .2 .4 .6 .8 1 x kdensity Treatment kdensity Comparison NORC | LENS PROGRAM BASELINE REPORT | 32 • Students read 51 cwpm in the treatment group and 60.7 cwpm in the comparison group. However, reading comprehension results are relatively low, for the treatment group students answered correctly only 37 percent of the items, while the figure for the comparison group is 50 percent. Finally, matching methods are successful in balancing most of the selected variables. An important exception is language spoken at home. This is because there are no communities in the comparison municipalities where Ixil is spoken. Hence, the model cannot balance local languages. However, for the impact evaluation, we will be able to offset these differences using the baseline data, in the context of the DID model. 6. Program evaluation 6.1 Research design overview We will use a mixed methods approach to address questions on the program’s relevance, effectiveness, efficiency, sustainability and impact. Quantitatively, we will analyze data from the student literacy assessments, student attendance, and teacher responses to the health and nutrition KAP survey to study how these outcomes change over waves of data collection. For example, for student reading skills, we will present changes over time of mean scores, the fraction of students that read no words, and the fraction of students reaching meaningful literacy thresholds (e.g. the target that SC will propose after baseline data is analyzed). We will graphically display descriptive statistics by wave of data collection for ease of exposition, as well as present results by municipality, to document heterogeneity between the target areas. We will also conduct regression analysis to determine how student sociodemographic characteristics, (e.g. age, gender, and parental literacy), and home literacy environment factors are correlated with outcomes of interest. Alongside the quantitative analysis, we will conduct FGDs with parents and KIIs with teachers, school principals, PTA members, LENS staff, local producers and government officials to add depth and nuance in explaining changes we see in the quantitative outcome measures. Protocols for FGDs and KIIs for each group will follow the program evaluation questions. Each protocol will be structured such that qualitative and quantitative data can be triangulated to provide insights to answer the different research questions. The instruments will be administered in Spanish and recorded with the respondent’s permission to collect as detailed information as possible. The audio recordings will then be transcribed for analysis. Qualitative analysis will take a multi-step approach. First, we will use a phenomenological analysis to identify key themes emerging from the KIIs and FGDs. Each key theme will align with specific aspects of the intervention, and the research questions. The key themes and evaluation questions that emerge will be used to develop a code frame, which will anchor the qualitative analysis, in which each code is a pre￾determined area of interest. The phenomenological approach helps identify topics that come up most frequently, and topics that were important to research participants, but not necessarily included in the protocols. We will then use a grounded theory approach to triangulate data across sources, identify explanations for the phenomena observed, and the relationships between these phenomena. We will use qualitative analysis software, such as NVivo, to code data from each transcript along the themes emerging from the research questions and pre-determined key themes, identifying patterns and underlying causes of the emergence of observed themes. 6.2 Sample and data collection For the baseline we collected data in 72 schools in the municipalities of Cunén, Chajul, Cotzal, Uspantán, Nebaj and Sacapulas. Fourteen of these 72 schools are also part of the treatment group of the impact NORC | LENS PROGRAM BASELINE REPORT | 33 evaluation. To ensure that we survey ten pilot program schools, we stratified the 270 LENS schools by municipality and pilot status, for a total of 12 strata.21 We sampled schools in proportion to the total number of schools in each strata. Table 8 shows the total number of schools in each municipality, and the number and fraction of sampled schools. In all municipalities roughly 26 percent of schools were sampled (72/270 = 0.266), except for Chajul and Uspantán, because four program evaluation schools in Uspantán refused to participate in the study, and were replaced using schools from Chajul. In addition, one school in Cunén that also refused participation was replaced with another school from the same municipality.22 The main reasons for these refusals were related to complaints about the school feeding program, in particular concerns about the appropriateness of the menus. Table 8. Number of samples schools by municipality Student survey Total Sampled Fraction sampled Chajul 40 15 0.38 Cotzal 26 7 0.27 Cunén 51 13 0.25 Nebaj 67 18 0.27 Sacapulas 37 10 0.27 Uspantán 49 9 0.18 Total 270 72 In schools with more than one 3rd grade classroom, enumerators selected one class randomly. Similar to the impact evaluation, the target was to survey ten 3rd grade students (five girls and five boys) and their teacher in each sampled school. However, a number of schools were too small so fewer than ten students were surveyed. To palliate overall sample reduction, in some large schools more than ten students were surveyed. In total 688 students in 72 schools were surveyed for the program evaluation. In 68 of these 72 schools we surveyed one 3rd grade teacher. In the remaining four schools it was not possible to survey the teacher either because she or he was absent or refused to participate. Additional teacher surveys were conducted in four replacement schools so the sample for the teacher survey is also 72.23 To supplement the quantitative data, we also conducted ten FGD with parents, 28 KIIs with 3rd grade teachers, and ten KIIs with local producers. Finally, we implemented a school observation checklist in 262 program schools at baseline, to record availability of food storage and manipulation facilities and equipment, among other schools’ characteristics. A summary of all the qualitative data collected for the Program Evaluation is presented below in Table 9. Table 9. Program Evaluation Data Collection Data Collection Number of Schools Total Participants Student Assessment 72 688 21 Four schools were dropped because we could not find them in the Ministry of Education data sets. 22 The reason we replaced the refusals from Uspantán with schools from a different municipality was because these refusals, rather than being school-specific, were driven by the teachers’ union and, as such, was a municipality-level problem. As a result, we opted to use replacements from Chajul, a different municipality. 23 Specifically, two schools in Uspantán were replaced with one in Chajul and one in Cotzal, one school in Sacapulas was replaced with another school in Sacapulas, and one school in Cunén was replaced with another school in Cunén. NORC | LENS PROGRAM BASELINE REPORT | 34 Data Collection Number of Schools Total Participants Parent FGD - 111 Teacher KII 28 28 Local Producer KII - 10 School Observation 262 - 6.3 Baseline results In this section we present baseline findings for the program evaluation. First, we discuss students’ background and reading skills, drawing from the analysis of the student survey, including the reading assessment, as well as focus groups discussions with parents. Second, we analyze teachers’ responses, both to the KAP instruments and the KIIs. Third, we present the results of the school observations. Lastly, we discuss findings of the interviews with local producers. Student background and reading skills Characteristics by Sex Table 10 shows descriptive statistics by students’ sex. All descriptive statistics use sampling weights, so the figures approximate the results for students in all 270 schools from which the 72 schools were sampled. Students are 9.6 years old, on average. Girls are seven percentage points more likely to speak Spanish at home than boys. Differences between girls and boys for other languages are small and not significant. About four fifths of the sample attended preschool, and half the students have repeated or are repeating a year. Attendance rate during the previous week was 96 percent. There are no major differences in households’ reading environment by students’ sex. There are books in seventy-one percent of the households. Most of these books are in Spanish.24 About half the students said someone reads with them at home, 61 percent of students’ mothers are literate and for students’ fathers this fraction is 85 percent. We do not find major differences in socioeconomic indicators between households of girls and boys. Almost 80 percent of households have electricity, and in most households someone owns a cell phone. Results for the asset index show that students’ households own, on average, 7.6 assets. Although the difference between girls and boys for the asset index is statically significant, this difference is very small. Table 10. Sociodemographic characteristics by students’ sex – PE Males Females Difference Age (years) 9.7 9.6 0.12 Language spoken at home Spanish 0.29 0.36 -0.07** K’iche’ 0.29 0.27 0.02 Ixil 0.31 0.30 0.00 Other 0.04 0.04 -0.00 NR 0.11 0.08 0.03 Attended preschool 0.77 0.79 0.02 Repeated or repeating a year 0.51 0.50 0.01 24 For example, 80 percent of the students that have children’s books at home said they have books in Spanish, 12 percent said K’iche and 12 percent Ixil. NORC | LENS PROGRAM BASELINE REPORT | 35 Males Females Difference Attendance last week (fraction) 0.96 0.97 0 Household reading environment There are books at home 0.71 0.71 -0.01 Household members read with child 0.51 0.55 -0.04 Mother's literacy 0.61 0.60 0.01 Father's literacy 0.81 0.88 -0.07* Socioeconomic indicators Household has electricity 0.79 0.81 -0.02 Household owns a cellphone 0.94 0.96 -0.02 Asset index (number of items) 7.80 7.41 0.39* N 340 348 Notes: For the differences tests, standard errors are clustered at the school level. Sample sizes are a few observations smaller for some items due to item-specific missing data. * p<.1 ** p<.05 *** p<.01 Table 11 shows results for nutrition-related indicators. Most students ate something before coming to school. During focus group discussions the majority of parents reported their children go to school having eaten breakfast, but there were some who mentioned that the schools sometimes have breakfast and sometimes have a snack (refacción), so they are not sure whether their child needs to eat before school or not, because each day is different. This suggests that parents should be better informed by the school, on what meals children are going to receive during the school day. Around 90 percent of students ate something at school the day before data collection. Of the students who received a meal the day before data collection, only 16-17 percent received a full meal, and about 45 percent received a small meal and a drink. Ninety-six percent of the boys and 98 percent of the girls said that they like the meals they receive in school. Table 11. Nutrition by students’ sex – PE Males Females Difference Did you eat something before coming to school today? 0.89 0.93 -0.04 Did you receive food at school yesterday? 0.90 0.89 0.01 What did you eat? A drink only (e.g. atol) 0.08 0.08 0.00 A small meal and a drink 0.46 0.45 0.00 A small meal 0.21 0.20 0.02 Full meal 0.16 0.17 -0.01 NR 0.09 0.10 -0.01 Do you like school meals? 0.96 0.98 -0.02 N 340 348 Note: For the differences tests, standard errors are clustered at the school level. Sample sizes are a few observations smaller for some items due to item-specific missing data. * p<.1 ** p<.05 *** p<.01 NORC | LENS PROGRAM BASELINE REPORT | 36 Although almost all students reported satisfaction with the school meals, the parent focus groups revealed that their views were less positive. In the majority of the focus groups, 8 of 10, participants stated that their children are not satisfied with the school meals. The two main reasons for this perception were (1) repetition of meals and (2) quality of the cooked food. One female parent from a K’iche’ predominant municipality said, “There have been many issues at the school because they repeat the [school meal] a lot. I’ve noticed that in multiple classrooms the food is wasted, it should not be like that. One should be conscious of what the children like and if possible not repeat the meals so much.” Another female parent from an Ixil predominant municipality qualified the issue by saying, “They like [the school meals], but they get bored. They like it! Maybe once or twice a week they get bored maybe every day.” For other parents, the main concern was quality of the cooked food. Many parents pointed out that some of the food served to their children was not prepared properly. One female parent mentioned that “because of the way it’s cooked, sometimes the atol is undercooked or poorly prepared”. The fact that students have much more favorable views on the quality of the provided meals could be the product of social desirability bias. When asked how school meals can be improved, many parents suggested diversifying school meals. Respondents were also specifically asked how local products can be incorporated to diversify school meals. All but one focus group mentioned local food items that can be incorporated, including fruits and vegetables. When asked about seasonality, focus group respondents mentioned that fruits tended to be available seasonally only, but that vegetables and herbs were available year-round. On a related note, when asked what challenges a school meal program like LENS could face, parents from four of the focus groups pointed to local acceptance or agreement over the appropriateness of the meals as a possible challenge.25 One female parent stated a challenge could be that, “the community does not accept the benefits of the program and that [the program] should know our needs.” The majority of the parents believed the program would be a benefit to the students for a variety of reasons, including improvements on nutrition, health, and support in the school. Figure 8 shows results for selected reading skills by gender. Albeit females have higher scores than males across three of the five subtasks, the differences are small and not statistically significant. 25 The majority of parents were not familiar with the LENS program (albeit some did recognize the Save the Children’s name), so enumerators gave a brief explanation so parents could answer program-related questions. NORC | LENS PROGRAM BASELINE REPORT | 37 Figure 8. Reading skills by students’ sex – PE Note: Sample sizes are 332 for males and 336 for females, although the number of observations may be smaller for some items due to item-specific missing data. The differences with respect to the sample sizes for the sociodemographic variables correspond to students that did not pass the initial instructions to answer the assessment. For the differences tests, standard errors are clustered at the school level.* p<.1 ** p<.05 *** p<.01 Overall, there are no major differences by sex, across sociodemographic characteristics or reading skills in the program evaluation schools. Characteristics by Municipality Table 12 shows descriptive statistics by municipality. In the first three columns the results for the three municipalities with Ixil communities, namely Chajul, Cotzal and Nebaj, are displayed. The second three columns show results where K’iche’ is the predominant local language, these are Cunén, Sacapulas and Uspantán. Across all six municipalities, there are no major differences in students’ age or sex. In Chajul, Cotzal and Nebaj about a fourth students speak Spanish at home. While in these three municipalities there are students that speak K’iche’ at home, especially in Cotzal, Ixil is the predominant local language. Of the three municipalities where there are no Ixil communities, in Cunén roughly two thirds of the students speak K’iche’ at home, while in Sacapulas and Uspantán only about a third speak K’iche’. Although for most variables there are no clear differences between Ixil and the K’iche’ municipalities, there are a few patterns that are worth highlighting. First, students in Ixil municipalities are more likely to have repeated a year than students in K’iche’ municipalities. Also, mothers in K’iche’ municipalities are more likely to be literate than in Ixil municipalities. Moreover, households in K’iche’ municipalities are more likely to have electricity and have more household assets, than households in the Ixil municipalities. These differences suggest that students in Ixil municipalities come from more disadvantaged backgrounds than students in K’iche’ municipalities. 51.7 28.0 38.1 52.9 40.5 52.0 27.7 39.5 54.5 39.2 0.3 -0.3 1.4 1.6 -1.3 Familiar Word Reading - % Correct Non-Word Reading - CWPM Listening Comprehension - % Correct Oral Reading Fluency - CWPM Reading Comprehension - % Correct Males Females Difference NORC | LENS PROGRAM BASELINE REPORT | 38 Table 12. Sociodemographic characteristics by municipality – PE Ixil K’iche’ Chajul Cotzal Nebaj Cunén Sacapulas Uspantán Male 0.49 0.50 0.52 0.51 0.52 0.50 Age (years) 9.6 9.9 9.9 9.6 9.3 9.6 Language spoken at home Spanish 0.19 0.36 0.23 0.34 0.53 0.44 K’iche’ 0.09 0.24 0.01 0.62 0.39 0.37 Ixil 0.51 0.41 0.70 0.00 0.00 0.00 Other 0.01 0.00 0.08 0.00 0.09 0.13 NR 0.23 0.01 0.07 0.06 0.12 0.11 Attended preschool 0.80 0.80 0.84 0.65 0.79 0.85 Repeated or repeating a year 0.62 0.57 0.58 0.44 0.32 0.41 Attendance last week (fraction) 0.96 0.97 0.95 0.97 0.96 0.99 Household reading environment There are books at home 0.78 0.70 0.70 0.67 0.72 0.71 Household members read with child 0.42 0.73 0.44 0.62 0.48 0.48 Mother's literacy 0.58 0.54 0.53 0.69 0.75 0.61 Father's literacy 0.83 0.86 0.82 0.94 0.87 0.64 Socioeconomic indicators Household has electricity 0.66 0.68 0.83 0.82 0.89 0.91 Household owns a cellphone 0.94 0.92 0.94 0.96 0.97 0.97 Asset index (number of items) 7.02 6.69 7.35 8.04 8.43 8.27 N 141 73 175 138 83 78 Note: Sample sizes are a few observations smaller for some items due to item-specific missing data. Table 13 shows nutrition indicators by municipality. There are no major differences in the likelihood that students eat something before going to school. Regarding eating something at school, in Chajul only 75 percent of students said they ate something at school the previous day, while in the other five municipalities this proportion is roughly 90 percent. It is also worth highlighting that students in K’iche’ municipalities are more likely to receive a full meal than students in Ixil municipalities. Table 13. Nutrition by municipality – PE Ixil K’iche’ Chajul Cotzal Nebaj Cunén Sacapulas Uspantán Did you eat something before coming to school today? 0.92 0.96 0.90 0.91 0.88 0.87 Did you receive food at school yesterday? 0.75 0.89 0.96 0.88 0.97 0.90 What did you eat? A drink only (e.g. atol) 0.11 0.14 0.06 0.05 0.00 0.15 NORC | LENS PROGRAM BASELINE REPORT | 39 Ixil K’iche’ Chajul Cotzal Nebaj Cunén Sacapulas Uspantán A small meal and a drink 0.33 0.33 0.55 0.46 0.61 0.35 A small meal 0.21 0.29 0.22 0.17 0.13 0.20 Full meal 0.17 0.10 0.07 0.27 0.19 0.23 NR 0.18 0.14 0.09 0.05 0.08 0.07 Do you like school meals? 0.99 0.94 0.96 0.97 0.97 0.99 N 141 73 175 138 83 78 Note: Sample sizes are a few observations smaller for some items due to item-specific missing data. Figure 9 shows reading skills scores by municipality. Panel A shows results for the three Ixil municipalities. Across all five subtasks Cotzal has the highest average scores, followed by Nebaj and Chajul. There are major disparities in reading skill across these three municipalities. For example, while students in Cotzal read, on average, 60.5 cwpm, in Chajul this figure is only 37.4 cwpm. Interestingly, despite the fact that students, especially in Cotzal, are able to read a fair amount of words per minute, their reading comprehension level is still relatively low; on average, they are only able to answer 44 percent of the questions correctly. Panel B shows results for K’iche’ municipalities. Uspantán has the highest average scores across all subtasks. Sacapulas outperforms Cunén in non-word reading and listening and reading comprehension, while the opposite pattern is observed for familiar word reading and oral reading fluency. NORC | LENS PROGRAM BASELINE REPORT | 40 Figure 9. Reading skills by municipality – PE Note: Total sample size is 668, although the number of observations may be smaller for some items due to item￾specific missing data. 57.2 32.4 43.4 60.5 43.7 51.1 28.0 42.0 52.9 42.6 41.1 21.9 24.7 38.0 24.3 Familiar Word Reading - % Correct Non-Word Reading - CWPM Listening Comprehension - % Correct Oral Reading Fluency - CWPM Reading Comprehension - % Correct A. Ixil predominant municipalities Cotzal Nebaj Chajul 58.7 30.4 51.2 63.0 56.2 47.3 27.6 44.9 48.5 46.6 53.6 26.8 31.9 55.9 33.6 Familiar Word Reading - % Correct Non-Word Reading - CWPM Listening Comprehension - % Correct Oral Reading Fluency - CWPM Reading Comprehension - % Correct B. K'iche predominant municipalities Uspantán Sacapulas Cunén NORC | LENS PROGRAM BASELINE REPORT | 41 It is interesting that despite the fact that students read a fair amount of words per minute, especially in municipalities like Uspantán and Cotzal, their reading comprehension scores are quite low. This could imply that students need to read faster to achieve acceptable levels of reading comprehension. Figure 10 shows average oral reading fluency by the number of reading comprehension questions correctly answered. It shows that students who answered 9-11 questions correctly read, on average, 77 words per minute, many more than the municipality-averages shown in Figure 9. Annex III shows this figure by language most spoken at home. Although the overall patterns are consistent with what is shown in Figure 9, for K’iche’ and Ixil the slope is less steep because there are so few students that answered correctly more than eight reading comprehension questions. Figure 10. Average reading fluency by number of reading comprehension questions correctly answered. To explore the relationship between sociodemographic characteristics and reading skills, we display bivariate regressions results between reading skills and selected socio-demographic characteristics in Table 14. Although all the correlations for males are negative (meaning that girls outperform boys), none of the coefficients are statically significant. For age, we ran regressions where the dependent variables are the different reading scores, and the covariates are dummy variables for the different ages; the omitted category is eight years, so coefficients should be interpreted as a function of the result for 8 years old students. All the age coefficients are negative; moreover, for most reading skills the coefficients increase with age in absolute value. This indicates that age and reading skills are negatively correlated. This could be because struggling students are more likely to repeat years, so the older a 3rd grader is, the lower her/his reading achievement would be. Speaking Spanish at home is positively correlated with reading skills; this is hardly surprising as the reading assessment was conducted in Spanish. Similarly, having attended preschool is positively correlated with all reading skills. Repeating one or more years is negatively correlated with reading 31 52 59 68 77 0 20 40 60 80 Correct Words Per Minute (ORF Sub-Task) 0-2 3-4 5-6 7-8 9-11 Reading Comprehension Raw Score NORC | LENS PROGRAM BASELINE REPORT | 42 skills. This reflects the same pattern found between age and reading skills, and suggests that repeating a year is not helping students catch up with students who have not repeated a year, at least not completely. Although the correlations between reading skills and having books at home is positive across all subtasks, none of the coefficients are statically significant, suggesting the relationship between having books at home and reading skill is weak at best. Other household indicators, like mother’s literacy and number of household assets, are more clearly correlated with reading skills. Both of the variables are positively correlated with reading skills, and the coefficients are significant across all specifications. Finally, there is no clear relationship between having eaten something before coming to school and reading skills. The coefficients are negative for four of the five subtasks, but the parameters is significant only for listening comprehension. Baseline data collected for the evaluation of IDEA confirms some of these results. In particular, IDEA data also shows that grade repetition is also negatively correlated with reading skills, girls (weakly) outperform boys in reading skills, and socioeconomic status (measured for IDEA as an index that combines type of home, household size, household amenities/possessions) is positively correlated with reading skills, just as the household asset index below in Table 14 is positively correlated with reading skills.26 Table 14. Correlations between reading skills and sociodemographic characteristics Familiar Word Reading % Correct Non-Word Reading - CWPM Listening Comprehension - % Correct Oral Reading Fluency - CWPM Reading Comprehension - % Correct Male -2.85 -0.85 -1.68 -3.29 -0.62 Age 9 yo -3.65 -1.69 -3.47 -4.42 -7.33 10 yo -8.51 * -3.61 -13.33 ** -10.98 ** -17.52 *** 11 yo -12.80 ** -5.62 * -12.28 * -15.46 *** -18.05 ** 12 yo -13.86 ** -4.57 -7.51 -20.55 *** -18.09 ** 13+ yo -29.64 *** -16.15 *** -21.01 *** -31.63 *** -31.33 *** Spanish is native language 12.40 *** 6.05 *** 27.37 *** 17.27 *** 27.98 *** Attended Any Pre-School 6.67 ** 3.79 ** 11.67 *** 9.20 ** 13.69 *** Ever failed a grade and had to repeat -11.48 *** -5.74 *** -10.57 *** -14.22 *** -14.55 *** Are there books in your house? 1.79 0.85 1.21 2.56 2.24 Literate Mother = YES 6.19 ** 3.53 ** 7.04 ** 8.09 *** 8.38 *** Household Asset Index (0=Poorest,13=Richest) 1.43 *** 0.74 *** 1.87 *** 2.15 *** 1.91 *** # of People in Household 0.09 -0.05 0.09 -0.16 -0.06 Ate Before School Today = YES 1.14 -0.44 -8.24 * -1.69 -6.31 Note: For the differences tests, standard errors are clustered at the school level. * p<.1 ** p<.05 *** p<.01 Teachers’ surveys and interviews Table 15 shows sociodemographic characteristics for the 72 surveyed teachers. Teachers are, on average, 37 years old and roughly half of them are males. They have an average of 11 years of experience teaching in primary schools. For a fourth of teachers Spanish is the mother tongue, for 19 percent it is K’iche’, for 46 percent it is Ixil, for 3 percent Uspanteco and for ten percent Sacapulteco. The prevalence of Ixil is 26 See Figure 15, Table C1 and Figure 17, respectively, in the Baseline report. NORC | LENS PROGRAM BASELINE REPORT | 43 relatively higher because we surveyed more schools in municipalities where the predominant local language is Ixil (41) than in municipalities where the predominant local language is K’iche’ (31). Teachers have relatively small classrooms, with an average of 18 students per classroom. Finally, teachers missed, on average, almost one day of class the week before data collection, which constitutes a relatively high absenteeism rate of 18 percent. Table 15. Teachers’ characteristics Mean Age (years) 36.9 Male 0.49 Years of experience (primary school) 11.4 Mother tongue Spanish 0.25 K’iche’ 0.19 Ixil 0.46 Uspanteco 0.03 Sacapulteco 0.10 Language(s) used in class Spanish 0.93 K’iche’ 0.25 Ixil 0.42 Uspanteco 0 Sacapulteco 0.04 Education level Primary level teaching certificate for urban areas ( MEPU ) 0.21 Primary level teaching certificate for rural and bilingual areas (MEPR ) 0.19 Secondary level teaching certificate 0.31 Some college 0.06 Technical degree 0.04 Professional degree 0.08 Other 0.11 Number of students in classroom 18.05 Number of school days missed last week 0.90 N 72 Note: Sample sizes are a few observations smaller for some items due to item-specific missing data. The survey also asked teachers which language or languages they use in class. We allowed for multiple responses in order to have a better picture of how teachers communicate in the classroom. While most teachers use Spanish in class, local languages are also prevalent. As Figure 11 shows, in the three K’iche’ municipalities, 94 percent of the teachers use Spanish, 52 percent use K’iche’, six percent use Uspanteco and 16 percent use Sacapulteco. In Ixil municipalities, 93 percent of the teachers use Spanish, 73 percent use Ixil and five percent use K’iche’. NORC | LENS PROGRAM BASELINE REPORT | 44 Figure 11. Language used in class, by local language Teachers’ previous exposure to la Taza Escolar and knowledge of food groups Given that the predominantly carbohydrate-based diets in the Western Highlands are associated with the rising prevalence of overweight, diabetes and other chronic diseases, 27 LENS will support teachers to recognize and promote a nutritionally balanced diet for students. La Taza Escolar (Student Cup) (Figure 12) is a graphical representation of foods and food groups, similar to the US food pyramid and Guatemala’s La Olla Familiar (Family Pot). Based on the National Food Guidelines (Guias Alimentarias Nacionales), la Taza Escolar has been embraced by MINEDUC to guide the selection of diverse foods to assure nutritionally balanced school meals. Figure 12. La Taza Escolar To measure whether teachers have prior exposure to the Taza Escolar graphic, we showed teachers Figure 12 and asked if they had seen it before. The majority of teachers (70 percent) reported having seen 27 Ricardo Uauy, Cecilia Albala, Juliana Kain, Obesity Trends in Latin America: Transiting from Under- to Overweight, The Journal of Nutrition, Volume 131, Issue 3, March 2001, Pages 893S–899S, https://doi.org/10.1093/jn/131.3.893S 0.94 0.93 0.52 0.05 0.00 0.73 0.06 0.00 0.19 0.02 0.00 0.10 0.20 0.30 0.40 0.50 0.60 0.70 0.80 0.90 1.00 K'iche is most prevalent local language (N=31) Ixil is most prevalent local language (N=41) Spanish K'iche Ixil Uspanteco Sacapulteco NORC | LENS PROGRAM BASELINE REPORT | 45 the Taza Escolar image prior to the survey. Figure 13 presents the breakdown of where teachers recalled seeing la Taza Escolar image for the first time. Among the teachers who had previously seen the image, 28.2 percent reported seeing it first in a book or during the teacher’s own studies, while nearly a quarter (26 percent) said they saw it in a MINEDUC training or materials. Thirteen percent reported seeing it through Save the Children activities or materials. The remaining were split between having seen it in an unspecified training (10 percent) and other/don’t know (23 percent). Figure 13. Where teachers saw the image for the first time Based on la Taza Escolar, MINEDUC developed a set of school menus with each meal composed of foods from different “levels” of the Taza Escolar, representing different food groups. To gauge teachers’ familiarity with the concept that school meals should include a diversity of food groups, we asked teachers to select the best example of a school lunch from among several options. Figure 14 shows that 59 percent correctly selected a bean and corn tamale with cheese and sauce, the one meal option that included several food groups and was specifically promoted in the MINEDUC school meal menu. This meal contrasted with the other options, including a hot oatmeal drink, a corn tortilla with avocado or spaghetti soup, which do not include several food groups. The fact that nearly half of teachers did not select the balanced meal suggests that even teachers who have seen la Taza Escolar lack understanding of how la Taza Escolar can be used to select a nutritionally complete meal for students. This implies there is an opportunity for LENS to provide teachers with nutrition education about balanced meals composed of several food groups depicted by la Taza Escolar. 25.6 12.8 10.3 28.2 23.1 0 10 20 30 Percentage of Teachers (%) Ministry of Education Training or Materials Save the Children Other Training or Unspecified Training Book or Teachers Own Studies Other / Dont Know Where did you first see the Taza Escolar? NORC | LENS PROGRAM BASELINE REPORT | 46 Figure 14. Teachers’ selection of best school lunch Teachers were asked to consider whether it is true or false that a child with a daily diet of oatmeal, corn flakes, tortilla with beans and rice for lunch and dinner is well-nourished. Figure 15 shows that only 45 percent of teachers correctly said “false”, suggesting that there are gaps in teachers’ nutritional knowledge. 11.8 8.8 58.8 14.7 5.9 0 20 40 60 Percentage of Respondents Oatmeal Tortilla with Avocado Bean Tamale w/ Cheese & Sauce Noodle Soup None of the Above What is the Best Example of a School Lunch? NORC | LENS PROGRAM BASELINE REPORT | 47 Figure 15. Teachers’ belief that a typical carbohydrate-based diet is nutritious Given that cereals, grains and tubers, such as maize, pasta, rice, potatoes and bread are predominant in the diets of the Western Highlands,28 the baseline sought to understand teachers’ knowledge of the function of different food groups, as a crucial aspect for promoting the need to consume foods from each food group to achieve a nutritionally balanced diet. Through a set of three questions, teachers were asked to identify which foods or food group: 1. primarily contributes to giving children energy for physical activity; 2. primarily helps children grow; and 3. primarily protects from illness due to its high quantity of vitamins, minerals and fiber? Teachers’ responses to question 2 – Which foods help children grow? - are shown in Figure 16. The correct options are animal protein, and beans and soya protein. Fifty-eight percent of teachers correctly identified animal protein (i.e. milk, meat, eggs and other animal source foods) as helping children grow, while only 18 percent recognized that the legumes beans, soy and fava/broad beans are also growth foods. More than half the teachers (58 percent) incorrectly selected leafy greens, vegetables and fruit. Overall, 59 percent of teachers selected at least one correct response, and 17 percent mentioned both correct responses. 28 FANTA. 2013. Summary Report: Development of Evidence-Based Dietary Recommendations for Children, Pregnant Women, and Lactating Women Living in the Western Highlands in Guatemala. Washington, DC: FHI 360/FANTA. 45.1 43.7 9.9 1.4 0 10 20 30 40 50 Percentage of Respondents False True Not Sure No response Daily Diet of Oatmeal, Cornflakes, and Tortilla with Beans & Rice is Nutritious NORC | LENS PROGRAM BASELINE REPORT | 48 Figure 16. Foods that help children grow according to teachers Notably, a greater percentage of teachers recognize animal source foods as important for growth, while few recognize the importance of legumes. This is an important gap in teacher’s knowledge about growth foods, considering that animal protein is scarce in the diets of the Western Highland, because of its relatively high cost and the challenges due to lack of refrigeration and consistent market access.29 Increasing the consumption of animal protein and the micronutrients it contains (e.g. iron, zinc and vitamin B12) is a priority to improve the quality of diets in the Western Highlands and address stunting.30 However, plant-based protein sources, particularly legumes, such as black beans, fava beans and soy (particularly in Incaparina fortified blended flour or Protemás Textured Soy Protein) are important sources of protein that are more accessible and affordable for families in the Western Highlands.31 LENS has the opportunity to increase teachers’ understanding and promotion of growth foods, particularly plant-based protein sources, in combination with lower-cost animal protein, such as eggs, or small animals like chicken. Teachers were also asked which food group principally provides energy to children for physical activity. As Figure 17 shows, only 18 percent of teachers correctly selected cereals, grains and tubers, the only correct answer provided. Thirty-two percent selected animal protein, and almost half (45 percent) selected herbs, vegetables and fruits, which in fact provide the least energy. As teachers were not familiar with foods that primarily provide energy, they lack the information to teach children that typical diets in Quiche commonly provide an overabundance of energy foods (i.e. carbohydrates). LENS could help 29 FANTA. 2015. Validation of Food-Based Recommendations Developed using Optifood for Groups at Nutritional Risk in the Western Highlands of Guatemala. Washington, DC: FHI 360/FANTA. 30 FANTA. 2015. Technical Brief: Reducing Stunting in the Western Highlands of Guatemala: Promoting the Consumption of Animal-Source Food to Optimize the Diets of Women and Young Children. Washington, DC: FHI 360/FANTA 31 FANTA. 2015. Validation of Food-Based Recommendations Developed using Optifood for Groups at Nutritional Risk in the Western Highlands of Guatemala. Washington, DC: FHI 360/FANTA. 57.7 35.2 57.7 18.3 11.3 0.0 0 20 40 60 Percentage of Teachers Who Answered 'Yes' (%) Animal Protein Cereals, Grains, and Tubers Herbs, Vegetables, and Fruits Beans and Soya Protein Fats None of the Above Which foods help children grow? Incorrect Answer Correct Answer NORC | LENS PROGRAM BASELINE REPORT | 49 teachers and students understand this imbalance in the typical diet, and give teachers the tools to promote foods that support growth (i.e. protein foods) and protection from illness (i.e. fruits, leafy greens and other vegetables). Figure 17. Foods that give children energy, according to teachers Teachers were also asked which foods protect children against sickness, due to their high content of vitamins, minerals and fiber. Figure 18 shows that two-thirds of teachers (64 percent) selected leafy greens, vegetables and fruit, the only correct answer. The most common incorrect response was milk, meat, eggs and other animal source foods, selected by almost a quarter of teachers. As the majority of teachers recognize the nutritional value of leafy greens, vegetables and fruit, LENS can build on this knowledge base by helping teachers more clearly differentiate the contribution of different food groups. For leafy greens, vegetables and fruit, LENS may highlight locally available foods, including foods grown in school gardens. 32.4 18.3 45.1 1.4 1.4 1.4 0 10 20 30 40 50 Percentage of Teachers Who Answered 'Yes' (%) Animal Protein Cereals, Grains, and Tubers Herbs, Vegetables, and Fruits Beans and Soya Protein Fats None of the Above Which foods give children energy? Incorrect Answer Correct Answer NORC | LENS PROGRAM BASELINE REPORT | 50 Figure 18. Foods that protect children from illnesses, according to teachers School hygiene To assess teachers’ knowledge of the importance of handwashing, the survey asked, without providing options, when it was important for children to wash their hands. According to the Global Handwashing Partnership, two critical times for handwashing have the greatest documented impact on child health, including: 1) after contact with feces (such as after using the toilet or changing diapers) and 2) before contact with food (such as eating or preparing food).32 In addition, the Centers for Disease Control promotes handwashing after sneezing, coughing or blowing one’s nose and after contact with animals, garbage or other dirty items. Figure 19 shows that 91.5 percent of teachers mentioned before eating, while only around half (53 percent) mentioned after using the latrine or toilet. Given that schools rely on teachers to define, encourage and enforce handwashing routines at school, these findings indicate that teachers require more support to clarify the priority moments for handwashing, possibly through games or other methods that help them visualize the critical pathways for disease, and which they can then share with their students. 32 https://globalhandwashing.org/about-handwashing/faqs/#when 22.9 10.0 64.3 1.4 1.4 0 20 40 60 Percentage of Teachers Who Answered 'Yes' (%) Animal Protein Cereal, Grains, and Tubers Leafy Greens, Vegetables, and Fruit Beans, Soy Meat, and Broad Beans None Which Type of Food Protects Children From Diseases? Incorrect Answer Correct Answer NORC | LENS PROGRAM BASELINE REPORT | 51 Figure 19. Moments when students should wash their hands In Key Informant Interviews (KII), all interviewed teachers spoke to the importance of cleanliness and hygiene, and most teachers specifically mentioned handwashing. More than half of the teachers had previously received training on school hygiene, including several who had attended trainings led by Save the Children. Teachers commonly mentioned that proper handwashing techniques were covered in the trainings and believed it was important that students practice “handwashing before and after going to the bathroom, before and after eating.” The teachers KAP survey also asked which moments in the day their students washed their hands at school the day prior to the survey. Again, options were not provided to the teachers. Figure 20 shows that 94 percent of teachers said their students washed hands before eating, but a much lower percentage said students washed their hands after using the bathroom (45 percent) or touching something dirty (20 percent). It is notable that there is both high teacher awareness of handwashing before eating and high reported student handwashing before eating. Qualitative data indicate that some teachers have classroom routines for all students to line up to wash hands before eating, which facilitates the practice and teacher’s verification. Also, students may visit the restroom individually during class, so teachers do not see whether they washed their hands or not. In order to promote handwashing, it may be helpful for LENS to focus on the area of greatest apparent deficit (handwashing after using the latrine) and solicit suggestions from students and teachers about how to verify handwashing, such as having a buddy system for handwashing, or having students mark on a chart whether they washed hands or not, (and perhaps whether soap and water were available). Regarding handwashing after sneezing, coughing or blowing one’s nose, although these moments are not fecal-oral pathways, is it an important pathway for transmitting infections in the classroom, and could be emphasized as a way of limiting transmission of other illnesses. 91.5 14.1 53.5 29.6 4.2 1.4 1.4 0 20 40 60 80 Percentage of Teachers Who Mentioned Response Option (%) Before Eating Before Cooking After Using the Restroom After Touching Something Dirty After Sneezing After Changing Diapers No Response / Dont Know Important Moments for Handwashing NORC | LENS PROGRAM BASELINE REPORT | 52 Figure 20. Moments when students washed their hands the day before data collection (as reported by the teacher) Nearly all teachers interviewed during KIIs stated they try to promote good hygiene habits, including handwashing, among their students. When asked how they promote good hygiene practices in the classroom, teachers pointed to various strategies they use to impart good hygiene practices to their students, such as using a chart to track students’ progress or creating games to encourage students. Several respondents explained they require their students to practice washing their hands daily or several times a day. One teacher said they teach “that they have to do [handwashing] every time before and after going to the bathroom, then before consuming the food. I tell them they are going to wash their hands, because there are children who touch all kinds of things, and at the time of the snack… not knowing how many things they have touched, I say, you have to wash your hands to be able to eat the snack.” Given the high prevalence of tooth decay among Guatemalan children,33 LENS intends to promote oral hygiene in the treatment schools, specifically the practice of students brushing teeth at school with a fluoride toothpaste. Brushing teeth with a fluoride toothpaste has been shown to reduce oral cavities.34 About a third of teachers (37 percent) reported that their students brushed their teeth at school yesterday (or on the last day of school). The fact that only about a third of teachers reported that their children brushed their teeth at school suggests that this practice is not yet well-established. LENS may choose to explore the barriers that prevent teachers from having a class brush their teeth, particularly since brushing their teeth requires each child to have a toothbrush and toothpaste. During KIIs several teachers said they took steps to promote tooth brushing among their students, and to assure that students brushed their teeth. During an interview, one teacher commented, “what we do in the morning is ask them if they brushed their teeth. If they have not brushed their teeth we take the children to 33 Archila, Luis & D Bartizek, Robert & W Gerlach, Robert & A Jacobs, Steven & Biesbrock, Aaron. (2003). Dental Caries in School-Age Children Residing in Five Guatemalan Communities. The Journal of clinical dentistry. 14. 53-8. 34 O'Mullane DM, Baez RJ, Jones S, et al. Fluoride and Oral Health. Community Dent Health. 2016;33(2):69-99. 94.4 5.6 45.1 19.7 0.0 0 20 40 60 80 100 Percentage of Teachers Whose Students Washed Their Hands (%) Before Eating Before Cooking After Using the Restroom After Touching Something Dirty After Sneezing Moments Students Washed Their Hands Yesterday NORC | LENS PROGRAM BASELINE REPORT | 53 brush them.” A few teachers mentioned they have their students bring a toothbrush to school with them or keep one in the classroom. Nutritional challenges relevant to schoolchildren Despite declines in anemia rates, child anemia remains a serious risk to children’s growth and development in Quiche, affecting 28 percent of children aged 6-59 months old, according to the most recent 2014-2015 Demographic and Health Survey (DHS). Anemia’s detrimental consequences on cognitive development and school outcomes are well-documented.35 36 Figure 21 summarizes results for three indicators on awareness of child anemia. First, 93 percent of teachers indicated that they were in agreement with the statement that “Anemia due to lack of iron is a serious condition in children.” This indicates a high level of teacher awareness of the seriousness of child anemia, providing a strong base for further teacher education, as other baseline questions, discussed below, indicate lower teacher knowledge of specific actions needed to prevent anemia. Figure 21 also shows that 94 percent of respondents correctly responded “true” to the statement that signs of anemia include tiredness, paleness and learning difficulties. Lastly, 80 percent of respondents correctly marked “true” to the statement that children can get anemia if their diets lack foods containing iron or are fortified with iron. Although a high percentage of teachers apparently are aware of the need for iron in the diet, teachers’ responses to other baseline questions, presented below, identify gaps in knowledge about which foods provide iron. Figure 21. Awareness of child anemia (%) 35 Black, R.E., et al. (2008). Maternal and child undernutrition: global and regional exposures and health consequences. The Lancet Maternal and Child Health Series 371 (9608), 243–260. 36 Sachdev H, Gera T, and, Nestel P. Effect of iron supplementation on mental and motor development in children: systematic review of randomised controlled trials. Public health nutrition 2005; 8 (2):117–32. 92.8 94.4 80.3 0 20 40 60 80 100 Percentage of Respondents Anemia is a serious condition in children Anemia symptoms include tiredness, paleness, and learning difficulties Children are more likely to have anemia if they do not consume enough iron NORC | LENS PROGRAM BASELINE REPORT | 54 To understand teachers’ current knowledge about anemia prevention, the survey asked them to select foods that are important for preventing anemia. The options provided were relatively important sources of iron in local diets,37 plus one incorrect option (apples). Results are shown in Figure 22. Even among foods such as liver and meat/fish, that are generally promoted for preventing anemia, few teachers correctly identified these foods as preventing anemia. Moreover, less than a quarter (24 percent) of teachers correctly selected at least two foods that prevent anemia, and few teachers (5 percent) selected at least three or all four correct options. Given the high prevalence of anemia in Guatemalan children and importance of anemia to students’ educational outcomes, there is a valuable opportunity to help teachers become more aware of the consequences and importance of anemia, as well as specific foods that prevent anemia. However, in promoting foods rich in iron that prevent anemia, it is important to note that previous research shows that cost is a primary barrier to increased consumption of animal source foods in Quiché. It is more feasible to promote consumption of fortified blended flours (e.g. Incaparina) and fortified oats, as well as black beans.38 LENS should also educate teachers, students and parents about the benefits and origin of the fortified rice and oil provided as US donated commodities. An opportunity exists for LENS to raise awareness of lower cost foods that prevent anemia, such as fortified foods, and legumes, and assure that eating red meat is not viewed as necessary to prevent anemia. Figure 22. Percentage of teachers’ who identified foods to prevent anemia 37 FANTA. 2013. Summary Report: Development of Evidence-Based Dietary Recommendations for Children, Pregnant Women, and Lactating Women Living in the Western Highlands in Guatemala. Washington, DC: FHI 360/FANTA. 38 Food and Nutrition Technical Assistance III Project (FANTA). 2015. Technical Brief: Micronutrient Malnutrition among Women and Young Children in the Western Highlands of Guatemala: What Are the Needs and What Can Be Done? Washington, DC: FHI 360/FANTA. 33.8 22.5 26.8 40.8 16.9 8.5 0 10 20 30 40 Percentage of Teachers Who Answered 'Yes' (%) Liver Meat/Fish Apples Incaparina and Other Fortified Products Beans Dont Know/ No Response Which foods are important for preventing anemia? Incorrect Answer Correct Answer NORC | LENS PROGRAM BASELINE REPORT | 55 Iodine deficiency causes irreversible damage to cognition, including lower IQ scores in schoolchildren39, and salt iodization is the major public health strategy to combat iodine deficiency. Since 1952, Guatemala has carried iodization of processed commercial salt, but program staff report that unprocessed artisanal salt is regularly consumed in some municipalities of Quiche. The survey included a baseline question to gauge teacher’s awareness of why it is important to consume iodized salt. As shown in Figure 23, only 8.5 percent of teachers correctly chose “for intelligence.” As few teachers are aware of the consequences of iodine deficiency, LENS can emphasize this issue related to the promotion of iodized salt. Figure 23. Teachers’ knowledge of the importance of iodized salt Figure 24 shows that 89 percent of respondents know that Incaparina is rich in calcium. Animal milk is typically promoted as a source of calcium. However, given that milk and milk products are rarely consumed in Quiche, due in part to cost barriers, lack of cold storage and preferences,40 it is important to raise awareness that Incaparina is a local calcium source, available at a lower cost than milk.41 39 Kapil U. Health consequences of iodine deficiency. Sultan Qaboos Univ Med J. 2007 Dec;7(3):267-72. PubMed PMID: 21748117; PubMed Central PMCID: PMC3074887. 40 SPRING. 2015. Formative Research: Key Influencers of Household Food Access in the Western Highlands of Guatemala. Arlington, VA: USAID/ Strengthening Partnerships, Results, and Innovations in Nutrition Globally (SPRING) Project. 41 FANTA. 2013. Summary Report: Development of Evidence-Based Dietary Recommendations for Children, Pregnant Women, and Lactating Women Living in the Western Highlands in Guatemala. Washington, DC: FHI 360/FANTA. 35.2 8.5 9.9 16.9 29.6 0 10 20 30 40 Percentage of Teachers Who Answered 'Yes' (%) To Grow For Intelligence For Teeth None of the Above Don't Know Why is it important to consume iodized salt? NORC | LENS PROGRAM BASELINE REPORT | 56 Figure 24. Percentage of teachers’ who agreed with the statement that Incaparina is rich in calcium. As shown in Figure 25, 76 percent of teachers expressed agreement with the statement, “To pay attention in class, it is important that children drink enough water.” As even mild dehydration is associated with reduced cognitive performance in school, it’s valuable for teachers to be aware of the benefit of encouraging children to consume enough water at school.42 42 Clinton S. Perry III et al. Hydration status moderates the effects of drinking water on children's cognitive performance. Appetite. Volume 95, 1 December 2015, Pages 520-527, https://doi.org/10.1016/j.appet.2015.08.006 1.4 88.7 8.5 1.4 0 20 40 60 80 100 Percentage of Teachers False True Not Sure No Response Incaparina is rich in calcium NORC | LENS PROGRAM BASELINE REPORT | 57 Figure 25. Percentage of teachers’ who agreed with the statement that “to pay attention in class, children need to drink enough water.” Figure 25 shows that 94 percent of teachers agreed with the statement, “My students need to learn about nutrition to prevent obesity.” About half of Guatemalan women of reproductive age are overweight or obese43 and a growing percentage of children are affected,44 making it highly relevant for teachers to be aware of the need to prevent obesity. 43 Ministerio de Salud Pública y Asistencia Social (MSPAS), Instituto Nacional de Estadística (INE), ICF International. 2017. Encuesta Nacional de Salud Materno Infantil 2014–2015. Informe Final. Guatemala: MSPAS/INE/ICF. 44 Muros, José & Briones, Mercedes & Rodríguez, Guillermo & Bouzas, P.R. & Giménez, Rafael & Cabrera, Carmen. (2016). Double burden of malnutrition in rural and urban Guatemalan schoolchildren. Nutricion hospitalaria: organo oficial de la Sociedad Espanola de Nutricion Parenteral y Enteral. 33. 345-350. 76.1 7.0 12.7 4.2 0 20 40 60 80 Percentage of Teachers Agree Neutral Disagree Don't Know Agreement with the statement 'To pay attention in class, it is important that children drink enough water.' NORC | LENS PROGRAM BASELINE REPORT | 58 Figure 26. Percentage of teachers’ who agreed that students need to learn about nutrition to prevent obesity. Due to the extremely high consumption of sugar in Western Highlands and the alarming prevalence of associated problems of diabetes, obesity and tooth decay, the survey asked teachers to select from a list of possible negative effects caused by consuming too much sugar. Figure 27 shows that the most recognized effect of excess sugar was diabetes, selected by 72 percent of teachers, followed by cavities, selected by about half (52 percent), while fewer teachers recognized obesity (32 percent) or hyperactivity (13 percent) as negative effects. Only 17 percent of teachers mentioned at least three correct responses. These findings suggest a need to help teachers relate common health risks to high sugar consumption, assuring that nutrition education is tailored to the highest priority health risks faced in the implementation area. 94.4 4.2 1.4 0 20 40 60 80 100 Percentage of Teachers Agree Neutral Disagree My students need to learn about nutrition to prevent obesity NORC | LENS PROGRAM BASELINE REPORT | 59 Figure 27. Negative effects of sugar When asked to express their opinion on whether drinking water instead of sweet drinks or soda helps to prevent diabetes, 93 percent of teachers correctly marked “true,” as shown in Figure 28. Teachers’ high level of knowledge about the association between sugar drinks and diabetes suggest that providing information about this issue may not be useful for LENS. Rather, LENS could add value by helping school communities exchange sweet drinks for water as much as possible, recognizing that sweet drinks are a social norm/expectation/preference that will be challenging to change. Additionally, 96 percent of teachers responded that they frequently or sometimes teach about the risks of consuming a lot of sugar, as shown in Figure 28. Given the gaps in teachers’ knowledge about the risks of consuming too much sugar, this question reveals the need for more emphasis on providing teachers with correct information and useful strategies or tools to enhance their nutrition education about excessive sugar. 52.1 71.8 32.4 12.7 14.1 0 20 40 60 80 Percentage of Teachers Who Answered 'Yes' (%) Cavities Diabetes Obesity Hyperactivity Fatigue What are the negative effects of too much sugar? Incorrect Answer Correct Answer NORC | LENS PROGRAM BASELINE REPORT | 60 Figure 28. Teaching the risks of too much sugar and teachers’ awareness that drinking water instead of soda helps to prevent diabetes Teachers’ attitudes and practices on student nutrition and health In recognition of the relationship between students’ diets, health and learning, LENS is interested in engaging teachers in supporting student’s improved health and nutrition. Therefore, the survey included a series of questions to gauge teachers’ current attitudes and practices related to nutrition and health. These questions asked teachers whether they agreed, were neutral or in disagreement with a set of statements related to their role in teaching about health and nutrition in their classroom. Figure 29 summarizes responses to these questions. Sixty-four percent of teachers were in agreement with the statement “There is a problem with child undernutrition in my school.” Given the alarming evidence from the Western Highlands about the consumption of highly processed “junk” foods, such as salty or sugar snacks,45 teachers were asked about the importance of teaching students about junk food. This question relies on teachers being aware of the consequences of junk food and how widespread consumption is among their students, as well as seeing themselves as responsible for the issue. Eighty-four percent of teachers agreed with the statement, “It is important to educate my students about junk food,” suggesting that they see the relevance of addressing junk food consumption through their teaching activities. Nearly all interviewed teachers (96 percent) expressed agreement with the statement, “My role as an educator allows me to communicate messages about nutrition in my school and community.” Similarly, a high percentage of teachers (91 percent) expressed agreement with the statement, “When I teach my students about nutrition, I can help to improve their health,” suggesting that teachers see the importance that nutrition education can have on the health of their students. These responses suggest an opportunity for LENS to build on 45 SPRING. 2015. Formative Research: Key Influencers of Household Food Access in the Western Highlands of Guatemala. Arlington, VA: USAID/ Strengthening Partnerships, Results, and Innovations in Nutrition Globally (SPRING) Project. 93.0 95.7 0 20 40 60 80 100 Percentage of Teachers Agrees drinking water instead of sugary drinks helps prevent diabetes Frequently or sometimes teaches about the risks of too much sugar NORC | LENS PROGRAM BASELINE REPORT | 61 teachers’ perception that they can be a conduit for sharing nutrition messages at school and in the community. Figure 29. Percentage of teachers who agree with nutrition-related statements In addition to having an interest or openness about teaching students about nutrition, teachers require sufficient knowledge, skills and appropriate materials to facilitate nutrition education. To analyze how prepared and equipped teachers feel about teaching students about nutrition, the survey asked teachers whether they agreed, were neutral or in disagreement with a set of related statements. The results are summarized in Figure 30. Regarding teachers’ sense of their own capacity (self-efficacy) to teach about nutrition, 82 percent of teachers expressed agreement with the statement, “I feel trained to be able to teach my students about good nutrition practices”. This suggests that the majority of teachers perceive themselves to be capable of providing nutrition education, although other baseline findings point to specific knowledge gaps. When asked about the time required for nutrition education, 69 percent teachers expressed agreement with the statement, “I have sufficient time in my classes to teach the students about nutrition. Also, 76 percent of teachers expressed agreement with the statement, “Teachers are responsible for teaching children about nutrition” while nearly the same percentage (78 percent) of teachers expressed agreement with the statement, “Parents are responsible for teaching children about nutrition.” These responses suggest that teachers already see themselves as responsible for education about nutrition and have similar expectations from parents, offering an opportunity for activities to engage both sets of stakeholders. 64.3 84.3 95.8 91.3 0 20 40 60 80 100 Percentage of Teachers Who Agree (%) There is Undernutrition Problem in my School Important to Teach my Students About Junk Food My Role as Teacher Enables Me To Communicate Nutrition Messages in School/Community I Can Help Improve Health By Teaching Nutrition NORC | LENS PROGRAM BASELINE REPORT | 62 Figure 30. Teachers’ perception of their own capacity, responsibility, and time availability for nutrition education During KIIs, teachers were asked if they had received training on school nutrition. Several teachers had participated in trainings related to student nutrition led by Save the Children or other organizations. Teachers mentioned learning about various topics in these trainings, including the nutritional properties of different foods, portion sizes, the effects of malnutrition, and what foods children should eat, among other topics. Most teachers who had previously received training expressed interest in receiving additional training related to student nutrition. In contrast to the quantitative data, this may indicate that, despite having already received training, many teachers still do not feel fully equipped to teach their students about nutrition. Furthermore, many teachers did not possess any classroom materials they could use to teach students about nutrition. Other teachers described having books, posters, and other materials, such as visual displays of the food pyramid, that they used to teach students about nutrition. Findings from KIIs also indicate that teachers believe it is important that they teach students about nutrition, though only a few teachers commented on parents’ responsibility or capacity to teach children about nutrition. The teachers who did refer to parents implied that the school could impart knowledge on parents and the community, in addition to the students - at times implying that the parents lacked the capacity to teach their children about nutrition. One teacher explained that they try to “talk to the children about nutrition because the community does not have the means as the parents do not have money.” Another teacher commented that they desired additional training that would “expand well on what is malnutrition and train the mothers,” and a third wanted training that would “help with the families in this village.” These responses indicate that some teachers feel parents require additional support to be able to educate their children about nutrition. 81.7 68.6 76.1 79.7 0 20 40 60 80 Percentage of Teachers Who Agree (%) I am Trained To Teach Healthy Nutrition Practices Sufficient Class Time to Teach Nutrition Teachers Are Responsible For Teaching Nutrition Parents Are Responsible For Teaching Nutrition NORC | LENS PROGRAM BASELINE REPORT | 63 Given LENS’ dual focus on literacy and student nutrition/health, teachers were asked about whether they use techniques for teaching literacy through using names of foods. Figure 31 shows that virtually all (99 percent) teachers responded that they frequently or sometimes teach to read and write using the names of foods. Similarly, 91 percent of teachers responded that they frequently or sometimes carry out games to promote messages about eating. LENS may find opportunities to support this type of games, by helping teachers maximize the potential of these games, or improving their approaches or materials, where possible. As handwashing practices require water and soap to be consistently available to students, LENS is interested in assuring that teachers ask students to report unavailability of soap and water at the school handwashing places. Virtually all (97 percent) of teachers responded that they frequently or sometimes ask their students to report if there is no water or soap to wash their hands at school. As we discuss in more detail in the next section, in only 32 percent of all 262 observed schools, enumerators observed handwashing facilities with both water and soap. Figure 31. Teaching hygiene and nutrition practices School garden Only about a quarter of teachers (28 percent) said there is a garden at their school to cultivate vegetables and fruit for students’ consumption. Among the teachers whose schools have a garden, three quarters (80 percent) reported that their students had worked in the school garden in the previous two weeks, as Figure 32 shows. Figure 32 also shows that one third of teachers (who have a school garden) responded affirmatively that their students consumed something produced in the school garden during the previous month. 98.6 97.2 91.5 0 20 40 60 80 100 Percentage of Teachers Who Do This Frequently or Sometimes I Teach Reading/Writing With Names of Foods I Ask Students To Report When No Soap/Water For Handwashing I Do Games To Promote Nutrition Messages NORC | LENS PROGRAM BASELINE REPORT | 64 Figure 32. Teachers reporting that students worked in the garden and that students consumed garden produce (only schools with gardens) Of the 14 teachers whose students did not consume any food produced by the garden in the previous month, nearly all (12/14) mentioned unavailability of garden produce due to seasonality and other causes. This suggests that LENS could work to maximize schools’ use of their gardens, possibility through cultivation techniques or diversifying production to extend the seasonal availability of crops. Also, it would be valuable to ensure students’ preferences are taken into account when selecting crops to cultivate, and/or using taste tests or games to encourage trying new foods. In KIIs, several teachers mentioned their school had a garden. One teacher commented, “We have a school garden, students plant it, work in it and cultivate it, and parents as well.” Another teacher recommended installing a garden at his or her school as a way to improve the quality of the food served at the school. School Observations We conducted school observations in 262 schools, 97 percent of the 270 target schools.46 Eight schools denied access to enumerators, as explained in subsection 4.4. The characteristics that were recorded can be grouped in three categories: cooking facilities and cooking practices, including food storage; hygiene, sanitation and water; and other school characteristics. Below we discuss the main findings. Cooking facilities and cooking practices 46 As explained before, four schools were dropped because we could not find them in the Ministry of Education data sets. 80.0 35.0 0 20 40 60 80 Percentage of Teachers (%) Students Worked in Garden in Last Two Weeks Students Ate Something From Garden in Last Month NORC | LENS PROGRAM BASELINE REPORT | 65 Almost all schools have a cooking space (259 out of the 262). Table 16 shows the main characteristics of these school kitchens. Ninety-two percent of the kitchens have a stove; of these, 61 percent use a regular firewood stove, and the rest use an improved firewood stove (ahorradora de leña). Eighty-four percent of the stoves have a chimney to mitigate indoor air pollution by drawing cooking smoke out of the kitchen. Over two thirds of kitchens have a sink and a similar percentage had either running or stored water in the kitchen at the time of the observation. However, only 43 percent of kitchens have soap for kitchen staff to wash their hands. Thirty-nine percent have a trash can, and 46 percent have the weekly menu on display. Table 16. School kitchen characteristics (N = 259) Characteristic Fraction Kitchen has a stove(a) 0.92 Firewood 0.61 Improved firewood (Ahorrador de leña) 0.38 Stove has a chimney 0.84 Kitchen has a sink 0.68 Presence of running or stored water 0.72 Presence of soap for hand washing 0.43 Presence of a trash can 0.39 Kitchen looks clean 0.65 A weekly menu is on display 0.46 In 259 schools enumerators were able to observe the food preparation process.47 Table 17 shows results for recorded practices. For some practices the number of observations is lower than 259 because it was not possible to observe all the practices in all schools. For example, enumerators were supposed to observe whether utensils were washed before starting food preparation, but were in time to observe this only in 123 schools. Out of these 123 schools, in 92 percent it was observed that utensils were properly washed.. In the majority of kitchens the stored water observed (if any) is visibly clean, and utensils and tables are visibly clean. However, in terms of staff attire, about half do not wear an apron and only 26 percent wear a hairnet or handkerchief over their hair. Table 17. Food preparation practices Characteristic Fraction N Food is kept covered 0.68 170 Utensils were properly washed 0.92 123 Stored water is visibly clean 0.83 175 Working tables are visibly clean 0.71 192 Dress code-kitchen staff Apron 0.49 195 Hairnet or handkerchief 0.26 191 47 Because food is usually cooked in the morning, enumerators were not able to observe the cooking process in all the visits. NORC | LENS PROGRAM BASELINE REPORT | 66 During the cooking process observation, enumerators asked one of the staff in each kitchen a few questions about food preparation and access to the kitchen. Table 18 summarizes the results of this short interview. In many schools parents, teachers and students/other children have access to the kitchen, in addition to the designated food preparation staff. It is possible that some parents help with cooking and/or bring ingredients, which is why they would enter the kitchen and may bring other children. Teachers may have oversight duties, which would explain their presence. Students or teachers may be in charge of taking the meals for distribution, which would explain their access to the kitchen. Enumerators also asked when kitchen staff wash their hands and did not read potential responses; 73 percent said before starting food preparation, 38 percent said after manipulating raw ingredients (e.g. raw chicken) and 24 percent after going to the bathroom. The expectation was that all respondents would answer that they wash their hands at key critical moments, such as before starting food preparation, after touching raw ingredients and after using the bathroom. Table 18. Kitchen staff interview (N=259) Characteristic Fraction Who has access to the kitchen? Parents 0.36 Teachers 0.61 Students/children of volunteers 0.29 Other 0.03 When do you wash your hands? Before cooking 0.73 After manipulating raw ingredients 0.38 After using the bathroom 0.24 Other 0.05 After the kitchen observation, enumerators asked to observe where food is stored. There is a food storage space in 251 of 262 observed schools (96 percent). Figure 33 shows that the majority of schools (77 percent) have a room to store food. Figure 33. Food storage space No food storage space\ 4% Cabinet 11% Room 77% Other 8% NORC | LENS PROGRAM BASELINE REPORT | 67 Enumerators recorded whether storage spaces are clean and well-equipped. Table 19 shows that the majority of storage spaces (60 percent) look clean and organized. In 80 percent of the storage spaces food is kept off of the ground, and there is measurement equipment in 91 percent of the food storage spaces. However, work still needs to be done to manage the storage space. Only half of the storage spaces had a daily stock control sheet, and in only 35 percent of cases was there a portioning table on display. Less than half of storage spaces had the Save the Children logo visible and only in 17 percent there was a checklist for visits by Save the Children technical staff. Table 19. Food storage characteristics (N=251) Characteristic Fraction Looks clean and organized 0.60 Food is stored off of the ground 0.80 There is a daily stock control sheet 0.50 Food measurement equipment 0.91 Portion table on display 0.35 Save the Children logo is visible 0.43 Save the Children visit control checklist 0.17 Iodine deficiency is a primary cause of reduced cognitive capacity and salt iodization is the primary prevention strategy.48 Therefore, enumerators recorded whether the salt used by the school was labeled as iodized. Although it was not possible to observe the label in 36 percent of the cases, when it was visible in almost all cases the salt was labeled as iodized, as Figure 34 shows. Figure 34. Iodized salt Water, sanitation and hygiene Ninety-eight percent of schools have sanitary facilities. The internationally accepted definition of improved sanitation facilities in a school setting includes flush or pour-flush toilets, pit latrines with slab, and composting toilets, all of which separate excreta from human contact.49 The majority of schools have 48 WHO webpage. Micronutrient Deficiencies. (n.d). Retrieved from https://www.who.int/nutrition/topics/idd/en/. 49 Improved” facilities in school settings include: flush/pour-flush toilets, pit latrines with slab, and composting toilets. “Unimproved” facilities include: pit latrines without slab, hanging latrines, and bucket latrines, or any other facility where human excreta is not separated from human contact. WHO/UNICEF. 2016. Core questions and indicators for monitoring WASH in Schools in the Sustainable Development Goals (Version June 2018). WHO: Geneva. Label on salt says is Iodized 60% Label does not says is Iodized 4% Not observable 36% NORC | LENS PROGRAM BASELINE REPORT | 68 a toilet or an improved latrine. More specifically, as Figure 35 shows, 42 percent of schools have flush toilets, 34 percent have manual pour-flush toilets, 12 percent have pit latrines with slab and two percent have compost latrines. Therefore, 90 percent of observed school sanitary facilities meet the definition for improved sanitation facilities (sanitation indicator 1), as part of monitoring countries’ progress toward the Sustainable Development Goals (SGD).50 Among the schools with unimproved latrines, the majority have pit latrines without slab, and the rest have other types of latrines. Figure 35. Sanitary services Table 21 shows the main characteristics of school bathrooms. Using the UNICEF/WHO criteria for SGD sanitation indicator 2 that defines usable toilets as available to students, functioning, and private,51 enumerators found that schools have, on average, 3.7 usable toilets/latrines. This is equivalent to roughly 46 students per toilet/latrine. In 84 percent of schools’ bathrooms are separated by students’ sex, which is SGD sanitation indicator 3. For MINEDUC the standard for sanitary facilities is that schools should have 1 toilet/latrine for every 25 girls, and 1 toilet/latrine and 1 urinal for every 50 boys. Unfortunately the survey did not record the number of urinals in each school, but we can estimate the fraction of schools meeting the MINEDUC’s standard. Assuming schools follow the guidelines for the distribution between girls toilets, boys toilets and urinals, we estimate 43 percent of schools meet the standard for boys, 48 percent meet the standard for girls and 41 percent meet the standard for both.52 50 UNICEF and WHO. 2016. Core questions and indicators for monitoring WASH in Schools in the Sustainable Development Goals (Version June 2018). WHO: Geneva. 51 “Usable refers to toilets/latrines which are (1) available to students (doors are unlocked or a key is available at all times), (2) functional (the toilet is not broken, the toilet hole is not blocked, and water is available for flush/pour￾flush toilets), and (3) private (there are closable doors that lock from the inside and no large gaps in the structure) at the time of the questionnaire or survey. If any of these three criteria are not met, the toilet/latrine should not be counted as usable.” WHO/UNICEF. 2016. Core questions and indicators for monitoring WASH in Schools in the Sustainable Development Goals (Version June 2018). WHO: Geneva. 52 By assuming schools follow the guidelines for the distribution between girls toilets, boys toilets and urinals, we are only saying that if there are 3 toilets/latrines in a school, we assume 2 are for girls, 1 is for boys, and there is one urinal. Note that we are not assuming schools meet the MINEDUC standard for the number of sanitary facilities, just that they distribute the number of toilets we observed they have, according to MINEDUC’s guidelines. Flush toilet 42% Pour-flush toilet 34% Compost latrine 2% Pit latrine with slab 12% Pit 8% Hangining latrine 1% Bucket 0.4% Not improved 10% NORC | LENS PROGRAM BASELINE REPORT | 69 The data also indicates that bathroom management and equipment can be improved. Only half were found to be clean, according to the SGD sanitation indicator SX5 for evaluating odor, pests and visible fecal matter.53 There were trash cans in 69 percent of the bathrooms, and only 13 percent of the bathrooms had checklist for daily verification of maintenance. Table 20. Toilets/latrines characteristics (N=258) Characteristic Number/fraction Average number of toilets/latrines per school 4.49 Average number of usable toilets/latrines per school (a) 3.75 Separated for boys and girls (fraction) 0.84 Bathrooms are clean (fraction) 0.51 Bathrooms have trash cans (fraction) 0.69 Bathrooms have a checklist for daily verification of maintenance (fraction) 0.13 (a) See footnote 51 Handwashing facilities In 245 schools (94 percent) students have handwashing facilities; 237 of these schools (90 percent of all 262 observed schools) have handwashing facilities that comply with the UNICEF/WHO standard definition for SGD hygiene indicator 1.54 Figure 36 shows the different types of facilities available. In almost half the schools, students have a sink with faucet to wash their hands. In 37 percent of schools students use a water tank (pila) with faucet. In 87 percent of schools where there are handwashing facilities enumerators found water available in these facilities, but found soap only in 36 percent. The SGD hygiene indicator 2 specifies that handwashing facilities should have both water and soap. Only in 35 percent of schools with handwashing facilities, or 32 percent of all 262 observed schools, this criterion is met. 55 53 “Clean: all toilets do not have a strong smell or significant numbers of flies or mosquitos, and there is no visible feces on the floor, walls, seat (or pan) or around the facility. Somewhat clean: there is some smell and/or some sign of fecal matter in some of the toilets. Not clean: there is a strong smell and/or presence of fecal matter in most toilets. WHO/UNICEF. 2016. Core questions and indicators for monitoring WASH in Schools in the Sustainable Development Goals (Version June 2018). WHO: Geneva. 54 “A handwashing facility is any device or infrastructure that enables students to wash their hands effectively using running water, such as a sink with tap, water tank with tap, bucket with tap, tippy tap, or other similar device. Note: a shared bucket used for dipping hands is not considered an effective handwashing facility,” UNICEF and WHO. 2016. Core questions and indicators for monitoring WASH in Schools in the Sustainable Development Goals (Version June 2018). WHO: Geneva. 55 “To be considered available, water and soap must be available at one or more of the handwashing facilities at the time of the survey or questionnaire. If girls and boys have separate facilities, soap and water should be at both. Soapy water (a prepared solution of detergent suspended in water) can be considered as an alternative for soap, but not for water, as non-soapy water is needed for rinsing. Surveys may choose to add other response categories for ash or alcohol hand rub, but these should be kept as separate categories from soap to support SDG monitoring.” UNICEF and WHO. 2016. Core questions and indicators for monitoring WASH in Schools in the Sustainable Development Goals (Version June 2018). WHO: Geneva. NORC | LENS PROGRAM BASELINE REPORT | 70 Figure 36. Characteristics of handwashing facilities (fraction, multiple responses allowed) (N=245) Access to drinking water Water for student consumption is available in only 173 schools (66 percent). Figure 37 shows the distribution of the different types of water sources for these 173 schools. The great majority of schools, 76 percent, have piped water. Other much less prevalent sources include rainwater (6 percent), protected well or spring (11 percent), and unprotected well or spring (4 percent). According to the WHO/UNICEF Joint Monitoring Programme, an “improved” drinking water source is protected from contamination, particularly fecal matter from piped or protected sources.56 Therefore, 93 percent of schools have water for students’ consumption, and 61 percent of all 262 observed schools, have an improved water source, (SGD water indicator 1). 56 Improved” water sources in school settings include: piped, protected well/spring (including boreholes/tubewells, protected dug wells and protected springs), rainwater catchment, and packaged or delivered water. “Unimproved” sources include: unprotected well/ spring, and surface water (e.g. lake, river, stream, pond). UNICEF and WHO. 2016. Core questions and indicators for monitoring WASH in Schools in the Sustainable Development Goals (Version June 2018). WHO: Geneva. 0.48 0.37 0.03 0.14 0.01 0.04 0.05 0.00 0.10 0.20 0.30 0.40 0.50 0.60 Sink with faucet Water tank with faucet Bucket with faucet Water tank with bucket Other NORC | LENS PROGRAM BASELINE REPORT | 71 Figure 37. Water sources (%) (N=173) Finally, 132 schools reported treating water for students’ consumption, representing 76 percent of schools that provide water to students, and 50 of all surveyed schools (SGD water indicator XW6a57). The most popular method of water treatment are clay filters (65 percent), followed by boiling water (38 percent), and adding chlorine (12 percent), as SGD water indicator XW6b58. Note that some schools use more than one method. In 144 schools, there was water available from the primary source during the school observation, equivalent to 83 percent of the schools that have water for students’ consumption, and 55 percent of all 262 schools. 57 SGD water indicator XW6a is based on the survey question “Does the school do anything to the water from the main source to make it safe to drink?” which was modified for the LENS survey to “Is the water treated to make it safe to drink?” (¿Se trata el agua para que sea segura para consumir?) 58 SGD water indicator XW6b is based on the survey question, “If yes, what treatment method is used?” (Que medida utiliza?) 5.8 75.8 11.0 4.0 3.5 0 10 20 30 40 50 60 70 80 Rain water Piped water Protected well or spring Unprotected well or spring Other NORC | LENS PROGRAM BASELINE REPORT | 72 Figure 38. Water treatment methods (%) (N=132) School meal program and other school characteristics At the end of the school observation enumerators asked the head teacher a few questions about school management, the school garden and the school meals program. Table 21 summarizes their responses. In most schools (97 percent) there is a school meals program, and in almost all cases head teachers reported that it is funded by MINEDUC. In the majority of cases, schools provide a small meal and a drink (67 percent) or a full meal (39 percent). Note that fractions for the types of meals add up to more than one. This could be because feeding programs could provide different meals over the year. Ninety-seven percent of head teachers reported that students contribute to the school feeding program, such as contributions of food or firewood. Notably, only 31 percent of schools have a garden; since gardens can be turned into assets for students’ nutrition, efforts should be made so more schools grow gardens. Given LENS’ interest in promoting child-to-child participatory actions to promote nutrition, hygiene and health practices, questions were included on whether the school uses a school hygiene traffic light checklist and has a student government. The traffic light checklist is a tool used by students to monitor the cleanliness of different areas of the school and to prompt actions to improve these areas. The majority of head teachers (87 percent) know what the traffic light is; however, among these schools, only 31 percent use a traffic light. Eighty-seven percent of schools have a student government. Regarding reading resources, 70 percent of schools have a library or reading room, and of these, 81 percent have more than 50 books. Table 21. Other school characteristics (N=262) Characteristic Fraction School has a school meals program 0.97 What meals are provided? A drink only (e.g. atol) 0.06 A small meal and a drink 0.67 0.65 0.38 0.12 0.12 0 0 0 0 0 1 1 1 Clay ecofilter Boil water Chlorine Other NORC | LENS PROGRAM BASELINE REPORT | 73 Characteristic Fraction A small meal 0.08 Full meal 0.39 Take-home meals 0.00 Other 0.01 Students contribute to the school meals program 0.97 School has a garden 0.31 School has a student government 0.87 Head teacher knows what the school hygiene "traffic light" checklist is (a) 0.87 School has a school hygiene "traffic light" checklist 0.31 School has a library 0.70 (a) The first surveys were conducted under the assumption that head teachers were familiar with the cleaning traffic light, so the originally survey did not ask if they knew what the traffic light was. During fieldwork enumerators realized that this was not the case, and that a non-negligible fraction of head teachers did not know what the traffic light was. Therefore, during fieldwork we included a question asking this. In the end, only 94 percent of head teachers were asked if they knew what the traffic light was. Local producers As part of the baseline evaluation, NORC’s data collection team interviewed 10 local producers to better understand their current situation as producers, as well as their capacity to provide food to schools for school feeding programs in the near future. To begin, producers were asked whether they were familiar with the LENS program. Two of the ten producers interviewed were aware of the program. One of the two producers familiar with the program said he/she was invited to a meeting about LENS because he/she was the president of the education advisory board. During that meeting they discussed LENS, the municipal menus, and whether or not there were local producers who provided food to the schools. Following this, the enumerators gave producers a brief introduction to the LENS program and asked them to describe some of the possible benefits of the program. Producers identified two distinct types of possible benefits of the LENS program. The first is a benefit to the children and their learning through access to school meals. In the words of one producer, “The benefit of better nutrition will be seen through the children’s increased academic performance and then they are more intelligent.” Aside from benefits to children, some producers identified possible economic benefits to themselves by having secure buyers for their products and reduced transportation costs as the producer would stay within the community. As part of the interview, each producer was asked details about their five most important crops. Table 22 summarize producers’ responses. The most popular crops are corn and beans, grown by five producers. Next are potatoes, tomatoes, carrots, chili peppers, and onions. All these products are ingredients in the menus proposed by the Ministry of Education for the school feeding program, 59 so they could easily be incorporated into these programs. To approximate how much producers could sell to schools, Table 22 shows average yearly production, after own consumption. 59 Menús para el programa de alimentación escolar 2019. Ministerio de Educación. NORC | LENS PROGRAM BASELINE REPORT | 74 Table 22. Harvest after own consumption, selected products (a) Crop type # Producers Average harvest after own consumption Crop harvest (unit) Bean 5 6 Quintals Corn 5 10 Quintals Potato 4 90 Quintals Tomato 3 192 Quintals Carrot 2 263 Units Chili pepper 2 5243 Units Onion 2 89 Quintals (a) Only results for products that were mentioned by two or more producers are included. Products mentioned only once include beets, cucumber, and cole, among others. Respondents were also asked about some of the challenges they face as agricultural producers. Reported challenges include: access to water, storage, transportation, sale prices, product quality, availability of resources, and lack of knowledge. One of the most discussed challenges was access to water. One producer framed the issue in terms of both not having access to water itself, and not having access to irrigation systems to pump water when it is available. The producer said, “During the summer, water is scarce and it’s not enough for watering, and it costs us to take water. Likewise, we need an irrigation system, because even if the water is available, if there isn’t an irrigation system then it’s not functional. ” Other frequently mentioned issues revolved around transportation and storage of products. Two producers mentioned that the quality of the products decrease when they are transported to the market. Finally, we also asked producers about their experience with projects similar to LENS, and selling food to schools in general. Of the 10 producers interviewed, three reported having previous experience selling their products to schools. One of these producers reported having sold beans to a school on only one occasion when there was a shortage, while the other two said they had regular relationships selling to schools. The two producers who have regular relationships selling their products to schools each mentioned previously being affiliated with an institution that helped connect them to schools. One producer specifically mentioned Save the Children, saying: “It was Save the Children personnel that contacted me to distribute some products. I think it was IDEA that Save the Children was working on with nutrition in schools, and based on that, they approached me about buying tomatoes. They bought from me and that’s how the relationship formed. There was nothing, no agreement but it was more for the quality of the product. Since they bought from me on that one occasion, and that’s how I connected with the schools.” Interestingly, these are also two of the three producers who report being registered with the Superintendent of Tax Administration (Superintendencia de Administración Tributaria – SAT). One of these three producers reported that it was easy to register with the SAT, while another mentioned it happened by chance while he was selling broccoli to a company as part of producer group. Of the seven producers who reported not being registered with the SAT, four said it was not necessary for their level of production. One of these producers told the interviewer, “We don’t register because we sell in local markets, we register if our products are sold in other places.” Two of the remaining producers mentioned they were not registered with the SAT because of the fees associated, one saying “I’m not familiar with the SAT and I’m scared to go register because it may cost a lot of money.” None of the 10 interviewed producers, including those who have previously sold food to schools, reported having the new MAGA accreditation that will be required to sell food to schools. When asked if they NORC | LENS PROGRAM BASELINE REPORT | 75 were familiar with the accreditation, the majority of the producers responded in the negative. One producer responded that they do not have the accreditation because the producers don’t know about it or what they have to do to obtain it. He went on to say, “It’s a mess, the government has not paid attention to [regulations]. If we are talking about MAGA, we don’t know where their offices are.” Another producer simply said, “I am unfamiliar with the certification, and as such I don’t have it.” There are a couple of reasons to think that local producers are in a good position to provide food to schools’ feeding programs. First, producers recognize potential economic benefits that could come from these programs, in terms of income stability and reduced transportations costs. Second, their products can be introduced in the school menus. That being said, it is important to highlight that it was challenging to recruit producers for the interviews, as explained in section 4.4. If this reflects an overall scarcity of local producers in the area, then the findings presented in this section should be taken with a grain of salt. For producers, selling to schools nearby would alleviate some of the challenges associated with transportation and storage of their products. However, based on the experiences of the two producers who currently provide food to schools, a stronger effort should be made by LENS and/or the Government of Guatemala to advertise the permits and accreditations needed to be a school meal provider. 6.4 Conclusions For the program evaluation the main findings are summarized as follows: Students’ background and reading skills • Students are 9.6 years old, on average. About four fifths of the sample attended preschool, and half the students have repeated or are repeating a year. • Roughly 90 percent of students ate something at school the day before data collection. During focus group discussions parents reported they aren’t sure whether their child needs to eat before school or not because schools sometimes offer breakfast and sometimes a snack (refacción). • Parents suggested menu diversification as a way to improve school meals. As opposed to the student responses, the majority of the focus groups, 8 of 10, participants stated that their children are not satisfied with the school meals. The two main reasons for this perception were (1) repetition of meals and (2) quality of the cooked food. • Students in Ixil municipalities come from more disadvantaged backgrounds than students in K’iche’ municipalities. In effect, students in Ixil municipalities are more likely to have repeated a year than students in K’iche’ municipalities, and mothers in K’iche’ municipalities are more likely to be literate than in Ixil municipalities. Moreover, households in K’iche’ municipalities are more likely to have electricity and have more household assets, than households in the Ixil municipalities. • There are major differences in reading skills between municipalities. For example, while students in Cotzal read, one average, 60.5 cwpm, in Chajul this figure is only 38 cwpm. • The data shows that students that answer 9-11 questions correctly read, on average, 77 words per minute. This could be used by Save the Children as benchmark to set program targets. • Bivariate regression analyses show that speaking Spanish at home, mother’s literacy and number of household assets, are positively correlated with reading skills. By contrast, age and having repeated a grade are negatively correlated with reading skills. NORC | LENS PROGRAM BASELINE REPORT | 76 Teacher surveys and interviews • Third grade teachers have relatively small classrooms, with an average of 18 students per classroom. • Teachers’ absenteeism rate is 18 percent, according to teachers’ own reports. • While most teachers use Spanish in class, local languages are also prevalent. In the three K’iche’ municipalities (Cunén, Sacapulas and Uspantán) 94 percent of the teachers use Spanish, 52 percent use K’iche’, six percent use Uspanteco and 16 percent use Sacapulteco. In Ixil municipalities (Chajul, Cotzal and Nebaj), 93 percent of the teachers use Spanish, 73 percent use Ixil and five percent use K’iche’. The use of local languages in class is not surprising as teachers in Guatemala are required to use local languages in early grades. • The majority of teachers (70 percent) reported having seen the Taza Escolar image prior to the survey, although 40 percent could not identify an example of a nutritionally-balanced meal. • There are important gaps in teachers’ knowledge on what types of foods help children grow, and give them energy. Fifty-eight percent incorrectly selected leafy greens, vegetables and fruit as types of food that help children grow. Also, only 18 percent of teachers correctly selected cereals, grains and tubers, as sources of energy. • The majority of teachers seem knowledgeable about which types of food protect children from diseases. Sixty-four percent correctly selected leafy greens, vegetables and fruit as foods that protect children from diseases, and the most common incorrect response was milk, meat, eggs and other animal source foods, selected by 23 percent. • Regarding school hygiene, a high percentage of teachers (87 percent) mentioned the importance of washing hands before eating, but only half mentioned the need to wash hands after using the bathroom. Similarly, nearly all teachers (94 percent) said their students washed their hands before eating on the previous day, but less than half mentioned that hand washing occurred after going to the bathroom, indicating a gap area in optimal practices. About a third of teachers mentioned that their students brushed their teeth at school the day before data collection. • In terms of awareness of micronutrients, teachers indicated their awareness of the seriousness of anemia (93 percent) and its symptoms (94 percent), but only a quarter could identify two or more foods that prevent anemia. Very few teachers (nine percent) knew why consuming iodized salt is important. Most teachers (89 percent) recognized Incaparina as a source of calcium and three￾quarters agreed that staying hydrated by drinking water can improve students’ attention in class. • Although almost all teachers said they teach students about the negative effects of too much sugar, there are gaps in teachers’ understanding about the negative impacts sugar consumption can have. While 70 percent see the association with diabetes, only a third recognize the relationship with obesity, and about half (52 percent) recognize sugar as a cause of cavities. • A high percentage of teachers agreed that students need nutrition education to prevent obesity (94 percent), and 84 percent recognize the importance of teaching children about the effects of junk food. Almost all teachers (96 percent) recognize they have a role as communicators of nutrition￾related messages in their communities, 82 percent feel trained to teacher about nutrition, and 69 percent say they have adequate time in class to teach about nutrition. NORC | LENS PROGRAM BASELINE REPORT | 77 • Only 28 percent of teachers reported that their schools had a garden. However, among schools with a garden, 80 percent of teachers said their students had worked in the garden in the last two weeks, and about a third of teachers (35 percent) said students consumed produce from the garden during the previous month. School observations • Most schools have a kitchen. Ninety-two percent of the kitchens have a stove; 68 percent have a sink and 72 percent had either running or stored water in the kitchen at the time of the observation. Only 43 percent of kitchens have soap for kitchen staff to wash their hands. Thirty￾nine percent have a trash can, and 46 percent have the weekly menu on display. • In the majority of kitchens the stored water observed (if any) is visibly clean, and utensils and tables are visibly clean. In terms of staff attire, about half do not wear an apron and only 26 percent wear a hairnet or handkerchief over their hair. • In 96 percent of the 262 observed schools there is a food storage space. Food is kept off of the ground in 80 percent of the storage spaces. Ninety-one percent have measurement equipment, 50 percent have a daily stock control sheet, 35 percent have a portioning table on display, 43 percent visibly display the Save the Children logo, and 17 percent have a checklist for visits by Save the Children technical staff. • Enumerators were able to observe the salt package in only 64 percent of the schools. Among these schools where the salt package was seen by enumerators, 94 percent of the salt was confirmed to be iodized. • Eighty-eight percent of schools have improved sanitation facilities. These include flush toilets, manual pour-flush toilets, pit latrines with slab and compost latrines. Among the schools with unimproved latrines, the majority have pit latrines without slab, and the rest have other types of latrines. • We estimate that 41 percent of schools meet MINEDUC’s standard for number of sanitation facilities (1 toilet/latrine for every 25 girls, and 1 toilet/latrine and 1 urinal for every 50 boys). • In 245 schools (94 percent) students have handwashing facilities. Only in 35 percent of schools with handwashing facilities, or 32 percent of all 262 observed schools, are there both water and soap for students to wash their hands. • Water for student consumption is available in only 173 schools (66 percent). Ninety-three percent of these schools, or 61 percent of all 262 observed schools, have an improved water source, such as piped water, rainwater, and protected well or spring. • In 97 percent of schools, school administrators reported that there is a school meals program, funded in most cases by MINEDUC. The majority of schools provide a small meal and a drink (67 percent), based example of a tamale and a hot drink for that option. Less than half of school administrators reported that their school currently provides a full meal (39 percent). • Seventy percent of schools have a library or reading room. Local producers • Producers identified possible economic benefits of having secure buyers for their products, including reduced transportation and storage costs. In addition, their products are commonly NORC | LENS PROGRAM BASELINE REPORT | 78 found as ingredients in the official school menus. This suggests that producers are in a good position to provide food to schools’ feeding programs. • It is important to highlight that it was challenging to recruit producers for the interviews. If this reflects an overall scarcity of local producers in the area, then LENS may find it more challenging to recruit local producers. • Producers also mentioned that the program are well positioned to improve students’ nutrition. • One of the most discussed challenges faced by local producers is access to water. This includes not having access to water itself, and not having access to irrigation systems to pump water when it is available. Other challenges mentioned include storage, transportation, sale prices, availability of resources, and lack of technical knowledge. • Three producers reported being registered with the Superintendent of Tax Administration (Superintendencia de Administración Tributaria – SAT). These producers did not mention that it was particularly difficult to register. However, two of the remaining producers mentioned they were not registered with the SAT because of the associated fees. • None of the 10 interviewed producers, including those who have previously sold food to schools, reported having the new MAGA accreditation that will be required to sell food to schools. 7. Programmatic recommendations The baseline findings revealed the following areas of opportunity for the LENS project: 1. Promote student feedback on menus and preparation. During FGD parents stated that their children are not satisfied with the school meals, primarily due to (1) repetition of meals and (2) inadequate quality of the cooked food (e.g. undercooking). Although LENS’ donated commodities are restricted to rice, oil and black beans, the project can promote additional local foods to assure a variety of dishes and also inform the school community about the particular benefits of the donated foods. LENS can also help schools collect and use feedback from children to improve recipes and/or meal preparation. 2. Support school communication on school meals. Parents reported they aren’t sure whether their child needs to eat before school or not because schools sometimes offer breakfast and sometimes a snack (refacción). This suggests that parents should be better informed about which meals children are going to receive during the school day. 3. Build on teachers’ interest in nutrition education and knowledge gaps. Findings from the teacher surveys indicate that nearly all teachers recognize the value of providing nutrition education to their students and view themselves, as well as parents, as having the responsibility to teach children about nutrition to improve their health. However, there were important gaps in teachers’ knowledge about key topics, such as anemia, obesity, diabetes and the foods to address these. LENS should find opportunities to capitalize on teachers’ base of interest, providing training on priority topics that teachers and students consider most relevant and which pose the greatest threats to student health. 4. Facilitate active learning about nutrition and health. Given that many teachers already mention engaging in some nutrition-related games with their students, LENS can help teachers maximize the potential of these games, by varying their approaches or materials, and assuring that nutrition education is tied to dynamic and practical approaches. NORC | LENS PROGRAM BASELINE REPORT | 79 5. Promote realistic, local solutions to nutrition challenges in Quiché. In tandem with standard nutrition messages of the Taza Escolar, it is essential that LENS promote locally-available, lower-cost nutrient-rich foods that are feasible for families, given documented income and access constraints faced by families in Quiché. In particular, LENS can increase understanding about the value of lower cost animal source foods and fortified foods, including the iron-fortified products provided by LENS. 6. Focus on gaps in school meals systems. Although most schools have kitchens and food storage spaces, there are multiple opportunities for LENS to improve the use of existing infrastructure, address gaps or strengthen processes. For example, the majority of school kitchens lacked a trash can, and soap was not present for school cooks to wash their hands. These findings suggest that, while most schools have already basic infrastructure in place, LENS could be instrumental in supporting schools to improve conditions. 7. Examine and address priority water, sanitation and hygiene constraints. Given that only a third of schools had both soap and water present for students’ handwashing, there is a need for LENS to address the factors that impede availability of water and (especially) soap, and help schools generate feasible, long-term solutions (including improved student monitoring). Regarding hygiene practices, handwashing before eating was better understood and practiced in schools, suggesting a need for LENS to emphasize handwashing after using the bathroom (such as process for students to mark whether water and soap were present at the handwashing place, each time they visit the bathroom). Other WASH challenges which require specific emphasis include lack of drinking water in a third of schools and a need for improved cleanliness/monitoring of school bathrooms. 8. Support optimal use of gardens. Only about a third of of schools have a garden, but schools with gardens report high use of students’ participation in gardening, although a lower number have recently consumed garden produce. This is an opportunity for LENS to support establishment of additional gardens and use of existing gardens for practical nutrition education. 9. Provide information and support to local producers to mitigate constraints. It was challenging to recruit local producers for the baseline interviews. LENS’ recruiting activities should elaborate on the benefits of program participation, as well as the potential risks and how they could be addressed. In addition, very few of the interviewed producers were registered with the SAT, and most did not even know about the accreditation MAGA requires to participate in schools feeding program. Efforts should be made by LENS and/or the Government of Guatemala to advertise the permits needed to be a school meal provider. 8. Limitations and recommendations for the evaluation All LENS schools were part of the IDEA program (2014-2018) so right from the start we know that treatment and comparison schools are different in some fundamental ways. Not only this affects comparability of treatment and comparison schools, but it can also affect some of the results we get in this baseline study. For example, students may have greater expectations about the size (and quality) of meals a school feeding program should provide, than their counterparts in comparison schools. Regarding comparability, the fact that we are using DID models will allow us to isolate any differences at baseline between treatment and comparison schools. In other words, we do not need to assume that schools are observationally equivalent at baseline for the impact evaluation, only that the treatment schools would observed the same change in the outcomes of interest than the comparison n schools, if the treatment schools did not receive treatment. Program substitution could affect the validity of the impact evaluation results. If other organizations are offering services similar to LENS in comparison schools, it would be ideal to collect data to at least control for this substitution. To do this we would need data on what other programs are being fielded in NORC | LENS PROGRAM BASELINE REPORT | 80 the comparison schools. In comparison schools only students are being surveyed, so we could ask them questions about feeding and literacy programs that are being fielded in their schools; however, it would probably be preferable to survey teachers to collect this type of data. SC should consider have the evaluation team survey teachers in comparison (and treatment) schools at midline and endline, so data on alternative school feeding programs and early reading interventions can be recorded. There are notable differences between treatment and comparison groups in terms of language spoken at home as there are no students that speak Ixil at home in the comparison group. However, for the impact evaluation, we will be able to offset these differences using the baseline data, in the context of the DID model. Student attendance is one of the key outcomes of the evaluation. The survey collected student attendance for the week previous to data collection using teachers’ attendance records, and the results indicate that attendance is quite high. These results could be an overestimation of the true attendance rates, because we are only collecting data for the students present the day of data collection. It is also possible that teachers tend to overestimate student attendance. As SC will start collecting attendance data more systematically as part of their monitoring activities, the evaluation team will be able to incorporate these data in the analysis.60 The school observations were programmed to be conducted at baseline only. However, it would be convenient that the evaluation team also conducts school observations at midline and endline, to record and analyze progress. SC should consider have the evaluation team conduct school observations at midline and endline, at least on the 72 schools that will be visited for the other activities. While KIIs interviews with teachers provided valuable insights on teachers’ views and practices regarding nutrition and hygiene, with fewer interviews the same information could probably be acquired. Previous research shows that 12 interviews are enough to discover between 80 and 90 percent of the themes.61 Along these lines, Save the Children should consider, for the midline and endline, reducing the number of KIIs not only with teachers, but with principals, PTA members, and government officials. Savings from these reductions in KIIs could help cover the costs of surveying teachers in impact evaluation schools and/or conducting school observations at midline and endline, as proposed above. Finally, the fact that it was not easy to recruit local producers for the interview implies that the documented results may not replicate if other local producers are interviewed. LENS’s efforts to recruit local producers for the program will intensify over the next two years, in preparation for the training activities that will be fielded in years 4 and 5 of the program. For the midline data collection, it would be ideal that the evaluation team has access to the pool of local producers recruited for the program, so a more representative sample of local producers can be interviewed. 60 In addition, once the Ministry of Education produces enrollment data for 2019, we will be able to calculate attendance rates for each of the classes where s, as the survey recorded the number of students present the day of data collection. 61 Guest, Greg, Emily Namey and Kevin McKenna (2017). How Many Focus Groups Are Enough? Building an Evidence Base for Nonprobability Sample Sizes. Field Methods 2017, Vol. 29(1) 3-22. NORC | LENS PROGRAM BASELINE REPORT | 81 Annex I. Instruments ESTUDIANTES62 CUESTIONARIO LENS Asistencia Enumerador, seleccione el número de días que el estudiante asistió a la escuela la semana pasada. 0 1 2 3 4 5 No sabe o no recuerda Refacción Hoy ¿comiste algo antes de venir a la escuela? Sí No No sabe o no recuerda Ayer ¿te dieron comida en la escuela? Sí No No sabe o no recuerda Ayer ¿qué te dieron de comida en la escuela? Una bebida sola (como atol o leche) Bebida y alguna comida pequeña Alguna comida pequeña Plato completo de comida No sabe o no responde ¿Te gusta la comida que te dan en la escuela este año? Sí No 62 The student questionnaire started with DIGEDUCA’s ELI, and is not shown. NORC | LENS PROGRAM BASELINE REPORT | 82 No sabe o no recuerda El año pasado ¿te daban comida en la escuela? Sí No [PROG: SKIP TO NEXT SECTION] No sabe o no recuerda [PROG: SKIP TO NEXT SECTION] El año pasado ¿qué te daban de comida en la escuela en un día cualquiera? (Encuestador, permitir respuesta múltiples.) Una bebida sola (como atol o leche) Bebida y alguna comida pequeña Alguna comida pequeña Plato completo de comida No sabe o no responde ¿Te gustaba la comida que te daban en la escuela el año pasado? Sí No No sabe o no recuerda Personal ¿Qué idioma hablas en casa la mayoría del tiempo? Español K'iche Ixil Otro No sabe o no recuerda Si responde otro, llenar el siguiente espacio. ¿Qué idioma habla tu maestro/a cuando te enseña en la escuela? (Encuestador, permitir respuesta múltiples.) Español K'iche Ixil Otro NORC | LENS PROGRAM BASELINE REPORT | 83 No sabe o no recuerda ¿Qué otro idioma? ¿Vives con tu mamá? Sí No [PROG: SKIP NEXT QUESTION] No responde [PROG: SKIP NEXT QUESTION] ¿Tu mamá puede leer y escribir? Sí No No sabe o no recuerda ¿Vives con tu papá? Sí No [PROG: SKIP NEXT QUESTION] No responde [PROG: SKIP NEXT QUESTION] ¿Tu papá puede leer y escribir? Sí No No sabe o no recuerda ¿Qué adulto cuida más de ti? Mamá Papá Abuela Abuelo Tía Tío Hermana Hermano Prima Primo Otro Nadie Si responde otro, llenar el siguiente espacio. ¿(Este adulto) puede leer y escribir? NORC | LENS PROGRAM BASELINE REPORT | 84 Sí No No responde Contándote a ti, ¿cuántas personas viven en tu casa? Hábitos de lectura ¿Hay un lugar en tu escuela dónde puedes leer o prestar libros? ¿No pienses en libros de texto, piensa en otros libros que puedes leer en la escuela o llevarte a casa? No hay lugar en la escuela [PROG: SKIP NEXT QUESTION] Esquina de lectura (lugar con libros dentro de la clase) Habitación de lectura (habitación dentro de la escuela con libros para leer) Biblioteca Otro No sabe o no recuerda [PROG: SKIP NEXT QUESTION] ¿Cada cuánto visitas (el lugar de lectura)? Nunca Una vez por semana Dos veces por semana Tres veces por semana Cuatro veces por semana Cinco veces por semana No recuerda o no sabe ¿Tienes materiales para leer en casa? Piensa en libros, periódicos o revistas que sean de tu familia, no los que puedes prestar en la escuela. Sí No [PROG: SKIP NEXT 7 QUESTIONS] No responde [PROG: SKIP NEXT 7 QUESTIONS] ¿En qué idioma están estos materiales? (Seleccione todas las opciones que apliquen) Español K'iche Ixil Otro No sabe o no recuerda ¿Tienes libros infantiles para leer en casa? NORC | LENS PROGRAM BASELINE REPORT | 85 Sí No [PROG: SKIP NEXT QUESTION] No responde [PROG: SKIP NEXT QUESTION] ¿En qué idioma están estos libros? (Seleccione todas las opciones que apliquen) Español K'iche' Ixil Uspanteco Sacapulteco Otro ¿Tienes otros tipos de libros para leer en casa? Sí No [PROG: SKIP NEXT QUESTION] No responde [PROG: SKIP NEXT QUESTION] ¿En qué idioma están estos libros? (Seleccione todas las opciones que apliquen) Español K'iche' Ixil Uspanteco Sacapulteco Otro ¿Tienes periódicos o revistas para leer en casa? Sí No [PROG: SKIP NEXT QUESTION] No responde [PROG: SKIP NEXT QUESTION] ¿En qué idioma están estos periódicos o revistas? (Seleccione todas las opciones que apliquen) Español K'iche' Ixil Uspanteco Sacapulteco Otro ¿Lees o intentas leer por tu cuenta en casa? Sí No No responde NORC | LENS PROGRAM BASELINE REPORT | 86 Al salir de la escuela, ¿participas en actividades de lectura fuera de tu casa? ¿Piensa en actividades como ir a la biblioteca o competencias de lectura? Sí No [PROG: SKIP NEXT QUESTION] No responde [PROG: SKIP NEXT QUESTION] ¿En qué actividades participas? (Encuestador, permitir respuestas múltiples) Clubes de lectura Actividades de lectura en la comunidad Actividades de lectura en la iglesia Visitas a la biblioteca Leer con amigos Leer con un mentor en la comunidad Competencias de cuenta cuentos Maratón de lectura Clubes de escritura Otro No sabe o no responde Si responde otro, llenar el siguiente espacio. ¿Qué fue lo último que leíste? Libro para niños Tira cómica Otro tipo de libro Revista Periódico Página o sitio de internet Otro No sabe o no responde Vivienda En tu casa, ¿hay una silla en buen estado? Sí No No responde En tu casa, ¿hay una cama en buen estado? Sí NORC | LENS PROGRAM BASELINE REPORT | 87 No No responde En tu casa ¿hay un reloj que funcione? Sí No No responde En tu casa ¿hay un radio que funcione? Sí No No responde En tu casa ¿hay una televisión que funcione? Sí No No responde En tu casa ¿hay una computadora que funcione? Sí No No responde En tu casa ¿hay una bicicleta que funcione? Sí No No responde En tu casa ¿hay una motocicleta que funcione? Sí No No responde En tu casa ¿hay un carro o camión que funcione? Sí No No responde En tu casa ¿hay un refrigerador que funcione? Sí No NORC | LENS PROGRAM BASELINE REPORT | 88 No responde En tu casa ¿hay una estufa que funcione? Sí No No responde ¿Tu casa tiene electricidad? Sí No No sabe No responde ¿Alguien en tu casa tiene celular? Sí No No sabe No responde NORC | LENS PROGRAM BASELINE REPORT | 89 CUESTIONARIO KAP PARA PROFESORES Cuestionario administrativo Fecha: Hora: Municipio: CodEst: Direccion: Entrevistador, ingrese su nombre por favor: NORC | LENS PROGRAM BASELINE REPORT | 90 Consentimiento Y Confidencialidad De Los Entrevistados Hola, mi nombre es [nombre de entrevistador] y trabajo para Juárez & Asociados. Estamos realizando un estudio del proyecto LENS de Save the Children para entender mejor cómo aprenden los niños. Su ayuda es muy valiosa para nosotros, sin embargo, no necesita participar si no lo desea. Su nombre no se registrará en este formulario, ni se mencionará en ningún lugar de los datos de la encuesta. El nombre de su escuela y su aula será registrado con el fin de poder conectar los datos entre escuela, aula y estudiantes. Lo que aprendamos será utilizado para ayudarnos a identificar el apoyo adicional que se necesita. La encuesta durará aproximadamente 40 minutos y le haremos preguntas sobre su experiencia como docente y sobre cuestiones relacionadas con la nutrición y la salud de los alumnos. Si acepta responder a las preguntas y cualquier pregunta lo hace sentir incómodo, podemos omitirlas. También puede optar por dejar de responder en cualquier momento. Por favor déjeme saber si tiene alguna pregunta antes de comenzar o durante la encuesta. ¿Está dispuesto a participar? No [PROG: END OF SURVEY] Yes NORC | LENS PROGRAM BASELINE REPORT | 91 A. Cuestionario demográfico ¿Qué idiomas predominan en la zona? (ENCUESTADOR, permitir respuestas múltiples) Español K'iché Ixil Uspanteco Sacapulteco Otro Ahora me gustaría hacerle unas preguntas sobre su persona: ¿Qué edad tiene? |_ _| años Registrar el sexo de la persona, por observación. Hombre Mujer ¿Cuántos años tiene de experiencia como maestro? |_ _| años ¿Cuántos años ha enseñado en primaria? |_ _| años ¿Cuál es su idioma materno? (Seleccione todas las opciones que aplique) Español K'iche' Ixil Uspanteco Sacapulteco Otro No sabe o no responde ¿Qué idioma habla cuando da clases? (Se puede permitir respuestas múltiples) Español K'iche' Ixil Uspanteco Sacapulteco Otro No sabe o no responde ¿Cuál es el nivel más alto de educación que ha completado? a) Magisterio en Educación Primaria Urbana, MEPU b) Magisterio en Educación Primaria Rural, MEPR c) Profesorado de segunda enseñanza d) Algún tiempo en la universidad e) Técnico universitario f) Licenciatura g) Otro NORC | LENS PROGRAM BASELINE REPORT | 92 h) No sabe o no responde ¿Ha recibido capacitación sobre cómo enseñar a leer, desde enero de 2019? Si No [PROG: SKIP NEXT QUESTION] No sabe o no responde [PROG: SKIP NEXT QUESTION] ¿Quiénes le han brindado la capacitación a partir de enero del 2019? (Encuestador, permitir respuesta múltiples.) Save de Children, IDEA USAID Leer y aprender Child Aid CRS/Caritas Ministerio de Educación u oficinas locales de educación Universidades Programa académico de desarrollo profesional docente (PADEP) Otro No sabe o no responde ¿En qué grados está enseñando este año escolar? (Encuestador, permitir respuesta múltiples. Si el maestro enseña varios grados en el mismo salón, marque “otro”, explique e indique que grados.) Preprimaria 1ro 2do 3ro 4to 5to 6to Otro No sabe o no responde ¿Qué sección de tercer grado imparte? A B C D Única Otros No sabe o no responde ¿Cuántas niñas hay inscritas en su sección? |_ _| niñas ¿Cuántos niños hay inscritos en su sección? |_ _| niños Pensando en la última semana, ¿cuántos días NO pudo asistir a la escuela? Por favor, no cuente los días en que toda la escuela estuvo cerrada por un feriado u otra razón. NORC | LENS PROGRAM BASELINE REPORT | 93 |_ _| días El año pasado, ¿fue usted profesor en esta escuela? Si No No sabe o no responde Reading Resources at School ¿Esta escuela tiene una biblioteca? (Un espacio donde los niños pueden leer y/o pedir prestado libros) Si No [PROG: SKIP TO NEXT SECTION] No sabe o no responde [PROG: SKIP TO NEXT SECTION] ¿Cuántos libros tiene la biblioteca de la escuela? (Maestro estima, no cuenta libros) 1-10 11-50 Más de 50 No sabe o no responde 30.1. ¿En qué idiomas están estos libros? (Encuestador, permitir respuesta múltiples.) Español K'iche' Ixil Uspanteco Sacapulteco Inglés Otro No sabe o no responde School feeding programs ¿Ha habido un programa de alimentación escolar en esta escuela desde enero del 2019? Si No [PROG: SKIP TO NEXT SECTION] No sabe no responde [PROG: SKIP TO NEXT SECTION] ¿Quiénes han proporcionado el programa de alimentación escolar en esta escuela desde enero de 2019? (Encuestador, permitir respuesta múltiples.) Save de Children, IDEA USAID Leer y aprender Child Aid CRS/Caritas Ministerio de Educación u oficinas locales de educación Programa académico de desarrollo profesional docente (PADEP) Otros No sabe no responde NORC | LENS PROGRAM BASELINE REPORT | 94 En un día típico, ¿qué comidas han proporcionado estos programas de alimentación escolar a partir de enero 2019? (Encuestador, permitir respuesta múltiples.) Una bebida sola (como atol o leche) Bebida y alguna comida pequeña Alguna comida pequeña Plato completo de comida No sabe Prefiere no responder NORC | LENS PROGRAM BASELINE REPORT | 95 Alimentación basada en la Taza Escolar La taza escolar es una representación del Ministerio de Educación de Guatemala para explicar cómo los niños y jóvenes en edad escolar deben comer. Por favor conteste las siguientes preguntas basándose en información que usted ha visto o recibido sobre la taza escolar. [Encuestador: Dale la copia de la taza escolar al maestro/a] ¿Ha visto esta imagen de la taza escolar? Si No [PROG: SKIP NEXT QUESTION] No responde [PROG: SKIP NEXT QUESTION] ¿En dónde vio esta imagen por primera vez? ¿Según la distribución de la taza escolar, cuál opción mencionada abajo es el mejor ejemplo de un almuerzo que se debe ofrecer en las escuelas? Atol de avena Tortilla con aguacate Tamalito de frijol con queso y salsa Sopa de fideo Ninguno No sabe o no responde NORC | LENS PROGRAM BASELINE REPORT | 96 Alimentación en la Escuela ¿Sus estudiantes recibieron alimentación en la escuela ayer? Si No [PROG: SKIP MENU QUESTIONS] No sabe o no responde [PROG: SKIP MENU QUESTIONS] ¿Qué recibieron? (NO lea las opciones) 1) Espagueti con espinaca y carne 2) Doblada de papa y queso 3) Guisado de verduras con huevo 4) Boxboles 5) Shepes con salsa 6) Siete camisas 7) Salpicón de rábano con carne vegetal 8) Empanada de frijol con hierbas 9) Guisado de fideo con bledo y pollo 10) Tamalito de chipilín con pollo 11) Arroz negro 12) Tortitas de ejote 13) Enchilada de verdura con carne 14) Tortilla con frijol y huevo 15) Chaomein con verdura y pollo 16) Doblada rellena de hortalizas y frijol 17) De manía con leche 18) Chuchitos de frijol y hierbamora con queso 19) Pupusas de hierba con queso 20) Arroz con hortalizas y huevo 21) Chaomein de res 22) Enchilada de carne vegetal 23) Fideos con carne y verduras o hierbas 24) Dobladas de hierbas con requesón 25) Tamalito de frijol con queso y salsa 26) Pan con picado de verduras y carne vegetal 27) Caldo de pollo con arroz y verduras o hierbas 28) Frijoles con huevo duro 29) Hierbas guisadas con huevo 30) Huevo con tomate y cebolla 31) Tamalitos de chipilín con queso y pollo 32) Empanada de frijol con hierbas 33) Otro 34) No sabe o no responde [PROG: OPEN SECONDARY MENU QUESTIONS BASED ON QUESTEION ABOVE] ¿Con que venía acompañado el Espagueti con espinaca y carne? (Encuestador, permitir respuesta múltiples.) Refresco de chilacayote Fruta de la temporada Nada NORC | LENS PROGRAM BASELINE REPORT | 97 No sabe o no responde Otro ¿Con que venía acompañada la Doblada de papa y queso? (Encuestador, permitir respuesta múltiples.) Atol de harina fortificada con leche Fruta de la temporada Nada No sabe o no responde Otro ¿Con que venía acompañado el Guisado de verduras con huevo? (Encuestador, permitir respuesta múltiples.) Atol de arroz en leche Tortilla Nada No sabe o no responde Otro ¿Con que venía acompañado los Boxboles? (Encuestador, permitir respuesta múltiples.) Atol de haba con leche Fruta de la temporada Nada No sabe o no responde Otro ¿Con que venían acompañado los Shepes con salsa? (Encuestador, permitir respuesta múltiples.) Atol de harina fortificada con manía Fruta de la temporada Nada No sabe o no responde Otro ¿Con que venía acompañado la Siete camisas? (Encuestador, permitir respuesta múltiples.) Refresco de papaya y piria Queso fresco Nada No sabe o no responde Otro ¿Con que venía acompañado el Salpicón de rábano con carne vegetal? (Encuestador, permitir respuesta múltiples.) Refresco de guayaba Tortillas Nada No sabe o no responde Otro ¿Con que venía acompañado el Empanada de frijol con hierbas? (Encuestador, permitir respuesta múltiples.) NORC | LENS PROGRAM BASELINE REPORT | 98 Refresco de manzana y membrillo Nada No sabe o no responde Otro ¿Con que venía acompañado el Guisado de fideo con bledo y pollo? (Encuestador, permitir respuesta múltiples.) Refresco de mora Nada No sabe o no responde Otro ¿Con que venía acompañado el Tamalito de chipilín con pollo? (Encuestador, permitir respuesta múltiples.) Atol de haba con leche Fruta de la temporada Nada No sabe o no responde Otro ¿Con que venía acompañado el Arroz negro? (Encuestador, permitir respuesta múltiples.) Refresco de naranja y limón Queso fresco Nada No sabe o no responde Otro ¿Con que venían acompañado las Tortitas de ejote? (Encuestador, permitir respuesta múltiples.) Atol de harina fortificada con leche Tortilla y fruta de la temporada Nada No sabe o no responde Otro ¿Con que venía acompañado la Enchilada de verdura con carne? (Encuestador, permitir respuesta múltiples.) Atol de plátano Fruta de la temporada Nada No sabe o no responde Otro ¿Con que venía acompañado la Tortilla con frijol y huevo? (Encuestador, permitir respuesta múltiples.) Atol de manzana Fruta de la temporada Nada No sabe o no responde Otro NORC | LENS PROGRAM BASELINE REPORT | 99 ¿Con que venía acompañado el Chaomein con verdura y pollo? (Encuestador, permitir respuesta múltiples.) Refresco de limonada Nada No sabe o no responde Otro ¿Con que venía acompañado la Doblada rellena de hortalizas y frijol? (Encuestador, permitir respuesta múltiples.) Atol de amaranto con leche Fruta de la temporada Nada No sabe o no responde Otro ¿Con que venía acompañado la De manía con leche? (Encuestador, permitir respuesta múltiples.) Refresco de horchata Ensalada de frutas con amaranto y miel Nada No sabe o no responde Otro ¿Con que venían acompañado los Chuchitos de frijol y hierbamora con queso? (Encuestador, permitir respuesta múltiples.) Refresco de manzana y membrillo Nada No sabe o no responde Otro ¿Con que venían acompañado las Pupusas de hierba con queso? (Encuestador, permitir respuesta múltiples.) Refresco de avena Fruta de la temporada Nada No sabe o no responde Otro ¿Con que venía acompañado el Arroz con hortalizas y huevo? (Encuestador, permitir respuesta múltiples.) Atol de pinol con leche Fruta de la temporada Nada No sabe o no responde Otro ¿Con que venía acompañado el Chaomein de res? (Encuestador, permitir respuesta múltiples.) Atol de avena O refresco de avena Nada No sabe o no responde NORC | LENS PROGRAM BASELINE REPORT | 100 Otro ¿Con que venía acompañado la Enchilada de carne vegetal? (Encuestador, permitir respuesta múltiples.) Atol de harina fortificada O refresco de harina fortificada Fruta Nada No sabe o no responde Otro ¿Con que venían acompañado los Fideos con carne y verduras o hierbas? (Encuestador, permitir respuesta múltiples.) Caliente de piña o refresco de Jamaica con piña Tortilla Nada No sabe o no responde Otro ¿Con que venían acompañado las Dobladas de hierbas con requesón? (Encuestador, permitir respuesta múltiples.) Arroz con leche U horchata de arroz con leche Nada No sabe o no responde Otro ¿Con que venía acompañado el Tamalito de frijol con queso y salsa? (Encuestador, permitir respuesta múltiples.) Atol de haba con leche o refresco de avena y fruta Nada No sabe o no responde Otro ¿Con que venía acompañado el Pan con picado de verduras y carne vegetal? (Encuestador, permitir respuesta múltiples.) Atol de harina fortificada con leche o refresco Fruta Nada No sabe o no responde Otro ¿Con que venía acompañado el Caldo de pollo con arroz y verduras o hierbas? (Encuestador, permitir respuesta múltiples.) Horchata de arroz con manía y leche Totilla Nada No sabe o no responde Otro ¿Con que venían acompañado los Frijoles con huevo duro? (Encuestador, permitir respuesta múltiples.) Caliente de manzana O limonada NORC | LENS PROGRAM BASELINE REPORT | 101 Tortilla Nada No sabe o no responde Otro ¿Con que venía acompañado las Hierbas guisadas con huevo? (Encuestador, permitir respuesta múltiples.) Ponche de frutas o refresco de piña Tortillas Nada No sabe o no responde Otro ¿Con que venía acompañado el Huevo con tomate y cebolla? (Encuestador, permitir respuesta múltiples.) Arroz con chocolate u horchata de arroz con leche Fruta Nada No sabe o no responde Otro ¿Con que venían acompañado los Tamalitos de chipilín con queso y pollo? (Encuestador, permitir respuesta múltiples.) Horchata de arroz Nada No sabe o no responde Otro ¿Con que venía acompañado la Empanada de frijol con hierbas? (Encuestador, permitir respuesta múltiples.) Refresco de manzana y membrillo Nada No sabe o no responde Otro ¿Con que venía acompañado el [otro]? ¿Los estudiantes en tu salón de clases se lavaron los dientes en la escuela ayer? (o el último día de escuela) Si No No sabe o no responde ¿En qué momentos se lavaron las manos los niños de su clase ayer? (o el último día de escuela) (Encuestador, NO LEA las opciones. Permitir respuestas múltiples.) Antes de comer Antes de cocinar Después de usar la letrina / inodoro. Después de tocar algo sucio, como animales o suciedad Después de estornudar No sabe o no responde NORC | LENS PROGRAM BASELINE REPORT | 102 ¿En qué momentos es más importante lavarse las manos? (Encuestador, NO LEA las opciones. Permitir respuestas múltiples.) Antes de comer Antes de cocinar Después de usar la letrina / inodoro Después de tocar algo sucio, como animales o suciedad Después de estornudar Otros Después de cambiar pañales No sabe o no responde ¿Qué alimentos ayudan a los niños a crecer? (Encuestador, permitir respuesta múltiples.) Leche, carne, huevo u otros derivados de animales Cereales, granos y tubérculos Hierbas, verduras y Frutas Frijoles, carne de soya y haba Grasas Ninguno No sabe o no responde ¿Qué grupo de alimentos les da principalmente energía a los niños para la actividad física? Leche, carne, huevo u otros derivados de animales Cereales, granos y tubérculos Hierbas, verduras y Frutas Frijoles, carne de soya y habas Grasas Ninguno No sabe o no responde ¿Qué grupo de alimentos por su alto contenido de vitaminas, minerales y fibra protegen de enfermedades? Leche, carne, huevo u otros derivados de animales Cereales, granos y tubérculos Hierbas, verduras y Frutas Frijoles, carne de soya y habas Grasas Ninguno No sabe o no responde ¿Cuáles son los efectos negativos de comer demasiada azúcar? (Encuestador, permitir respuesta múltiples.) Problemas en los dientes / caries Riesgo de diabetes / azúcar alto Aumento de peso / obesidad Hiperactividad / exceso de energía Les da sueño No sabe o no responde ¿Qué alimentos son importantes para prevenir la anemia? (ENCUESTADOR, permitir respuestas múltiples) NORC | LENS PROGRAM BASELINE REPORT | 103 Hígado/riñón/corazón Carne roja/pescado/pollo Manzana Incaparina y otros productos fortificados Frijoles No sabe o no responde ¿Por qué es importante consumir sal yodada? Para crecer Para la inteligencia Para los dientes Ninguno No sabe o no responde ENCUESTADOR LEA LA FRASE: "Pensando en los niños de edad escolar en su escuela, indique si cada oración es verdadera (V), falsa (F), o no está seguro (NES). Cuando el niño come todos los días: mosh, corn flakes, tortilla con frijol y arroz para almuerzo y cena. Este niño está bien alimentado." Verdadero Falso No está seguro No responde Al igual que la leche, la Incaparina es rica en calcio. Verdadero Falso No está seguro No responde A los niños les puede dar anemia si les falta alimentos que tienen hierro o fortificados con hierro Verdadero Falso No está seguro No responde Los síntomas de anemia en niños escolares incluyen el cansancio, palidez y dificultades para aprender Verdadero Falso No está seguro No responde Tomar agua pura en lugar de fresco o gaseosa ayuda a prevenir la diabetes. Verdadero Falso No está seguro No responde NORC | LENS PROGRAM BASELINE REPORT | 104 ENCUESTADOR LEA LA FRASE: Pensando en los niños de su escuela, indique si está de acuerdo, neutral, en desacuerdo con las siguientes oraciones. En mi escuela hay problema de desnutrición en los niños De acuerdo Neutral Desacuerdo No sabe o no responde Es importante educar a mis estudiantes sobre la comida chatarra. De acuerdo Neutral Desacuerdo No sabe o no responde La anemia causada por falta de hierro es una condición seria en los niños De acuerdo Neutral Desacuerdo No sabe o no responde Mi rol de educador me facilita comunicar mensajes sobre nutrición en mi escuela y comunidad De acuerdo Neutral Desacuerdo No sabe o no responde Cuando enseño a mis estudiantes sobre nutrición, puedo ayudar a mejorar su salud. De acuerdo Neutral Desacuerdo No sabe o no responde Me siento capacitado para poder enseñarle a los estudiantes sobre buenas prácticas de la nutrición saludable De acuerdo Neutral Desacuerdo No sabe o no responde Para prestar atención en clase, es importante que los niños tomen suficiente agua De acuerdo Neutral Desacuerdo No sabe o no responde Tengo suficiente tiempo en mis clases para enseñarle a los estudiantes sobre nutrición De acuerdo Neutral NORC | LENS PROGRAM BASELINE REPORT | 105 Desacuerdo No sabe o no responde Mis alumnos necesitan aprender sobre nutrición para prevenir el sobrepeso De acuerdo Neutral Desacuerdo No sabe o no responde Los maestros son responsables de enseñarles a los niños sobre nutrición De acuerdo Neutral Desacuerdo No sabe o no responde Los padres son responsables de enseñar a sus hijos sobre nutrición De acuerdo Neutral Desacuerdo No sabe o no responde ENCUESTADOR LEA LA FRASE:"Con qué frecuencia Ud. realiza cada una de las siguientes actividades. Indique si lo hace a menudo, algunas veces, o nunca. Enseño a leer y escribir utilizando nombres de alimentos A menudo Algunas veces Nunca No sabe o no responde Les pido a mis estudiantes reportar si no hay agua o jabón para lavarse las manos en la escuela A menudo Algunas veces Nunca No sabe o no responde Enseño sobre los riesgos de comer mucho azúcar A menudo Algunas veces Nunca No sabe o no responde Realizo juegos para promover mensajes sobre alimentación A menudo Algunas veces Nunca No sabe o no responde NORC | LENS PROGRAM BASELINE REPORT | 106 Algunas escuelas cultivan verduras y/o frutas para el consumo de los estudiantes. ¿Hay un huerto en tu escuela? Si No [PROG: END SURVEY] No sabe o no responde [PROG: END SURVEY] En las últimas dos semanas, ¿los estudiantes trabajaron en el huerto escolar? Si [PROG: SKIP NEXT QUESTION] No No sabe o no responde ¿Por qué los estudiantes no trabajaron en el huerto escolar? (NO lea las opciones) Agua insuficiente Tiempo insuficiente Falta de materiales / herramientas de jardinería No permitido por la escuela Estudiantes no interesados Huerto demasiado pequeño Otro No sabe o no responde En el último mes, ¿sus estudiantes consumieron algo producido por el huerto escolar? Si [PROG: END SURVEY] No No sabe o no responde ¿Por qué los estudiantes no consumieron nada producido por el huerto escolar? (ENCUESTADOR, permitir respuesta múltiple. NO leer respuestas) Porque el huerto no produjo nada Porque no les gusta a los estudiantes lo que produce el huerto [PROG: END SURVEY] Porque no es temporada de cosecha [PROG: END SURVEY] Porque es muy temprano para cosechar [PROG: END SURVEY] Otro [PROG: END SURVEY] No sabe o no responde [PROG: END SURVEY] ¿Por qué no produjo nada el huerto? NORC | LENS PROGRAM BASELINE REPORT | 107 GUÍA DE OBSERVACIÓN ESCOLAR 1 Nombre del enumerador: ________________________________________ Posición geo referencia: _________________________________________ Fecha de la aplicación: FECHA __ / __ / ____ HORA __:__:__ AM/PM DATOS DEL ESTABLECIMIENTO Seleccione de la lista la escuela que está visitando. Municipio _______________________________________________ CodEst _______________________________________________ Dirección _______________________________________________ Nombre Observador Observador, ingrese su nombre por favor: _______________________________ Instalaciones Escolares: Espacio para almacenar alimentos 1 Sí No (Enumerador, Si la respuesta es “No,” salta al #17.) La escuela cuenta con un lugar para almacenar los alimentos (almacén). 2 El lugar es: Sí No Comentarios Gabinete Habitación 3 Otro ¿Cuál? Sí No Comentarios NORC | LENS PROGRAM BASELINE REPORT | 108 4 El lugar para almacenar alimentos se utiliza exclusivamente para almacenar alimentos. 5 El lugar para almacenar alimentos está limpio y organizado (comida guardada adecuada mente y libre de restos de comida) 6 ¿Se observa evidencia de humedad o huele a humedad en el lugar para almacenar alimentos? 7 En el lugar para almacenar alimentos, los alimentos se almacenan fuera del piso en repisas o dentro de un mueble seguro y limpio. 8 Se observan sustancias fecales o alguna otra evidencia de roedores o cucarachas (o alguna otra peste o animal) en el lugar para almacenar alimentos. 9 El lugar para almacenar alimentos tiene un control diario actualizado disponible en el almacén. 10 El lugar para almacenar alimentos tiene un equipo de medición de porciones (taza de medir, bascula, o pesa). 11 El equipo de medición de las porciones de comida está limpio y en buena condición. 12 En el lugar para almacenar alimentos, los alimentos se encuentran sellados y no se observan derrames. 14 La tabla de cálculo de porciones se encuentra exhibida. 15 El espacio para almacenar alimentos está identificado con el logo (imagen) de Save the Children. 16 El lugar para almacenar alimentos tiene una lista de verificación de visitas de los técnicos de Save the Children. Instalaciones Escolares: Espacio para cocina 17 Sí No [PROG: IF NO, La escuela tiene un espacio para SKIP TO 30] cocina. NORC | LENS PROGRAM BASELINE REPORT | 109 Sí No Comentarios 28 El espacio para cocinar está al aire libre. 18 El espacio de cocina se usa exclusivamente para cocinar. 19 El espacio para cocinar tiene puerta. 20 El espacio de cocina tiene piso de cemento o cualquier otro material para recubrir el piso. 22 El espacio para cocina está organizado (platos y cubiertos tapados). 23 El espacio de cocina está visiblemente limpio (libres de restos de comida). 24 El espacio de cocina cuenta con bote de basura. 25 El espacio de cocina tiene lavamanos (sea agua entubada o agua almacenada). 26 En el espacio de cocina, hay agua para limpieza (sea agua entubada o agua almacenada). 27 El espacio de cocina tiene un menú semanal. La etiqueta dice yodada. La etiqueta no dice yodada. No se observa la etiqueta. 29 Observe si el paquete de la sal dice ‘sal yodada.’ (Nota: Si no es posible ver el paquete, indica “no se observa.”) Instalaciones Escolares: Equipo de Cocina 30 Sí No [PROG: IF NO, SKIP TO 36] La escuela tiene estufa. Gas Leña Ahorrador de leña Otro [PROG: IF “Other” ask question 32] 31 La estufa es de: 32 ¿De qué tipo es la estufa? NORC | LENS PROGRAM BASELINE REPORT | 110 33 Sí No Comentarios ¿La estufa tiene chimenea? Instalaciones Escolares: Servicios sanitarios para los estudiantes 36 Sí No [PROG: IF NO, SKIP TO 48] ¿Hay servicios sanitarios para los estudiantes en la escuela? 37 Inodoro Letrina Los servicios sanitarios son: (Marque el más común.) 38 ¿Qué tipo de letrina se utiliza? Seleccione una Letrina de pozo Letrina elevada Letrina de cubeta Letrina de losa Letrina de compostaje Inodoro con descarga directa Inodoro con descarga manual (el usuario vierte agua para la descarga) 39 ¿Qué tipo de inodoro se utiliza? Respuesta numérica 40 Número de inodoros/letrinas individuales para uso de estudiantes 41 Número de inodoros/letrinas individuales que se encuentran funcionando para uso de estudiantes (disponible, funcional, con cerrojo) Sí No Comentarios 42 Los inodoros/letrinas se encuentran separados para hombres y para mujeres. 43 Los inodoros/letrinas tienen puertas con cerrojos. 44 El espacio está visiblemente limpio. (Todos los baños no tienen un olor fuerte o un número significativo de moscas o NORC | LENS PROGRAM BASELINE REPORT | 111 mosquitos, y no hay heces visibles en el piso, las paredes, el asiento o alrededor de las instalaciones.) 45 Los inodoros/letrinas/baños tienen bote de basura. 46 Los inodoros/letrinas tienen una lista de verificación del mantenimiento diario del baño/letrina. Instalaciones Escolares: Instalaciones para lavado de manos 48 Sí No [PROG: IF NO, La escuela tiene un lugar para que los SKIP TO 51] estudiantes se laven las manos. 49 Para lavarse las manos, los estudiantes utilizan: (Esta pregunta se refiere a la infraestructura.) Sí No Comentarios Lavamanos con grifo Pila con grifo Tanque de agua con grifo Cubeta con grifo Contenedor de agua elevado, con un pequeño agujero Pila o cubeta con palangana Pila o cubeta (se meten las manos) 49.1 Otro ¿Cuál? 50 Para lavarse las manos, los estudiantes tienen acceso a: (Encuestador, Seleccione todos que apliquen.) Sí No Comentarios Jabón Agua o contenedor con agua Ninguno Instalaciones Escolares: Agua para consumo de los estudiantes 51 Sí No [PROG: IF NO, La escuela tiene agua para consumo de los SKIP TO 86] estudiantes. NORC | LENS PROGRAM BASELINE REPORT | 112 52 La principal fuente de agua en la escuela es: Sistema de captación de agua de lluvia Agua entubada Pozo de agua o nacimiento de agua protegido Pozo de agua o nacimiento de agua sin protección Lago, río, arrollo 53 Otro Por favor especifique. Sí No Comentarios 54 La escuela tiene cisterna para almacenar agua. 55 ¿Se encuentra actualmente disponible el agua, proveniente de la fuente primaria? 56 ¿Se trata el agua para que sea segura para consumir? 57 ¿Qué medida utiliza? Selección multiple Ecofiltro de arcilla Hervir el agua Clorar el agua Desinfección solar 58 Otro Por favor especifique: [PROG: IF “ECOFILTRO” NOT SELECTED, SKIP TO NEXT SECTION] 59.1 ¿Cuándo fue la última vez que cambiaron el ecofiltro? (mes y año o dejar en blanco si no se sabe) MM/AAAA 59 Sí No Comentarios ¿El ecofiltro tiene agua? Preguntas para el/la director/a Encuestador, hágale las siguientes preguntas al director/a de la escuela. 86 Sí No No sabe o no responde ¿Hay un programa de alimentación escolar en esta escuela en este ciclo escolar 2019? NORC | LENS PROGRAM BASELINE REPORT | 113 87 ¿Quién proporciona el programa de alimentación escolar en esta escuela en este ciclo escolar 2019? A Save de Children, IDEA B USAID Leer y Aprender C Child Aid D CRS/Caritas E Ministerio de Educación u oficinas locales de educación F Programa académico de desarrollo profesional docente (PADEP) 87.1 Otro(s) Por favor especifique H No sabe o no responde 88 ¿Qué comidas han proporcionado estos programas de alimentación escolar? A Bebida sola (como atol) B Bebida y comida pequeña C Comida pequeña sola D Plato completo E Alimentos para llevar a casa 89 Otro Por favor especifique 90 Sí No No sabe o no responde ¿Los estudiantes contribuyen con comida o leña para la alimentación escolar? 91 ¿Qué contribuyen? Sí No No sabe o no responde 93 ¿Hay gobierno escolar para la escuela? 93.1 ¿Sabe qué es el semáforo de limpieza? 94 ¿Hay semáforo de limpieza en uso? [PROG: IF NO, SKIP TO 85] 95 (Encuestador: pide ver el semáforo y observe.) El semáforo se encuentra exhibida en un lugar abierto. NORC | LENS PROGRAM BASELINE REPORT | 114 Jardines de la escuela 85 Sí No Comentarios La escuela tiene un huerto. Biblioteca escolar 96 Sí No Comentarios ¿Esta escuela tiene una biblioteca? (Recursos de lectura) 97 1-10 11-50 Más de 50 ¿Cuántos libros tiene la biblioteca de la escuela? [Enumerador, pide que lo dejen contar los libros (hasta un máximo de 50)] NORC | LENS PROGRAM BASELINE REPORT | 115 GUÍA DE OBSERVACIÓN ESCOLAR 2 Seguridad alimentaria y prácticas de higiene: Preparación de comida Sí No 60 ¿Durante la visita, se observó la preparación de alimentos? [PROG: IF NO, END OBSERVATION] 60.1 ¿Se prepara comida en la escuela para consumo de los estudiantes? Sí No No se observa Comentarios 61 Todas las personas que manejan los alimentos utilizan un delantal/gabacha visible y limpio. 63 Todas las personas que manejan los alimentos usan redecilla o pañuelo para el cabello. 65 Durante la observación, ¿al menos una persona que maneja los alimentos se lavó las manos? 65.1 ¿Se lavó correctamente las manos? (jabón y agua que corre) 67 La comida se mantiene cubierta para protegerla de contaminación. 73 El equipo de comida, los utensilios y las superficies de contacto fueron lavados, enjuagados y desinfectados apropiadamente. [Si el observador no puede observar el proceso de preparación de comida, marque "no se observa" y preguntará sobre el proceso utilizado. Detalla el proceso usado en el comentario que sigue.] NORC | LENS PROGRAM BASELINE REPORT | 116 75 Toda agua almacenada está limpia y libre de grasa y de partículas de comida. 77 Las superficies de trabajo están visiblemente limpias (sin restos de comida). 79 Los botes de basura se mantienen cubiertos. 81 Los botes de basura están apropiadamente llenos, sin rebalsar. Encuestador, haga las siguientes preguntas a una de las personas que prepara la comida: 83 ¿Quién tiene acceso a la cocina? (Enumerador, permitir respuestas múltiples) Personal que prepara la comida Padres de familia Maestros Niños (estudiantes o hijos voluntarios) 83.1 Otro Por favor especifique: 84 ¿Cuándo se lava las manos? (Enumerador, NO lea las opciones.) Marque todas las que correspondan Antes de preparar la comida Después de tocar alimentos crudos Después de usar el baño 84.1 Otro Por favor especifique: Nunca 84.2 ¿Dónde se lava las manos? (Enumerador, observe si hay…) Marque todas las que correspondan Jabón Agua o contenedor con agua Ni jabón ni agua No se observa NORC | LENS PROGRAM BASELINE REPORT | 117 KII MAESTROS Hola, mi nombre es [nombre de entrevistador] y trabajo para Juárez & Asociados. Estamos realizando un estudio del proyecto Lectoescritura y Nutrición Sostenible (LENS) de Save the Children para entender mejor cómo aprenden los niños. Su ayuda es muy valiosa para nosotros, sin embargo, no necesita participar si no lo desea. Su nombre no se registraran en este formulario, ni se mencionarán en ningún lugar de los datos de la entrevista. La entrevista durará aproximadamente una hora y le haremos preguntas sobre su experiencia como docente y sobre cuestiones relacionadas con la nutrición y la salud de los alumnos. Si usted permite, la entrevista será grabada. Si acepta responder a las preguntas y cualquier pregunta lo hace sentir incómodo, podemos omitirlas. También puede optar por dejar de responder en cualquier momento. Por favor déjeme saber si tiene alguna pregunta antes de comenzar o durante la entrevista Su identidad se mantendrá confidencial asegurándonos que ninguno de sus comentarios permita que usted o su escuela sean identificados. ¿Tiene alguna pregunta? ¿Está dispuesto a participar? ______________________________________________________________________________ ____ Inicio 1) ¿Conoce el programa de Lectoescritura y Nutrición Sostenible LENS? (ENCUESTADOR: Si no, salte a #2). a. ¿Es LENS la forma como le llaman al programa? b. ¿Podría describir brevemente sus expectativas de LENS? c. ¿Qué beneficios espera del programa para su escuela? d. ¿Cuáles son los retos que considera que LENS puede encontrar en su escuela? Nutrición e Higiene de la Escuela Ahora hablemos sobre la nutrición e higiene dentro de la escuela. 2) ¿Esta escuela proporcionó algún tipo de alimentos a los estudiantes el año pasado? ENCUESTADOR: Si el maestro no sabe porque no estuvo en esta escuela el año pasado, salte a pregunta #3. Si no, ¿por qué no? a. ¿Qué alimentos proporcionó? b. ¿Era una alimentación adecuada y saludable? ¿Por qué si o no? c. ¿Qué sugerencias tiene para mejorar la comida de la escuela? NORC | LENS PROGRAM BASELINE REPORT | 118 d. En su escuela, ¿quién decide que alimentos servir? 3) ¿Qué tan importante es que su escuela brinde alimentación a los estudiantes durante su jornada escolar? ¿Por qué? 4) ¿Qué alimentos se venden en su escuela? a. ¿Cuánto cuestan? (agrupar y poner rango) b. ¿Quiénes los venden? c. ¿Hay restricción de que se vende o quienes venden? d. ¿Qué opina Ud. de estos alimentos? e. ¿Tiene sugerencias acerca de los alimentos que se deben vender en su escuela? 5) ¿Ha recibido capacitación sobre la nutrición escolar? Si no, ¿por qué no? a. ¿Qué temas cubrieron en la capacitación? b. ¿Quién dio la capacitación? c. ¿Qué partes de la capacitación le parecían más útil? d. ¿Sobre qué temas de nutrición escolar le gustaría recibir capacitación? 6) ¿Qué materiales tienen para enseñar temas de nutrición a los niños? 7) ¿Qué hábitos de higiene promueven en su escuela? a. ¿Como los promueve? 8) ¿Ha recibido capacitación en prácticas de higiene escolar? Si no, ¿por qué no? a. ¿Qué temas cubrieron en la capacitación? b. ¿Quién dio la capacitación? c. ¿Hay algo de la capacitación que fue útil? d. En general, no hablando solo de las capacitaciones previas, ¿Sobre qué temas de higiene escolar le gustaría recibir capacitación? Alimentos locales y la Diversidad de la Dieta 9) ¿Cuáles alimentos que se producen en esta zona, serían buenos de incluir en el menú de la escuela? a. ¿Siempre están disponibles? b. ¿Quiénes producen estos alimentos? c. ¿Cómo podrían ser utilizados o preparados? (recetas, etc…) d. ¿Qué tanto les gustarían estos alimentos a los estudiantes? Estado de Implementación del Programa [SALTAR SI AL PRINCIPIO AFIRMARON NO CONOCER EL PROGRAMA] Ahora hablemos sobre el proyecto LENS 11) ¿Ha recibido alguna capacitación por parte del proyecto LENS? a. Si ha recibido, ¿de qué fue la capacitación? NORC | LENS PROGRAM BASELINE REPORT | 119 b. ¿Hay algo de la capacitación que fue útil? ¿Cómo? c. ¿Como haría la capacitación más útil? d. APLICA A TODOS: ¿está enterado de capacitaciones futuras en las que participará? ¿Saben los temas o algunos detalles? 12) ¿Conoce los círculos de aprendizaje? a. Si lo está, ¿ha participado previamente en algún círculo de aprendizaje? i. Por favor describa su experiencia. 13) ¿Ha recibido su escuela materiales de lectura por parte del proyecto LENS? a. Si ha recibido, ¿qué tipo de materiales recibió? b. ¿Qué tan apropiados son los materiales para enseñar a leer a los estudiantes? ¿Cómo? Libros y otros materiales de Lectura Finalmente, hablemos sobre los materiales de lectura en su escuela. 14) ¿Su escuela tiene una biblioteca o un espacio para que los estudiantes lean o piden libros prestados? Si no, ¿por qué no? a. ¿Qué tipos de libros tiene? b. ¿En qué idiomas están los libros? c. ¿Qué tan útiles son los materiales para promover la lectura? 15) ¿Fuera de la escuela, hay otra biblioteca cercana? a. Si la hay, ¿Dónde está? ¿Qué tan accesible es para sus estudiantes y sus padres? 16) ¿Existen otros programas u actividades de lectura en su escuela? (tales como campamentos de lectura o concursos de lectura). Si no, ¿por qué no? a. Si existen, por favor, describa los programas o actividades. b. Desde su perspectiva, ¿qué tan útiles son para promover la lectura? Conclusión Gracias por hablar conmigo hoy. Sus opiniones son muy valiosas. 17) ¿Hay algo más que quisiera mencionar acerca de la lectura y nutrición en su escuela? NORC | LENS PROGRAM BASELINE REPORT | 120 GUÍA DE GRUPOS FOCALES CON PADRES DE FAMILIA A. Introducción y consentimiento (10 minutos) Introducción (resumida por el facilitador) Hola, gracias por hacer tiempo para participar en esta conversación de grupo focal (FGD). Soy [NOMBRE], el moderador. Trabajo con Juárez & Asociados y NORC de la Universidad de Chicago. Estamos trabajando para evaluar un programa de alfabetización y nutrición llamado Lectoescritura y Nutrición Sostenible (LENS) implementado por Save the Children. Hemos pedido a ustedes que nos hablen sobre sus opiniones sobre el proyecto LENS. El propósito de este estudio es comprender cómo los niños aprenden a leer. Lo que aprendemos se utilizará para mejorar el diseño del programa y también podría ayudar a diseñar mejores programas en el futuro. Su identidad se mantendrá confidencial y nos aseguraremos de que ninguno de sus comentarios se remita a usted. Nuestra conversación se grabará con esta tableta [presente tableta] para que pueda revisar la conversación y tomar notas. Nada de lo que diga estará asociado a su nombre. Del mismo modo, le pedimos que sea respetuoso con las opiniones de los demás y que no repita fuera de esta sala lo que se dice hoy aquí. Es importante que nos dé su opinion sincera. La conversación durará aproximadamente una hora y media y nos detendremos para tomar un descanso en el medio. Hasta este momento, ¿tiene algo que requiera nuestras aclaraciones o tiene alguna pregunta? [dejar espacio para preguntas]. Si no hay más preguntas, procederemos. Consentimiento informado Por este medio pido su permiso verbal para continuar con la conversación de grupo focal (FGD). Si no acepta participar en la discusión del grupo focal, siéntase libre de abandonar la sala. No estaremos en contacto con usted o su familia si no desea participar. PERMITIR A LOS MIEMBROS QUE NO ACEPTEN SALIR DE LA SALA. Certifico que he discutido los procedimientos de consentimiento con los encuestados, ____ en total, y solo continué con el consentimiento de todos los miembros presentes o que permanecen en la sala. Firma (facilitador) Fecha: NORC | LENS PROGRAM BASELINE REPORT | 121 A. Preguntas de evaluación (45 minutos) Ahora vamos a hablar sobre el programa de Lectoescritura y Nutrición Sostenible (LENS). Antes de comenzar, me gustaría establecer reglas para esta conversación. Cada uno de ustedes tendrá la oportunidad de hablar y compartir sus pensamientos. Queremos asegurarnos de que todos se sientan cómodos compartiendo sus pensamientos, incluso si son diferentes. Recuerde, no hay respuestas correctas o incorrectas. Por favor, no critique las respuestas de nadie, ni interrumpa cuando estén hablando. Por favor, no discuta nada de lo que digan fuera de esta sala. ¿Están todos de acuerdo? [Moderador: asegura que todos estén de acuerdo] Apertura 1. ¿Está familiarizado con el programa de Lectoescritura y Nutrición Sostenible (LENS)? a. ¿Es LENS como se llama este programa? b. ¿Puede decirme qué sabe de este programa? [Después de que los participantes respondan, complemente la descripción del programa (por ejemplo, explique que LENS proporcionará comidas en las escuelas y mejorará la calidad de la instrucción para la lectura, a través de la provisión de herramientas de enseñanza y materiales de lectura complementarios). c. ¿Puede describir brevemente sus expectativas del programa? i. ¿Cuáles son algunos de los beneficios que espera ver del programa? ii. ¿Cuáles son algunos desafíos que podría enfrentar el programa? Adecuación de las comidas del programa Ahora vamos a discutir el papel del proyecto relacionado con la nutrición escolar. 2. Si sus hijos recibieron una comida escolar el año pasado, ¿qué les proporcionaron? a. ¿A sus hijos les gustó la comida provista en la escuela? 3. Si sus hijos comen en la escuela, ¿de igual forma comen algo antes de ir a la escuela? Si es así, ¿qué tipo de alimentos suelen comer en casa antes de la escuela? 4. ¿Con qué frecuencia los niños compran alimentos mientras están en la escuela? ¿Qué tipo de comida? 5. ¿Contribuyeron las familias a la comida escolar el año pasado? [Sondear: por ej. Leña, ayuda con la preparación, trayendo unos cuantos tomates, etc.] ¿Cómo contribuyó su familia a la comida escolar el año pasado? NORC | LENS PROGRAM BASELINE REPORT | 122 a. ¿Cree que su familia podría seguir haciendo esta contribución? 6. Aparte de estas contribuciones, ¿participaron los padres en el programa de comidas escolares? Si es así, ¿en qué tipo de actividades participaron los padres? a. ¿Qué se podría hacer para involucrar más a los padres? ¿Cómo espera participar en el programa de comidas escolares este año? 7. ¿Sabe quién preparó las comidas escolares? ¿Son voluntarios o se les paga? 8. ¿Cuál es su opinión sobre las formas en que se preparan los alimentos en la escuela? 9. ¿Cuál es su opinión sobre la higiene? ¿Tiene alguna sugerencia sobre cómo mejorar la higiene? 10. ¿De qué manera funcionaron bien las comidas escolares? ¿Cuáles fueron los retos? 11. ¿Qué sugerencias tiene para las comidas escolares en el futuro? Oportunidades percibidas para integrar alimentos / recetas locales en las comidas escolares 12. En su escuela, ¿cómo se toman las decisiones sobre qué alimentos servir? 13. ¿Puede pensar en algún alimento producido en su comunidad que sea bueno incluir en la comida escolar? a. Si es así, ¿qué alimentos son? ¿Están siempre disponibles o solo algunas veces? ¿Cuándo? Satisfacción y adecuación de los resultados del programa 14. ¿Puede pensar en algo que su hijo le haya dicho que aprendió sobre nutrición en la escuela? a. ¿Le pidieron que pruebe una nueva práctica o actividad de nutrición en casa? 15. ¿Puede pensar en algo que su hijo le haya dicho que aprendió sobre salud en la escuela? NORC | LENS PROGRAM BASELINE REPORT | 123 16. ¿Qué tan satisfecho está con lo que su hijo ha aprendido en la escuela sobre salud y nutrición? ¿Por qué? 17. ¿Qué tipos de recursos de lectura (por ejemplo, libros, libros de texto, revistas) se proporcionaron a sus hijos como parte del programa LENS? ¿Qué tan satisfecho está con esos recursos de lectura? Gracias a todos por su participación. 18. ¿Hay algo más de lo que te gustaría mencionar que aún no hayamos hablado? C. Cierre Agradezca a todos por su participación, explique que ha sido muy útil y explique nuevamente para qué se utilizará la información que han compartido. NORC | LENS PROGRAM BASELINE REPORT | 124 KII PRODUCTORES LOCALES Gracias por tomarse el tiempo de platicar conmigo el día de hoy. Mi nombre es [nombre de entrevistador] y trabajo para Juárez & Asociados. Estamos realizando un estudio de Save the Children para mejor entender su experiencia con el programa de Lectoescritura y Nutrición Sostenible (LENS). Su ayuda es muy valiosa para nosotros, sin embargo, no necesita participar si no lo desea. El propósito de esta entrevista es comprender de mejor manera su experiencia con el programa de Lectoescritura y Nutrición Sostenible (LENS). Su participación es voluntaria y usted puede dejar de participar en el momento que lo desee. La entrevista tomará aproximadamente una hora y si usted permite, la entrevista será grabada. Se elaborará un reporte con los resultados de este estudio, sin embargo, su identidad se mantendrá confidencial asegurándonos que ninguno de sus comentarios pueda ser rastreado a usted o su organización. ¿Tiene alguna pregunta? ¿Está dispuesto a participar? Opening 1) Registrar el sexo de la persona, por observación. 2) ¿Qué edad tiene? 3) ¿Está familiarizado con el programa de Lectoescritura y Nutrición Sostenible (LENS)? Si si: a. ¿Es LENS la forma como le llaman al programa? b. ¿Podría describir brevemente el programa? [Después de que el entrevistado conteste, complementé su descripción del proyecto, en particular explique qué LENS va promover la compra de productos agrícolas locales para los programas de alimentación escolar.] i. ¿Cuáles son sus expectativas del programa? ii. ¿Qué beneficios espera del programa? iii. ¿Cuáles son los retos que como productor podría enfrontar como consecuencia de participar en el programa? Prior Experience Ahora, hablemos de su actividad agrícola. 4) ¿Cuáles son los cinco cultivos más importantes que usted produjo en los últimos 12 meses? NORC | LENS PROGRAM BASELINE REPORT | 125 a. CROP PRODUCTION Listado de cultivos mencionados por el entrevistado ÁREA DE TIERRA UTILIZADA PARA CRECER ESTE CULTIVO (TODAS LAS PARCELAS ; HECTAREAS) [Si es cero, continúe con el siguiente cultivo] Si te da el área en cuerdas, incluye la conversión de cuerdas a metros cuadrados. ¿CUÁNTO COSECHÓ? ¿CUÁNTA COSECHA VENDIÓ? ¿CUANTO RECIBIÓ POR LA VENTA DE SU COSECHA EN TOTAL? ¿CUÁNTO DE SU COSECHA, UTILIZÓ PARA CONSUMO? Cantidad Unidades 10 Unidad 11 Libras 12 KG 13 Quintales 14 Otro Cantidad Unidades 10 Unidad 11 Libras 12 KG 13 Quintales 14 Otro (QUETZALS) Cantidad Unidades 10 Unidad 11 Libras 12 KG 13 Quintales 14 Otro |__|__|__|.|__|__| |__|__|__|__| |__|__| |__|__|__|__| |__|__| |__|__|__|__|__|__| |__|__|__|__| |__|__| |__|__|__|.|__|__| |__|__|__|__| |__|__| |__|__|__|__| |__|__| |__|__|__|__|__|__| |__|__|__|__| |__|__| |__|__|__|.|__|__| |__|__|__|__| |__|__| |__|__|__|__| |__|__| |__|__|__|__|__|__| |__|__|__|__| |__|__| |__|__|__|.|__|__| |__|__|__|__| |__|__| |__|__|__|__| |__|__| |__|__|__|__|__|__| |__|__|__|__| |__|__| |__|__|__|.|__|__| |__|__|__|__| |__|__| |__|__|__|__| |__|__| |__|__|__|__|__|__| |__|__|__|__| |__|__| NORC | LENS PROGRAM BASELINE REPORT | 126 5) ¿Quién es el dueño de la tierra donde usted cultiva? a. ¿Paga por utilizar la tierra? 6) ¿Cuáles son las principales dificultades que tiene para producir? (Ejemplo. falta de ingresos para comprar semillas y fertilizantes, dificultades para transportar semillas y fertilizantes, acceso al agua, acceso a crédito, etc.) 7) ¿Cómo y dónde almacena su producto? a. ¿Cuáles son las dificultades que afronta para poder almacenar su producto? b. ¿Es habitual que se arruina/pierde parte de su producto? 8) ¿Cómo transporta su producto? a. ¿Cuál es el costo de trasportarlo y qué tan lejos/ a dónde lo transporta? b. ¿Cuáles son las dificultades que afronta para transportar su producto? (Ejemplo. Carreteras en mal estado, inundaciones, etc) 9) ¿Se encuentra actualmente registrado en la SAT (Superintendencia de administración Tributaria) como productor agrícola? a. Si está registrado, ¿cuándo se registró? ¿qué tan fácil fue registrarse? ¿Qué asistencia, si hubo, recibió durante el proceso? b. Si no está registrado, ¿por qué no? (Indague para ver si conocen el sistema) i. ¿Cree que sería útil recibir capacitación sobre cómo registrarse en el SAT? Oportunidades y Retos Asociados a la Provisión de Comida a Escuelas Ahora le hare unas preguntas sobre su posible interés en participar en LENS. 10) ¿Ha participado antes en un proyecto como LENS suministrando alimentos a escuelas? a. ¿Quién fue su contraparte del lado de la escuela? b. ¿Qué hizo en ese proyecto? (Si no entrego comida a la escuela, saltar a la pregunta 11) c. ¿Qué tipo de alimentos, con qué frecuencia y cuánto alimento proporcionó a la escuela en ese proyecto? d. ¿Qué tipo de registro o contratación se le requirió como parte de ese proyecto? e. ¿Cuáles fueron los mayores beneficios de ese proyecto para usted? f. ¿Cuáles fueron los mayores retos que enfrento usted en ese proyecto? 11) Si no ha participado, o no está participando actualmente en un programa de alimentación escolar (como productor), cuáles son las razones por las que no lo está haciendo? (e.g. regulaciones son difíciles de cumplir, su producción no es suficiente para participar, dificultades para almacenar/transportar alimentos). 12) ¿Cómo se diferencia el producir alimentos para las escuelas de producir alimentos para consumo propio o para una empresa privada? a. ¿Los mecanismos de contratación son diferentes para los productores de alimentación escolar de los que usa habitualmente? NORC | LENS PROGRAM BASELINE REPORT | 127 b. ¿Qué prefiere? ¿Por qué? c) ¿Cree que sería útil recibir capacitación sobre mecanismos de contratación de programas de alimentación escolar? 13) ¿Tiene recomendaciones sobre cómo facilitar la relación entre productores locales y programas de alimentación escolar, incluida la relación con los directores y las organizaciones de padres de familia? 14) En adelante, para participar como productor en programas de alimentación escolar, es necesario que el Ministerio de Agricultura Ganadería y Alimentación (MAGA) le expida un certificado. ¿Está familiarizado con este proceso? Si sí: a. ¿Cuenta con esta certificación del MAGA, o ha intentado obtener esta certificación? b. Si está certificado, ¿cuándo consiguió la certificación? ¿qué tan fácil fue conseguirla? ¿Qué asistencia, si hubo, recibió durante el proceso? c. Si no está certificado, ¿por qué no? (Indague para ver si conocen el sistema) 15) ¿Cuál es su opinión sobre las regulaciones gubernamentales para pequeños y medianos productores? a. ¿Qué tipo de apoyo necesita para cumplir con los requisitos de los pequeños y medianos productores agrícolas? b. ¿Cree que sería útil recibir capacitación sobre cómo enfrentar las regulaciones gubernamentales para los pequeños y medianos productores? 16) ¿Cree que es posible que algunos agricultores reciban un trato preferencial en el proceso de selección para el programas de alimentación escolar? a. En caso afirmativo, ¿tiene alguna sugerencia sobre cómo evitar esta situación? Conclusión Gracias por platicar conmigo hoy, sobre el programa de Lectoescritura y Nutrición Sostenible (LENS). 17) ¿Hay algo más que desea mencionar que aún no hayamos hablado? NORC | LENS PROGRAM BASELINE REPORT | 128 Annex II. Results for all ELI subtasks Table II.1. Reading skills by treatment status - IE Treatment Comparison Difference Letter Naming - % Correct 69.6 75.1 -5.53* Letter Naming Task 2 - % Correct 57.7 63.8 -5.68* Upper/Lowercase Pair Letter Sounds - % Correct 54.2 53.3 0.88 Letter Sounds Task 2 - % Correct 43.2 48.1 -4.84* Initial Phoneme Identification - % Correct 78.3 80.4 -2.10 Phoneme Separation - % Correct 43.4 55.8 -12.36*** Familiar Word Reading - % Correct 48.8 57.3 -8.43** Non-Word Reading - CWPM 26.2 30.4 -4.14** Listening Comprehension - % Correct 36.0 46.3 -10.30** Oral Reading Fluency - CWPM 51.0 60.7 -9.64** Reading Comprehension - % Correct 36.4 50.3 -13.88*** Dictation - % Correct 0.67 0.92 -0.25*** Note: Sample sizes are 455 for the treatment group and 463 for the comparison, although the number of observations may be smaller for some items due to item-specific missing data. The differences with respect to the sample sizes for the sociodemographic variables correspond to students that did not pass the initial instructions to answer the assessment. For the differences tests, standard errors are clustered at the school level. * p<.1 ** p<.05 *** p<.01 Table II.2. Reading skills by gender - PE Males Females Difference Letter Naming - % Correct 75.9 74.2 1.71 Letter Naming Task 2 - % Correct 61.9 59.7 2.22 Upper/Lowercase Pair Letter Sounds - % Correct 56.1 52.8 3.25 Letter Sounds Task 2 - % Correct 42.4 41.0 1.40 Initial Phoneme Identification - % Correct 79.1 74.9 4.20* Phoneme Separation - % Correct 45.7 44.3 1.38 Familiar Word Reading - % Correct 51.7 52.0 -0.29 Non-Word Reading - CWPM 28.0 27.7 0.33 Listening Comprehension - % Correct 38.1 39.5 -1.42 Oral Reading Fluency - CWPM 52.9 54.5 -1.59 Reading Comprehension - % Correct 40.5 39.2 1.26 Dictation - % Correct 0.74 0.72 0.02 Note: Sample sizes are 332 for males and 336 for females, although the number of observations may be smaller for some items due to item-specific missing data. The differences with respect to the sample sizes for the sociodemographic variables correspond to students that did not pass the initial instructions to answer the assessment. For the differences tests, standard errors are clustered at the school level. * p<.1 ** p<.05 *** p<.01 NORC | LENS PROGRAM BASELINE REPORT | 129 Table II.2. Reading skills by municipality - PE Chajul Cotzal Cunén Nebaj Sacapulas Uspantán Letter Naming - % Correct 74.1 79.7 68.2 79.4 73.9 76.1 Letter Naming Task 2 - % Correct 53.0 67.1 56.2 64.5 56.5 66.3 Upper/Lowercase Pair Letter Sounds - % Correct 43.3 53.8 63.0 50.3 55.4 58.0 Letter Sounds Task 2 - % Correct 26.7 37.8 51.6 35.0 46.8 52.0 Initial Phoneme Identification - % Correct 61.8 79.0 82.8 71.9 81.1 85.9 Phoneme Separation - % Correct 33.0 40.4 43.3 45.9 57.3 54.2 Familiar Word Reading - % Correct 40.3 57.2 53.6 51.1 47.3 59.0 Non-Word Reading - CWPM 21.4 32.4 26.8 28.0 27.6 30.6 Listening Comprehension - % Correct 24.7 43.4 31.9 42.0 44.9 49.2 Oral Reading Fluency - CWPM 37.4 60.5 55.9 52.9 48.5 63.2 Reading Comprehension - % Correct 23.7 43.7 33.6 42.6 46.6 52.3 Dictation - % Correct 0.43 0.80 0.62 0.78 0.85 0.96 Note: Sample size is 668, although the number of observations may be smaller for some items due to item-specific missing data. The difference with respect to the sample sizes for the sociodemographic variables correspond to students that did not pass the initial instructions to answer the assessment. NORC | LENS PROGRAM BASELINE REPORT | 130 Annex III. Average reading fluency and reading comprehension, by language most spoken at home 37 52 55 74 82 0 20 40 60 80 Correct Words Per Minute (ORF Sub-Task) 0-2 3-4 5-6 7-8 9-11 Reading Comprehension Raw Score Spanish 35 58 66 64 59 0 20 40 60 80 Correct Words Per Minute (ORF Sub-Task) 0-2 3-4 5-6 7-8 9-11 Reading Comprehension Raw Score Kiche NORC | LENS PROGRAM BASELINE REPORT | 131 34 47 60 68 69 0 20 40 60 80 Correct Words Per Minute (ORF Sub-Task) 0-2 3-4 5-6 7-8 9-11 Reading Comprehension Raw Score Ixil