1 FINAL REPORT Program Evaluation Food For Education, Phase II Huehuetenango, Guatemala Comercializadora Quetzalteca de Bienes and Servicios S.A. Final Evaluation for FY 2013 McGovern-Dole Food for Education Phase II Project in Huehuetenango, Guatemala Program: McGovern-Dole International Food for Education and Child Nutrition Agreement Number: FFE-520-2013/034-00 Funding Year: Fiscal Year 2013 Project Duration: 2013-2016 Implemented by: TechnoServe Evaluation Authored by: Comercializadora Quetzalteca de Bienes and Servicios (EQSA) S.A. DISCLAIMER: This evaluation was conducted by an independent third party firm. The views expressed in this report do not necessarily reflect the views of the United States Department of Agriculture or the United States Government. Accessibility Note: An accessible version of this report can be made available upon request by emailing FAS.monitoring.evaluation@usda.gov. 2 Glossary ATC Administrative Technical Coordinator CNB National Base Curriculum COCODE Community Development Council CPF Parents Council CSB Corn Soy Blend DIGEDUCA General Bureau of Educational Evaluation and Research DMM Municipal Woman’s Bureau ENCOVI National Survey of Living Conditions ENSMI National Maternal Child Health Survey FFE Food For Education FNS Food and Nutrition Security IDH Human Development Index INE National Statistics Institute M&E Monitoring and evaluation MOE Ministry of Education MTE Mid-term Evaluation PO Parent Organization PCI Project Concern International RENAP National Population Registry SESAN Secretariat of Food and Nutrition Security SPSS Statistical Package for the Social Sciences UNDP United Nations Development Program USDA United States Department of Agriculture WE Women Empowered 3 Table of Contents 1. Executive Summary .................................................................................................. 6 1.1 Introduction .............................................................................................................................. 6 1.3 Results ...................................................................................................................................... 6 1.3.1 Contribution to Feed the Future .............................................................................................. 6 1.3.2 Improvement of literacy in school age children ...................................................................... 6 1.3.2.1 Improve Classroom Attention .......................................................................................... 7 1.3.2.1.1 Reduce short term hunger.................................................................................................. 7 1.3.2.1.1. Increased access to food ................................................................................................... 7 1.3.2.2 Improvement in school attendance ....................................................................................... 7 1.3.2.2.1 Reduction of health related absences ................................................................................ 7 1.3.2.2.2 Improve school infrastructure ........................................................................................... 8 1.3.2.2.3 Increased school enrollment .............................................................................................. 8 1.3.2.2.4 Increase community understanding of the benefits of education ...................................... 8 1.3.3 Increase in use of diet and health practices ...................................................................... 8 1.3.3.1 Increase in knowledge of health and hygiene practices ................................................... 8 1.3.3.2 Increase in knowledge of safe food preparation and storage practices ............................ 9 1.3.3.3 Increase in knowledge of nutrition ................................................................................... 9 1.3.3.4 Increase in target schools with improved equipment for food preparation and storage . 10 1.3.4 Empowerment of women ............................................................................................... 10 1.3.5 Evaluation of the sustainability tool ............................................................................... 10 2. Introduction and Objectives of the Evaluation ........................................................ 11 3. Context of the department of Huehuetenango ......................................................... 11 3.1 Population ................................................................................................................................ 12 3.2 Food and Nutrition Security (FNS) .......................................................................................... 12 3.3 Education .................................................................................................................................. 13 4. General information for the program municipalities .............................................. 14 5. Evaluation methodology .......................................................................................... 14 5.1 Document Review ........................................................................................................................ 14 5.2 Determination of the sample .................................................................................................. 14 5.3 Instruments ............................................................................................................................. 15 5.4 Methodology for Measuring Sustainability ............................................................................ 17 6. Results ..................................................................................................................... 18 6.1 Improvement of literacy of school age children (MGD SO1) ................................................ 18 6.1.1 Improve classroom attention (MGD 1.2) ....................................................................... 21 4 6.1.2 Improvement in school attendance (MGD 1.3) .............................................................. 25 6.2 Increase the use of dietary and health practices (MGD SO2) ................................................ 29 6.2.1 Improve knowledge of health and hygiene practices (MGD 2.1) .................................. 32 6.2.2 Increase knowledge of safe food preparation and storage practices (MGD 2.2) ........... 34 6.2.3 Increase nutrition knowledge (MGD 2.3) ...................................................................... 35 6.2.4 Increase access to adequate food preparation, tools and storage equipment (MGD 2.6) 37 6.3 Key Results ............................................................................................................................ 38 6.3.1 Increase government support (MGD 1.4.3) .................................................................... 38 6.3.2 Increase the commitment of Local Organizations and Community Groups (MGD 1.4.4) ... 39 6.4 Sustainability .......................................................................................................................... 42 6.5 Evaluation of the Monitoring and Evaluation (M&E) system ............................................... 44 6.6 Lessons learned ...................................................................................................................... 45 6.7 Synthesis of accomplishment of MTE recommendations ...................................................... 46 7. Conclusions ............................................................................................................. 48 8. Recommendations ................................................................................................... 52 Annex 1. Comparison of Results Indicators ................................................................... 56 Annex 2. Information about the Consultant Team ......................................................... 60 Annex 3. Qualitative Evaluation Tools .......................................................................... 61 Annex 4. Quantiative Evaluation Tools ......................................................................... 66 5 Index of Tables Table 1. Nutritional Status in FFE Municipalities ................................................................................. 12 Table 2. Initial and Final Enrollment Records FFE Municipalities ....................................................... 13 Table 3. Summary of general information for FFE municipalities ........................................................ 14 Table 4. Distribution of the sample by municipality .............................................................................. 15 Table 5.Criteria for classification of reading comprehension ................................................................ 16 Table 6. Reading Comprehension .......................................................................................................... 18 Table 7. Results of reading tests ............................................................................................................. 18 Table 8. Reading Test Results by Sex and Grade .................................................................................. 19 Table 9. Number of people who benefit directly/indirectly from the interventions funded by USDA .. 20 Table 10. Results of indicators that contribute to Feed The Future ....................................................... 20 Table 11. Level of student hunger .......................................................................................................... 21 Table 12. Percentage of students consuming daily meals at school ....................................................... 22 Table 13. Percentage of students who are identified as being inattentive in class ................................. 24 Table 14. Causes of school inattention ................................................................................................... 24 Table 15. Indicator for Improved school attendance .............................................................................. 25 Table 16. Results for the absences for illness indicator ......................................................................... 25 Table 17. Students benefiting from infrastructure .................................................................................. 26 Table 18. Percentage of students enrolled in school as a result of USDA support ................................ 27 Table 19. Number of parent members of a PO....................................................................................... 28 Table 20. Results of benefits of education indicator ............................................................................. 29 Table 21. Indicator for schools with clean latrines ................................................................................ 30 Table 22. Indicator for soap and water in a hand washing station ......................................................... 30 Table 23. Indicator for schools with clean kitchens and utensils ........................................................... 31 Table 24. Results for health / hygiene practices indicator ..................................................................... 32 Table 25. Results for indicator on sources for health information ......................................................... 33 Table 26. Comparison of health information sources ............................................................................ 34 Table 27. Indicator for Nutrition Practices ............................................................................................. 34 Table 28. Results for food preparation indicator .................................................................................... 35 Table 29. Results of indicator for sources of nutrition information ....................................................... 36 Table 30. Indicator for better food preparation and storage ................................................................... 37 Table 31. Increase in governmental support .......................................................................................... 38 Table 32. Increase in participation of local organizations ...................................................................... 39 Table 33. Results from application of sustainability tool in FFE schools .............................................. 43 Table 34. PCI’s response to MTE recommendations ............................................................................. 46 Index of Illustrations Illustration 1. Location of Huehuetenango ............................................................................................. 11 Illustration 2. Reading Achievement lectura .......................................................................................... 13 Illustration 3. Indicator for hungry or very hungry ................................................................................ 21 Illustration 4. Percentage of absences due to illness .............................................................................. 26 Ilustration 5. Enrollment of elementary students ................................................................................... 27 Illustration 6. Benefits of elementary education ................................................................................... 29 Illustration 7. Health and hygiene practices ........................................................................................... 32 Illustration 8. Sources of health information .......................................................................................... 33 Illustration 9. Good nutrition practices................................................................................................... 35 Illustration 10. Food storage and safety precautions .............................................................................. 36 Illustration 11. Sources for nutrition information .................................................................................. 36 Illustration 12. Sustainability situation of evaluated schools ................................................................. 42 6 1. Executive Summary 1.1 Introduction The Food For Education (FFE) program has been implemented by Project Concern International (PCI), in the department of Huehuetenango, specifically in three municipalities of the Mam ethnic group: San Gaspar Ixchil, Cuilco and San Pedro Necta; and three municipalities of the Q´anjob´al ethnic group: Santa Eulalia, Santa Cruz Barillas and San Pedro Soloma. Funding for this was provided by the United States Department of Agriculture (USDA). The program seeks to improve literacy of school age children and increase the use of good health and nutrition practices. A total of 58,044 direct beneficiaries were reached which included: 38,020 students, 1,492 teachers, 1,202 collaborating mothers, 1,470 members of Parent Organizations (PO), 2,211 Women Empowered, 13,625 parents and 24 government officials. The second phase of this PCI program ran from 2013 to 2016. This external evaluation was carried out to evaluate the expected and unexpected results, its impact, sustainability and lessons learned in order to systematize learning for future programs that will be implemented in similar contexts, and in particular the FFE third phase program (EDUCAMOS) currently being implemented by PCI. To achieve the purposes of the evaluation, quantitative measurements were taken to determine accomplishment of the results indicators and to have a comparison with what was seen in the baseline study and mid-term evaluation (MTE). This information is complemented with a qualitative evaluation that considered the opinions of the key stakeholders in the educational community, Municipal Councils, and Ministry of Education (MOE) staff, including the Administrative Technical Coordinators (ATC) and PCI implementation staff. A specific tool was used to measure sustainability. The Women Empowered (WE) initiative was also analyzed. 1.3 Results 1.3.1 Contribution to Feed the Future This is composed of three indicators related to a) Number of associations formed (Municipal Sustainability Commissions) (6); b) Number of beneficiaries of USDA assistance (58,044), and c) people trained in child health and nutrition (35,057). All three results exceeded the final targets. 1.3.2 Improvement of literacy in school age children The results obtained reflect that girls and boys greatly exceeded the values found in the MTE, with a positive differential of 35.9% for girls and 38.1% for boys. Nevertheless, with regard to the target established at the baseline there was a differential of -9.5% in girls and -11.5% in boys. Thus, the established target was not achieved. The main reason was that the program had 7 set very high targets from the baseline: 61% for girls and 57% for boys, which was not realistic when comparing with data from the General Bureau of Educational Evaluation and Research (DIGEDUCA), which for 2014 established a national average for reading achievement of 40.4%. The average for Huehuetenango was 23.84% and the average for the FFE municipalities was 16.81%. 1.3.2.1 Improve Classroom Attention 34% of students were identified as being inattentive in class and the final target was 54%, which means that the target was exceeded by 20 percentage points. Of this total, 15% show distraction by looking around, 8% by doing something else, 8% talk to their friends, 2% do something to distract the class, and 1% showed signs of tiredness. In the interviews with directors, teachers and parents they stated that the food provided by PCI with contributions from the MOE and the parents themselves, is contributing directly to children paying more attention in class and improving their performance. 1.3.2.1.1 Reduce short term hunger The indicator measures the percentage of students who are hungry or very hungry1 . The final target was 5% and in the final evaluation this 5% was achieved. The children who responded that they were hungry and the 1% who said they were very hungry were specific to certain locations and may be the result of circumstances on the day that data was collected in a community of San Pedro Soloma, where not enough food was prepared so some students were still hungry. 1.3.2.1.1. Increased access to food The indicator for this result measures the percentage of students consuming daily meals at school. The target of 100% was achieved; all the schools had permanent consumption of school breakfasts. 1.3.2.2 Improvement in school attendance This indicator measures the percentage of boys and girls who have regular school attendance (at least 80% attendance). The school records show a 97.3% attendance for girls and 98% for boys, which exceeds the target by 7.3 and 8 percentage points, respectively. 1.3.2.2.1 Reduction of health related absences A total of 13.1% of the students stated that they had missed class for any reason the week prior to the evaluation, nevertheless, only 8.6% of the cases were due to illness, with 2.5% due to family travel and 2%, for other reasons. The indicator is specific for illness and the final target 1 Established using evaluation survey tools, see Annex 3 8 was 15%, thus the indicator was exceeded by 6.4 percentage points. In the focus groups, parents indicated that hygiene habits are being applied in the home which could reduce the frequency of illnesses, especially those caused by lack of personal hygiene. 1.3.2.2.2 Improve school infrastructure Infrastructure construction or rehabilitation exceeded the target. According to program records, 9,126 students benefited with kitchens (target: 6,390) and 15,711 with water systems (target: 8,520). 1.3.2.2.3 Increased school enrollment This indicator sought to increase school enrollment by 1%, and was not achieved. Reduction in enrollment is a national phenomenon. Data from the six FFE municipalities obtained from the MOE show that from 2013 to 2015 there was a reduction of -4.5% for boys and -3.85% for girls. 1.3.2.2.4 Increase community understanding of the benefits of education The final target was 1,150 parents who are members of the PO and this was exceeded by 320 with an achievement of 1,470 parents. The focus groups with parent leaders showed that the majority of members have a good understanding of the program and are willing to give their time, effort and in some cases, economic resources to support program activities and the development of the school. In the majority of cases there is a cordial relationship between the PO and teaching staff and they support school activities. Parents’ identification of the importance of education has increased significantly. The Final Evaluation shows that 84.6% recognize at least three benefits of elementary education, compared to 13.1% in the MTE. 1.3.3 Increase in use of diet and health practices The indicators for this strategic objective were achieved and exceeded. Significant improvement was noted when compared to the MTE. The measurement shows that the target for clean latrines was exceeded by 26 percentage points, in soap and water at hand washing stations by 36 percentage points and in clean kitchens and utensils by 30 percentage points. These high results in such a short time seem questionable, but they were justified because the program strongly promoted improved activities to reach the targets based on the MTE recommendations. This was ratified by teachers and parents. 1.3.3.1 Increase in knowledge of health and hygiene practices Parents in the communities can identify at least three health and hygiene practices and can identify at least one local source of information on this topic. Both indicators were exceeded with 18 and 29 percentage points, respectively, compared to the target. 9 There was a significant increase in identification of sources of information. They mentioned places such as health centers, convergence center/health post, the church and hospitals. In the baseline and MTE there was a category called “Other”. In the Final Evaluation, this category showed an increase compared to the MTE and 63.5% selected PCI as a source of information and the home (family) as the second source for this category with 31.9%. In the MTE, 42.9% of parents who marked this category mentioned PCI as a source of information. To promote sustainability, knowledge of other permanent sources at the local level should continue to be reinforced. 1.3.3.2 Increase in knowledge of safe food preparation and storage practices The indicator for this result exceeded expectations by 48.6 percentage points, as 88.6% of mothers identified at least three important nutrition practices (target: 40%). The collaborating mothers are identified with the program and practice what they learn in the trainings. The PO stated in the focus groups that food management is one of the experiences that will last beyond the current program, given that it has allowed them to organize themselves to be present in the school, to provide support to the collaborating mothers and manage the products from the MOE. The contribution of vitamins and minerals, and energy foods are the most mentioned practices with 97.4% and 89.5%, respectively, followed by protein foods, 83.2%. The directors mentioned that at the beginning of the program they had difficulty with variety in the menus and the children got tired of eating the same thing. They noted that the school fairs are a good source of information as in these events they have the opportunity to exchange recipes and knowledge in order to prepare different menus using the same products. 1.3.3.3 Increase in knowledge of nutrition The percentage of students (and their parents) who can identify at least one local source of information about nutrition and diet was exceeded by 28.3 percentage points. The people interviewed mentioned the health center most frequently (43.3% of the responses), followed by school with 40.7%. The category of “others” (77.2% of responses) included PCI as a source of information. The percentage of mothers who can identify at least three key practices for safe food preparation and storage exceeded the expected levels as the final target was 80% and the result is 99.1%. This high accomplishment was confirmed in the on-site observations where the kitchens were in compliance with the standards: people who had direct contact with food were wearing hairnets, used the PCI apron, trash cans were available, the food was processed in a hygienic manner in plain view and the utensils were washed and dried. There might be some bias due to knowing that the evaluation team was coming although parents commented in the focus groups that the program has helped them adopt adequate food management practices. 10 1.3.3.4 Increase in target schools with improved equipment for food preparation and storage The percentage of schools with improved food preparation and storage stayed the same, with 100% of schools in both the final evaluation and MTE. All the schools visited demonstrated good food preparation skills and although the utensils show wear, they are functional and are kept hygienically. In general the storage rooms are in good condition and adequate based on the particular conditions of each place. 1.3.4 Empowerment of women Six communities with WE confirmed that women in groups are adopting a savings habit, and they agree that this is the activity they like most because they now have funds available to use in any emergency or to invest in a productive activity. Some of the surveyed groups are beginning to award loans among themselves with mutually agreed upon interest rates that are usually around 5%. According to program data, 59% of groups are awarding loans, a significant increase from 2014, where only 29% of groups were awarding loans. In the surveyed groups, few loans are destined for education and most are focused on income generating projects. In the six groups it was confirmed that political empowerment was the weakest component. The women stated that in only a few cases are they part of the PO. In general they do not participate in decision making organizational structures because they say they don’t have time and because the men do not offer opportunities for their participation. 1.3.5 Evaluation of the sustainability tool The application of the sustainability tool in the 28 schools showed that the condition that scored highest (or showed greatest sustainability) was that of the hygiene practices. The second condition with a high score was adequate infrastructure, where PCI’s efforts, in conjunction with the community and municipalities, were significant for improving kitchens and water systems. Literacy improvement had medium to low scores which means that direct actions are needed to promote improvement of ways to address this topic, using appropriate methodologies for the context of each area. The school breakfast also received medium to low scores. The condition that scored lowest was municipal and community support, which highlights the need to continue strengthening local community leadership so that they are able to request and obtain the necessary municipal support and integrate all the members of the community in the effort to have a self-sustaining development program. 11 2. Introduction and Objectives of the Evaluation PCI implemented the second phase of the FFE program from 2013-2016, with funding from the US Department of Agriculture (USDA). The program area is in the department of Huehuetenango, which is the second most populated department of the country, representing 7.9% of the national population. The program has two strategic objectives:  Improve the literacy of school age children  Increase the use of health and nutrition practices This final external evaluation was conducted to evaluate the achievements, results, and program impacts through the following objectives:  Evaluate the relevance, efficacy, efficiency, impact, and sustainability of the activities  Evaluate the achievement of expected results based on the Results Framework  Analyze the unexpected results that have contributed to transforming the communities  Identify the lessons learned during the life of the program  Suggest recommendations for future programs with similar contexts This document reports on the quantitative and qualitative results of the program’s Final Evaluation, which is based on an analysis of accomplishment of the indicators, observation of the physical infrastructure, analysis of the support tools and especially, perceptions from the beneficiaries regarding the changes that have been obtained to date, and whether there has been community impact or not. The evaluation document is organized through a description of the context of the target department and municipalities, the evaluation methodology is then discussed, followed by a description of the evaluation results by indicator. Finally, conculsions and recommendations are presented. 3. Context of the department of Huehuetenango Huehuetenango is located in the northwestern region of Guatemala. It has 33 municipalities and covers 7,400 km², which is 6.8% of the country. Its topography is very irregular. The Cuchumatanes Mountains run through the department, reaching the highest elevation in Central America. The climate is very diverse, with both hot and cold areas. Illustration 1. Location of Huehuetenango 12 3.1 Population According to the Nation Population Registry (RENAP 2016), Guatemala’s population is estimated at 17 million people, with Huehuetenango representing 7.9% of the total, and an annual population growth rate of 2.4% (52.5% female and 47.5% male). Of those, 47.4% are under 18 years of age, 48.1% are between 19 and 64, and 4.5% of the population are over 64. According to the Human Development Report (HDR 2015)2 , Huehuetenango has the highest concentration of ethno-linguistic diversity in the country with 10 ethnic groups: Akateko, Awakateko, Chalchiteko, Chuj, K´iche´, Mam, Popti´, Q´anjob´al, Tektiteko and Mestizo. Indigenous groups make up 56% of the population and 44% are mestizos. Seventy percent of the population lives in rural areas. 3.2 Food and Nutrition Security (FNS) According to the Fourth National Height Census published by the Secretariat of Food and Nutrition Security (SESAN 2015), Huehuetenango is in fourteenth place at the national level for chronic malnutrition in children from six to nine years of age. Ten municipalities are rated as having ‘very high’ food insecurity. The following table shows the municipalities that are covered by the FFE program, in descending order by category of nutritional vulnerability. Table 1. Nutritional Status in FFE Municipalities3 2 Published by the UN Development Program (UNDP) 3 Fourth National Height Census 2015. SESAN Municipality Total children evaluated Normal Stunting Nutritional Vulnerability Category Total children Normal % Total Moderate Severe Total children with stunting % Total children Moderate stunting % Total children Severe Stunting % Santa Eulalia 1,350 385 28.5 965 71.5 623 46.2 342 24.3 Very high San Gaspar Ixchil 329 114 34.6 215 65.4 156 47.5 59 17.9 Very high Santa Cruz Barillas 3,407 1,208 35.5 2199 64.5 1,446 42.4 753 22.1 Very high San Pedro Necta 1,172 524 44.7 648 55.3 446 38.1 202 17.2 Very high San Pedro Soloma 1,755 912 52 843 48 641 36.5 202 11.5 Very high Cuilco 2,078 1,159 55.8 919 44.2 969 33.5 223 10.7 High 13 3.3 Education The 2015 HDI showed that Huehuetenango had an education index of 0.461, which places the department in third from last place at the national level, above Alta Verapaz and Chiquimula. Illiteracy has declined: in 2009, the rate was 28.4%, while in 2013 it was 23.3%. According to the MOE, the number of students declines from the beginning to the end of the school year for elementary education. This is observed in FFE municipalities in data from 2014. Table 2. Initial and Final Enrollment Records FFE Municipalities4 Municipality Enrollment records 2014 Initial Final Difference Cuilco 9,943 9,484 -459 San Pedro Necta 6,211 6,085 -126 San Pedro Soloma 9,486 8,828 -658 Santa Eulalia 8,161 7,915 -246 Santa Cruz Barillas 18,341 17,658 -683 San Gaspar Ixchil 1,597 1,495 -102 According to 2014 DIGEDUCA data, only 40.4% of sixth grade students in the nation displayed a satisfactory reading level. Huehuetenango is below the national level at 23.84%. Illustration 2 presents a comparison of the FFE municipalities. San Pedro Necta is 1.41% above the national average with a total of 41.81% and exceeds Huehuetenango’s average by 17.97%, meaning it has the highest sixth grade reading level of these municipalities. The remaining municipalities are below the departmental average, with Cuilco the next highest at 18.92%, followed by Soloma with 17.66%, and Santa Cruz Barillas and Santa Eulalia with the lowest rates at 5.26% and 5.56%, respectively. The average of these six FFE municipalities is 16.81%, lower than the mean for Huehuetenango, indicating a significant deficiency in the reading ability of sixth grade students. 4 Source: Enrollment data from the MOE 2014 0 10 20 30 40 50 Source. From evaluation with DIGEDUCA 2014 data Achievement Percentage Illustration 2. Reading Achievement lectura 14 4. General information for the program municipalities Table 3. Summary of general information for FFE municipalities5 Information FFE municipalities San Andrés Cuilco San Pedro Necta San Gaspar Ixchil San Pedro Soloma Santa Eulalia Santa Cruz Barillas Predominant Ethnicity Mam / Mestizo Mam Mam Q´anjob´al Q´anjob´al Q´anjob´al Distance from department capital (Km) 90 55 44 60 67 150 Location in the department of Huehuetenango North West region Southern region Southern region Northeast region Northeast region Northern region Access: Type and status of roads Paved in adequate condition Paved in adequate condition Paved in adequate conditions Paved in poor condition Paved in poor condition Paved and unpaved in poor condition Projected population to 2016 (INE) 64,233 inhabitants 36,837 inhabitants 7,363 inhabitants 56,748 inhabitants 55,194 inhabitants 177,353 inhabitants Population Men 46% 49.7% 45.3% 46.1% 49.4% 51.1% Population Women 54% 50.3% 54.7% 53.9% 50.6% 48.9% # elementary schools 105 44 12 50 56 190 5. Evaluation methodology 5.1 Document Review EQSA conducted a desk study to review existing information provided by PCI to analyze the program structure based on the Results Framework, reports for USDA, beneficiary data bases, baseline and MTE reports. Coordination meetings were held with PCI staff, before, during and after the fieldwork stage of the evaluation. 5.2 Determination of the sample The following formulas were used to determine the sample: Where n is the size of the sample, E is the margin of error, N is the size of the population, r is the positive variability of interest in the evaluation and Z (c/100) is the critical value for the 5 Source: From evaluation, with data from Municipal Development Plans 2015 15 confidence level C. This calculation is based on the normal distribution. To select the number of schools, a percentage weight was given to each of the municipalities based on the number of participant schools, then the approximation was made to know the number assigned to each. The results are presented below: Table 4. Distribution of the sample by municipality6 Municipality Schools Percentage weight Score Sample San Gaspar Ixchil 12 0.0412371 1.15464 2 San Pedro Necta 39 0.1340206 3.75258 4 Cuilco 91 0.3127148 8.75601 9 San Pedro Soloma 34 0.1168385 3.27148 3 Santa Eulalia 45 0.1546392 4.32990 4 Santa Cruz Barillas 70 0.2405498 6.73540 6 Totals 291* 28 The following sample was defined with a 95% confidence level for all sources:  381 students for surveys, achieved 359  28 directors, achieved 28  84 teachers, achieved 80  296 fathers and/or mothers for surveys, achieved 351  84 members of the PO, achieved 119  28 focus groups with parents, achieved 28  84 Collaborating mothers, achieved 116 For the number of schools, the same number was used as in the MTE. To define the schools for evaluation, ALAZAR.INFO software was used for randomization. The use of this system made it possible to establish the names of the 28 first choice schools and six replacements, which did not end up being used. 5.3 Instruments Data collection was carried out using five techniques: i. Surveys: with closed questions were used to measure the results indicators with students, parents and collaborating mothers who work as volunteers for food preparation. The interviewers were native speakers of either Mam or Q´anjob´al, to ensure effective communication as they knew the linguistic idioms of each location and this promoted confidence in those being interviewed. The same tools were used as in the MTE to ensure an 6 Of the 294 schools in the program universe it was determined that the 291 schools that had been in the program since 2014 would enter into the drawing 16 objective comparison of the indicators. Surveys were conducted using paper forms, and entered using double data entry for tabulation and analysis in SPSS. ii. Focus groups: This technique uses meetings with a group of individuals gathered to talk in a guided manner about different topics related to the project evaluation. The data collection method used the targetplan tool and translators were used when needed. The tool had to be adapted because the participants were unable to write or structure ideas in a short time period. Those who participated in these meetings were members of the PO, parents, members of Women Empowered groups and PCI staff. iii. Semi-structured Interviews: these were held with Directors and a sample of teachers from the 28 schools that were evaluated, with the ATC, members of local government, MOE staff in Huehuetenango and USDA Washington staff. iv. Observation: This was used to determine the functionality of the infrastructure and gardens in the schools, as well as to measure indicators related to school hygiene, the conditions of latrines, and food preparation. v. Secondary sources: This included a review of related documents, including the data base or statistics for indicators that are not measured in the field. Attendance records were reviewed in the schools. The field work was carried out by two teams that functioned autonomously: one in the Mam area and the other in the Q´anjob´al area. The teams rotated (except for the translators) so each could have a complete view of both regions. To determine the reading ability of students who had completed sixth grade, students were presented with reading material based on the expected level for sixth grade students and they were asked to read out loud. Their reading was recorded. The reading evaluation for these children focused on their ¨reading ability¨ (the child’s competency in reading) and this was analyzed in three ways: speed, accuracy and comprehension based on the indicators established by the Ministry of Education for completion of sixth grade. The same formula used in the baseline and MTE was used: Reading ability = speed (0.1) + accuracy (0.2) + comprehension (0.7)7 Table 5.Criteria for classification of reading comprehension8 Classification Range Standard 81 to 100 points Medium 61 to 80 points Low 41 to 60 points Very low 0 to 40 points 7 Technical report from MTE 8 Source: Baseline 17 The test was applied according to the definition of the indicator, with boys and girls who had completed sixth grade. As in the Baseline and MTE, the test was given to students who were in the first year of middle school or who were not studying. Statistical significance of comparisons of results of literary tests between the baseline, MTE and final evaluation was established using Pearson’s chi-squared tests, considering only tests carried out with students who had completed sixth grade. 5.4 Methodology for Measuring Sustainability This evaluation used the instrument FFE II Sustainability Evaluation Tool9 for which was provided by PCI to measure and determine the situation of five specific conditions and activities that promote sustainability: i. School breakfast provided daily to 100% of students: Preparation, daily, contributions and increase in non-USDA resources (parents, MOE, gardens), planned purchases of foods and timely disbursements by the MOE. ii. Improvement of student literacy: Reading in class and at home, community promotion of reading, availability of material, monitoring of teacher attendance and parent attendance at meetings. iii. Hygiene and nutrition practices for school performance: Hand washing, teachers teaching, parents with knowledge about health, hygiene and nutrition practices and school with clean interior and exterior. iv. Adequate infrastructure: Kitchen, store room, latrines and hand washing stations exist and are functional and safe water is available for consumption. v. Municipal and community support: Functioning of empowerment groups, increase in municipal contributions, sustainability commissions with a functioning work plan and institutional/municipal support for the school. For each of the five conditions, five activities that are considered key for achievement of the condition are evaluated, thus a total of 25 activities are evaluated. Each activity evaluated is assigned a value of four points, giving a possible total of 100 points. In the communities evaluated, the instrument was applied and each activity was rated with 0 to 4 points. The total was presented over 100 points, based on observations in each school for the five conditions and 25 activities. For each municipality’s score, the average of the communities’ score for each activity was calculated and totaled to determine the score relative to the possible 100 points. 9 Annex 5 18 6. Results 6.1 Improvement of literacy of school age children (MGD SO1) Table 6. Percentage of students, who at the end of sixth grade have grade level reading comprehension as defined by national norms, in schools supported by USDA10 Result Baseline MTE Final Evaluation Final target Difference from target (percentage points) Girls 8.8% 15.6% 51.5% 61% -9.5 Boys 2.5% 7.4% 45.5% 57% -11.5 As part of the findings and analysis, the following should be noted: i. The results from the Baseline and MTE noted in the indicators table is the average score (on a scale of 100 points). In this regard, a very high target was set from the time of the baseline. ii. The indicator corresponds to the percentage of students who have obtained the standard score (81 to 100 points). iii. While the comparisons of results between a) the MTE and final evaluation, and b) the baseline and final evaluation are statistically significant (p<0.05), comparison of results between the baseline and MTE are not statistically significant (p>0.05) The percentages of students who have completed elementary school and are in middle school or who are not studying are presented in the following score categories: Table 7. Results of reading tests Rating Frequency % Cumulative % Very low 8 6.6 6.6 Low 7 5.8 12.4 Medium 47 38.8 51.2 Standard 59 48.5 100.0 Total 121 100.0 The resulting 48.5% of students in FFE municipalities who reached the standard rating shows improvement from the national and departmental level, given that based on a study carried out by the “Entrepreneurs for education” organization at the national level, 40.40% of sixth grade students achieved the standard qualification in reading ability. Additionally, students in the FFE municipalities showed a little more than double the improvement with relation to the departmental mean (23.84%). This also exceeds the mean of the six municipalities (16.81%), based on 2014 DIGEDUCA data. 10 Reading tests with 121 students who are in first year of middle school or have completed sixth grade and are not studying. This number is similar to the Baseline (125) tests. In the MTE, 247 tests were given 19 A greater percentage of the students who continue their studies achieve the standard rating (81 to 100 points) required by the MOE. Of these students, 70% of girls achieved the standard rating, and 51.3% of boys did the same. Of the students who do not continue studying, 23.1% of the girls achieve the standard rating while 31.3% of the boys do so. Table 8. Test Results by Sex and Grade Sex/Rating Study Situation Currently Total Not Studying Currently studying Females Very low Frequency 5 0 5 % within grade 19.2% 0.0% 7.6% Low Frequency 3 1 4 % within grade 11.5% 2.5% 6.1% Medium Frequency 12 11 23 % within grade 46.2% 27.5% 34.8% Standard Frequency 6 28 34 % within grade 23.1% 70.0% 51.5% Total Frequency 26 40 66 % within grade 100% 100% 100% Male Very low Frequency 3 0 3 % within grade 18.8% 0.0% 5.5% Low Frequency 1 2 3 % within grade 6.3% 5.1% 5.5% Medium Frequency 7 17 24 % within grade 43.8% 43.6% 43.6% Standard Frequency 5 20 25 % within grade 31.3% 51.3% 45.5% Total Frequency 16 39 55 % within grade 100% 100% 100% Part of the success in reading comprehension improvement in FFE municipalities, compared to the national, departmental and municipal mean (DIGEDUCA, 2014 data), is due program activity, as shown in the qualitative data analysis. Interviews with directors, teachers and parents determined that this success was a result of the strategy of reading for 30 minutes daily in school, followed by giving parents the responsibility of supporting and motivating their daughters to read at home for 15 to 20 minutes. In all 28 schools evaluated, only 3.57% shared that the children do not read at home. Parents also mentioned that in some cases the children first help the family with household chores (i.e. carrying water from its source to the home) before doing homework or reading. Some parents’, especially mothers’, willingness to help with reading and writing indicate that 20 they recognize the importance of learning Spanish at school. At times, when there is a high load of school work and household chores, they prefer to let the children play before reading because they also recognize the importance of leasure time. In interviews with the teachers, there is a tendency to improve teaching quality in addition to reading results. In school, everyone has a minimum of 30 minutes of reading. The time is dedicated to understanding what they read and not speed. Sometimes they alternate days between pure reading and dictation so that reading skills are combined with writing. In the interviews with directors, it was noted that all the schools except the Parochial School of Santa Eulalia have books from the MOE for the children to use. The books tend to not be sufficient or adequate for the ages of the children. The Bible and newspapers are regularly read. In the majority of schools, books are not loaned out for children to take home as they might get damaged or lost and because they are part of the school inventory. Reading speed continues to be a big problem, as the main focus is on comprehension indicated by test results. Students are not able to read the minimum of 124 words per minute required for the sixth grade and there are errors in reading accuracy especially in students with an indigenous native language. The interviews with directors and teachers made it clear that PCI carries out direct actions to promote reading, mainly reading contests and teacher training. Table 9. Total number of people who benefit directly/indirectly from the interventions funded by USDA Type of beneficiaries Baseline MTE Final Evaluation Final target Difference from target Direct 0 - 58,044 54,765 3,279 Indirect 0 - 189,305 198,070 -8,765 Table 10. Results of indicators that contribute to Feed The Future Indicator Baseline MTE Final Evaluation Final target Difference from target Number of public-private associations formed as a result of USDA support11 0 - 6 4 2 Number of beneficiaries who participate in networks of secure productive chains as a result of USDA support 0 - 58,044 54,765 3,279 Number of people trained in child health and nutrition, as a result of USDA support 0 - 35,057 24,150 10,907 11 Source: Sustainability Commission Plan data bases. Associations refers to the Municipal Sustainability Commissions (One per municipality) 21 6.1.1 Improve classroom attention (MGD 1.2) 6.1.1.1 Reduce short term hunger Table 11. Percentage of students in selected schools who indicate that they are "hungry" or "very hungry" during the school day Baseline MTE Final Evaluation Final target Difference from target 9.3% 19.7% 5% 5% 0 For this evaluation, as in the baseline and MTE, it was determined that the best time to speak with the students was after breakfast and recess (around 11 a.m.) because at that time they could verify whether the food sustained the children. Focus groups participants and teachers expressed that food served at school is a strong motivation for the children who get up early to go to school and receive classes as well as eat breakfast. In the Chuchjil school of Soloma, a higher number of students than average indicated that they were hungry. This might be because they could not go for seconds at the school provided meal, meaning they were still hungry the day of the evaluation. According to what the directors shared, some children save part of the food to take home to share with their siblings. After explaining the importance of school feeding, the number of children taking food home has been reduced. According to the focus groups, the POs participate and actively collaborate in the whole process of providing breakfasts. Teachers also carry out their tasks. Among the children, 5% said they were hungry and 1% stated that they were very hungry, as shown in Illustration 3. Illustration 3. Indicator for hungry or very hungry No 95% Little 4% Very 1% Yes 5% 22 The people interviewed who function as ATC or are part of the local government, and even the Municipal Mayor of Santa Eulalia, agreed that the greatest impact of the program in the communities has been the reduction of hunger in school children. They suggest that the program be expanded, especially to remote schools with a lot of poverty. 6.1.1.1.1 Increase access to foods In the visits, it was determined that all 28 schools gave priority to the school meal and served it every day that the children attended school. Table 12. Percentage of students consuming daily meals at school Baseline MTE Final Evaluation Final target Difference from target 0 - 100 100 0 Access to food is possible thanks to the combination of contributions from PCI, the MOE and parents. In the focus groups it was noted that in the 28 surveyed schools, contributions are provided based on parents’ means, and include firewood, fruit, vegetables and supplies. According to the focus groups, 100% of the POs feel that the school feeding is one of the activities where they are most involved; they take turns being present in the schools to verify that all the children are fed. Collaborating mothers particularly take turns supporting breakfast preparation and hygiene maintenance. According to the directors, 100% create their commissions at the beginning of the year in January, without exception and whether or not it is multi-graded with a teacher for each grade or two teachers for the whole school. The school snack is always included by the commission, as they are generally convinced that it is one of the differentiating factors of their school. Provide school breakfasts The school breakfast is served to all the children with no exceptions. This is the primary program achievement according to the directors, of which 83.33% indicated that school feeding is fundamental for the children’s development and improvement in their learning abilities. Of the parents, 100% recognized the importance and value of the breakfast provided to their children given limitations in the home where the children are not always able to have breakfast. 23 The majority of the participants in focus groups and interviews coincided in calling the school feeding “school breakfast.” This is probably due to the fact that it is the first meal the children receive in the day. It is not related to the schedule or the content. In the focus groups and interviews, parents and directors agreed that school feeding is the basis for the children’s learning process. If they are not fed, they don’t have the capacity to learn, and then they are tired and sleep in class or simply lose interest and stop coming to school. In general, the school is where the children eat well and with variety. The ATC and Municipal Councils also emphasized the importance of the school breakfast. Establishment of school gardens In schools with gardens, 100% of the parents indicated that one of their contributions to the food program is some form of work with the gardens, i.e. time to prepare the land, provision of black soil, paying the salary of someone to care for the garden, and teaching the children how to grow food items in the garden. Of the 28 schools observed, 10 have school gardens. Those without a garden cited the lack of land for planting, the lack of water, and the slope of the land. In the parochial school of Santa Eulalia, approximately 400 plants are available for school feeding. It is the only school in which there is a person paid for by the parents to maintain the garden in optimal conditions for production. Although the gardens do not provide a large quantity of products, 100% of the gardens provide vegetables for their schools. Additionally, they are used to teach the children to plant and produce, and they are used for practice in courses on productivity and development. Some gardens are located outside the school on land temporarily loaned by parents. This requires more time to go to the site to work, leading to a potential loss of the products. In interviews, teachers expressed that the next step for the school gardens would be to convert them into family gardens so the experience can be replicated in homes, especially during the rainy season. The garden products can also be used to support the project’s sustainability. Temux Grande produced 800 broccoli plants, which were sold in the local market. The earnings were used to buy food supplies for the school feeding program and new plants for the garden. Other garden products such as radishes and peas are used for the children’s food. This process should be replicated in other schools as well. In Soloma, some children used to have skin discolorations which disappeared after they began receiving the breakfasts. They recognize that PCI promoted the solution to this problem. 24 Table 13. Percentage of students who are identified as being inattentive in class Baseline MTE Final Evaluation Final target Difference from target (Percentage points) 60.4 48.1 34 54 20 Observations were carried out in 25 schools (89.3% of the project’s schools) to 558 students. In the other three (10.7%), classroom attention observations were not possible for a variety of reasons. It was determined that 66% were attentive in their respective classes. The causes for inattention are presented in the following table: Table 14. Causes of school inattention Percentage Observation 15% Were distracted looking around 8% Were distracted and moving around 8% Were distracted talking 2% Do something that distracts the group 1% Show signs of sleepiness or tiredness 34% Of total students were distracted 66% Of total students pay attention to their teachers n=558 Students Influences on students’ distraction and/or inattention: i. Multi-grade schools, where the teacher gives work to one group while attending another group, loses student’s attention ii. Multi-grade classrooms where the content is the same for all the grades but where the difficulty level and depth changes iii. The pedagogical technique for desk location was inadequate in two of the schools observed. iv. Religion plays a role in students’ participation in some educational activities. For example, in one school, parents forbade their children from participating in civic activities, from doing group work, and from being recognized for outstanding achievements v. The didactic materials displayed on classroom walls are deficient in content vi. In some cases, the teacher had low proficiency in the topic While 34% showed signs of distraction, only 1% were distracted because they were sleepy or tired. The rest were normal, age-related behaviors because they are healthy and have the necessary strength and energy. More appropriate classroom management skills are required on part of the teachers. Key Finding: In focus groups, it was noted that the children have more energy during the day and study more. 25 The teaching staff indicated that teaching quality and children’s attention has improved since the beginning of the program. The availability of food has contributed to improved attention and participation, as they are no longer distracted by hunger and thus do better in all their studies, including literacy. In general they pay more attention and are able to concentrate. Sometimes students are distracted due to normal childhood energy levels, but they are quickly able to refocus. 6.1.2 Improvement in school attendance (MGD 1.3) Children’s school attendance has improved based on verification of the teachers’ roll books in the evaluated schools and comparison with the data in the information corners of each school visited. Table 15 displays the percentage of students with regular school attendance in USDA supported schools, defined as 80% or higher attendance rates. Table 15. Indicator for Improved school attendance Gender Baseline MTE Final Evaluation LOP target Difference from target (percentage points) Boys 85.1% 94.6% 97.3% 90% 7.3 Girls 84.9% 94.3% 98% 90% 8 In the MTE, this indicator was already high, and it improved even more in the final evaluation. In the focus groups with parents, it was noted that the children get up early and are more motived to go to school with the expectation that they will receive a meal there. In interviews, some teachers mentioned cases in which the impact of school feeding is so significant for the children that even when they might be sick with a cold, (and understanding that it is not correct) they come to school to get their meal, eat it, and then go home to rest. 6.1.2.1 Reduce health related absences (MGD 1.3.2) Table 16 shows the percentage of students who state that illness is the main reason for being absent from school: Table 16. Results for the absences for illness indicator Baseline MTE Final Evaluation Final target Difference from target (percentage points) 19.5% 4.3% 8.6% 15% 6.4 Based on what was shared by teachers, children’s absences due to illness are low. The directors say that while students are motivated to learn, they are also motivated by the meals they receive, which improves their health and prevents them from getting sick. Parents also noted that absence due to illness is relatively low and that the children are healthy because they wash their hands and eat well. School directors also think that the children get fewer intestinal diseases because 26 they wash their hands, consume purified water (boiled, filtered) and because of the food provided by PCI. A key action carried out by PCI according to the interviews with the ATCs, is the promotion of consumption of safe water which contributes to the reduction of many of the gastrointestinal illnesses common in Guatemala and in the program’s schools. Among the students, of the 13.1% stated that they had missed school in the week prior to the evaluation, only 8.6% was due to illness, 2.5%, due to family travel, and 2% for other reasons. Illustration 4. Percentage of absences due to illness Regarding the days that they were absent (for all reasons), 68.1% of the students missed only one day of class, while the greatest number of days absent was 15 days for a student who said he had chickenpox. Of the common illnesses leading to absences, fevers accounted for 44%, colds and cough, 12%, gastrointestinal illnesses such as diarrhea and/or stomach ache, 19%, toothache, 6%, and the remaining percentage consists of various illnesses. The indicator refers to “illness” without specifying what kind of illness, and the baseline and MTE reports do not describe the illnesses detected. 6.1.2.2 Improve school infrastructure (MGD 1.3.3) Table 17 displays the number of students who benefitted from the rehabilitation or construction of kitchens or water stations/systems as a result of USDA support: Table 17. Students benefiting from infrastructure Indicator Baseline MTE Final Evaluation Final target Difference Kitchens 0 - 9,126 6,390 2,736 Water stations or systems 0 - 15,711 8,520 7,191 No 87% Illness 9% Family Trip 2% Other 1% Work at home 1% Yes. 13% No Illness Family Trip Other Work at home 27 In the parent focus group responses, the importance of infrastructure support was recognised, as their own resources are insufficient and there is no support from the local governments for these projects. It was also noted that in four of the six municipalities the construction of kitchens was recognized as an important benefit. Of the 28 schools observed, all have infrastructure in acceptable conditions for the preparation of school breakfasts. Sixteen schools (57.14%) have constructions made with parent, municipal and/or MOE support, and 12 (42.86%) were built by PCI, parents and sometimes with municipal support. All the kitchens are functioning. The focus groups and interviews showed that some communities are concerned about the need for water. For example, in Quixabaj one of the most important impacts of the program was due to the installation of a water storage tank (Rotoplas), according to the group of parents who participated. In Cuilco, a municipality with water problems in some communities, respondents stated that an important benefit of the program was the provision of water storage tanks. In the community of Buena Vista (San Gaspar Ixchil) respondents believe that it is one of the needs that hasn’t yet been resolved and that they need water to be able to produce the school garden and increase hygiene in the school. 6.1.2.3 Increased school enrollment (MGD 1.3.4) Table 18. Percentage of students enrolled in school as a result of USDA support12 Enrollment Baseline MTE Final Evaluation Target Difference Boys N/A N/A -4.05% 1 -4.05% Girls N/A N/A -3.85% 1 -3.85 Schools in the six municipalities showed a reduction in school enrollment rates. This tendency is present at the national level, based on what can be observed in Illustration 5, which includes data presented in the INE Education Compendium for 2015. The number of students enrolled in elementary school is determined based on the total number of students enrolled each school year as of March 30. Between 2011 and 2015 there has been a 9.9% decline. The parental focus groups described a small decrease in enrollment. These groups indicated that families needed money and thus those with 12 Source: MOE Enrollment records 2013 to 2015 for schools in the six FFE municipalities 2,350,000 2,400,000 2,450,000 2,500,000 2,550,000 2,600,000 2,650,000 2,700,000 2010 2011 2012 2013 2014 2015 2016 Source: INE Education 2015 Ilustration 5. National Enrollment of elementary students 28 greatest need traveled to seek seasonal work, even though this coincided with the end and the beginning of the school year. Another common response was that some parents need their children to work to help support the family. 6.1.2.4 Increase community understanding of the benefits of education (MGD 1.3.5) Table 19 displays the number of parents in the target communities who are members of the PO (or similar group in the school community): Table 19. Number of parent members of a PO Baseline MTE Final Evaluation Final target Difference from target 0 - 1,470 1,150 320 According to MOE norms, each school should have parent representation by means of the PO or other entities (councils, committees), which are made up of five members: 1 President, 1 Secretary, 1 Treasurer and 2 members at large. The PO, when compared to other organizational forms, promotes sustainability, as it receives economic resources from the MOE which can be used according to the priorities and purchase list that accompanies the disbursement. The members serve for up to four years, though there is the possibility of re-election and in some cases rotation of roles. Other organizational forms have more limited resources, and members do not serve for as long, which limits their sense of ownership. Of the four communities of Santa Eulalia, three have a PO, and the Parochial School has a Board of Directors. In the six communities of Barillas, men’s and women’s participation is balanced in the PO. In Yulconop, there are no women participants, and its community of Quixabaj is the only one with only male members. In the rest of the schools, participation is relatively balanced between men and women. In the three schools of San Pedro Soloma, there is a Committee, a Board of Directors and a PO. All of them only have one year of duration. In San Pedro Necta, of the four schools visited, two have POs and two have Parent Councils, with four and two years of duration, respectively. Of the nine schools visited in Cuilco, seven have POs with a duration of four years (six schools) and two years (one school). The other two have parent councils with a duration of two years. Key Finding: The community organization is one of the great contributions that the program is leaving as a legacy for the communities and this is recognized by parents, directors and teachers, who indicate that the POs were activated and organized to work in the program and participate in activities such as: food storage management, food transport, trainings on various topics, gardens and infrastructure. The organizations do not exist in name only, they work, they take turns and they look out for the wellbeing of the school and the children. Before PCI’s presence, they only met when the directors called meetings but did very little for the school. Now there is a commitment to work. 29 The results of a survey to find the percentage of parents in the target communities who can name at least three benefits of elementary education are shown in Table 20: Table 20. Results of benefits of education indicator Baseline MTE Final Evaluation Final target Difference from target (Percentage points) 58.7% 13.1% 84.6% 60% 24.6 Reading and writing were the most mentioned benefits (94%), followed by the ability to speak well (66.1%) and respect for others (48.7%). In the focus groups with parents, it was noted that PCI had held various meetings and trainings for fathers and mothers on this topic because they wanted all the children to be enrolled in school, thus they informed parents about what they would learn there. The benefits parents listed are presented below: Illustration 6. Benefits of elementary education In the focus groups, parents stated that PCI has trained them at various times so that now they understand the benefits, although this does not necessarily result in greater enrollment or reduction of migration given that their poverty forces them to seek labor and income opportunities. 6.2 Increase the use of dietary and health practices (MGD SO2) The percentage of selected schools that clean the latrines at least once a day are displayed in Table 21: 94 14 66.1 4.8 24.2 9.7 12.3 8.5 40.7 48.7 45 32.8 1.7 0 20 40 60 80 100 Read and Write To do accounts and calculations Speak well Know and value your body Know about hygiene Know how to behave well To have good health To know about nutrition Be good citizen Respect to others To find employment To be able to continue studying Other % 30 Table 21. Indicator for schools with clean latrines13 Baseline MTE Final Evaluation Final target Difference (percentage points) 54% 39.3% 86% 60% 26 The observations showed that 24 schools (86% of the total) had clean latrines, and the children were organized by grades to clean them at least once a day. Two schools that had toilets built by PCI were not using them due to lack of water (Shequemebaj, Cuilco) or because of beliefs about paranormal phenomena (Yoch, Santa Eulalia14). Four of the schools observed (14%) had dirty latrines because water is scarce, so they are only cleaned once or twice a week, which is inadequate for the students. Five schools have School Governments (four in Barillas, one in Cuilco) composed of students from all the grades that monitor the cleanliness of the school and use the “cleanliness traffic light.” In the focus groups it was seen that the PO and collaborating mothers, take turns at being present in the school, to support and monitor program functions. The traffic light system is a positive and functional tool that is helping give visual support to improving hygiene in general, including the kitchens, and has attracted the attention of the educational community for improving school hygiene. The instrument also promotes mobilization of the educational community to get involved and adopt hygiene habits. In all the schools, the latrines are labeled for separate use by boys, girls, and teachers. They also have the minimum required privacy conditions, especially for girls. The parent focus groups commented that one of the elements defined as having “impact for people” was the construction of toilet facilities, especially in the communities of Barillas, Santa Eulalia, San Pedro Soloma and Cuilco. The ATC for Soloma, María Elena Matías Antonio, made the recommendation that cesspits, although clean given their characteristics, should be eliminated from the schools and substituted with formal systems because the cesspits always attract insects and contamination. Table 22 shows the percentage of schools with soap and water in a hand washing station frequently used by students: Table 22. Indicator for soap and water in a hand washing station Baseline MTE Final Evaluation Final target Difference from target (percentage points) 17.4% 25% 71% 35% 36 13 Evaluation team observation 14 The piping was previously running through a burial ground, leading to beliefs of paranormal interference in the project. PCI is rerouting the piping to avoid this problem 31 Observations in the 28 schools determined that 100% have a place for hand washing, in the form of either a specific washing area or a pila. However, only 71% of the schools had soap. The observations showed that 18 schools (64%) carried out correct hand washing. Ten schools do so without following the steps taught by PCI or do not practice hand washing. Significant factors for this included water scarcity, lack of interest on the part of the teachers, and the children’s excitement when the food was ready. In two schools, Quixabaj and Yoch (Santa Eulalia), the students have to haul water for hand washing and for hygiene in the latrines. In the parents’ focus groups, 83% (five of six municipalities) stated that they observed a positive change in the children at home, where they are now washing their hands before meals. This change was promoted by PCI in the schools and the habit has carried over to the household level. The topic of handwashing implies the use of soap and water. For this reason, none of the focus groups or interviews mentioned soap as being necessary for handwashing or bathing. Table 23 shows the percentage of schools (or families) in the target communities with clean kitchens and cooking utensils, prior to use, according to accepted standards: Table 23. Indicator for schools with clean kitchens and utensils Baseline MTE Final Evaluation Final target Difference from target (percentage points) 60.7% 50% 100% 70% 30 The schools visited had clean kitchens and hygienic utensils ready for use. It was observed that the collaborating mothers and members of the PO were attentive to this issue. They could have known that the evaluation team was coming and cleaned more thoroughly, but some situations can’t be improvised and thus the conclusion was that the kitchens tend to be kept in good conditions regarding cleanliness, including cleanliness of the utensils and food preparation. A lack of ventilation, on the other hand, was observed in some schools. The parents’ focus groups in all the municipalities, with the exception of Barillas (where utensils were not mentioned), indicated that they are the ones who contribute kitchen utensils to the project and who monitor their cleanliness. In all the kitchens, a poster was seen with the Golden Rules for food preparation, which could contribute to hygiene maintenance. The collaborating mothers of the six municipalities indicated that the PCI staff trained them on this topic. The majority of utensils were worn but still functional, and in all cases, they are clean at the end of the day. Teachers in San Pedro Necta brought up the possibility of receiving support to acquire utensils for the kitchen because the parents are very poor and generally can’t contribute. The collaborating mothers, members of the PO, and hired staff (in some schools) who work in the kitchen all give special care to keeping things clean, including the kitchen utensils. 32 6.2.1 Improve knowledge of health and hygiene practices (MGD 2.1) The percentage of parents in the selected communities who can identify at least three important health/hygiene practices is shown in Table 24: Table 24. Results for health / hygiene practices indicator Baseline MTE Final Evaluation Final target Difference from target (percentage points) 59% 87.80% 93.20% 75% 18.2 There has been an increase in knowledge when compared to the MTE, which, according to the PCI supervisors, is due to the intensity of trainings after the MTE. The practices mentioned by parents are shown below in Illustration 7: Illustration 7. Health and hygiene practices The practices with the greatest number of responses were daily baths and handwashing. A lesser portion mentioned brushing teeth, combing hair and changing clothes. The practices mentioned least were: using shoes, using latrines and/or toilet, and ‘other’. PCI should reinforce parents’ knowledge of the least mentioned practices given their importance for the students’ and families’ health and hygiene. Hygiene is one of the most mentioned topics in the focus groups and interviews; the POs mentioned it as one of the topics taught by PCI that they easily remember. Additionally, parents’ focus groups mentioned hygiene as one of the positive changes that their children have experienced and that these will be permanent habits with or without the program. Parents also recognized that there had been a transformation in the life of the community in general, because they are replicating this in their homes. The Directors say that it is one of the main program achievements, and the teachers consider it to be a strategy for avoiding illnesses and school absences. 95.7 84.9 47.3 60.1 75.2 13.1 15.7 32.8 0 10 20 30 40 50 60 70 80 90 100 Daily Bath Hand Washing Brushing Teeth Combing Hair Changing Clothes Wearing Shoes Using the Toilet Other % 33 The percentage of students (and their parents) in the selected communities who can identify at least one local source of information about good health practices is displayed in Table 25: Table 25. Results for indicator on sources for health information Baseline MTE Final Evaluation Final target Difference from target (percentage points) 61.8% 48.95% 98.75% 70% 28.75 It was again observed that the trainings given by PCI have had a positive impact, as there was a high level of knowledge about information sources. Additionally, in all the schools there were posters in the native language with information about health practices, including where to request information. The local information sources that were identified are presented below. Illustration 8. Sources of health information Schools, institutes or private schools (49.9%) were mentioned, as well as health centers (46.7%) and health posts or convergence centers (34.8%). Libraries were barely mentioned, as they are scarce in the rural areas where the evaluation was carried out. The category of “OTHER” corresponded to PCI as a source of information (63.5%), and the remaining percentage mentioned the household or family, which were not in the menu of options for previous evaluations. Because PCI will leave the communities at the end of the program, it will be important to continue sharing with parents the available sources of information to promote sustainability and use. The changes in comparison with the Baseline and MTE can be seen below: 34.8 46.7 19.9 2.3 49.9 18.8 0.6 1.1 82.1 0 10 20 30 40 50 60 70 80 90 Health post / Convergence center Community clinic Hospital NGO clinics / Private clinics School Church Library Does not know Other 34 Table 26. Comparison of health information sources Source of information Baseline % MTE % Final Evaluation % Schools/private schools/institutes 51.8 60.9 49.9 Health Post/Convergence Center 47.9 25.4 46.7 Health Center 18.6 12.1 34.8 Hospital 0 1.3 19.9 NGO/private clinics 1.9 3.4 2.3 Church 2.6 3.0 18.8 Other 1.0 11.1 82.1 6.2.2 Increase knowledge of safe food preparation and storage practices (MGD 2.2) The percentage of mothers (or caregivers) in the target communities who can identify at least three important nutrition practices or dietary recommendations is shown in Table 27: Table 27. Indicator for Nutrition Practices Baseline MTE Final Evaluation Final target Difference from target (percentage points) 13.2% 76.8% 88.6% 40% 48.6 These positive results are due to the fact that PCI carried out multiple trainings on the topic, which was mentioned by teachers and parents. Additionally, program staff mentioned that they worked with both morning and afternoon sessions to cover as much of the population as possible. This was in compliance with the MTE recommendation to reinforce trainings on nutrition and diet for all stakeholders. When parents were consulted about what considerations, nutrition, or feeding practices should be taken into account for their children, the main responses dealt with providing foods with high vitamin and mineral content (97.4%), followed by foods that provide energy (89.5%) and in third place, foods with protein of high biological value (83.2%). At the local level they did not give the responses exactly as they were listed in the menu of responses. They either used local idioms or they gave examples of what a person needs for good nutrition. Understanding of the local language was critical for comprehending the parents’ responses and for building trust with the respondents. The language varied in the different regions. The Mam language spoken in the Quetzaltenango region varies a bit from the Mam spoken in Huehuetenango. The same occurs with regard to Q´anjob´al, which varies depending on the region where it is spoken. Details about the practices or recommendations shared are presented below: 35 Illustration 9. Good nutrition practices 6.2.3 Increase nutrition knowledge (MGD 2.3) The percentage of mothers (or caregivers) in target communities who can identify at least three key, safe food preparation and storage practices is shown in Table 28: Table 28. Results for food preparation indicator15 Baseline MTE Final Evaluation Final target Difference from target (percentage points) 13.2% 94.4% 99.1% 80% 19.1 The indicator was exceeded significantly compared to the baseline and improved with regard to the MTE. In all the schools, the collaborating mothers indicated that PCI trains them several times a year on this topic and that in the monthly technician’s visit they review the kitchen and offer recommendations for improvement. Because of this, they know a lot about how to prepare the food. PCI’s routine activities promoted a positive change in this indicator. The kitchens in the schools where the evaluation was carried out are being used adequately. The individuals who have direct contact with food wore hair nets, used their PCI apron or their own, had waste baskets, and the food was processed hygienically and was visible. There might be a bias because they knew that the evaluation team was coming, but all the interviews confirmed that in general, adequate food preparation is the norm. There were no animals or small children in the kitchens. Some schools do not have piped water but organize themselves with support from the students to bring water. 15 Surveys from 116 collaborating mothers 0 20 40 60 80 100 120 Other With local flavors and / or texture Hygienic and suitable temperature With local forms of preparation Suitable according to age and / or gender Good supply of vitamins and minerals High value proteins Good energy supply 36 The responses provided by the collaborating mothers are presented below. Illustration 10. Food storage and safety precautions The percentage of students (and their parents) in the selected communities who can identify at least one local source of information about nutrition and diet is shown in Table 29: Table 29. Results of indicator for sources of nutrition information Baseline MTE Final Evaluation Final target Difference from target (percentage points) 58.1% 57.6% 96.8% 70% 26.8 There was high knowledge of sources for information on this topic due to the many trainings PCI gave to parents. Additionally, parents and students can consult illustrative materials in the local language posted in the schools. The main sources for information on nutrition and diet identified by the parents are displayed in Illustration 11: Illustration 11. Sources for nutrition information 71.6 69.8 71.6 54.3 62.9 85.3 67.2 31.0 55.2 Food must be covered Use of piped water The food is processed on separate tables There must be a covered trash can There should be no food waste Cooks have to wash their hands before… When cooking, cooks do not wear rings Microbiological analysis of water There should be no loose animals in the… 28.2 43.3 21.7 3.7 40.7 12 0.9 77.2 Health post / convergence center Social Clinic Hospital NGO clinic/ Private clinic School Church Library Other 37 The people interviewed mentioned the health center most frequently, with 43.3% of the responses, followed by the school with 40.7%. Under “others,” talks by PCI registered 77.2%. There was no specific line for PCI given that the same question structure was used as in the Baseline and MTE in order to make the respective comparisons. 6.2.4 Increase access to adequate food preparation, tools and storage equipment (MGD 2.6) The percentage of target schools with improved food preparation and storage equipment is shown in Table 30: Table 30. Indicator for better food preparation and storage Baseline MTE Final Evaluation Final target Difference from target (percentage points) N/A 100% 100% 29% 71 The goal is to cultivate healthy and nutritious eating habits in students. In all the schools they mentioned that PCI makes a big effort to ensure that the food is prepared adequately, that the school is kept clean and that the school stores sell healthy foods. This is in line with the actions that the MOE promotes for healthy schools - reduced consumption of junk food, sugars and processed juices. Food supply. At the time of the visit, 22 schools (78.57%) had food. Six schools (21.43%) did not have food in the store room and were waiting for disbursement of MOE products and from PCI. Nutritional Quality. In the majority of schools visited, school feeding is composed of a nutritious milk-like food16 and a carbohydrate food for energy (91% and 95%, respectively). About 30% include eggs and 68% fruit or vegetables. CSB is used in all the schools as a beverage. Rice is a key part of the children’s breakfast in all the schools and is generally mixed with beans, depending on supplies. With support from parents or the MOE, rice can also be mixed with eggs, vegetables and other foods. In two schools (Najab, Quixabaj) rice is accompanied with french fries, which is appealing to the children but not necessarily healthy. In Santo Domingo Sabunul, CSB is mixed with fruit and used as a beverage which is different and appealing to the children. Preparation. All the schools have a formal kitchen space. Additionally, 100% of the people responsible for food preparation have received formal training, which includes personal hygiene and food recipes. In two schools (Yoch and Temux Grande), the observers noted that the rice was not fully cooked. 16 Normally milk or INCAPARINA, a disolvable mix of maize and soya flour fortalized by vitimins and minerals 38 6.3 Key Results 6.3.1 Increase government support (MGD 1.4.3) Table 31 shows the value of public resources assigned to elementary education (includes teaching staff, infrastructure, materials and supplies, etc.): Table 31. Increase in governmental support Baseline MTE Final Evaluation Final target Difference from target 0 - $2,645,617 $50,000 $2,595,617.06 The team viewed the support documents issued, signed, and sealed by the MOE of Huehuetenango, which establish a gradual increase in resources for FFE schools from 2014 to 2016 for contributions to school feeding, resources for teachers, school maintenance, and school supplies. The increase and assignment of financial resources are key, and the more consistent the contributions are, the more effectively school feeding will be covered, contributing directly to sustainability. 6.3.1.1 Links to Administrative Technical Coordinators Interviews were possible with the Administrative Technicians in the majority of municipalities. The connection they have with the PCI program is important, and their contribution towards coordination with teachers and the relationships with general education authorities is key for program management. The following professionals were interviewed: i. José Rigoberto Inés Domingo from Santa Eulalia ii. María Elena Matías Antonio from San Pedro Soloma iii. Olda Herrera López from San Pedro Necta iv. Grisdeli Castillo from San Gaspar Ixchil v. Melvin Castillo from Santa Cruz Barillas It was noted that the link with PCI facilitates activities in the schools that are covered by the program. All coincided in saying that the relationship was formal with defined communication channels. In some cases, such as in Soloma, one of the main activities was the school garden, where the children can develop their abilities and learn technology related to agriculture, as well as being able to replicate this at home and improve their diet. The macro tunnels are an important activity, and they have studied the cases of some organizations that do something similar in other departments such as Quetzaltenango, Totonicapán and Sololá. They say that the school fairs are very innovative and important as they showcase the gardens, preparation of a menu for the children’s food, and a reading contest. 39 Sustainability is a topic that the ATC are aware of and they have developed a sequence for how the process should be carried out. They know how to organize themselves and what should be done. This knowledge is not available in the schools at the level of directors or POs. They should work together on a sustainability program, and the ATC of Santa Eulalia suggested that they should choose one school per municipality to make it a “demonstration school” so people can see that sustainability is possible. The ATCs said that in the future it would be possible to improve programs of this type, especially if they are supported by PCI, as it considers the point of view of community members that are the most distant from the department capitals. In Santa Eulalia, where many communities are at a great distance from the municipality, the PCI FFE program covers three with extremely difficult access. Coverage should be extended because of the need that exists. The garden methodology is key and should be expanded, depending on the characteristics of each locale. Relationships with the Ministry of Agriculture (MOA) should be strengthened and cooperation agreements signed. In the lowlands of the municipalities that have conducive conditions, special encouragement should be given to the establishment and production of fruit trees. All the ATCs were committed to supporting their respective schools. In spite of the numerous activities in their daily schedule, they say they are very willing to support programs that transcend within the communities and can change the future for school children who are acquiring the habits of studying and progressing. 6.3.2 Increase the commitment of Local Organizations and Community Groups (MGD 1.4.4) The number of POs contributing to the schools as a result of USDA support is shown in Table 32: Table 32. Increase in participation of local organizations Baseline MTE Final Evaluation Final target Difference 174 - 294 290 4 All the program schools have representative organizations for parents. The majority have POs, and others have education councils, parent councils, or school committees. These organizations now exist in the schools, and the means of selecting people for the different posts is functional (although the director leads some activities that are inherent in the PO, e.g. the purchase of MOE foods). The next step, then, would be strengthening the structures to enable them to be decision makers and for them to be responsible for managing their agenda focused on school improvement, the internal workings of the parent assembly, and to achieve increased activities aimed at student well-being. 40 Women’s Empowerment In Phase ll, the FFE program did not have a defined strategy for incorporating a gender focus. In spite of that, PCI’s Women Empowered (WE) self-managed savings groups were integrated into “Food for Education” as a way to increase women’s participation and decision making in schools and in their communities. The WE model aims to strengthen social cohesion and mobilizes community members to solve problems, advocate for support and contribute to building community resilience. Information was collected in 6 of the 28 schools visited (21.43%) where WE groups exist. Economic empowerment: Of the six groups visited, all the women save between Q5.00 and Q15.00 a week. The majority of groups meet every two weeks. The women say they appreciate the WE initiative because it allows them to acquire the savings habit and enables them to have their own money, which they can use for many things such as investment in food and clothes for the family. Some women pursue small businesses, which can include bookstores (which actually sell paper goods such as notebooks, writing implements, paper, glue, etc.), small stores, which in cities are called neighborhood stores, and sales of chicken, which is the most commonly consumed meat in the communities. Other ideas for productive activities included: a community pharmacy, a bakery, and in one of the groups they shared the vision of creating a women’s community cooperative. According to program records (at 30th December 2016), across all groups in the project, the total value of savings was Q321,088 (approx. US$43,79217) and the total value of loans was Q159,628 (approx. US$21,771). The average return on savings for all groups was 5.4%. There has been a significant increase over time in lending in the groups, with the percentage of groups who issue loans nearly doubling between 2014 and 2017 (29% to 59%). The percentage of groups members with outstanding loans has also significantly increased, from 4.2% in 2014 to 16.8% in 2017. Overall program data shows that the majority of loans are used for productive, income generating purposes (50%), whilst 20% of loans are used for food security and a further 20% for health purposes. Overall, 11% of loans are used for educational purposes. Another advantage is that they have funds and can support other women who need an affordable and timely loan without complicated administrative paperwork, to have money to cover needs that may arise. In the focus group interviews, none of the groups mentioned taking out a loan for their children’s education. They emphasized that they expect their children to finish elementary education but they do not have resources to enable them to continue their studies. This would generally mean 17 Based on exchange rate of $0.14 = Q1 41 going to a school in the municipal seat, which implies expenses for room, board and transportation, and that investment is beyond their means. One of the six groups mentioned that they are afraid to award loans because they haven’t built trust among themselves, so the social capital is weak. The other five stated that not all the women are interested in requesting credit. An important aspect, and key strategy of the WE methodology, that strengthens the internal democracy of the groups are the regulations that improve and guide their functioning. These are established in their minutes book, and some groups keep them visible on a poster. Social Empowerment: During the focus groups in the six communities visited, the participants mentioned gaining knowledge about self-esteem and the right to live free from violence, specifically interfamily violence. Based on their trainings, they state that they now know they have the right to a job and to have economic initiatives that generate income for them and their families. They mention the right to natural resources. They are in the process of strengthening their capacity to be active in public venues, although in some cases they are still shy or the men do not give them access to opportunities. Despite reporting strengthened self esteem, focus groups participants stated that it’s important to address these issues with their husbands so they can be sensitized and achieve better distributions of power in the family and the community. In the parents’ focus groups, the participants agreed that it involves a dialogue between parents to determine what the right decisions are for their children. Men and women participated in these groups18, and for this reason there appears to have been progress in distribution of power in the family. Political Empowerment: The six focus groups confirmed that this component is the weakest. Women state that the majority of them do not participate in other groups within their communities for a variety of reasons. Some of these include:  They don’t have time to participate  The men in the community groups don’t make space for womens’ participation and decision making  Instead of supporting them, in one community they charged them for signing any project proposal they wanted to present to support entities In the six groups interviewed, it was noted that the women need abilities’ strengthening for dialogue, prioritization, and negotiation of problems. The majority didn’t know how to answer when asked how they address school and community problems. 18 Program records indicate that 6% of WE group participants are men 42 This will require a parallel process with the men, both husbands as well as local leaders, so that they include the women’s groups in decision making for community development. Male engagement has been recoginsed by PCI as a key strategy to support womens’ empowerment. The majority of women in the WE groups participate in school activities as collaborating mothers, and in a few cases as members of the POs. 6.4 Sustainability To analyze the sustainability of the evaluation schools, the PCI tool for classifying them by results was applied to have a better view of the status of each one. Five main criteria and 25 indicators were evaluated. The results are shown in Illustration 12: Illustration 12. Sustainability situation of evaluated schools The highest scoring condition was that of hygiene practices that have been adopted by all the key stakeholders: POs, parents, directors, teachers and students. It was evident that they have taken ownership of the practices and they apply them in daily life. The second highest condition was adequate infrastructure. PCI made a notable effort in conjunction with the community and municipalities to equip the schools with the infrastructure needed to improve the school environment. Regarding improved literacy, the scores were medium or low. This should be taken into account in future education related programs and because it is related to the 11 11 11 12 8 11 13 15 15 13 10 11 20 20 17 19 20 16 19 16 16 14 16 13 6 4 3 2 6 1 Santa Eulalia San Pedro Soloma Santa Cruz Barillas Cuilco San Gaspar Ixchil San pedro Necta School Feeding Reading Improvement Hygiene Practices Infrastructure Municipal Support 43 improvement of education quality, which at the national level, is highly deficient. School feeding also received medium to low scores. While all the communities in one way or another provide contributions for the breakfasts, this is not geared towards sustainability because of the dependence on foods from PCI, the lack of consistency in government contributions, the poverty of the families, and the lack of other funding sources. Finally, the condition that scored lowest was municipal and community support, which highlights the need to continue strengthening local community leadership so that they can request the required municipal support and thus integrate all the members of the community in the efforts to have a self-sustaining development program. Based on an ideal weighting to determine whether a school is on a realistic road to sustainability, three categories are proposed: A: Schools with 70 or more points, B: Schools with 60 to 69 points, and C: Schools with 59 or fewer points. The results for the evaluated schools are: Table 33. Results from application of sustainability tool in FFE schools Municipality Community Total Classification Santa Eulalia Parochial 76 A 70 + points San Pedro Soloma Najab 76 Santa Cruz Barillas Santa Cruz Barillas 76 Santa Eulalia Temux 72 Santa Eulalia Quixabaj 72 Santa Cruz Barillas Yulconop 72 Cuilco Hermosillo 72 Cuilco Santo Domingo 68 B 60 to 69 points Cuilco Posonicapa Chiquito 68 San Pedro Soloma Bacau 64 Santa Cruz Barillas Buenos Aires Chiblac 64 Cuilco Yulvá 64 San Gaspar Ixchil Buena Vista 64 Santa Cruz Barillas Nuevo Manantial 60 The instrument for measuring sustainability effectively shows which communities are on the road to sustainability, as well as those where strengthening of specific aspects is needed. On comparing the scores from the PCI sustainability tool obtained in each intervention community with the observations and results from focus groups and in-depth interviews, the correlation of the criteria used in this tool was clear. Currently the five criteria have the same weight but PCI should evaluate whether any should be prioritized more than others and if so, make the relevant adjustments. 44 Municipality Community Total Classification Cuilco Nueva Reforma 60 Cuilco Ixtatilar 60 San Pedro Necta El Palmar 60 Santa Eulalia Yoch 56 C 59 or fewer points San Pedro Soloma Chuchjil 56 Cuilco El Carmen 56 San Gaspar Ixchil Ixpaltzaj 56 San Pedro Nectá Vista Hermosa 56 Santa Cruz Barillas Yulwitz Grande 52 San Pedro Nectá Joya Grande 52 Santa Cruz Barillas Nueva Unión las Palmas 48 Cuilco Villa Nueva 48 Cuilco Shequemebaj 44 San Pedro Necta Santa Clara 40 One of the conditions needed for the achievement of long term program sustainability is the existence of one or more types of development capital in each community: social capital, human capital and financial capital. 6.5 Evaluation of the Monitoring and Evaluation (M&E) system The PCI M&E unit uses an electronic system called Mobenzi, which is very practical as it collects data in the field that enables the program to have a vision of what is happening in the schools and feeds the data bases for program beneficiaries and activities. For example, for the indicators based on records, the system allows the data to be exported to Excel, SPSS, Epiinfo, etc. In the school observations, the evaluation team saw the School Information Corner instrument which is updated monthly to monitor the activities in the schools and assign performance scores. The directors stated that when the field technician updates the monthly data they review it in conjunction with the PO to agree on ways to improve. All the Information Corners were found to be up to date. The data includes information about performance indicators such as school attendance, store room and kitchen quality, school hygiene, etc. It is an instrument for analysis of the quality of implementation in the school. It was noted that for 2017 there is a new design of the Information Corners that includes new measurement indicators. 45 6.6 Lessons learned During the evaluation process, the team had direct contact and interactions with approximately 1,852 people. In this process, several important lessons were learned and should be shared to achieve better results in the future, or to maintain or exceed what the program has done successfully so far. 6.5.1 Committed field staff One of the key elements for carrying out program activities is having staff who are highly committed, know the environment, are committed to the communities they are assigned to, and willing to deal with the challenges due of topography and the difficulties in reaching the schools, weather problems, and work schedules. If the staff fail, the program will also fail. 6.5.2 Sustainability In this type of program, it is necessary to introduce and broadly incorporate the idea that the support is temporary and thus communities must seek sustainability. Parents and teachers are aware of the importance of the school breakfasts for improving school attendance and performance, thus they also must be made aware of the need to think about the future. Beneficiaries see the program as a donation, and while it lasts, one has to take advantage of it. In general, they do not have the vision of seeking self-help, although to some degree they do it “obligated or not by the management” by giving their contributions in time, in kind, or monetarily, but the overall concept does not exist. Paternalism has been incorporated into the culture and it takes work to eradicate it in the communities. It is important to build the schools’ self-sustainability structure so that the greater community of the COMUSAN, the COCODES, the local government, the ATC, the Departmental Bureaus and local players such as the POs can work, understand, have a vision and prepare a plan for a sustainable future for school feeding. Currently, Sustainability Commissions have been formed and have an interesting structure, which should be strengthened until they become functional; nevertheless, only the ATCs seemed clear about them. There was no evidence of knowledge of this structure at other levels. 6.5.3 Community participation One of the greatest difficulties in working toward and positively influencing children’s future is the participation of parents who have a certain degree of leadership in the community who prefer to hold on to customs from the past and who are afraid of change. They tend to limit the participation of others. A sensitization process with parents who do participate is advisable so that they can become change agents regarding the need to be present, to make sure that children come to school, and who support this kind of program. The program should use an adequate 46 strategy in the community to design the training program for different levels and audiences so that they really achieve the expected impact. 6.5.4 Infrastructure Support for construction or rehabilitation of infrastructure requires the definition of well established actions and commitments with all the responsible stakeholders before the projects are assigned, and these should reflect the needs expressed by the local players. This will ensure that all the projects are utilized and that the community takes ownership and responsibility for them. With regards to the school garden infrastructure which is of great importance for school and family diets (if they are replicated in the home), they should be adapted to each community’s conditions based on access to, or lack of water, soil structure, and slope, so that it is a useful tool that can be replicated at the household level and support future sustainability of the schools. 6.5.5 Take the mountain to the authorities In the cities, and especially for those central authorities headquartered in the capital city, there is a lack of knowledge about the reality of the program coverage in communities located in the mountains of Huehuetenango. It is important to ensure that strategic actors, as well as those who are key for approving projects or supporting the creation of infrastructure in the communities, see the actual realities either by traveling there or by audio visual means that enable them to become aware of the needs and thus achieve commitments, signed agreements and timely support, etc. 6.5.6 The authorities It is important to achieve the commitment of local authorities to define who will take responsibility for school feeding, and this person should be part of the sustainability commission. In the interview with local authorities from the six municipalities, it was noted that while they know about the program and give support whenever they are requested to transport foods, they are not committed to education itself (with the exception of San Pedro Necta). It is a task that they are not very interested in and they only promote what is programed if a budget line has been assigned. 6.7 Synthesis of accomplishment of MTE recommendations Table 34. PCI’s response to MTE recommendations Recommendation Accomplished? Means of addressing More resources and support are needed in the home to improve reading ability Partial, no resources assigned PCI designed a reading comprehension strategy with in and out of classroom activities, although they did not invest resources at the household level which they could support in the future. Children’s enrollment should be promoted at the preschool level Yes PCI carried out various activities to promote enrollment at the pre- as well as elementary school level. 47 Recommendation Accomplished? Means of addressing Promote strict control of teachers’ attendance No PCI has included an indicator for teacher attendance in the next phase of the program. This indicator did not exist in FFE II. Teach teachers ways of controlling students and keeping them engaged in the classroom Yes The reading comprehension strategy includes dynamic activities, which resulted in increased attention in class. Promote open reading spaces in the community or school Yes, in some schools PCI held reading camps in some schools on a pilot basis to determine their usefulness. They were seen to be important and should be implemented in the next phase of the program. Establish more information sessions aimed at parents so that they understand the advantages of education, especially preschool education Yes PCI provided sufficient information for parents and this is reflected in the fact that the target for the indicator significantly exceeded expectations. Promote basic education for girls No PCI addresses pre- and elementary school education Promote the knowledge of educational indicators with teachers and parents Yes Information corners were present in the schools and various indicators were displayed and discussed by the technicians with the educational community. Promote parent participation in the school Yes PCI has held community meetings with good participation by parents who are interested in the program, the food and the trainings. Continue the feeding program, improve coordination of MOE efforts Yes In all the schools complementarity was noted between the foods from PCI and the MOE, and parents provided contributions in kind as well as cash. Coordinate school feeding so that it is more effective and integrated Yes There was no differentiation between PCI and MOE foods. In fact they complemented each other to vary the nutritious menus and avoid boring the children. Increase the frequency of trainings and participants, as well as the replication methods. Yes PCI increased the topics and forms of training, successfully exceeding the indicators for this recommendation. Promote greater parent participation in the FFE program, continue promoting the contribution of kitchen supplies, but seek diversity and simplicity; improve storage for firewood. Yes High parent involvement was observed in all the schools, in all the processes from contribution of supplies to work on infrastructure and gardens, food transportation and management, filling out records, etc. The firewood is stacked in specific spaces and is contributed by parents. Promote improvement of store room maintenance and share the norms for appropriate store room management Yes 100% of the evaluated store rooms complied with the quality standards for food storage Evaluate trainings Yes Using simple techniques, the technical staff asks questions to measure what has been learned. Example: use of the “question dice” Review the steps that have been taught and evaluated for handwashing and if needed, change them Yes The steps for handwashing were based on official MOH standards. Some schools suffer from lack of available water. 48 Recommendation Accomplished? Means of addressing Ensure that the members of the CPF understand the importance of microbiological water analysis as well as the results from each test. Yes PCI has included the topic in its trainings and as a result the microbiological analysis was mentioned in some of the survey responses. Ensure knowledge transfer when members of the PO rotate in order to avoid losing ground with achievements Yes PCI carries out training for new members of the PO as they need to take immediate action related to food management. Reinforce trainings on nutrition and diet required in the school for all players. Yes PCI has complied satisfactorily with the recommendation and has exceeded the MTE results Reconsider the nutrition trainings Yes PCI addressed the topic based on the context of each community and used examples with local resources and explanations of technical language Improve bathroom cleanliness Yes PCI designed the cleanliness traffic light tool which includes monitoring of bathroom hygiene. The issue of lack of water in some schools is ongoing. The kitchen area should be supervised and teachers, directors, the cleaning commission and school government should be made aware in order to comply with the rules, emphasizing the most important ones for food preparation or child safety. Yes In addition to the trainings, illustrated material was distributed with the golden rules for school feeding which facilitated satisfactory compliance with the indicator. Promote agreements to secure land for the gardens, cultivation of plants that don’t require seed purchase and that have high nutritional value, promote new recipes for traditional crops. Yes The schools lack physical spaces for establishing gardens, and thus the majority are located on land loaned by parents. PCI promotes planting of local species. A recipe book that uses local resources has been prepared although in some cases it is not being used. The mothers learn better in the practical trainings that PCI gives. Strengthening of leadership roles in the Municipal Sustainability Commissions is needed Partial At the time of the evaluation, this entity was not mentioned in the school, only at the ATC level. This recommendation needs to continue being addressed. Continue promotion of school fairs Yes The fairs are held each year and they are seen as a place for sharing/teaching recipes and other knowledge. 7. Conclusions a. General Aspects: 1. Based on what was observed and what was stated in the focus groups and in-depth interviews, the following key activities and strategies were most effective in achieving results: i. Formation and strengthening of POs or similar entities ii. Community organization and involvement in productive networks (WE groups) 49 iii. Trainings on health and nutrition topics iv. Support in activities to reinforce literacy v. Distribution of foods that have an impact on children’s performance and attention vi. Increase in knowledge of:  Important hygiene and health practices  Food preparation  Nutritional practices  Local sources of information about health and nutrition vii. Investment in school infrastructure, in the following order:  Safe Water  Improvement or Construction of kitchens  Improvement of store rooms  Construction of toilet facilities viii. Community organization and training, as a cross cutting issue in each school 2. An important achievement is the coordination with, and involvement of the ATC to develop joint activities and involve the schools in strategic actions that help improve literacy indicators. 3. The government of Guatemala, through the MOE, is involved with reading by sending books to the schools for student use, and supports the validation of strategies to improve reading comprehension and coordinates different activities that PCI implements. At the household level, there is limited availability of adequate reading material. 4. According to the focus groups carried out with key stakeholders, PCI took into account the basic needs of the educational community when developing the program. For example, in prioritization of infrastructure construction, in the manner that food is prepared, in trainings about good health and nutrition practices, among others. 5. Logistics for food management has been implemented entirely by the community, coordination with the parent leaders, the PCI program and in the majority of cases with the local government. 6. The MOBENZI system rapidly and efficiently provides useful information giving reliable support, which improves implementation decisions and offers greater knowledge of what is happening at the field implementation level. Additionally, the school information corners facilitate greater involvement of teachers and parents for improving their performance. b. Results and impact 7. Students’ reading ability was increased and reached national standards according to the published indicators. i. Speed and accuracy are areas that require further work in spite of improvement in comprehension. ii. Students’ inattention in class had decreased in each of the evaluations, and there is a trend towards further decreases, exceeding the established target. 50 iii. Impact was seen in the reduction of absences due to illness and in cases of students who were hungry or very hungry. 8. Based on the focus groups, surveys and observations, it was noted that students have adopted nutrition, health and hygiene behaviors and practices with spillover to their parents. 9. There was an increase in knowledge of the benefits of education as a result of the increased trainings for parents on this topic. 10. Based on the interviews, focus groups and observations it was concluded that the degree of satisfaction of the beneficiaries or direct participants (parent leaders, parent assemblies, students) is high based on their involvement in the process. 11. The different direct beneficiaries are involved in the program in the following ways: i. Leaders of parent assemblies (POs, Committees, Councils, etc.), take turns being present one or several days every week to manage foods, get training and organize parents to build infrastructure, work on gardens or infrastructure, transport food and support school hygiene. When necessary, they come several times a week if called by the director or by PCI staff. ii. The WE groups are organized and trained. Some members are part of the parent organizations or are collaborating mothers. The WE organization as such is beginning to be involved in the development of the school. iii. Parents receive training and support infrastructure, gardens, and breakfasts. They contribute supplies or money for the breakfasts. iv. The collaborating mothers contribute time for food preparation. v. The teachers support program record keeping, form food and hygiene commissions, use the garden for teaching and are strengthening their capacities. vi. The students learn from the garden, receive breakfasts and improve their learning. 12. The school gardens are successful for food production to complement the breakfasts and as a pedagogical tool. So far, they have not had the expected impact economically or with regard to sustainability. c. Sustainability 13. The key stakeholders identify with and take ownership of many of the FFE II program actions, complying with their role as stipulated in the plans and based on the organization to which they belong: Parent leaders organized in POs or similar entities, Parent assembilies, Directors and teaching staff, Collaborating mothers, WE groups and Students 14. The key stakeholders are currently active in carrying out planned program activities. The program should generate in them the capacity to be visionaries who can analyze, make decisions, and plan for the future of the school as well as the community. They need more training and formation to be able to internalize the need to make the activities sustainable and independent. 15. Some of the members of the WE groups participate as collaborating mothers and as mother leaders who support program activities. The women indicate that they have their husbands’ 51 backing to participate in the WE groups. In the focus groups, only two groups stated that there were women participating in the parent leadership entities (PO or similar). i. Economic empowerment is high – they have savings and award loans. ii. The strengthening process must continue for the social and political components in order to achieve the proposed objective. iii. Strengthening is needed for sensitizing husbands so that they respect and comply with women’s rights. iv. Women’s low participation in the COCODES is due to internal segregation in the communities, where men don’t take women’s opinions into account. 16. The sustainability tool gives a quick diagnosis of the status of the schools with regard to sustainability. Considerations to keep in mind: i. Design a classification system, so that after receiving schools’ scores, they can be classified to establish needs according to their level. ii. Analyze the elements of sustainability that PCI intends to leave in the schools so they can prioritize and act. There is also a need to define what is meant by sustainability. d. Gender 17. Cross cutting gender activities have been carried out throughout the program: i. Infrastructure: bathrooms have clear physical divisions to provide security and recognition of girls’ needs ii. Women’s participation in parent organizations has been promoted iii. Women’s training has been strengthened iv. Women have been empowered, seeking their integration in program activities which can contribute to sustainability given that women are strengthened and the savings habit is institutionalized as well as analyzing social problems. They are beginning to occupy leadership positions. v. Parent participation in their children’s education has been promoted by means of the meetings called by the parent organization (PO or similar), teachers, and PCI. e. Final Conclusion The FFE program II has had an impact in the target schools and municipalities, as evidenced by the high level of accomplishment of the results indicators and even more important, in the testimonies of all the educational communities. Thus, the model that has been implemented in the 294 schools can be replicated in other municipalities and schools, not only in Huehuetenango but also in other departments of Guatemala. Additionally, PCI has complied with the majority of the MTE recommendations, which contributed to satisfactory accomplishment of the majority of indicators. 52 8. Recommendations a. For community organization and institutional coordination 1. Support the formation of the legal status of the PO so that each school has the right to choose the foods that the students need and to ensure alignment with MOE interests in this regard. At the same time, support capacity strengthening so that the POs can perform their duties, including leadership for preparation of purchase plans which till now has been carried out exclusively by the director and teachers on the school feeding commission. 2. Continue the close coordination and communication with MOE staff including the ATC, which will facilitate alignment with their educational initiatives (Let’s Read Together, formation of POs, healthy schools, etc.), and the program will benefit because their actions will have institutional approval and validation from this governmental entity. 3. Seek greater participation and involvement of the Mayor and his/her Council members, informing them periodically about the activities and results achieved and making them aware of the importance of educational improvement actions, including support for school feeding, scholarships, infrastructure funding, seeds for gardens, teachers’ salaries, etc. 4. PCI should continue propitiating participation by the educational community in program planning and execution. It is important to facilitate continued awareness of the interests of the direct beneficiaries who value the prioritization of their needs as well as respecting their desire to participate in activities such as reading camps, school fairs, and gardens, etc. 5. Continue using the MOBENZI technology and share the monitoring results with municipal authorities, the MOE, USDA, teachers and parents, so that they are aware of this effective system for control of activities at the field level. b. Accomplishments and pending issues 6. Given the positive results with regard to health, nutrition and hygiene practices, the program should maintain constancy in the trainings as well as informational and sensitization activities for parents and students while involving teachers and volunteer staff to ensure that levels of understanding are maintained and put into practice. Also continue reinforcing the information about safe food preparation with collaborating mothers. Consider providing training for new parents who join the schools. 7. In spite of the fact that students show improvement in reading comprehension, they still need activities that will improve their speed and accuracy. To this end, teachers should be trained in techniques or exercises to use in the reading process. This should be done in the initial grades to guarantee that they learn to read well from first grade onward. 8. In spite of the problem of decreasing enrollment at the national level, it is possible to take direct action in each community. Together with the parent leaders, teaching staff and community authorities (Auxiliary Mayor, COCODE), school censuses should be carried out to ensure that all school age children are registered in school. Parents should know about and respect the Right to Education that is guaranteed in the Constitution of Guatemala. They 53 should also be informed about the dangers of migration to the United States and Mexico (drug trafficking). In special cases of extreme poverty, scholarships can be promoted with support from the institution, municipal governments or private sector. 9. Given that some children miss school due to gastrointestinal illnesses, PCI can, by itself or in association with the Ministry of Health, carry out deworming or vitamin application activities that can ensure better biological assimilation of the foods provided. 10. Expand the school gardens based on the climate and context of each community, verifying climate conditions, land situation, water availability, etc. Where conditions permit, foods can be produced using hydroponics. Additionally, year round production should be promoted via staggered planting and harvests depending on the species, in order to accomplish several purposes: i. Have food available during the entire school year ii. Take advantage of the garden as a continuous teaching tool iii. Sensitize parents and teachers about the importance of the garden iv. Have a permanent learning space for household gardens v. Consumption of nutritious foods which are necessary for a healthy diet vi. Reduce vegetable contributions from parents and make their importance more visible vii. Promote sustainability with involvement of teachers and parents 11. Design a methodology for literacy in keeping with the diversity of the two regions where the program operates. This should be promoted beginning in the early grades to ensure greater impact on student performance improvement. This should include fun activities that encourage student participation. 12. Include a formal area for improving the teaching staff’s capacities. It was observed that the teaching materials are not adequate or are not used correctly, and they lack motivational techniques to encourage children’s interest and attention. This has direct repercussions on the quality of student learning, which will take its toll at the secondary level. 13. Address with the MOE, ATC, and teachers the implications of inattention in class and to identify and carry out short term actions to reduce inattentiveness. For example, better distribution of classroom space, training for teachers on dynamic teaching techniques, review of the timing of school breakfasts, sharing knowledge in extra-curricular activities, reading outdoors, etc. will reduce inattentiveness. 14. Parents, teachers, and school authorities should become direct implementers and active elements to improving education and school feeding, as they are going to be trained in sustainability topics and already have full knowledge of how to manage school feeding. This will make it much easier for many of the activities that the program promotes to continue after PCI leaves. 15. Since food was found to be prepared inadequately in three schools, training on recipe preparation should be reinforced to ensure that the consumers are satisfied and that the food is able to be digested well. 54 16. Given that some schools requested books or libraries for their students, it is important to seek mechanisms or funding sources to provide the schools with books so that they have permanent reading material. They already have some text books but lack sufficient supplies of other books to vary reading and increase reading ability levels, especially to improve speed and accuracy. c. For Sustainability 17. Evaluate the level of sustainability of each school and prioritize actions based on a mapping of needs and the results of that evaluation. The current tool for measuring sustainability is useful but should be analyzed to see if the same weight should be given to the established conditions and criteria. Additionally, the meaning of sustainability that PCI wants to promote in the schools should be analyzed. 18. A public policy that supports continuity of the program should consider at least the following elements: i. Promote and support strengthening of organized parent organizations in POs or similar entities. ii. Introduce the option or requirement for taking into account direct purchases of produce in the community and if that is not available, open the option for purchases outside the community. This should be negotiated with the MOE. iii. Implementation of school gardens in the schools where the minimum required conditions are present, on school or parents’ land, and with adequate technologies based on the local characteristics. iv. Design a specific mechanism for financial decentralization towards the departmental bureaus so that the purchase lists have an endogenous element which will facilitate adaptation to the different local cultures and contexts. v. Conduct a baseline survey of all the educational, nutritional indicators, etc. vi. Design the means for participation of the local government, organizations, social groups and the community. vii. Introduce the productive component of Local Economic Development as a cross cutting issue for parents. viii. Consider the opinions and validation of all relevant stakeholders, including the educational community. 19. With regard to the sustainability tool, it would be important to carry out a classification of the schools according to the results obtained to identify their sustainability status. Those with the lowest scores have more conditions that need strengthening. Analyze the proposal for classification that is presented in the corresponding chapter. 20. The program should promote knowledge of local sources of information on health, nutrition and hygiene, other than PCI, as it is not a permanent source. Training on these topics should continue. 55 21. Consumption of safe water should be promoted in 100% of the schools, with sustainable systems that could include boiling or chlorination, and even Ecofiltros as long as there is a commitment by parents to replace the filter units when needed and by teachers to maintain the units and thus ensure their optimal life span and adequate functioning. d. For Women’s Empowerment 22. PCI should continue working on the political empowerment of the WE groups. In spite of being long term processes, it is important to achieve the goals that this empowerment seeks. For example, success stories where women have positions in community or school organizations can be shared, where mother leaders support program activities or simply where they are organizing themselves and supporting school or community improvement activities. Political empowerment includes working closely with husbands and community leaders, so that they will support the women’s work, take their opinions into account, and prioritize projects that meet their needs. 23. With regards to economic empowerment, the program should seek mechanisms to help women overcome their fear of disbursing loans or the lack of trust among themselves. They should also strengthen the women’s capacities to undertake productive enterprises either as individuals, which is where they are most comfortable, or as a group, to have greater sales volume in the commercial activities. They should be trained on how to keep accounts and prepare basic, understandable business plans. 24. PCI should analyze a modality similar to the WE initiative that would include men. In some of the focus groups, they recognized the benefits of being part of these groups. This doesn’t mean changing the existing methodology, but rather seeking mechanisms for forming groups with men, which could be done on a pilot basis as mixed groups or groups only made up of men. Given that the youth are a dynamic entity in the communities, the program could also experiment with groups made up of youth, men and women. 25. For sustainability purposes, PCI should link the WE groups with permanent entities in the municipalities, for example the Municipal Women’s Office (DMM) or other local organizations that support women’s endeavors. e. Final Recommendation 26. Replicate the FFE II implementation model in other schools in Huehuetenango or other departments, given the results achieved in improvement of health, nutrition and hygiene practices, and of educational indicators such as the improvement in reading comprehension and classroom attention and the reduction of illnesses and hunger in students. Additionally, the WE groups should be replicated, as this energizes the family economy and is also an efficient mechanism and a good example for integrating women as a key element in community development. 56 Annex 1. Comparison of Results Indicators Result Indicator Baseline Mid-term Evaluation Final Evaluation Final target Difference Contribution to Feed The Future Number of public-private associations formed as a result of USDA support 0 - 6 4 2 Number of beneficiaries who participate in networks of secure productive chains as a result of USDA support 0 - 58,044 54,765 3,279 Number of people trained in child health and nutrition as a result of USDA support 0 - 35,057 24,150 10,907 Improvement of literacy in school age children Percentage of girls who at the end of sixth grade demonstrate reading comprehension at grade level, as defined by national norms, in the schools supported by USDA. 8.8% 15.6% 51.5% 61% -9.5 Percentage of boys who at the end of sixth grade demonstrate reading comprehension at grade level as defined by national norms, in the schools supported by USDA. 2.5% 7.4% 45.5% 57% -11.5 Total number of people who benefit directly from the interventions funded by USDA 0 - 58,044 54,765 3,279 Total number of people who benefit indirectly from the interventions funded by USDA 0 - 196,170 198,070 -1,900 Increase in governmental support Value of public resources assigned to elementary education (includes teachers, infrastructure, materials and supplies, etc.) 0 - $2,645,617 $50,000 $2,595,617.06 57 Result Indicator Baseline Mid-term Evaluation Final Evaluation Final target Difference Increase in participation of local organizations and community groups Number of POs contributing to the school as a result of USDA support 174 - 294 290 4 Improve classroom attention Percentage of students who are identified by their teachers as inattentive in class 60.4% 48.1% 34% 54% 20 Reduce short term hunger Percentage of students in selected schools who indicate that they are "hungry" or "very hungry" during the school day 9.3% 19.7% 5% 5% 0 Increase access to food Percentage of students eating daily meals at school 0 - 100% 100% 0 Improvement of school attendance Percentage of boys with regular school attendance (80%) due to USDA support 85.1% 94.6% 97.3% 90% 7.3 Percentage of girls with regular school attendance (80%) due to USDA support 84.9% 94.3% 98% 90% 8 Reduction of absences related to health Percentage of students who state that illness is the main reason for their school absences 19.5% 4.3% 8.6% 15% 6.4 Increase in use of health and dietary practices Percentage of selected schools that clean latrines at least once a day 54% 39.3% 86% 60% 26 Percentage of schools with soap and water in a hand washing station frequently used by students 17.4% 25.0% 71% 35% 36 Percentage of schools (or families) in the target communities that have clean kitchens and cooking utensils prior to use, according to accepted standards 60.7% 50% 100% 70% 30 58 Result Indicator Baseline Mid-term Evaluation Final Evaluation Final target Difference Increase in knowledge of health and hygiene practices Percentage of parents in the selected communities who can identify at least three important health/hygiene practices 59% 87.8% 93.2% 75% 18.2 Percentage of students (and their parents) in the selected communities who can identify at least one local source of information about good health practices 61.8% 48.95% Parents 98.3% Students 99.2% Average: 98.75% 70% 28.3 Increase in knowledge of safe food preparation and storage practices Percentage of mothers (or caregivers) in the target communities who can identify at least three important nutrition practices or dietary recommendations 13.2% 76.8% 88.6% 40% 48.6 Increase in nutrition knowledge Percentage of students (and parents) in the selected communities who can identify at least one local source of information about nutrition and diet 58.1% 57.6% Parents 98.3% Students 95.3% Average: 96.8 70% 25.3 Percentage of mothers (or caregivers) in target communities who can identify at least three key practices for safe food preparation and storage 13.2% 94.4% 99.1% 80% 19.1 Increase in access to adequate food preparation, tools and storage equipment Percentage of selected schools with improved food preparation and storage N/A 100% 100% 29% 71 Improvement of school infrastructure Number of students who benefit from the rehabilitation 0 - 9,126 6,390 2,736 59 Result Indicator Baseline Mid-term Evaluation Final Evaluation Final target Difference or construction of kitchens as a result of USDA support Number of students who benefit from the rehabilitation or construction of water stations or systems as a result of USDA support 0 - 15,711 8,520 7,191 Increase in enrollment Increase in percentage of boys enrolled in school as a result of USDA support 0 - -4.05% 1 -4.05% Increase in percentage of girls enrolled in school as a result of USDA support 0 - -3.85% 1 -3.85 Increase in community understanding of the benefits of education Number of parents in target communities who are members of the PO (or similar school community group) 0 - 1,470 1,150 320 Percentage of parents in the target communities who can name at least three benefits of elementary education (data collected by survey) 58.7% 13.1% 84.6% 60% 24.6 60 Annex 2. Information about the Consultant Team 1. Lic. Julio Antonio de la Roca Cacacho 2. Lic. Jose Gil Mena Alberto 3. Licda. Irma Violeta Hernández 4. Lic. Víctor Manuel Palacios Marroquín 5. Licda. Mayra Aminta Gómez Alegría 6. Licda. Dacia Marroquín Mazariegos Translation team for the Q´anjob´al area 7. Evelyn Norasilda Rivera Bhighzar 8. Juana Cristina Antonio Mateo Translation team for the Mam area 9. Dorca Lily Morales Hernández de Ramírez 10. Rosita Yesenia Domingo Ramírez 61 Annex 3. Qualitative Evaluation Tools Instrumento cualitativo No. 1-EF-FFE Directores/directoras: 1. Cuáles son los principales logros del programa PCI.? a. Cuáles son los retos a vencer en el futuro? 2. Cuál es el nivel de participación de los maestros en las actividades del programa? 3. Como se han organizado ustedes para el trabajo con PCI? a. Han hecho comisiones i. Cuáles? ii. De qué duración? 4. Han registrado mejoras en los indicadores de enseñanza - aprendizaje? a. Cuáles? 5. Es oportuno el aporte del subsidio de MINEDUC a la escuela? 6. En qué medida se tomó en cuenta las necesidades de la población beneficiaria en el programa de PCI? 7. Cuáles son las actividades del programa que recomendaría expandirse y / o limitarse para asegurar un uso más eficiente de los recursos al tiempo que se garantizan los mismos resultados o mayores? 8. ¿En qué medida los actores clave se apropian de las actividades del programa? (estudiantes, padres de familia, maestros, autoridad local) 9. Qué pasará cuando el proyecto finalice, como lograrán continuar con el programa de alimentación a los niños a. Existe alguna comisión para sostenibilidad? b. Se reúnen al menos 3 veces al año c. Esta comisión tiene un plan de trabajo d. Recibe la escuela algún apoyo aparte de PCI, MINEDUC y MSPAS 10. Se puede decir que el 100% de estudiantes cuenta con material de lectura permanente y disponible para alumnos 11. Se lleva un sistema de control, libro de registro o cuaderno de asistencia de maestros 12. Se recibe un aporte municipal y este tienen un aumento del 3% anual para transporte de alimentos, mano de obra calificada, infraestructura, compra de alimentos locales, etc. 13. Podría identificar y relatar dos historias de éxito dentro del término del proyecto de PCI Instrumento cualitativo No. 2-EF-FFE Maestros 1. Cuál es el nivel de participación en las actividades del programa por parte del claustro? 2. Qué actividades y estrategias fueron efectivas para lograr los resultados del programa? a. En donde se debe mejorar? Por qué? 62 3. Como se considera el impacto en la inscripción y asistencia a la escuela con el funcionamiento del programa? 4. Ha logrado el programa mejorar los resultados para la lectoescritura, la calidad de la enseñanza y la atención de los estudiantes? 5. Hasta qué punto se han promovido y adoptado comportamientos y prácticas (por ejemplo, en nutrición, salud e higiene) por los beneficiarios? 6. Cuál es el nivel de satisfacción de los niños? 7. En qué medida los actores se apropian de las actividades del programa? 8. Tienen los estudiantes al menos una hora de lectura? a. 30 minutos en clase y 30 fuera de la escuela 9. Se han establecido campamentos de lectoescritura al nivel comunitario u otras iniciativas parecidas para fomentar la lectura en la comunidad? 10. Que buenas prácticas de salud, higiene y nutrición a. HIGIENE: lavado de manos, cepillado de dientes, higiene personal, baño diario, peinado, cambio de ropa, uso de calzado, uso de letrina. b. SALUD: desinfección de agua para consumo humano, prevención de enfermedades, limpieza en la escuela, manejo de la basura. c. NUTRICIÓN: Alimentos con aporte energético, alimentos con proteínas, alimentos con vitaminas y minerales, alimentos según edad o sexo, alimentos con formas locales de preparación, alimentos servidos con higiene y temperatura adecuada, alimentos con sabor, textura, color local mente aceptados 11. Cuál es su apreciación de las actividades a. Huertos pedagógicos b. Desayunos escolares c. Mejora de infraestructura (cocina y agua) d. Eventos de capacitación e. Semáforo de limpieza f. Ferias escolares g. Invernaderos h. Madres liderezas voluntarias i. Campamiento de lecto escritura j. Fortalecimiento de capacidades institucionales Instrumento cualitativo No. 3-EF-FFE Padres/madres 1. Cuál es el aporte que dan los padres/madres para la actividad del programa 2. Como se lograría que el programa siga funcionando cuando finalice la participación de PCI 3. Hasta qué punto se han logrado un cambio positivo de comportamiento y prácticas de nutrición, salud e higiene en los niños? 4. En qué medida los padres/madres se apropian de las actividades del programa? 5. Participa usted en OPF, en COCODE o en otra clase de organización? 6. Cuál es el mayor beneficio que observa del programa 7. Tienen los estudiantes al menos una hora de lectura? a. 30 minutos en clase y 30 minutos en casa 63 Instrumento cualitativo No. 4-EF-FFE Miembros de OPF 1. Como se encuentra organizado el OPF 2. Cada cuanto tiempo hay cambio en los puestos 3. Se involucran personas nuevas o solamente se rotan los puestos dentro de los mismos participantes 4. Cuanto tiempo invierte usted a la semana en la organización 5. Ha recibido capacitación por parte de PCI a. Cuales 6. Cuál ha sido su mejor experiencia con el programa a. Cual ha sido una experiencia negativa 7. Los padres de familia asisten por lo menos 2 veces por año a reuniones de los OPF? a. Participan los maestros? 8. Hacen planes de compra para la refacción escolar? Instrumento cualitativo No. 5-EF-FFE Concejos Municipales 1. Cuál ha sido el aporte de la municipalidad para apoyar el programa de PCI 2. Cree usted que ésta clase de intervención es beneficiosa para los niños de las escuelas del municipio? a. Apoyarían otra clase de programas escolares? 3. Tienen ustedes convenios firmados con PCI a. Cuál es el marco del mismo 4. Cuáles son las actividades del programa que recomendaría ampliar y / o limitar, para asegurar el uso eficiente de los recursos que garanticen los mismos resultados o mayores? 5. Que acciones podría llevar a cabo el gobierno local para asegurar la continuidad de esta clase de programas 6. Cree usted que sería importante la formulación de una política pública para impulsar un programa de alimentación escolar 7. De acuerdo a su criterio, que funcionó y que no funcionó en la ejecución del programa 8. Que otras acciones le gustaría que el programa implementara Instrumento cualitativo No. 6 -EF-FFE Coordinadores Técnico Administrativos (CTA) 1. Como CTA, cuál es su nivel de vinculación con el programa de PCI de FFE II a. En que actividades ha apoyado al programa 2. Cuál es el canal de comunicación que utiliza con el programa? 3. Cuáles son las actividades del programa que recomendaría ampliar y / o limitar, para asegurar el uso eficiente de los recursos que garanticen los mismos resultados o mayores? 4. Que propone para tener una coordinación interinstitucional eficiente 64 5. Como podría hacerse sostenible el programa 6. Que aspectos deberá mejorar PCI en el futuro para hacer más eficiente el programa 7. Podría identificar y relatar dos historias de éxito dentro del término del proyecto de PCI Instrumento cualitativo No. 7-EF-FFE Personal de PCI 1. Qué mecanismos y metodología de implementación del menú de actividades cree que se debería seguir para la efectividad de los mismos 2. Qué actividades y estrategias claves fueron más efectivas para lograr cada resultado? a. ¿Cuáles fueron los menos efectivos? i. ¿Por qué? 3. Cómo se ha llevado a cabo la ejecución y el cumplimiento de las actividades del programa a nivel técnico / programático, organizativo y contractual? 4. Analizar el sistema de MyE del programa, que incluye sistemas de gestión de la información, instrumentos de mantenimiento de registros, entrada y salida de datos, uso, ventajas y desventajas de los sistemas automatizados. (MOBENZI, CMOVES, SAVIX). 5. En qué medida, PCI abordó las recomendaciones de evaluación de medio término, específicamente relacionadas con el establecimiento de sesiones de información para padres para comprender los beneficios de la educación (Recomendación 6), aumentar la frecuencia de las capacitaciones y la participación, ), promover mejoras en el mantenimiento de bodegas a nivel escolar (Recomendación 14)? 6. La herramienta de evaluación de la preparación para la sostenibilidad, diseñada por PCI, es una forma eficaz de medir la preparación para la sostenibilidad y las escuelas de posgrado? a. ¿Qué recomendaciones tiene para mejorar la herramienta? 8. ¿Cómo ha promovido el programa la equidad de género? a. ¿Qué resultados e impactos diferenciales tienen las actividades del programa en mujeres y niñas? b. ¿En hombres y niños? Instrumento cualitativo No. 8-EF-FFE MINEDUC. 1. Cuáles son las necesidades de apoyo que deben de existir de manera mutua entre el Ministerio y PCI 2. Que elementos han sido esenciales para lograr una buena coordinación con PCI a. Porque b. Que se debería de mejorar 3. Como ha contribuido el trabajo de PCI al CNB a. Y a otras necesidades de las escuelas 4. ¿Cómo contribuye el Ministerio de Educación a lograr el resultado de "Mejorar la calidad de la enseñanza de lectoescritura"? a. ¿Se está tratando adecuadamente? 65 b. ¿Cómo puede PCI abordar este resultado en la siguiente fase de implementación del programa? ¿Qué experiencias exitosas hay en la región con respecto al enfoque para este resultado? Instrumento cualitativo No. 9-EF-FFE Personal de USDA. 1. Cuáles fueron los mecanismos de coordinación e implementación que se hicieron con PCI a. Las que mejor se aplicaron b. Las que se deben de mejorar 2. Conoce la situación actual de los programas de alimentación en Guatemala a. Cuál es la percepción de futuro de esta clase de programas 3. Qué clase de coordinación y comunicación existe con MINEDUC 4. Cuáles son las herramientas que se utilizan para el análisis de resultados de los programas 5. Que retos se deberán afrontar de acuerdo a su punto de vista, para la sostenibilidad de esta clase de programas 6. Seguirán en el apoyo al importante segmento de la educación infantil en Guatemala a. Que otras áreas abarcarán Instrumento de captura de datos 1. Grupo Focal: Se llevó a cabo la metodología de grupo focal con METAPLAN, en donde las aportaciones de los participantes del grupo focal se recogen en primer lugar en palabras y luego en tarjetas. Debido a la poca o nula capacidad de escribir de los participantes, se hizo un hibrido de la metodología de captura, siendo el moderador y las auxiliares las que escribían en las tarjetas y posteriormente se colocaban de en el papelografo. La metodología se llevó a cabo con los OPF o similares, con los padres y madres de familia que asistieron a la convocatoria realizada, y en metodología completa, con los supervisores de campo de PCI. 2. Entrevistas a fondo: con Directores, Maestros, gobierno local (Alcalde o Concejal), CTAs, con personal de MINEDUC (con la Jefe del departamento de Organización Escolar) y personal de USDA, 66 Annex 4. Quantiative Evaluation Tools 1. Estudiantes Cuestionario para estudiantes Municipio: Cuilco Comunidad: Gaspar Ixchil San Escuela: Pedro Necta San Entrevistada/o: Pedro Soloma San Edad: Cruz Barillas Santa Eulalia Santa 0. Buenos días, ¿cómo te sientes el día de hoy? 1. ¿Estás de acuerdo en participar en este estudio respondiendo a las preguntas? Si No 2. ¿En qué grado estás? Cuarto Quinto Sexto 3. En este momento, ¿tienes hambre? No (Pasar a siguiente pregunta) Si ¿Tienes… Mucha Hambre Poca 4. ¿Cuáles son esos lugares donde se puede aprender o conseguir información sobre las prácticas o hábitos de salud e higiene que debe tener toda persona? (MARCAR LAS QUE RESPONDA EL/LA ESTUDIANTE) 1. Puesto de salud / Centro de convergencia 2. Centro de salud 3. Hospital 4. Clínicas de ONG / Clínicas privadas 5. Escuelas / Institutos / colegios 6. Iglesias 7. Bibliotecas 8. Otro (Especifique) _____________________________________ 9. No sabe ¿Cuántas marcó? 67 6. ¿Cuáles es lugar/es donde se puede aprender, o conseguir información sobre buenas prácticas o hábitos de alimentación, dieta y nutrición? 1. Puesto de salud / Centro de convergencia 7. Bibliotecas 2. Centro de salud 8. Otro (Especifique) 3. Hospital 9. No sabe 4. Clínicas de ONG / Clínicas privadas ¿Cuántas marcó? 5. Escuelas / Institutos / colegios 6. Iglesias 7. La semana pasada, ¿faltaste algún día a clases? Si (Pasar a la siguiente pregunta) No (Dar gracias y terminar) 8. ¿Cuántos días faltaste? 9. ¿Por qué faltaste a clase? Por enfermedad 9.1 ¿De qué te enfermaste? Trabajo en casa Del estómago Trabajo en parcela Fiebre No tuve ganas Gripe viaje familiar Tos otros Dolor de muela Dolor de oído 2. Padres de familia Cuestionario para padres de familia Municipio: Cuilco Comunidad: San Gaspar Ixchil San Pedro Necta San Pedro Soloma Santa Cruz Barillas Santa Eulalia 0. Buenos días, ¿cómo se siente el día de hoy? 1. ¿Está de acuerdo en participar en este estudio respondiendo a las preguntas? Si No 68 2. ¿Cuáles son esas prácticas o hábitos de salud o higiene que debe tener toda persona? 1. Baño diario 5. Cambio de ropa 2. Lavado de manos 6. Uso de calzado 3. Cepillado de dientes 7. Uso de letrina o inodoro 4. Peinado 8. Otra (Especifique) ¿Cuántas marcó? 3. ¿Cuáles son esos lugares donde se puede aprender o conseguir información sobre las prácticas o hábitos de salud e higiene que debe tener toda persona? 1. Puesto de salud /Centro de convergencia 6. Iglesias 2. Centro de salud 7. Bibliotecas 3. Hospital 8. No sabe 4. Clínicas de ONG / Clínicas privadas 9. Otro (Especifique) 5. Escuelas / Institutos / colegios ¿Cuántas marcó? 4. ¿Qué cosas o aspectos de nutrición o alimentación se deben tomar en cuenta? 1. Buen aporte energético 5. De acuerdo con formas locales de preparación 2. Proteína de alto valor biológico 6. Servidos con higiénicas , y temperatura adecuadas 3. Buen aporte de vitaminas y minerales (vitamina “A”, hierro y zinc) 7. Con sabor, textura, y/o colores localmente aceptados 4. Adecuados según requerimientos de edad y/o sexo 8. Otro (Especifique) 5. ¿Cuáles es lugar/es donde se puede aprender, o conseguir información sobre buenas prácticas o hábitos de alimentación, dieta y nutrición? 1. Puesto de salud /Centro de convergencia 6. Iglesias 2. Centro de salud 7. Bibliotecas 3. Hospital 8. No sabe 4. Clínicas de ONG / Clínicas privadas 9. Otro (Especifique) 5. Escuelas / Institutos / colegios ¿Cuántas marcó? 6. Cuando se hacen preparaciones o comidas para grupos grandes de niños/as, ¿Qué cuidados básicos de almacenamiento y seguridad higiénica de alimentos y preparaciones, se deben tomar en cuenta? (inocuidad de alimentos) 1. La comida debe taparse 2. Uso de agua entubada, si no hay agua entubada, el agua está almacena en recipientes donde no quepan las manos, para servirla usan algún chorrito o tipi tap. 3. La comida se procesa en mesas o tablas separadas según sea carne, verdura o fruta 4. Debe de haber un basurero, tapado 69 5. No debe de haber residuos de comida o basura en el piso 6. Los cocineros tienen que lavarse las manos antes de cocinar, y hacerlo bien (lavado con jabón y agua corriendo, y por lo menos 20 segundos, y secarse bien con toallas limpias no con gabacha) 7. Al cocinar, las cocineras no usan anillos ni tienen pintadas las uñas 8. Se hace análisis microbiológico al agua (que no haya presencia de bacterias E.coli o coliformes) 9. No deben de haber animales sueltos en la cocina (perros, gatos, gallinas, etc). 10. Otro (Especifique) __________________________________ 11. No sabe / No responde 7. ¿Qué cosas aprenden o que ganan para su vida los niños/as que van a estudiar en la escuela primaria de esta comunidad? (beneficios de la educación) Aprenden: 1. A leer y escribir 8. Sobre como nutrirse bien/ evitar desnutrición 2. A Hacer cuentas y cálculos 9. Ser buenos ciudadanos 3. A hablar bien 10. Respetar los demás 4. A conoce y valorar su cuerpo 11. Para entrar a un trabajo 5. Prácticas de higiene 12. Cómo base, para poder estudiar más 6. Cómo alimentarse bien 13. Otro (Especifique) __________________________________ 7. Tener buena salud / prevenir enfermedades 8. Ha observado si su(s) hijo(s) leen por lo menos 30 minutos en la casa? Si No Muchas Gracias por su tiempo! 70 Annex 5. Sustainability Evaluation Tool Herramienta de evaluación de sostenibilidad FFE II 2016 Condición No. Actividades Definición Punteo 1. Desayuno escolar proveido diariamente al 100% de estudiantes 1 Los padres están preparando y sirviendo la comida escolar todos los días de clase (verificar calendario) Escuela con regularidad en el consumo de desayunos. Revisar calendario de consumo. SI: Todos los días se sirven desayunos, independientemente si es con alimentos PCI, MINEDUC o combinación de ambos. 4 2 Por lo menos 30% de los alimentos utilizados para la preparación del desayuno escolar vienen de recursos no￾USDA (padres, huertos, grupos ME, comunidad, MINEDUC) (Efectuar cálculos con el director) Escuela con aportes regulares de padres para preparar el desayuno Revisar menús en calendarios, revisar COLOS, preguntar a madres. Efectuar cálculos con director/a. SI: 30% de aportes provienen de padres, huertos, MINEDUC, etc. 4 3 Comités de refacción escolar y los consejos de padres de familia implementan planes de compras (incluyendo compra de alimentos nutritivos) para utilización optima del subsidio del MINEDUC para conformar la ración base Escuela con planificación de sus compras, combinando alimentos formadores, energéticos y protectores. Revisar productos en bodegas, facturas, calendario de consumo, etc. SI: Hay variación en las compras (no duplicación de productos) 4 4 Se ha incrementado la cantidad de alimentos de fuentes no USDA (Ej. padres, huertos, grupos ME, comunidad, MINEDUC) desde la medición del año anterior. (verificar salas situacional,registros de la escuela,colos, etc Escuela donde hay aumento de contribuciones de padres, producción en huertos, aportes MINEDUC, otros. Revisar COLOS, consultar con director/a y CPF, madres, sala situacional, otros. SI: Hay evidencia o testimonios de un aumento respecto a año anterior 4 5 Desembolso oportuno del subsidio MINEDUC a la escuela (Escuela priorizadas: Q1.61/día área urbana, Q2.08 área rural; Escuelas no priorizadas: Q1.11 área urbana, Q1.58 área rural) (consultar con el director/a) Escuela con regularidad en recibir aportes del MINEDUC, por lo que se tiene disponibilidad de alimentos en todo momento. Consultor con director/a. SI: Hay evidencia de que los desembolsos han sido oportunos. 4 2. Mejoramiento de Lecto￾escritura de Estudiantes 6 Los estudiantes leen por lo menos 60 minutos diarios (30 min dentro de la clase; 30 min fuera de la clase) (observación y consulta con profesores) Escuela que promueven lectura permanente, como indica la pregunta. Consultar con director/a, profesores, niños, niñas, padres; observación SI: En todos los grados se leen 30 minutos y si la mayoría de estudiantes/padres indican que se lee en casa/otro lugar 4 71 7 Se han establecido campamentos de lectoescritura al nivel comunitario u otras iniciativas parecidas para fomentar la lectura en la comunidad? Escuela con actividades extra-aula para fomentar la lectura. Consultar con director/a, profesores, padres, estudiantes. SI: Hay evidencia de que las actividades se realizan en forma planificada, al menos 1 vez al mes. 4 8 100% de estudiantes con material de lectura permanente y disponible para alumnos (consultar con director/a, observación) Escuela con libros, revistas u otros materiales de lectura ubicados en un lugar específico y que es utilizado por los estudiantes. Verificar con director/a, observación. SI: Hay evidencia de material de lectura en un sitio específico. 4 9 Asistencia de los maestros es monitoreada y reportada a las escuelas y al nivel municipal (consultar con directores/as) Escuela con sistema de control (libro de registro, cuaderno de asistencia, otro), donde se anota diariamente la asistencia de docentes. Verificar registros con director/a. SI: Hay evidencia del registro y su envio a CTA, otros. 4 10 Los padres asisten por lo menos 2 reuniones de los comités de padres (incluyendo maestros) por año (consultar con director o CPF) Escuela que promueve reuniones periodicas con padres/madres de familia y éstos asisten con regularidad. Verificar con CPF, director/a, libros de actas, etc. SI: Más del 90% de padres, asisten al menos, a 2 reuniones por año; incluyendo la mayoría de maestros (90%) 4 3. Prácticas de Higiene y Nutrición para Rendimiento Escolar 11 Los alumnos practican un buen lavado de manos en la escuela (una demostración, en un grado seleccionado al azar) Escuela con práctica permanente de lavado de manos, realizándolo correctamente. Se observa o se realiza una demostración en un grado seleccionado al azar. SI: Más del 90% se lavan las manos correctamente. 4 72 12 Los maestros o directores, enseñan al menos tres buenas prácticas de salud, higiene o nutrición (preguntar a un maestro,si menciona al menos TRES seleccionar SI menos de tres Seleccionar NO): HIGIENE: lavado de manos,cepillado de dientes,higiene personal,baño diario,peinado,cambio de ropa,uso de calzado,uso de letrina. SALUD: desinfección de agua para consumo humano,prevención de enfermedades,limpieza en la escuela,manejo de la basura. NUTRICIÓN: Alimentos con aporte energético,alimentos con proteínas,alimentos con vitaminas y minerales,alimentos según edad o sexo,alimentos con formas locales de preparación,alimentos servidos con higiene y temperatura adecuada,aliementos con sabor,textura,color local mente aceptados Escuela que fomentan prácticas de salud, higiene y nutrición, en todos los grados. Seleccionar un grado al azar y preguntar al maestro, cuáles prácticas de salud, nutrición o higiene enseña regularmente a sus alumnos. (Ver listado) SI: El maestro/a menciona al menos tres buenas prácticas de salud, higiene o nutrición. 4 13 Los padres o madres, mencionan al menos tres buenas prácticas de salud, higiene o nutrición (consultar a un padre + una madre, si menciona al menos TRES seleccionar SI menos de tres Seleccionar NO): HIGIENE: lavado de manos,cepillado de dientes,higiene personal,baño diario,peinado,cambio de ropa,uso de calzado,uso de letrina. SALUD: desinfección de agua para consumo humano,prevención de enfermedades,limpieza en la escuela,manejo de la basura. NUTRICIÓN: Alimentos con aporte energético,alimentos con proteínas,alimentos con vitaminas y minerales,alimentos según edad o sexo,alimentos con formas locales de preparación,alimentos servidos con higiene y temperatura adecuada,aliementos con Escuela donde los padres/madres conocen buenas prácticas de salud, nutrición e higiene. Entrevistar a un padre y a una madre (total: 2) y preguntar ¿Cuáles son las prácticas de salud, higiene o nutrición que debemos practicar? (Ver listado) SI: El padre + la madre (los dos) respondieron, al menos 3 de las posibles respuestas. 4 73 sabor,textura, color, localmente aceptados 14 La escuela cuenta con interiores limpios (incluyendo aulas, cocina y bodegas de alimentos) (observación en la escuela) Escuela que promueve la limpieza en los interiores de la misma (aulas, cocina y bodega de alimentos). Observar que los tres ambientes estén limpios (sin evidencia de basura, roedores, planificación de limpieza por estudiantes, etc.) SI: Los tres ambientes están limpios. 4 15 La escuela cuenta con exteriores limpios (incluyendo letrinas, estaciones de lavado de manos, patio, y alrededores de le escuela) (observación en la escuela) Escuela que promueve la limpieza en los exteriores de la misma (letrinas, estaciones de lavado de manos, patio y alrededores de la escuela). Observar que los cuatro ambientes estén limpios (sin evidencia de basura, planificación de limpieza por estudiantes, etc.) SI: Los cuatro ambientes están limpios. 4 4. Infraestructura Adecuada 16 Cocina con estufa existente y funcionando (observación) Escuela con estufa funcionando, independientemente si fue apoyada por PCI o no. Observación en el área de cocina. SI: Cocina funcionando adecuadamente: libre de humo, animales no entran, hay espacio para trabajar adecuadamente, no se cocina en el suelo, etc. 4 17 Bodega escolar existente y funcionando (observación) Bodega funcionando adecuadamente con alimentos PCI, MINEDUC, aportes de CPF. Observación en el área de bodega. SI: Bodega funcionando adecuadamente en la mayoría de los aspectos: buena ventilación, libre de humedad, limpieza permanente, alimentos alejados de la pared, libre de roedores, los productos están ordenados, etc. 4 18 Disponibilidad de agua segura para consumo humano (observación) Escuela con acciones para consumo de agua segura. Observación de las áreas de consumo de agua para estudiantes y los mecanismos de consumo. SI: Escuela con filtro, ecofiltro, recipientes con agua hervida, clorada u otra forma que asegure consumo de agua segura. 4 19 Letrinas existentes y funcionando (Observación) Escuela con letrinas que funcionan adecuadamente. Observación en área de letrinas/inodoros/baños, otros. SI: Letrinas limpias o con evidencia de limpieza permanente por estudiantes, tiene basurero y lavamanos. 4 74 20 Lavamanos existentes y funcionando (Observación) Escuela con lavamanos funcionando adecuadamente. Observación. SI. Lavamanos tienen agua permanente, los chorros están en buen estado, no hay fugas de agua, es útil para todos los estudiantes. 4 5. Apoyo municipal y comunitario 21 Grupos de Mujeres Empoderadas funcionando y los miembros de la familia usan el ahorro para apoyar a sus hijos en la escuela (ver libro de actas, preguntar a mujeres participantes del grupo) Escuela / comunidad con grupo de ahorro ME activo y donde se realizan préstamos para educación. Verificar en Libro de actas del grupo o consultar con las participantes. SI: al menos una persona ha realizado un préstamo para educación. 4 22 Aumento del 3% anual del aporte municipal a las escuelas para transporte de alimentos, mano de obra calificada para infraestructura, compra de alimentos locales, etc. (consultar con director/a) Escuela que es apoyada por su municipalidad en concepto de transporte de alimentos, mano de obra calificada para infraestructura, compra de alimentos, construcción de aulas, etc. Verificar con director, CPF, COCODE, otros; revisar COLOS/anotaciones del técnico, etc. SI: Hay evidencia de que la escuela ha tenido más del 3% de aporte municipal, respecto al año anterior. 4 23 La Comisión de Sostenibilidad u otra similar está establecida y se reúne por lo menos 3 veces al año (consultar con director/a, puede ser otra comisión similar a nivel comunitario o municipal) Escuela con participación en comisiones de sostenibilidad u otra comisión (comunitaria, municipal) que fomente la continuidad de actividades del programa de alimentación escolar. Consultar con Director, CPF, COCODE, otros. SI: Hay al menos, un participante (maestro, padre/madre de familia, otro miembro comunitario) que esté participando de la comisión de sostenibilidad / otra organización similar, que haya participado al menos en 3 reuniones, en el último año. 4 24 La Comisión de Sostenibilidad tiene un plan de trabajo para la sostenibilidad de la alimentación escolar y otras actividades del programa (preguntar con director/a) Escuela con miembros participando en comisiones de sostenibilidad u otras similares (comunitario, municipal) y que cuenten con un plan de trabajo para la sostenibilidad de la alimentación escolar y otras actividades del programa (consultar con director/a, CPF, otros). SI: Hay evidencia del plan de trabajo y que esté en implementación. 4 75 25 Escuela recibe apoyo de otras iniciativas/programas dentro del departamento. (Preguntar con director/a) Escuela beneficiada de instituciones del sector público o privado (diferente a MINEDUC/MSPAS), para apoyar la alimentación u otra actividad escolar (construcción de aulas, dotación de agua segura, etc.) SI: Hay evidencia de beneficio institicional, privado o público, para beneficio de la comunidad educativa (alimentación, infraestructura, insumos, materiales didácticos, etc.) 4 Total 100