Jorge Matute, Héctor Gamero, Christa Haeussler, Fabiana Flores 03/07/2014 BASELINE REPORT Food for Education – FFE Project - Phase II Huehuetenango, Guatemala PCI Guatemala Funded by the United States Department of Agriculture Cooperative Agreement No. FFE-520-2013/034-00 Baseline Report for FY 2013 McGovern-Dole Food for Education Phase II Project in Huehuetenango, Guatemala Program: McGovern-Dole International Food for Education and Child Nutrition Agreement Number: FFE-520-2013/034-00 Funding Year: Fiscal Year 2013 Project Duration: 2013-2016 Implemented by: TechnoServe Evaluation Authored by: Ciensa-JMatute Jorge Matute Héctor Gamero Christa Haeussler Fabiana Flores DISCLAIMER: This evaluation was conducted by an independent third party firm. The views expressed in this report do not necessarily reflect the views of the United States Department of Agriculture or the United States Government. Accessibility Note: An accessible version of this report can be made available upon request by emailing FAS.monitoring.evaluation@usda.gov. i Table of Contents Acknowledgements ........................................................................................................................................iii Acronyms.......................................................................................................................................................iii Executive Summary........................................................................................................................................iv Introduction...............................................................................................................................................iv Methodology .............................................................................................................................................iv SPECIFIC METHODOLOGIES....................................................................................................................... v Results and findings..................................................................................................................................... v Qualitative Component............................................................................................................................ v Quantitative Component.........................................................................................................................vi Conclusions...............................................................................................................................................xii Recommendation ......................................................................................................................................xii I. Introduction and Background........................................................................................................................ 1 II. Methodology .............................................................................................................................................. 1 Qualitative methodology................................................................................................................................. 1 Quantitative methodology.............................................................................................................................. 2 II.1 Sampling Design .................................................................................................................................... 2 II.2 Sample Sizes.......................................................................................................................................... 3 Sample sizes by target populations........................................................................................................... 3 II.3 Sample selection within the clusters....................................................................................................... 4 II.4 Indicators and instruments..................................................................................................................... 4 II.5 Data entry and statistical analysis........................................................................................................... 6 II.6 Field team, training/standardization and field work ................................................................................ 6 Specific Methodologies ................................................................................................................................. 10 Literacy Calculation for the Students..........................................................................................................13 III Characterization of the Quantitative Sample ..............................................................................................14 IV Results and findings ..................................................................................................................................15 Qualitative Component ................................................................................................................................. 15 Quantitative Component...............................................................................................................................18 I Enrollment and school coverage (Increased Student Enrollment)...........................................................19 II Improved Student Attendance .............................................................................................................24 III Health and nutrition educational activities (Contribution to Feed the Future / Improved Knowledge of Health/Hygiene Practices)......................................................................................................................31 IV Improved Attentiveness......................................................................................................................41 ii V Improved Reading ability of school-age children..................................................................................44 VI School Feeding (Reduced short term hunger, increased knowledge of safe food prep and storage practices, increased access to requisite food prep and storage tools and equipment)................................48 VII School Infrastructure (Increased use of health and dietary practices/Improved school infrastructure)..56 V. Conclusions..............................................................................................................................................61 VI. Recommendation....................................................................................................................................62 Annexes........................................................................................................................................................63 ANNEX 1 - FIELD REPORT FOR PCI BASELINE SURVEY IN COMMUNITIES OF HUEHUETENANGO .........................64 Annex 2 - Sampling Design for PCI Baseline Study 2014...................................................................................69 Sampling Design ....................................................................................................................................69 Selected Schools .....................................................................................................................................70 Annex 4 Reading used for the Evaluation .......................................................................................................72 Annex 5 Summary Table of Indicators for FFE II Program.................................................................................73 iii ACKNOWLEDGEMENTS This report provides a detailed description of the baseline for Phase II of the PCI Food for Education (FFE) Program that is being carried out in Huehuetenango, Guatemala. The consultant group CIENSA-JMatute wishes to express its thanks to the Program staff for their collaboration as well as that of all those who were directly or indirectly involved in the study, i.e., students, teachers, directors, parents, and community leaders. We are also grateful for the collaboration provided by the representatives of the local Ministry of Education (MINEDUC) office which was visited by our anthropologist. ACRONYMS COCODE Community Development Committee (Comité Comunitario of Desarrollo) FFE Food for Education FG Focus Groups MINEDUC Ministry of Education MSPAS Ministry of Health OR Odds Ratio PCI Project Concern International TAC Technical Administrative Coordinator for education at the local level (Coordinador Técnico Administrativo of educación al nivel local) USAID United States Agency for International Development USDA United States Department of Agriculture iv Baseline Report Food for Education (FFE) Project - Phase II in Huehuetenango, Guatemala PCI Guatemala JMatute-CIENSA EXECUTIVE SUMMARY Introduction In April 2014, PCI Guatemala hired CIENSA-JMatute to carry out a baseline study for the Food for Education Project Phase II (approved/funded by USDA/FAS Grant Agreement: FFE-520-2013/034) which began this year in Huehuetenango, Guatemala. Methodology This baseline study was carried out using quantitative methods (interviews and statistical measurements) to calculate indicators and for measuring knowledge, attitudes and practices, and qualitative methods were used for the experiences and perceptions of relevant Huehuetenango Education Ministry officials identified by PCI Guatemala. Five individual interviews were carried out with: two Technical Administrative Coordinators (TAC) from the Municipalities of Cuilco and Santa Cruz Barillas and three officials from the Technical Pedagogical Bureau located in Huehuetenango. The second phase of this Food for Education Program will be carried out with 279 schools in six municipalities of the department of Huehuetenango. Of these 279 schools, 174 participated in Phase I and 105 new schools are being added for Phase II. The sampling design for measurement of indicators in this baseline took into account the status of each school (Phase I participant or new) and the region where it is located (Mam/Q’anjobal). For this purpose a stratified design was used with the strata being defined as a combination of ‘school status’ and region. Four strata were identified. Within each stratum schools were selected using probability-proportional-to-size (PPS) sampling. Within the sampling design, each school was then considered as a cluster. Additionally, the communities where the selected schools were located were automatically selected for the sample of parents at the community level, and each community was also considered as a cluster. The target populations for measuring the indicators and the sample sizes for in each are as follows: o Directors and teachers: n=73 o Students (4th to 6th grade): n=358 o Parents belonging to the Parent Council: n= 59 o Schools records (documentation): n= 26 schools1 o Schools (observation process): n = 25 schools (one school could not be evaluated because of its Mother’s Day celebration) • In the community: 1 The records from one of the schools were made available during the weekend since it was not possible to visit the school on a school day due to their Mothers Day celebration. v o Parents (mostly mothers): n= 477 o Children between 5 and 12 years of age: n= 441 o Youth between 13 and 15 years of age: n= 177 SPECIFIC METHODOLOGIES In this study two measurement methodologies were implemented for indicators that required specific, out-of-the-ordinary strategies (not surveys, interviews, or document reviews). The first of these was to measure students’ attentiveness in class in order to determine the percentage of children who are inattentive in class. For this strategy, three different classrooms were chosen at random in schools with more than three classrooms. In schools with three or fewer classrooms, all were included in the study. In each classroom, one class period was chosen at random and in the school with only one classroom, three different class periods were chosen. Over a 10 minute period, the observer randomly chose one minute during which to observe the students’ behavior with regard to the class that was being taught. It is important to mention that these observations were only carried out when the teacher was giving a “normal” class, in which he/she interacted with students frequently using audio-visual aids such as the blackboard. The observations were carried out during class periods such as math, social studies, natural sciences and language. Classes with dictation or artistic subject matter or where students needed to come and go from the classroom were avoided. The second special methodology was to measure the reading ability of students who had completed sixth grade. Students who had passed sixth grade in 2013 in the target schools were identified and were located in the community (or in their secondary school, if they were attending). These students were presented with reading material based on the expected level for sixth grade students and they were asked to read out loud. Their reading was recorded. The reading evaluation for these children focused on their ¨reading ability¨ (the child’s competency in reading) and this was analyzed in three ways: speed, accuracy and comprehension based on the indicators established by the Ministry of Education for completion of sixth grade. Results and findings Qualitative Component The five officials from the Ministry of Education who were interviewed were familiar with the Food for Education program and had participated in meetings with technical staff, in trainings, in the preparation of workshops, in the Sustainability Commission and in fairs. According to those who were interviewed, this is a program that PCI is supporting with food commodities and at the same time promotes healthy eating habits to strengthen students’ academic performance. All the interviewees emphasized that the department of Huehuetenango has high levels of poverty and malnutrition and thus the school breakfast or snack is the first meal of the day for a large majority of students. They also agreed that this is an incentive for students and helps to lower dropout rates as “the school breakfasts sustain children’s attendance rates” as well as “strengthening educational quality”. Those who were interviewed also recognized the program’s importance given that it supports families which tend to be large, with nutritive foods that help strengthen academic performance: “obviously good health is a key factor for intellectual development”. They also think that it is very effective since in the three years since the program began, the number of students who finish sixth grade has increased. Previously, most students only finished third grade and then their parents sent them out to work.” The school breakfast or snack is an incentive for the children because, “the school day is long and the majority of children arrive without having eaten”, “thus the children anxiously await the vi moment when they will receive breakfast” and “the snack or breakfast are often the only reason some children go to school”. With regard to the efficiency of the program, one of the participants indicated that more time is needed to assess whether or not the program has been efficient, but that according to what he had heard, the results are acceptable. With regard to the program’s sustainability and the support that the MINEDUC can offer, one of the interviewees said that, “we must be realistic” and understand that the current situation in the communities is precarious and for this reason the process is not sustainable. They also noted that the lack of adequate training has implications for achieving sustainability. Those who were interviewed indicated that at this time the MINEDUC has no program or project that is evaluating progress in reading and writing since it is still in the implementation process. They noted that the Ministry of Education is using a strategy to improve reading and writing in primary grades and as part of this strategy they spoke of the importance of placing bilingual teachers in the early grades, “to eliminate culture shock and barriers to learning.” A book on Teaching Strategies for Reading Comprehension has also been published. The lack of human resources is also a limiting factor given that there are many students but few teachers. There is an enormous need for contextualized staff who are bilingual in Mayan languages, “Huehuetenango is unique (a little Guatemala) with 10 languages spoken here”. Finding trained, competent staff comfortable in the context of the area where they would need to work is a huge challenge. Furthermore, the communities are very remote and access is difficult so teachers that the Ministry does hire do not want to work there. To cover the lack of teachers in the schools, the Ministry trained multi-grade teachers. There is a guide for working with one group of children in the classroom while letting another group work with a leader. Some teachers have also taken the initiative to change the school schedule so that they work with one group in the morning and another in the afternoon and some communities have decided to hire a teacher from the community itself to increase staff retention. Quantitative Component Enrollment and school coverage Table 1 Enrollment and school coverage Indicator Percentage o mean Percentage of 7 year olds enrolled in school in 2014 (community sample) 97.2% Percentage of 7 year old girls enrolled in school in 2014 (community sample) 94.9% Mean change in total school enrollment in 2014 compared to 2013, entire school (school records) -5.5% Mean change in school enrollment of boys in 2014 compared to 2013, entire school (school records) -5.6% Mean change in school enrollment of girls in 2014 compared to 2013, entire school (school records) -5.2% Mean change in total first grade enrollment in 2014 compared to 2013, entire school (school records) -6.3% Mean change in first grade enrollment of boys in 2014 compared to 2013, entire school (school records) -0.04% Mean change in first grade enrollment of girls in 2014 compared to 2013, entire school (school records) -3.4% Parents who value school education for their children: mention at least three benefits or reasons (community sample) 55.8% vii According to the results obtained at the community level, the number of seven year old children enrolled in first grade this year (2014) is greater in the Q’anjobal region (97.8%) than in the Mam region (93.6%). According to school records, in 2013 there was no increase in school enrollment compared to 2012. In 2014 there was a significant decrease in total enrollment of children (all grades) compared to 2013, with an average for the entire study population of 5.5%. When talking to directors and teachers, 84.4% felt that there was an increase in enrollment in 2014 compared to 2013 in the Mam region while only 13.5% felt there was an increase in the Q’anjobal region. More parents in the Q´anjobal region stated that they had problems or difficulties for enrolling their children than those in the Mam region (25.7% compared to 6.5%, respectively). The majority of parents in the Q’anjobal region (82.3%) appreciated or valued school attendance for their children because they want them to learn to talk well, while a lower percentage of parents in the Mam region stated this reason (56.7%). On the other hand, in the Mam region parents valued school attendance because it helps their children respect others and acquire a solid foundation to be able to study more in the future (27.7% compared to 10.7%, and 34.0% compared to 10.7%, for these two reasons, respectively). Improving school attendance Table 2 Improving school attendance Indicator Percentage Percentage of boys who attended school at least 80% of the time during the last 10 class days prior to the baseline measurement, April 2014 (community sample) 98.0% Percentage of girls who attended school at least 80% of the time during the last 10 class days prior to the baseline measurement, April 2014 (community sample) 97.4% Percentage of boys and girls who attended school at least 80% of the time, during 2013 (cohort sample, school records) 84.7% Percentage of students (boys) who claim to have attended school every day of the previous week (sample of students from 4th to 6th grades) 85.1% Percentage of students (girls) who claim to have attended school every day of the previous week (sample of students from 4th to 6th grades) 84.9% Percentage of students (boys) who stated that in the last four weeks they missed school for health reasons (sample of students from 4th to 6th grades) 19.1% Percentage of students (girls) who stated that in the last four weeks they missed school for health reasons (sample of students from 4th to 6th grades 20.1% The results show very high and similar percentages for good school attendance in 6 to 12 year old students according to the community survey (above 85 percent in all cases). Nevertheless, a higher percentage of children in the Q´anjobal area compared to the Mam area reported that they had attended at least 80% of classes in the last 10 days of school prior to baseline measurement (98.4% compared to 94.1%, respectively). It is important to highlight that the percentages of attendance for children from 5 to 12 years of age based on the methodology of “cohort or follow up group,” were also very high. At the opinion level, the results show that a large proportion of the directors and teachers interviewed (above 85%) reported that school attendance was very good during 2013. In spite of this, it was observed that a similar number of these professionals (around 87%) reported that on the day of the survey there was at least one student absent in their respective classrooms and health reasons were the most frequently mentioned reason for this. In the Mam region there was a 2.8 times greater probability (Odds Ratio - OR) of reporting that students were absent due to health reasons than in the Q´anjobal region. It was noted that approximately half of the directors and teachers interviewed in both linguistic regions felt that health reasons were a principal cause for absences. viii A large percentage of students stated that they had attended class the week prior to the survey. The percentage in the Q´anjobal region was higher than in the Mam region (88.5% vs. 79.5%). It was also observed that approximately 15% of the students interviewed stated that some of their classmates “didn’t come to school today” for health reasons. School absences for this same cause during the previous week were reported by half of the children who were interviewed. Educational activities related to health and nutrition Table 3 Educational activities related to health and nutrition Sample of students from 4th to 6th grades Indicator Percentage Percentage of students (boys and girls) who know the main health/hygiene practices or habits that everyone should have. Mention at least 3 of the 7 71.4%) Percentage of students (boys) who know the main health/hygiene practices or habits that everyone should have. Mention at least 3 of the 7 60.4% Percentage of students (girls) who know the main health/hygiene practices or habits that everyone should have. Mention at least 3 of the 7 85.5% Percentage of students (boys and girls) who know the main health/hygiene practices or habits that everyone should have. Mention at least 4 of the 7 39.8% Percentage of students (boys) who know the main health/hygiene practices or habits that everyone should have. Mention at least 4 of the 7 30.6% Percentage of students (girls) who know the main health/hygiene practices or habits that everyone should have Mention at least 4 of the 7 51.7% Percentage of parents who know the main health/hygiene practices or habits that everyone should have. Mention at least 3 of the 7 (sample of members of Parent Council) 55.4% Percentage of parents who know the main health/hygiene practices or habits that everyone should have. Mention at least 4 of the 7 (sample of members of Parent Council) 29.6% Percentage of parents who know the main health/hygiene practices or habits that everyone should have. Mention at least 3 of the 7 (community sample) 58.7% Percentage of parents who know the main health/hygiene practices or habits that everyone should have Mention at least 4 of the 7 (community sample) 24.7% A high proportion of the students (above 80%) stated that they were trained in some of the areas of hygiene, preventive health, food preparation or nutrition; nutrition was the area that was least mentioned (68.9%). The values for training in preventive health and food preparation were higher in the Mam region compared to the Q´anjobal area: (88.2% vs. 75.9%, and 82.5% vs. 64.9%). A very high percentage of children (over 90%) stated that they knew the principal health/hygiene practices that everyone should have. Nevertheless, these results show that only 37.6% of the Mam children and 41.2% of the Q´anjobal students were actually able to cite four of the seven practices, with girls having the highest percentages. These results also showed that the hygiene practice of daily baths is the most widely known among the Mams, and that of combing hair is the most commonly known among the Q´anjobals. As is to be expected, the latter was reported by a higher proportion of girls than boys. Approximately 30% of the members of the Parent Councils knew the health/hygiene practices that everyone should have. It can be noted that the Parent Council members in the Mam region were more aware of the practice of daily baths than those in the Q´anjobal region (100% compared to 47.0%). Only a quarter of the fathers/mothers interviewed in the community knew four of the seven health/hygiene practices or habits. The parents in the Mam region were more aware of the habits of daily baths and changing clothes than those of the Q´anjobal region (91.4% vs. 74.1, and 74.0% vs. 45.0%, respectively). Attentiveness in class ix Table 4 Attentiveness in class Indicator Percentage Mean of the Percentage of students who were inattentive (presented at least one of the five criteria) (observations in the classroom) 60.4% Percentage of students (boys and girls) who thought that their attention in the last class period was good (sample of students from 4th to 6th grades) 87.8% Percentage of students (boys) who thought that their attention in the last class period was good (sample of students from 4th to 6th grades) 83.8% Percentage of students (girls) who thought that their attention in the last class period was good (sample of students from 4th to 6th grades) 93.1% The indicator for inattentiveness in students was higher for those observed in the Q´anjobal region (77.4%) compared to 36.6% in the Mam region, a gap of 40.8% between the two groups. A high percentage of the students (above 85%) considered that their attention in the class they were receiving or had just received was good, with girls rating themselves higher than boys (93.1% compared to 83.8%). The principal causes of students’ inattentiveness just prior to the interview (those who considered their attention as merely okay or bad) were that: a) they don’t understand or it is hard for them to understand what the teachers are presenting (46.3%); b) the topic of the class doesn’t motivate them (40.7%), and c) they are very restless and it’s hard to sit still (23.0%). Reading ability in students who completed sixth grade Table 5 Reading ability in students who completed sixth grade Sample of students who completed 6th grade in 2013. The mean was on a scale of 0 to 100 points. Indicator Percentage or mean Mean of the percentage of reading ability -1 in students (boys and girls) 64.7 points Mean of the percentage of reading ability -1 in students (boys) 63.7 points Mean of the percentage of reading ability -1 in students (girls) 66.0 points Percentage of students (boys and girls) who comply with the expected reading standard (at least 81 points) reading ability -1 5.7% Percentage of students (boys) who comply with the expected reading standard (at least 81 points) reading ability -1 2.5% Percentage of students (girls) who comply with the expected reading standard (at least 81 points) reading ability -1 9.7% Mean of the percentage of reading ability -2 in students (boys and girls) Weighting: speed 10%, accuracy 20%, comprehension 70% 55.9 points Mean of the percentage of reading ability -2 in students (boys) Weighting: speed 10%, accuracy 20%, comprehension 70% 54.3 points Mean of the percentage of reading ability -2 in students (girls) Weighting: speed 10%, accuracy 20%, comprehension 70% 58.0 points Percentage of students (boys and girls) who comply with the expected reading standard (at least 81 points) reading ability -2 5.3% Percentage of students (boys) who comply with the expected reading standard (at least 81 points) reading ability -2 2.5% Percentage of students (girls) who comply with the expected reading standard (at least 81 points) reading ability -2 8.8% Percentage of directors and teachers who claim that the reading ability of the students who complete sixth grade meets the expected standards (sample of teachers and directors) 24.5% According to the indicators for reading ability 1 and 2, girls have a higher percentage in the standard category than boys. Especially with regard to reading ability 2, it can be seen that in general, girls score x higher than boys. Similarly, girls had higher marks for accuracy, with the percentages for them being greater (30.9%) in the standard category and lower in the low category (6.4%), while boys had percentages of 13.7% and 24.6%, respectively. This can be linked to the fact that at these ages girls tend to be more mature than boys. Social roles and programs that have been implemented to support girls’ education at earlier ages should also be taken into account. Reading ability -2 is better in the Mam region than in the Q’anjobal area. It is important to highlight that in terms of reading ability-2, based on the points average (55.9 points), the children’s levels were generally low. Teachers in the Mam and Q’anjobal groups feel that improving comprehension should be given more emphasis than the other skills. Perhaps they are not aware of the relationship that exists with the other skills in order to develop efficient readers with the expected competency level by the end of primary school. This was confirmed especially in the Mam (89.9%) group as they stated that specialized texts on reading processes were a priority although a lesser percentage in the Q’anjobal group (54.8%) also mentioned this. School Feeding Table 6 School Feeding Indicator Percentage Percentage of students (boys and girls) who ate breakfast at home (today) (sample of students from 4th to 6th grades) 96.3% Percentage of students (boys and girls) who were hungry or very hungry at the time of the interview (sample of students from 4th to 6th grades) 9.3% Percentage of students (boys) who were hungry or very hungry at the time of the interview (sample of students from 4th to 6th grades) 9.5% Percentage of students (girls) who were hungry or very hungry at the time of the interview (sample of students from 4th to 6th grades) 8.9% Percentage of teachers and directors who think that there are students today who are hungry or very hungry at the time of the interview (sample of teachers and directors) 51.4% Percentage of parents who know at least 3 of the 15 evaluated conditions for ensuring food safety (Food safety knowledge indicator) (sample of Parent Council) 74.6% Percentage of parents who know at least 6 of the 15 evaluated conditions for ensuring food safety (Food safety knowledge Indicator) (sample of Parent Council) 23.0% Percentage of parents who know at least 3 of the 15 evaluated conditions for ensuring food safety (Food safety knowledge Indicator) (Community sample) 68.9% Percentage of parents who know at least 6 of the 15 evaluated conditions for ensuring food safety (Food safety knowledge Indicator) (Community sample) 19.0% Almost all the children interviewed said they had eaten breakfast that day. Approximately one quarter of these said they brought a snack with them to school and in most cases this was food prepared at home. A higher percentage of the children in the Q’anjobal region compared to those in the Mam region indicated that they took money to school to buy a snack (53.6% vs. 39.5%). It was noted that the majority of these children liked it when the school breakfast or snack was served at 10 a.m. or after 11. A higher percentage of Mam children indicated that the best time for the school breakfast/snack was between 10 and 11 a.m. (90.0% vs. 47.1%). Of the children interviewed in both regions, 12 indicated that they were hungry or very hungry at the time of the survey (all were interviewed first thing in the morning). The majority of the directors and teachers interviewed (61.4%) indicated that the school breakfast or snack is served between 10 and 11 a.m. A little more than half of these directors and teachers stated that in their classroom there were students “who were hungry or very hungry”, at the time they were xi interviewed; the majority of those who shared this opinion worked in schools in the Mam region (81.2% versus 32.9% in Q’anjobal). The results show that less than a quarter of the Parent Council members (23.0 %), know at least six of the 15 basic aspects needed to comply with safe food preparation. The four aspects that were mentioned most often were: cleanliness of the work area, hand washing before cooking, the use of aprons, and the use of hair nets for people who cook or prepare food. The absence of illness in those who are preparing food, the presence of a refrigerator for safe food storage and the micro-biological analysis of the water to be used were rarely or ever mentioned by those who were interviewed. A little less than 20% of the parents in the community know at least six of the 15 basic aspects needed to comply with safe food preparation. The aspects that were mentioned most frequently were: cleanliness of the work area, hand washing before cooking, the use of aprons, and the use of hair nets for people who cook or prepare food. Again the least mentioned were: availability of refrigerators, micro-biological analysis of water and the absence of illness in those who prepare food. School Infrastructure Table 7 School Infrastructure Observations at the school Indicator Percentage Percentage of schools that have running water 86.9% Percentage of schools that have water at the time of the visit 66.5% Percentage of schools that have a water tap where foods are prepared in the school 56.3% Percentage of schools that have latrines and/or flush toilets 100% Percentage of schools that have clean latrines and/or clean flush toilets 54.0% Percentage of schools that have a hand washing station near the latrines and/or flush toilets 69.6% Percentage of schools that have a hand washing station near the latrines and/or flush toilets with soap and water available there 17.4% Percentage of schools that have a place assigned for storage of the foods used to prepare school meals 81.3% Percentage of schools that have a designated place for food preparation (kitchen) 96.7% Percentage of schools that have appropriate utensils for food preparation 88.2% The majority of the schools which were visited have running water systems. A low percentage (28.9%) of the schools reported that the running water system was built or rehabilitated by the PCI/USDA FFE Program (more than half of the schools in Phase II of the Program participated in Phase I), and in a little over half of these, water taps were observed in the food preparation area. With regard to basic hygiene conditions, these results show that all the sample schools have latrines and/or flush toilets. A little more than half of the schools (54.0%) had clean latrines or toilets2 , and around 70% of these had hand washing stations nearby for hand washing although only 17.4% had both water and soap available for this important practice. The majority of these schools (81.3%) had a storage room designated for storing the foods that would be used for preparation of the school meal, with qualities such as adequate separation of supplies or foods, cleanliness and order, and appropriate roof quality. Almost all of the schools (96.7%) had a kitchen for preparation of the school breakfast or snack and only 19.1% of these reported that the kitchens were built or improved with help from the PCI/USDA FFE Program. On the other hand, only 60.6% of the kitchens that were observed complied with at least four 2 Observation of these installations was made around 7:30 a.m., upon arrival at the schools. xii of the following six conditions: a) has appropriate food preparation utensils, b) has sufficient physical space, c) has adequate security conditions for those who work there, d) has good lighting, e) has good ventilation, and f) has good hygiene (at the time of the visit). Conclusions The goal of a baseline survey is to present details of the current situation for all project indicators. Given this, the most important conclusion is that achieving these impacts and effects will be a huge challenge for the program. Thus it would be wise to revise the commitments and goals so that they can be realistic without negating the challenge implied in achieving change. In particular, we want to share a conclusion with regard to the impact indicator which we consider to be most important and the most difficult to achieve – Reading Ability: The expected competency level established by the MINEDUC is far from being achieved by the children evaluated in both the Mam and Q’anjobal groups. When taking into account the general results, it can be concluded that upon finishing sixth grade, the children in these populations are not prepared to use the reading process as a tool to build deeper knowledge. The systematic errors show weak areas in the teaching that limit children in their development as literate readers. This process is most pronounced in the fourth through sixth grades although the teaching process in the earlier grades should also be strengthened. Recommendation Given that the end line study of the first phase was recently carried out, and the same kinds of weakness in terms of documenting processes and activities in the schools were found in this study, we once again recommend that the program promote good documentation systems in the schools where it works so that they will be available for future evaluation activities. 1 Baseline Report Food For Education (FFE) Project - Phase II in Huehuetenango, Guatemala PCI Guatemala JMatute-CIENSA I. INTRODUCTION AND BACKGROUND In April 2014, PCI Guatemala hired CIENSA-JMatute to carry out a baseline study for Phase II of the Food for Education program that began this year in Huehuetenango, Guatemala. The objective of the baseline study was to document at the beginning of the program, the status of the effect and impact indicators agreed upon with the donor, in order to be able to measure changes achieved over the course of the Program. Thus this study has a strong Quantitative Component and a smaller Qualitative Component, which consisted of five interviews with key personnel from the Ministry of Education in Huehuetenango. The Program is based on a plan prepared and approved/funded by USDA (USDA/FAS Grant Agreement: FFE-520-2013/034). Annex 3 presents the Program’s Monitoring and Evaluation Plan. II. METHODOLOGY This baseline survey was carried out using quantitative methodology (surveys and statistical measurements) and also qualitative methodology for calculating indicators. The qualitative methods were used to assess the knowledge, attitudes and practices, as well as experiences and perceptions of Ministry of Education staff in the Department of Huehuetenango in order to complement the Evaluation carried out by Michele González Arroyo between February and April 2014 (for the end line evaluation of Phase I). QUALITATIVE METHODOLOGY The methodology focused on individual interviews. Those interviewed were identified by PCI Guatemala staff and the consultant made the necessary appointments. Five individual interviews were held with the Technical Administrative Coordinators of the Municipalities of Cuilco and Santa Cruz Barillas and three officials from the Technical Pedagogical Bureau located in Huehuetenango. The interview guides were designed in order to collect information that the PCI Guatemala staff felt was important and which was agreed upon prior to conducting the study. The consultant guided the interviews in order to collect the information needed by PCI Guatemala. The field phase was carried out from May 6 to 9, 2014, with the support of a person assigned to record and transcribe the focus group sessions. The investigator collected the information virtually by means of interviews carried out via SKYPE, and the field staff set up the conferences with the people to be interviewed. Then the information was transcribed and reviewed in order to prepare the report. 2 Data entry, consolidation and analysis were carried out based on the specific questions for this study. QUANTITATIVE METHODOLOGY The representativeness and confidence level of the sample are important factors, not only for the actual development of a study, but also for documentation of the processes used. The sampling design and sample selection are critical for achieving a representative and reliable sample. II.1 Sampling Design The second phase of the Program will be carried out with 279 schools distributed in six municipalities of the department of Huehuetenango. Of the 279 schools, 174 participated in the first phase and thus, the program is adding 105 new schools. Given this, the sampling design for this baseline took into consideration the status of the school (Phase I participant or new school) and the region where it is located (Mam/Q’anjobal). A stratified design was used with the strata being defined as a combination between the status of the school and the region. Four strata were identified: Table 1: Stratification of schools Strata Municipality Number of new schools for 2014* Total number of students in the strata* Number of schools in the # sample** Presence in Phase I Region I YES Mam Cuilco 36 12,085 6 San Gaspar Ixchil 12 San Pedro Necta 32 Total 80 II YES Q’anjobal San Pedro Soloma 15 17,187 7 Santa Cruz Barillas 36 Santa Eulalia 43 Total 94 III NO Mam Cuilco 53 San Pedro Necta 6 4,215 7 Total 59 IV NO Q’anjobal San Pedro Soloma 18 6,158 6 Santa Cruz Barillas 26 Santa Eulalia 2 Total 46 * Information provided by PCI: Schools FFE II con codigo MINEDUC 2014-03-04.xlsx ** Number of schools according to calculated sample size REPRESENTATIVENESS and confidence level of the sample (JMatute) A sample is representative of a population when all the members of the population have equal probability of being selected. The confidence level of the sample is based on the probability of obtaining the population values within the confidence intervals. This is calculated prior to sample selection. 3 Within each stratum, schools were selected by probability-proportional-to-size (PPS sampling), considering each school as a cluster as a part of the sample design. Additionally, the communities where the selected schools were located were automatically selected for the sample of parents at the community level, with each community also being considered as a cluster. Annex 2, Sampling Design presents a list of the selected schools. II.2 Sample Sizes There were four target populations for measurement of indicators: • Schools • Directors and teachers • Students • Parents (in the community and members of the Parent Council) Since the schools were the principal unit for analysis, this section presents the manner in which the sample size of 13 schools per stratum (total 26) was determined. The sample size of 26 schools was determined using the following statistical parameters: • Confidence level = 95% • Effect of sampling design on variance = 1 • Unknown variance • Finite population: 279 • Estimation error (for schools) 18% The sample size for directors and teachers (it was assumed that each school would have at least two people: Director/teachers), was calculated as 52 teachers based on the following statistical parameters: • Confidence level = 95% • Effect of sampling design on variance = 1 • Unknown variance • Finite population: aprox = 558 (assuming an average of 2 per school) • Estimation error 13 For students, the sample size was set at five per grade (4th to 6th) = 15 per school. Total 390 Students: • Confidence level = 95% • Effect of sampling design on variance = 3 • Unknown variance • Finite population: above 7,000 students • Estimation error 9 For parents, the sample was determined by means of a random selection of households in the community where the school was located, where the household family had a school age child (5 to 13 years of age). The goal was to find 20 households per community. The total sample size was: 520 parents: • Confidence level = 95% • Effect of sampling design on variance = 3 • Unknown variance • Finite population: a7,000 • Estimation error 7.4% Sample sizes by target populations • School: o Directors and teachers: n=73 o Students (4th to 6th grade): n=358 o Parent members of Parent Council: n= 59 4 o School records (documentation): n= 26 schools3 o The school (observation processes): n= 25 schools (one school could not be evaluated due to Mothers Day celebration) • In the community: o Parents (mostly mothers): n= 477 o Children between 5 and 12 years of age: n= 441 o Youth 13 to 15 years old: n= 177 II.3 Sample selection within the clusters The target populations were randomly selected within each of their clusters (school or community). In the case of the school records or documents, some indicators, such as Attendance, required the extraction of random samples of students by grade, with which a cohort was designed to evaluate attendance during 2013. The selection of the moments and classrooms for evaluating inattentiveness was also random (see the corresponding chapter). Further details on the field methodology are found in Annex 1, Field work report. Prior to data collection in the field, the two survey teams (one for Mam speakers and the other for Q’anjobal) were trained and standardized in the use of the instruments (See Annex 1). II.4 Indicators and instruments The first activity to be carried out was the preparation of the instruments based on the indicators PCI defined as the most important to be measured in this baseline survey. These indicators will serve to measure changes over the life of the program (effect and impact) in the target populations. A total of 12 instruments were prepared for data collection. The 12 instruments were distributed in the target populations as follows: (See Annex 2, Instruments) • Schools: o Instrument # 9: School records (administrative) o Instrument # 8: Observations at the school (infrastructure and hygiene) o Instrument # 10: School records (Class attendance during 2013, for cohort study ) o Instrument # 6: Perception of attentiveness in class • Directors and teachers o Instrument # 7 • Students (4th to 6th grade) o Instrument 4 • Parents: o At the school:  Instrument # 5: Parent Council o In the community:  Instrument # 0: Characterization of the sample 3 The records for one of the schools were made available over the weekend since their Mother’s Day celebration made it impossible to visit on a school day. 5  Enrollment and Attendance: • Instrument # 1: Adolescents (13 to 15 years) • Instrument # 2: Children (6 to 12 years )  Instrument # 3: Parents  Instrument # 11: Literacy of children who completed 6th grade in 2013 Table 2: Indicators and the instruments used to measure them Result Indicator Instrument Chapter in the 1 2 3 4 5 6 7 8 9 10 11 document Improved literacy of school age children 1 Percentage of students who, by the end of 6th grade, demonstrate they can read and understand text at the level defined by national standards V Evaluation of literacy Improved attentiveness 2 Percentage of students identified as inattentive in class IV Attentiveness in class Reduced short-term hunger 3 Percentage of students who indicate they are “hungry” or “very hungry” VI School Feeding Increase in knowledge of program safe food preparation 4 Percentage of adults who can identify at least three safe food prep and storage practices Improved student attendance 5 Percentage of students who attend at least 80% of class sessions II Attendance, promotion, continuity and drop out rates Reduced health-related absences 6 Percentage of students who state that the primary reason for their absences is health￾related Increased use of health and dietary practices 7 Percentage of schools with clean latrines (at least once per day) VII School Infrastructure 8 Percentage of schools with soap and water in hand washing stations 9 Percentage of schools with clean food prep and serving equipment (according to accepted standards) Increased access to requisite food prep and storage tools and equipment 10 Percentage of schools with the requisite food prep and storage tools and equipment Improved school infrastructure 11 Percentage of schools with kitchens and potable water systems Improved knowledge of health and hygiene practices 12 Percentage of parents who can identify at least three important health/hygiene practices III Health and nutrition educational activities 13 Percentage of students/parents who know of at least one source of information on good health / nutrition practices Increased Student Enrollment 14 Percentage of boys/girls enrolled in the school I Enrollment and 15 school coverage Percentage of parents who can mention at least three benefits of sending their children to school Once the draft instruments were approved by PCI the validation process was begun. The validation of the data collection instruments is a very important step since it will determine data collection quality. 6 The purpose and justification for the validation step prior to the data collection stage in this study was the need to clearly understand how the primary schools4 worked, especially those where PCI has their school feeding program in the communities of Huehuetenango. Another objective was the validation of the data collection instruments at both the school and community levels. It is important to note that this validation took place in conjunction with the end line study for Phase I of the program. The majority of the instruments used for the baseline are the same as those used for the end line study. The new instruments (such as those used for literacy and attentiveness assessment) were validated during the training exercise. II.5 Data entry and statistical analysis Double data entry was used along with verification processes in order to guarantee a 0% error rate. EPI INFO version 6.04d was used for this. This software was also used for the statistical analysis: Based on the original data bases, the indicators that are shown in the Results Tables were created and these were calculated taking into account the sampling design. Thus the analysis was carried out using the CSAMPLE module of EPI INFO, while identifying the strata and clusters and assigning weight variables. This weighting factor is important in order to maintain the representivity of the results, so that all those who are interviewed (from the different target groups) have the same selection probability. The weighting is defined as the inverse of the selection probability. It is important to mention that for each of the target populations a different weighting was calculated due to the selection processes that were carried out to obtain the respective sample. II.6 Field team, training/standardization and field work The data collection phase of an evaluation study, with the baseline being the first measurement, is key for adequate future assessment of an intervention’s impact and effects. For this reason CIENSA-JMatute gives special emphasis to the quality and scope of the activities in this phase. Composition of field teams Two teams were prepared for data collection: A and B (one each for the Mam and Q´anjobal area communities). Each of these teams was composed of a supervisor, an editor and four interviewers.5 All team members were organized for performance of specific roles in order to optimize the data collection activities in each of the schools and their respective communities. Annex 1 presents full details about the team members. The field work began on April 22, 2014, and ended on May 7. It is important to note that the majority of the field team was bilingual with regard to the languages of the two regions and they were standardized in the management of the instruments in their respective languages. Work in Schools 4 It was especially important to understand the workings of the enrollment and internal management and documentation processes in these schools given that these would provide the basic inputs for filling out some of the study instruments. 5 Except for the supervisor, all members of Team A spoke Mam as their native language and four of the members of Team B were native Q´anjobal speakers. 7 The teams’ arrival at each of the schools chosen for the sample was scheduled in advance (See Annex 1), in coordination with the appropriate entities6 . The supervisor from each team was responsible for coordinating activities with the director of each school7. Field work took one day per school/community. The activities in the schools were carried out in the mornings and the community surveys were conducted in the afternoons. Data collection at each school began at 7:30 a.m. with the site observation of latrines/toilets and interviews with fourth, fifth and sixth grade students being the first activities of the day in each site. Contacts were made during the morning hours in order to locate students who had completed sixth grade the previous year (2013), in order to test their reading skills. In the majority of cases, the interviews with the director, the teachers, and the members of the Parent Council were conducted mid-morning. Some of these interviewees had to be located in the community in the afternoons (if they had not come to the school). It should be noted that the majority of these interviews were carried out in the local language, (Mam or Q´anjobal) given that most of the Parent Council members only speak their native language. Table 3 summarizes the system used in the schools for data collection. Table 3. Work in the Schools Activity Team member responsible Target population Questionnaires used 1. Interview with boys and girls (These were carried out in the first 2 or 3 hours of the morning) Editor and interviewer Students of 4th, 5th and 6th grades 4 2. Review of Attendance and PRIM Records (Carried out throughout the morning) Supervisor and interviewer Directors of each school 9 and 10 3. Observation of hygiene/ school infrastructure / food preparation (This was done during the first 3 hours of the morning) Interviewer School director /teacher Member of Parent Council + 8 4. Interviews with directors and teachers (Carried out mid-morning) Editor and interviewers School director and teachers + 5 5. In-class Observations (To detect children with inattentive behaviors) Interviewers Boys and girls receiving a normal class 6 6. Interviews with Parent Council (Carried out first thing or mid-morning) Interviewers Parent Council Members+ 7 + A key condition for the interviews with directors and teachers was that they had been in their positions during all of 2013; this condition also applied to Parent Council members. Work in Communities Once the activities were concluded in each school, the entire team proceeded to collect data for the community survey. Based on observation and/or a walk through of the sample communities and with 6 These arrangements were made by the team supervisors using cell phones. Occasionally visits had to be reprogrammed or adapted to the field calendar based on the opportunities and/or difficulties expressed by the directors of the schools included in the sample. 7 These names were provided by PCI staff. 8 support from the local authorities or leaders8 , each community was divided into several parts or sectors to be able to randomly select two or three. Data collection was carried out in each of the selected sectors by means of visits to every home in the sector. At every home the team knocked on the door and depending on the presence or absence of the respective target populations, the corresponding data collection instrument was used via direct, face-to-face interviews. In each of these households the complete set of interviews (when questionnaires 0, 1, 2 and 3 were applied), took approximately 25 to 35 minutes. The names of these questionnaires, the mode of application and the mechanics of the community survey are summarized in Table 4. One person from each group (Mam and Q´anjobal) was trained and standardized to be responsible for conducting the literacy tests in each community with a sample of no more than five students who had completed sixth grade in 2013. The application of this test was done individually with each child or youth who had been located and summoned to a designated place in each community9 . The interpretation of these tests was carried out in two steps. The first took place at the community level and the second (which required deeper analysis) was carried out in the CIENSA offices. This second analysis was carried out based on recordings made while the children were reading. The application of these reading tests was supervised by a psychology and education professional who is part of the CIENSA group. Table 4 Work plan in the communities Activity carried out Person/s responsible Target population Questionnaires used 1. Community Interview (Normally carried out in the afternoon in each community) Supervisor, editor, and interviewers Homes with adolescents and school age children who were in school in 2013 0, 1, 2 and 3 2. Test of reading ability and comprehension 1. Interviewer (Group A) 2. Editor (Group B) 3.Education and psychology professional Students (Youth) who finished sixth grade in the community school in 2013 11 Names and characteristics of the questionnaires that were used 1. Questionnaire 0 Filter for Characterization and Coverage: for characterization of the population and school coverage in each of the communities 2. Questionnaire 1 Questionnaire for adolescents: For attendance records and promotion of adolescents between 13 and 15 years of age 3. Questionnaire 2 Questionnaire for school age children: For attendance records and promotion of children between 5 and 12 years of age 4. Questionnaire 3 Questionnaire for community survey: for parents, with regard to their children’s school participation (adolescents and/or school age) during 2013 and 2014 5. Questionnaire 11 Instrument for literacy assessment: For boys and girls (adolescents) who had completed sixth grade in the target school in each of the sample communities. 8 In the majority of cases these local authorities or leaders were members of the COCODES and/or members of the Parent Council. The latter were contacted when they were interviewed in the school. 9 The application of the literacy test was done in community locales such as schools, multi-use rooms and the parents’ homes. 9 Given that the school visits could only be carried out during the week (Monday to Friday), and that the field work began on a Tuesday10, the field teams used part of two weekends to carry out community surveys. Some of these were carried out in the mornings especially on Saturdays. This was all previously coordinated with the respective contacts at the local level. Quality Control Mechanisms In order to guarantee the quantity and quality of the data from the schools and communities, several quality control measures were used. The primary methods were supervision and editing. The supervisors from each group and the Psychology Professional from CIENSA were responsible for the first of these. Editing or deep review of the information collected was carried out in the schools and communities by the editors of each group. This was extremely useful for timely and immediate discovery of errors or inconsistencies which might have gone undetected, negatively affecting data collection. With these quality control measures, the level of standardization obtained by the interviewers was maximized during the training stage which had a positive impact on data collection. The editors were responsible for keeping track of the questionnaires that were used in each school and community, as well as filling out the summary sheet where the products from each day’s work were listed. PRODUCTS OBTAINED Table 5 Products Obtained by Field Team A (Mam Area) RT = Reading Test; T = Total No. School/Community (Municipality) Data Collection Instruments Applied (ID) RT Community School 0 1 2 3 T 4 5 6 7 8 9 10 T 1 El Palmar (San Pedo Necta) 3 26 2 12 13 53 11 3 1 3 1 1 1 21 2 Nimá (San Pedro Necta) 5 18 7 12 13 50 15 4 1 0 1 1 1 23 3 Ojo of Agua de las P (San Pedro Necta) 6 26 7 15 19 67 13 3 1 2 1 1 1 22 4 Huixoc (San Pedro Necta) 5 26 3 15 16 60 15 4 1 3 1 1 1 26 5 Los Alisos (San Pedro Necta) 5 25 3 19 20 67 15 4 1 3 1 1 0 25 6 Hoja Blanca (San Pedro Necta) 4 25 2 15 15 57 15 3 1 1 1 1 0 22 7 Chacpantzé (San Gaspar Ixchil) 5 29 11 20 20 80 15 4 1 3 1 1 0 25 8 Monte Rico (Cuilco) 5 22 5 14 14 55 14 3 1 2 1 1 0 22 9 Nueva Esperanza (Cuilco) 1 26 5 16 17 64 15 1 1 3 1 1 0 22 10 Yerba Buena (Cuilco) 5 28 5 19 20 72 15 4 1 3 1 1 0 25 11 Buena Vista (Cuilco) 5 26 8 21 21 76 15 4 1 1 1 1 0 23 12 Cholivá (Cuilco) 5 26 2 15 15 58 3 3 13 Coxtón (Cuilco) 4 25 5 19 19 68 1 1 2 10 Due to difficulties in contacting the schools on the first day, the field work began on Tuesday, April 22 and ended on May 7. Each of the field teams remained in their respective regions during the entire data collection period. 10 Table 6 Products Obtained by Field Team B: (Q´anjobal Area) RT = Reading Test; T = Total No. School/Community (Municipality) RT Data Collection Instruments Applied (ID) Community School 0 1 2 3 T 4 5 6 7 8 9 10 T 1 Seclecán (Sta. Cruz Barillas) 3 25 6 18 19 68 14 3 1 3 1 1 1 24 2 El Quetzal (Sta. Cruz Barillas) 5 28 9 19 23 79 15 4 1 3 1 1 1 26 3 Jolomtaj (Sta. Cruz Barillas) 4 26 7 19 21 73 10 3 1 3 1 1 1 20 4 Monte Bello M: (Sta. Cruz Barillas) 3 20 8 17 19 64 14 1 1 3 1 1 1 22 5 San Juan La Ceiba (Sta. Cruz Barillas) 3 23 8 18 19 68 15 1 1 3 1 1 1 23 6 La Campana I (Sta. Cruz barillas) 5 22 8 16 18 64 15 2 1 3 1 1 1 24 7 Tziquiná Grande (Sta. Eulalia) 3 20 9 17 19 65 15 1 1 3 1 1 1 23 8 Santa Eulalia (Sta. Eulalia) 5 23 8 17 19 67 15 4 1 3 1 1 1 26 9 Pueblo Nuevo (San P. Soloma) 4 26 13 19 20 78 15 3 1 3 1 1 1 25 10 Sjolom Ku (San P. Soloma) 4 24 9 18 20 71 15 3 1 1 1 1 1 23 11 Yulchen (San P. Soloma) 2 27 5 18 19 69 15 3 1 0 1 1 1 22 12 Gelná (San P. Soloma) 2 25 11 18 20 74 15 2 1 0 1 1 1 21 13 Yac Xap (San P. Soloma) 5 23 9 17 19 68 15 3 1 0 1 1 1 22 SPECIFIC METHODOLOGIES Special measurement methodologies were applied for indicators which required specific, out-of-the￾ordinary strategies (not surveys, interviews, or document reviews). This section will describe the methodology used for two of the quantitative chapters that are presented in the results section. Attentiveness in Class In order to measure this factor, at each school (except the one that was having a Mother’s Day celebration instead of regular classes) three different classrooms were chosen at random in cases where there were more than three classrooms. When a school had three or fewer classrooms, all were included. In each classroom a class period was randomly selected, or if there was only one classroom, three different class periods were selected. Once the class and period were selected for the observation, the observer entered the class and waited for at least five minutes after his/her entrance so that students were used to his/her presence and returned to their normal routines. During the next 10 minutes, the observer randomly selected one minute and during that time observed the behavior of the students with regard to the class being taught. It should be noted that these observations were only carried out in classes where the teacher was giving a normal class and interacting with the students, often using audio visual aids such as the black board. The observations were conducted during classes such as math, social studies, natural 11 sciences and language. Observation was avoided in classes where dictation was being given or those of an artistic nature, or when students had to constantly come and go from the classroom. During the minute of evaluation the observer counted how many students were: • Distracted: looking toward the doors, windows or ceiling, whispering, thinking or meditating, playing with something, eating, drinking or chewing • Paying attention to something else: looking, reading or strongly focusing on something, writing or drawing something without raising their head, lying down on the desk or table • Doing something distracting: laughing out load or with audible sound, tapping or kicking something, walking or running inside the classroom • Not motivated: showing signs of drowsiness, yawning, nodding off, or sleeping in the classroom • Distracting with others: talking to a classmate, touching a classmate, throwing something to a classmate, hitting another student. Literacy in students who have completed 6th grade The Program is being asked to improve literacy levels in students at project schools (Improved Literacy of School-age children). In order to demonstrate this effect or impact in the target population, the field team sought out students who had completed Grade 6 in 2013, excluding any students who were repeating the grade. This was done at the community level and some students had to be located at the institute in which they were continuing with secondary education. For this purpose, the selected schools were asked to provide a list of the students who had passed sixth grade in 2013 and then the field team tracked them down. The evaluation focused on reading ability: as the level of competency that the child has for reading and for which three elements were analyzed: speed, accuracy and comprehension. Based on the indicators established for the end of sixth grade by the Ministry of Education, the level of competency could be described as: ¨Read critically, identifying important ideas and information that allow me to communicate in a functional manner and to become informed while increasing and deepening my knowledge.¨ (CNB)11 The indicators that accompany reading competency are based on the following standards: (Taken from: Texto Comunicación and Lenguaje 6º Primaria) • Standard 4: express critical judgments based on the content and relationship between parts of the text. • Standard 5: Read silently, at least 10 recreational books of appropriate level, without mumbling at a speed of at least 400 words per minute with support from the graphic elements of the text. As can be seen, it is expected that the reading ability at the end of sixth grade will allow continued learning of other subjects as is stated in the competency. Based on this, a sixth grade level reading text was chosen and an instrument was designed to measure and record the desired skills. 11 CNB: National Base Curriculum. MINEDUC 12 • Reading speed. This refers to the rhythm with which the reader decodes each word. For this the observer recorded the number of words read out loud during the first minute of reading. In order to establish the baseline values, reading speed was calculated as a percentage of the standard expected value (125 words/minute)12. Thus the reading speed was obtained in the following manner: Speed = Words read in the first minute/125 x 100 • Reading accuracy. Reading accuracy refers to the ability the reader should have to be able to decode each word correctly. This includes intonation (reading which respects punctuation) and excludes the knowledge of the words being read. Errors in this skill can appear at the sound, syllable or word level and are classified as: substitutions (changing one sound for another); inversions (changing order); guessing (inventing sounds, syllables or words similar to what is written); additions (adding of sounds, syllables or words); omission (not reading sounds, syllables or words); intonation (reading which respects punctuation signs and accents) and regressions (repetition of syllables or words). There are seven types of error. Two analysis processes were used in this study to determine reading accuracy: o The observer followed the child’s reading noting on a separate text document the words that the children read inaccurately (any of the seven errors). o Additionally, the child’s reading was recorded and a CIENSA educator later listened to it in the office and reviewed/corrected the field assessment. Accuracy was calculated taking into account the total number of words in the text (575 words). Additionally the errors of a sub-sample were reviewed in order to identify those which were most serious or where most help would be needed with the goal of understanding the reading environment and determining possible causes for gaps in reading ability. Additionally this would help detect children with learning problems that require special attention in the reading process. It should also be noted that when the error count was made, only one error was counted per word (including those of intonation) even though there were often more than one error per word. Accuracy = 100 – (number of errors/575 x 100) Note: There were a total of 575 words in the reading text used. 12 125 words/minute is the minimum for the reading speed standard required for passing sixth grade. Reference: Document provided by PCI: Guia rapida para pruebas of lectura FFE 2014-03-07.docx 13 Images of one of the documents used in the field to measure accuracy • Reading Comprehension. This skill is extremely complex. To measure it we must evaluate the result of several process that go from understanding the content to use of analysis and synthesis processes which allow the reader to abstract the main ideas and/or make a judgment about the content of what was read. In this study, the children were asked 10 questions related to what they read in order to assess the degree of comprehension. The comprehension rate was calculated in the following manner: Comprehension = Number of Correct Responses/10 x 100 Literacy Calculation for the Students Based on the document that PCI shared for calculating student reading ability, the three skills should have the same weight in calculating reading ability. While we agree with this premise, the consulting team feels that the comprehension and accuracy skills which are linked in this case, are more important (especially comprehension) and should have greater weight. All of this is focused on the degree of competence expected on finalizing primary education. 14 For this reason, there are two indicators used in measuring reading ability that enrich the analysis for future planning of steps to take and which help determine the causes of low comprehension scores. • Reading Ability-1 This indicator, which corresponds to the one that PCI has presented, uses equal weighting for the three skills: Reading ability -1 = Speed + Accuracy + Comprehension/3 x 100 • Reading Ability -2 This indicator corresponds to the consultants’ proposal Reading Ability -2 = Speed (0.1) + Accuracy (0.2) + Comprehension (0.7) A child who reads at a speed of 129 words per minute, doesn’t make any errors and answers all the comprehension questions correctly will receive 100 points for reading ability. III CHARACTERIZATION OF THE QUANTITATIVE SAMPLE This section presents the statistical description of the sample for this study. The quantitative sample was obtained by visiting 26 schools and the communities where they are located. Samples obtained in the schools: • 358 Students from 4th to 6th grade: (Questionnaire 4) o 196 boys and 162 girls o 118 in 4th grade, 121 in 5th grade, 119 in 6th grade. • 73 from directors and teachers: (Questionnaire 5) o 13 directors (the majority of whom are also teachers) and 60 teachers o 36 men and 37 women • 59 Parents who work with the Parent Council: (questionnaire 7) o 55 have been in their role since last year o 13 in the role of President, 12 Secretaries, 13 Treasurers, and 8 Members-at-Large • 25 observations (one per school): (Questionnaire 8: Infrastructure and hygiene in the school) • 26 document reviews (one per school): (Questionnaire 9: School records, and interview with a key person at the school) • 75 observations (three per school): (Questionnaire 6: Attentiveness in class) • 149 Students for the cohort study implemented to evaluate school attendance in 2013 o 20 from Mam region, and 129 from Q’anjobal Community Samples: • A total of 643 homes were visited, of which 516 had interviews because they fit the criteria of having at least one child between 5 and 15 years of age: (Questionnaire 0: Selection of sample and characterization) Description of those interviewed: (from the total of 643) o 331 were Mam and 312, Q’anjobal o 71 were men and 445, women 15 In the 643 homes the following information was collected: • 177 children between 13 and 15 years: (Questionnaire 1) o 93 boys and 84 girls o 102 were 13, 54 were 14, and 21 were 15 years old • 441 children between 5 and 12 years: (Questionnaire 2) o 227 boys and 214 girls o 10 were 5, 47 were 6, 79 were 7, 80 were 8, 59 were 9, 66 were 10, 50 were 11, and 50 were 12 years old • 477 parents: (Questionnaire 3) o 70 men and 407 women IV RESULTS AND FINDINGS QUALITATIVE COMPONENT The five officials from the Ministry of Education who were interviewed were familiar with the Food for Education program and had participated in meetings with technicians, in trainings, in preparation of workshops, in the Sustainability Commission and in fairs. According to those who were interviewed, this is a program that PCI is supporting with food commodities and at the same “promoting” healthy eating to strengthen students’ academic performance. All those interviewed emphasized that the department of Huehuetenango has many very poor communities with a high degree of malnutrition and that often the school breakfast or snack may be students’ first meal of the day. The interviewees also agreed that these meals are an incentive for students that lowers drop out rates as it “maintains student attendance through the school breakfasts” while also “strengthening educational quality”. They also acknowledged the importance of the program, because it is supporting families, many of which are very large, with nutritional foods that help students improve academic achievement. “Obviously health is important for intellectual development”. They also think that it is very effective, given that in the three years that the program has been operating, the number of students completing sixth grade has increased since they previously only went as far as third grade and then their parents sent them to work. The school breakfast or snack is an incentive for the children because “the day is long and the majority of them come to school without having breakfast”. Thus “the children anxiously anticipate the moment when they will get their snack” and some children may attend school “just because of the snack or breakfast they will receive”. They explained that the program is also working with school gardens (to promote new food and nutrition habits.) And they are promoting community organization, the use of organic products and protection of the environment. With regard to the program’s efficiency, one of the participants indicated that more time is needed to assess whether or not the program has been efficient, but that according to what he had heard, the results are acceptable. However, two other participants representing the MINEDUC commented that “the government’s initiative to provide resources directly to the school was not efficient. It required that the schools purchase from formal businesses which can provide invoices, and this type of business in rural areas mainly sell processed and packaged goods, as opposed to fresh produce.” Additionally, 16 they expressed concern that the USDA/PCI commodities and the goods purchased with the government’s support (DIGEFOCE) had to be kept separate in the schools’ warehouses which could represent a lack of efficiency. One of them also suggested the elaboration of a menu and budget. With regard to the sustainability of the program and the support that the MINEDUC can provide, one of those being interviewed said that “we need to be realistic” and understand that the current situation in the communities is precarious and thus the process cannot be sustainable. They think that in the future there might be some possibility of sustainability if MINEDUC provides fifty percent and the communities provide the other half. Nevertheless, this would require that they could generate income by means of a business or cooperative which would require a mid- or long-term process. The idea of training students in school gardens could begin a process for creating future capacity. Those who were interviewed indicated that at this moment, the MINEDUC doesn’t have any program or project that is evaluating student progress in reading and writing since it is still in the implementation stage. Nevertheless, they are coordinating with the Technical Administrative Coordinators (TAC) so that they can carry out visits and workshops and evaluate the process since they consider that it is one of the “Ministry of Education’s star projects”, strengthened by the presence of “NGOs like PCI”. They added that they would like to begin searching for “some institution that could do some sort of consultancy, but we don’t have funds for this and would depend on an external entity”. Another of the respondents said that the Mid-Term Evaluation would be carried out in 2015 and they would like an external entity to measure reading and writing progress at the different levels. They mentioned that the Ministry of Education is using a strategy to improve reading and writing skills in primary school students. This strategy uses several techniques. One of these is to implement reading and writing dynamics so that they will become a habit with the children. On a daily basis they should read for 30 minutes and ensure that the children understand what they are reading. They described various techniques the teachers are using to achieve this objective. Last year they gave a training on how to involve parents and they were given orientation about how to write stories. They have tried to get an adult who is authorized by the school to come to tell stories and anecdotes in the classroom or via radio as they have support for this kind of transmission. Another technique is called “family stories”, where an adult helps a child write a story which is then entered in a department or national level contest. Another technique involves reading a story in parts and having the children act it out. For example, they may give the title of a story to a child and then ask what they think the story is about. He or she would try to come up with something similar to what they have read. This technique has had very positive results. While it looks like the child is playing, he or she is actually participating in an entertaining activity where everyone is learning and shy children are equally engaged. All these activities are part of a strategy which focuses on helping the child to enjoy reading while increasing their initiative and creativity. In this vein, they spoke of the importance of placing bilingual teachers in the early grades “to avoid shocks and barriers”. With regard to teacher training, they have given priority to first grade and in May they expect to begin holding reading and writing workshops for second and third grade teachers as well as a math workshop for first grade teachers. “These are strategies that help teachers to have tools in order to provide quality education in the classroom”. They further stated that the Ministry of Education published a book titled, Learning Strategies for Reading Comprehension (Estrategias of Enseñanza para la Comprehension Lectora), which is a guide with strategies to be applied during and after reading. This has been provided to all teachers as a tool. 17 They also stated that in order to accomplish the planned strategies, the MINEDUC needs human, material and economic resources with the greatest limiting factor being financial resources. As a concrete example, they said that so far this year they haven’t had funds available even when the MINEDUC asks them to go to the field, gather teachers and take materials to them. Huehuetenango has a large population spread over a sizeable geographic area. The lack of human resources is also a limiting factor given that there are many students and few teachers. There is a great need for contextualized staff who are fluent in the local Mayan languages. “Huehuetenango is unique (a little Guatemala) in that there are 10 languages spoken here”. It is challenging to find trained staff skilled in working within the context of the schools where they need to work. Additionally the communities are remote and hard to access so even when the Ministry does find personnel to contract, the teachers don’t want to go to those areas. They noted that a lack of adequate training is an obstacle for sustainability of the educational process. Personnel need to be adequately trained and many teachers retire, transfer to other locations and the new teachers are not well trained – “their training is superficial”. They feel that teachers need to be trained to go directly to the classroom. They state that training is very important in order to utilize the valuable human resources they have. They also shared a need for resource materials. The MINEDUC edits very good books but they aren’t accessible for everyone due to the lack of funding. There is also a lack of desks, especially for left￾handed students. Many of the Municipalities have been supportive in building classrooms but there are still not enough. Even though they lack many resources, the MINEDUC staff has tried to mitigate the situation using various strategies. For example, in order to overcome the lack of transportation, some have been able to connect with other organizations in the area to get rides or in other cases they have been able to get support from the Municipalities to cover the cost of fuel when using their own vehicles. In order to cover the lack of teachers, the Ministry did a training for multi-grade teachers. And guidelines have been prepared on how to work with one set of students in the classroom while another group is working separately with a leader. Other teachers have changed the schedule so that they have one group in the morning and another in the afternoon. Some communities chose to hire a teacher from the community itself so that they would be motivated to stay for a longer time. With regard to teacher attendance at the trainings, they suggested that the locations for these trainings be decentralized since many teachers are unable to attend because of the distance from their work sites. The MINEDUC’s support consists of training teachers in use of the guides, in distributing texts and in giving the schools a materials stipend of Q202 ($26) per year (which they consider to be very low). The results indicate that the program is very valuable, but since it is a medium-term process, it will need support from organizations like PCI. Although the communities have offered support according to their means, they are living in very precarious situations and it is hard for them to give even the amount they are being asked for. The school garden trainings are very valuable if they take into account access to water and the cultural patterns of the zone. 18 QUANTITATIVE COMPONENT This section first presents the results and findings from measurement of the quantitative indicators and later will present those from the Qualitative Component (interviews with MINEDUC staff). In terms of the Quantitative Component it can be seen that each of the Tables not only includes the value of the indicator but also documents the sample size used to calculate it as well as the statistical confidence level of 95%. The confidence levels serve to rapidly determine any significant difference between the groups as well as to determine how good the estimation is, based on the statistical error and taking into account the range. Most importantly, the ranges allow us to know the values of the indicators in the study population. The results and findings of the quantitative methodology are summarized in seven chapters: I. Enrollment and School Coverage II. Attendance, Promotion, Continuity and Drop Out Rates III. Health and Nutrition Educational Activities IV. Attentiveness in Class V. Reading Evaluation VI. School Feeding VII. School Infrastructure 19 I Enrollment and school coverage (Increased Student Enrollment) This section presents information associated with the coverage that the school has in the community where it is located and enrollment at the school level. The objective is to determine any increases in school enrollment. School coverage was measured by means of interviews at the community level, while enrollment was assessed by reviewing records. Increased student enrollment is measured in this study by means of comparing enrollment numbers from one year to the next. When the calculation is made only using school records to reflect the enrollment numbers, it doesn’t truly answer the question of whether parental attitudes towards school attendance (school coverage) have changed as these numbers do not clarify whether enrollment increased because there are more school age children in the community or because of a change of attitude regarding the value of school attendance. In this section emphasis is given to the enrollment of seven year old children in first grade considering that this is the most objective information that will reflect attitudinal changes in the population regarding school enrollment. As has been mentioned, community data was obtained by means of a survey regarding the school’s coverage in relation to the number of school age children (five to 12 years old) and adolescents (13-15 years) who attend school. The latter are considered in the study given that not all children enter first grade at age seven but may do so when they are older. Additionally it takes in to account the fact that some might repeat a grade. Tables 1a and 1b present the changes that were seen in the enrollment rates from 2012 to 2013 and from 2013 to 2014. Table 1a corresponds to the results of the community survey and 1b to the results obtained from school records. For both analyses, the change is expressed as percentage of change in relation to the previous year: Percentage of relative change = # of students in most recent year – # of students in previous year/# of students in previous year x 100 20 Table R1: Student enrollment and school coverage in the communities. Community sample The confidence level of 95% is found in parenthesis. (The groups with statistical differences are shown in green) Indicator Pci1lb1.rec, pci1b2.rec Children from 5 to 12 years Children from 13 to 15 years Region Sex Total (n=441) Region Sex Total (n=122) Mam (n=210) Q’anjobal (n=231) Male (n=227) Female (n=214) Mam (n=38) Q’anjoba l (n=84) Male (n=65) Female (n=57) Ind1, ind1x Children enrolled in the community school in 2014 (2014 coverage) 97.8% (95.9 to 99.7) 98.7% (96.6 to 100) 98.6% (97.4 to 99.8) 98.1% (95.3 to 100) 98.3% (96.9 to 99.7) 94.7% (86.3 to 100) 99.3% (97.8 to 100) 98.2% (94.6 to 100) 97.6% (93.9 to 100) 97.9% (95.4 to 100) Those who passed sixth grade in 2013 were excluded Ind1a Children enrolled in the community school in 2013 (2013 coverage) n=203 N=228 n=224 N=207 n=431 n=66 n=111 n=93 n=84 n=177 81.6% (73.8 to 89.5) 88.4% (83.4 to 93.5) 83.5% (77.4 to 89.7) 87.7% (82.1 to 93.2) 85.5% (81.4 to 89.7) 100% (-) 98.0% (95.1 to 100) 97.5% (94.0 to 100) 98.7% (96.9 to 100) 98.7% (96.9 to 100) Percentage of change for school coverage in 2014 relative to 2013 19.8% 11.7% 18.1% 11.9% 15.0% -5.3% 1.3% 0.7% -1.1% -0.8% 7 year old children enrolled in first or second grade Ind1x7 Children enrolled in the community school in 2014 (2014 coverage) n=166 N=218 n=198 N=186 n=384 93.6% (90.9 to 96.2) 97.8% (95.7 to 99.8) 97.2% (94.3 to 100) 94.9% (90.9 to 98.9) 96.1% (94.1 to 98.0) Ind1ax7 Children enrolled in the community school in 2013 (2013 coverage) n=134 N=177 n=156 n=155 n=311 95.4% (92.5 to 98.2) 92.5% (88.2 to 96.9) 93.0% (88.7 to 97.4) 94.3% (89.6 to 99.1) 93.7% (90.9 to 96.5) Percentage of change for school coverage in 2014 relative to 2013 -1.9% 5.7% 4.5% 0.6% 2.6% 21 Table R1a: Increased school registration in project schools School records The confidence level of 95% is found in parenthesis. (The negative values that were different from the previous year are in red.) Indicator Pcilb9a, pcilb9b Region Total (n=26 schools) Sex Mam (n=13 schools) Q’anjobal (n=13 schools) Male (n=26 schools) Female (n=26 schools) Ind1T114 Ind1O14 Ind1A14 Mean of student enrollment in 2014, entire school 129.9 students (99.3 to 160.5) 209.8 students (89.7 to 329.8) 169.7 students (107.4 to 232.0) 86.9 boys (55.1 to 118.7) 82.8 girls (52.2 to 113.3) Ind1T13 Ind1O13 Ind1A13 Mean of school enrollment in 2013, entire school 135.4 students (106.2 to 164.5) 223.4 students (96.2 to 350.5) 179.2 students (113.2 to 245.2) 92.0 boys (59.0 to 125.0) 87.2 girls (54.2 a 120.3) Ind1T12 Ind1O12 Ind1A12 Mean of school enrollment in 2012, entire school n=13 schools n=9 schools N=22 schools n=22 schools n=22 schools 137.0 students (107.9 to 166.2) 225.2 students (36.4 to 414.0) 171.2 students (91.8 to 250.7) 88.0 boys (46.5 to 213.8) 83.2 girls (45.2 to 121.2) In12a13T In12a13O In12a13A Mean change in student enrollment between 2013 and 2012, entire school n=13 schools n=9 schools N=22 schools n=22 schools n=22 schools 0.4% (-3.6 a 4.5) 2.6% (-3.9 a 9.1) 1.3% (-2.2 a 4.8) 0.5% (-4.4 a 5.5) 2.3% (-1.6 to 6.3) In13a14T In13a14O In13a14A Mean change in student enrollment in 2014 compared to 2013, entire school -4.3% (-8.2 to -0.4) -6.8% (-10.6 to -2.9) -5.5% (-8.4 to -2.6) -5.6% (-8.3 to -2.8) -5.2% (-9.7 to -0.7) Enrollment in first grade Ind1T114 Ind1O14 Ind1A14 Mean of student enrollment in 2014, first grade 23.9 students (16.7 to 31.2) 35.3 students (16.3 to 54.4) 29.6 students (19.6 to 39.6) 16.1 boys (10.6 to 21.7) 13.5 girls (9.0 to 18.0) Ind1T13 Ind1O13 Ind1A13 Mean of student enrollment in 2013, first grade 25.2 students (19.0 a 31.3) 40.3 students (17.9 a 62.6) 32.7 students (21.0 a 44.3) 16.3 boys (10.6 a 21.9) 16.4 girls (10.3 to 22.4) Ind1T12 Ind1O12 Ind1A12 Student enrollment in 2012, first grade n=13 schools n=9 schools n=22 schools n=22 schools n=22 schools 21.6 students (16.0 to 27.1) 42.7 students (9.3 to 76.1) 29.8 students (15.1 to 44.4) 14.6 boys (7.8 to 21.5) 15.1 girls (7.2 to 23.0) Ic12a13T Ic12a13O ic12a13A Mean change in student enrollment between 2013 and 2012, first grade n=13 schools n=9 schools n=22 schools n=22 schools n=22 schools 19.2% (-12.6 to 50.9) 3.8% (-19.4 to 27.0) 11.5% (-9.3 to 35.7) 10.6% (-16.7 to 37.8) 23.4% (-2.5 to 49.4) Ic13a14T Ic13a14O ic13a14A Mean change in student enrollment between 2014 and 2013, first grade -3.0% (-12.8 to 6.8) -9.7% (-25.9 to 6.4) -6.3% (-16.2 to 3.5) -0.04% (-12.0 to 11.9) -3.4% (-23.2 to 16.3) 22 Table R1b: Opinion with regard to increased student enrollment in program schools Sample of directors and teachers The confidence level of 95% is found in parenthesis. (The groups with statistical differences are shown in green.) Indicator Piclb5a Directors and teachers Region Total (n=73) Mam (n=40) Q’anjobal (n=33) Directors and teachers who think that student enrollment for 2014 increased compared to 2013 Ind1p 84.4% (63.9 to 100) 13.5% (0.0 to 27.7) 38.0% (6.7 to 69.4) Number of schools where the majority of interviewed staff (directors and teachers) feel that student enrollment increased in 2014 compared to 2013 6 schools of 12 3 schools of 13 9 schools of 25 Table R1c: Opinion with regard to student enrollment. Community sample The confidence level of 95% is found in parenthesis. (The groups with statistical differences are shown in green.) Indicator Piclb3a Directors and teachers Region Total (n=477) Mam (n=222) Q’anjobal (n=255) Parents who state that they had problems or obstacles to enrolling their children in school Ind1q 6.8% (3.1 to 10.5) 25.7% (20.0 to 31.5) 17.8% (13.7 to 21.8) Problems encountered for enrolling children: Only those which were mentioned are included Lack of money to support them 5.4% (3.0 to 7.7) 25.7% (20.0 to 31.5) 17.2% (13.4 to 20.9) Lack of interest on the part of the children 0% (-) 1.7% (0.0 a 3.6) 1.0% (0.0 to 2.2) Constant illness in child 0% (-) 1.2% (0.0 a 3.1) 0.7% (0.0 to 1.9) Distance between home and school 0.2% (0.0 to 0.6) 0% (-) 0.1% (0.0 to 0.2) Parents who value the education their children are getting at school (Mentioned at least three benefits the children receive by attending school) Ind1r 66.6% (59.0 to 74.3 48.0% (35.0 to 61.0) OR=2.2 55.8% (47.5 to 64.1) What they value: Only the benefits they mentioned are included Children learn to read and write 97.0% (94.8 to 99.1) 98.7% (97.0 to 100) 98.0% (96.7 to 99.3) Children learn to do math and keep accounts 15.3% (9.4 to 21.2) 11.1% (5.8 to 16.4) 12.9% (8.9 to 16.8) Children learn to talk well 56.7% (47.1 to 66.4) 82.3% (74.8 to 89.9) 71.6% (6.4 to 76.8) Children learn to value and know their own bodies 2.3% (0.4 to 4.3) 1.4% (0.0 to 3.4) 1.8% (0.5 to 3.1) 23 Indicator Piclb3a Directors and teachers Region Total (n=477) Mam (n=222) Q’anjobal (n=255) Children learn hygiene practices 8.0% (4.9 to 11.1) 4.6% (2.4 to 6.8) 6.0% (4.2 to 7.8) Children learn about good nutrition 3.6% (0.6 to 6.5) 0.5% (0.0 to 1.6) 1.8% (0.5 to 3.1) Children learn how to stay healthy/to prevent disease 1.7% (0.1 to 3.2) 1.2% (0.0 to 3.1) 1.4% (0.1 to 2.7) Children learn how to eat well/avoid malnutrition 0.3% (0.0 to 1.0) 0% (-) 0.1% (0.0 to 0.4) Children learn to be good citizens 17.5% (13.8 to 21.3) 13.0% (7.4 to 18.6) 14.9% (11.4 to 18.4) Children learn to respect others 27.7% (22.2 to 33.3) 10.7% (5.2 to 16.3) 17.9% (12.7 to 23.1) Children learn how to get a job 40.1% (32.0 to 48.1) 32.2% (23.4 to 41.0) 1.8% (0.5 to 3.1) Children get a foundation to be able to study more 34.0% (27.9 to 40.1) 10.7% (6.0 to 15.5) 20.5% (15.5 to 25.6) 24 Principal findings for enrollment and school coverage According to the results obtained at the community level, the number of seven year old children enrolled in first grade this year (2014) is greater (P<0.05) in the Q’anjobal region (97.8%) than in the Mam region (93.6%). Seven year olds in the Q’anjobal region are three times more likely (Odds Ratio - OR) to be enrolled in first or second grade than children in the Mam region. According to school records, there was no increase in enrollment in 2013 compared to 2012 (P>0.05). This situation was the same for overall enrollment as well as for first grade enrollment. According to the same records, in 2014 there was a significant decline in total enrollment (all grades) when compared to 2013, with an average for the entire study population of 5.5%. In the opinion of the directors and teachers in the Mam region, 84.4% felt there was an increase in enrollment in 2014 compared to 2013, which is statistically different (P<0.05) from the perception in the Q’anjobal region (13.5%), and also contradicts the findings from the school records. In this case it would be reasonable to state that a teacher in the Q’anjobal region has 34.4 time greater probability (OR) of being correct with regard to the enrollment figures than a teacher in the Mam region. When parents were interviewed about student enrollment, more parents in the Q´anjobal region stated they had problems or obstacles related to enrolling their children than parents in the Mam region (25.7% compared to 6.5%, P <0.05, respectively). The parents who were interviewed in the communities of the Q’anjobal region have 4.8 times more probability (OR) of mentioning difficulties than those of the Mam region (P<0.05). The reason that seemed to mark the notable difference between these two populations was the lack of money for supporting their children in school (P<0.05). Again, the parents in the Q’anjobal region had 6.1 times greater probability (OR) of stating this as a problem than the parents in the Mam region. The results also show that more parents in the Q’anjobal region (82.3%) place high value on their children learning to speak well than the parents in the Mam region (56.7%, P<0.05). Parents in the Q’anjobal region have 3.6 times more probability (OR) of mentioning this as a reason for enrolling their children in school than the parents in the Mam region. On the other hand, in the Mam region parents place higher value on their children’s school attendance in order to learn to respect others as well as for acquiring a solid foundation for future studies (27.7% compared to 10.7%, and 34.0% compared to 10.7%, P<0.05 for these two reasons, respectively). In terms of the Odds Ratio, it can be seen that parents in the Mam region have 3.2 and 4.3 times more probability of stating these reasons for valuing school, respectively, than their counterparts in the Q’anjobal region. II Improved Student Attendance Successful learning depends on students’ presence in school. This section presents the results of the four indicators that measure student behavior directly associated with their academic progress. • Attendance. One project indicator is: Percentage of children who attend at least 80% of classes. The measurement of this indicator was done by means of two independent samples in order to establish the values for the 2013 school year: 25 o The first measures mothers’ perceptions regarding their children’s attendance at school and this was calculated based on their recall of events during the community level survey. In this case it was considered that a child did not meet the requirement when the person being interviewed stated that the child had missed three or more months of school (of a total of 10 months: January to October). o The second sample was constructed following the cohort methodology in which a random sample of students who were enrolled in 2013 (based on that year’s attendance records made available by the sample schools) were selected and their attendance during that year was corroborated on 10 previously defined days (one per month). The cohort study was composed of random samples of five students per grade and this sample was called the follow up cohort for evaluating student attendance. It should be noted that not all schools had the documentation necessary to be able to carry out this cohort follow up. In the beginning, a third sample was contemplated to measure student attendance for 2013. This would have been obtained by reviewing the attendance records at the schools. Given that the schools do not keep records about the percentage of student attendance during the year, use of this sample was not possible. It is important to note that this section also includes information related to the indicator for absences for health reasons. • Promotion. Promotion corresponds to the percentage of children who pass their grade. This indicator was calculated in the community survey for the 2013 school year. Additionally the perception of directors and teachers was taken into account. • Continuity. Continuity corresponds to the percentage of children who are enrolled for the following school year whether or not they passed their grade. This indicator was calculated through the community survey for students from 2013 going into the 2014 school year. Additionally the perceptions of directors and teachers were taken into account. • Drop out rate. This corresponds to the percentage of children who abandon school before finishing the school year. This indicator was calculated through the community survey for the 2013 school year. Additionally the perception of directors and teachers was taken into account. 26 Table R2: Attendance, promotion, continuity, and drop out rates. Community sample The confidence level of 95% is found in parenthesis. (The groups with statistical differences are shown in green.) Indicator Pci2lb1.rec, pci2lb2.rec Children from 5 to 12 years of age in 2014 Children from 13 to 15 years of age Region Sex Total (n=441) Region Sex Total (n=175) Mam (n=210) Q’anjobal (n=231) Male (n=227) Female (n=214) Mam (n=66) Q’anjobal (n=109) Male (n=91) Female (n=84) Student attendance in 2013 and 2014 Children with good attendance during 2013. (6 to 12 years old in 2014) Mother’s perception Ind2 n=164 n=204 N=188 n=180 n=368 100% (-) 99.8% (99.7 to 100) 99.9% (99.6 to 100) 100% (-) 99.9% (99.8 to 100) 96.3% (93.7 to 98.8) 98.6% (96.5 to 100) 98.0% (95.2 to 100) 97.4% (94.6 to 100) 97.7% (96.0 to 99.4) Children who attended class a minimum of 80% of the time during the last 10 days of class prior to the baseline survey, April 2014 Information from the mother Ind2a 94.1% (91.8 to 96.5) 98.4% (96.9 to 100) 97.2% (95.1 to 99.3) 95.9% (93.7 to 98.2) 96.6% (95.0 to 98.2) Promotion in 2013 Children enrolled in the school in 2013 who passed their grade Ind4x13 n=164 n=204 N=188 n=180 n=368 94.9% (88.4 to 100) 89.3% (80.3 to 98.3) 92.1% (84.7 to 99.6) 90.6% (82.9 to 98.3) 91.4% (84.8 to 98.0) 86.5% (79.2 to 93.8) 94.2% (90.4 to 98.0) 92.0% (87.2 to 96.7) 90.3% (85.0 to 95.5) 91.1% (86.6 to 95.6) Continuity Children enrolled in the school in 2013 who were enrolled again in 2014, whether or not they passed their grade in 2013 Ind5x13 n=164 n=204 N=188 n=180 n=368 55.7% (37.8 to 73.7) 79.1% (68.8 to 89.3) 71.5% (59.7 to 83.2) 69.2% (57.1 to 81.3) 70.4% (61.7 to 79.1) 97.2% (94.7 to 99.7) 98.5% (96.0 to 100) 98.3% (96.8 to 99.8) 97.7% (94.3 to 100) 98.0% (96.3 to 99.7) 27 Table R2a: Student attendance Sample of cohort from school records (Attendance books) The confidence level of 95% is found in parenthesis. Indicator Pcilb10a.rec Children from 5 to 12 years of age Region Total (n=149) Mam (n=20) Q’anjobal (n=129) Children who attended a minimum of 80% of classes during 2013 Ind2 90.0% (74.1 to 100) 84.2% (66.7 to 100) 84.7% (68.6 to 100) Table R2b: Opinion with regard to attendance, promotion, continuity and drop out rate. Sample of directors and teachers The confidence level of 95% is found in parenthesis. (The groups with statistical differences are shown in green.) Indicator Pcilb5i2 Directors and teachers Region Total (n=73) Mam (n=40) Q’anjobal (n=33) Student Attendance Directors and teachers who considered that student attendance in 2013 was very good Ind2p 89.3% (78.6 to 99.9) 88.2% (72.9 to 100) 88.6% (78.4 to 98.8) Number of schools where the majority of staff interviewed (directors and teachers) considered that student attendance in 2013 was very good 11 schools of 12 10 schools of 13 21 schools of 25 Directors and teachers who reported that the day of the visit there was at least one student absent Ind2pa 86.6% (71.7 to 100) 87.7% (67.1 to 100) 87.3% (73.3 to 100) Number of students absent One 18.7% (2.0 to 35.5) 27.7% (14.5 to 40.9) 24.3% (13.3 to 35.2) 2 or 3 51.8% (28.6 to 75.0) 50.7% (25.4 to 76.0) 51.1% (33.2 to 69.1) 4 or more 16.1% (6.8 to 25.5) 9.2% (0.0 to 21.5) 11.9% (2.7 to 21.1) Reasons for absences on the day of the visit Illness 44.1% 16.4% 27.0% Drop outs Children who dropped out of school before the end of the 2013 school year (whether or not they passed) Ind6 n=164 n=204 N=188 N=180 n=368 0% (-) 0% (-) 0% (-) 0% (-) 0% (-) 2.1% (0.0 to 4.6) 0.2% (0.0 to 0.7) 0.7% (0.0 to 1.7) 1.3% (0.0 to 3.0) 0.5% (0.0 to 2.0) 28 Indicator Pcilb5i2 Directors and teachers Region Total (n=73) Mam (n=40) Q’anjobal (n=33) Only those that were mentioned are included Work at home 8.8% 2.9% 5.2% Work in the fields 3.9% 0% 1.2% Temporary migration 2.6% 11.2% 7.9% Directors and teachers who reported that at least one student was absent for health reasons the day of the visit Ind2pb 47.4% (35.6 to 59.2) 28.9% (14.8 to 43.1) 36.0% (25.0 to 47.1) Directors and teachers who reported that in the previous week at least one student had been absent for health reasons Ind2pc 44.8% (33.8 to 55.7) 22.3% (13.8 to 30.8) 30.9% (23.5 to 38.3) Directors and teachers who consider that the school breakfast serves as a very important incentive for children to attend school Ind2pd 83.6% (74.5 to 92.7) 98.8% (96.2 to 100) 93.0% (87.1 to 98.8) Directors and teachers who consider that if the quality of the school breakfast were improved, school attendance would increase Ind2pe 86.0% (75.3 to 96.7) 96.0% (89.3 to 100) 92.2% (85.0 to 99.3) Directors and teachers who consider that health reasons are one of the principal causes of absenteeism Ind2pf 49.9% (27.4 to 72.4) 52.6% (28.7 to 76.6) 51.6% (34.6 to 68.5) Promotion Directors and teachers who consider that there was at least a 75% promotion rate in 2013 Ind4p 96.3% (92.4 to 100) 92.7% (83.7 to 100) 94.1% (88.5 to 99.6) Number of schools where the majority of staff interviewed (directors and teachers) consider that there was at least a 75% promotion rate in 2013 12 schools of 12 13 schools of 13 25 schools of 25 Drop out rate Directors and teachers who consider that the drop out rate in 2013 was equal to, or less than, 10% Ind6p 64.7% (45.7 to 83.7) 54.1% (33.1 to 75.1) 58.1% (43.9 to 72.4) Number of schools where the majority of staff interviewed (directors and teachers) consider that the drop out rate in 2013 was equal to, or less than, 10% 9 schools of 12 6 schools of 13 15 schools of 25 29 Table R2c: Student Attendance. Sample of students from 4th to 6th grade The confidence level of 95% is found in parenthesis. (The groups with statistical differences are shown in green.) Indicator Pcilb4a Region Total (n=73) Sex Boys (n=196) Girls (n=162) Mam (n=40) Q’anjobal (n=33) Attendance to la school Students who state that they attended class every day in the previous week Ind2p 79.7% (73.7 to 85.8) 88.5% (83.0 to 94.0) 85.0% (80.0 to 90.1) 85.1% (80.1 to 90.2) 84.9% (75.6 to 94.2) Students who state that they missed class last week for health reasons Ind2pa 10.6% (5.5 to 15.7) 8.7% (4.9 to 12.5) 9.5% (6.3 to 12.6) 7.7% (3.8 to 11.7) 11.7% (3.5 to 19.9) Number of days missed the previous week for health reasons One day 8.5% 6.6% 7.3% 6.6% 8.2% Two or three days 1.6% 1.4% 1.5% 0.4% 2.9% Four or more 0.5% 0.7% 0.7% 0.7% 0.6% Students who state that some of their classmates didn’t attend class today Ind2p1 79.8% (59.1 to 100) 58.8% (40.7 to 77.0) 67.1% (53.3 to 80.9) 67.5% (50.4 to 84.6) 66.7% (55.6 to 77.7) Students who state that some of their classmates didn’t attend class today due to health reasons Ind2p1a 17.6% (9.9 to 25.4) 14.2% (9.0 to 19.5) 15.6% (11.2 to 20.0) 10.0% (1.6 to 18.3) 22.9% (14.1 to 31.6) Students who state that some of their classmates didn’t attend class last week due to health reasons Ind2p2 56.2% (40.5 to 71.8) 45.9% (41.3 to 50.5) 50.0% (42.3 to 57.6) 55.5% (44.6 to 66.5) 42.7% (29.1 to 56.3) Students who state that in the last four weeks they have missed classes due to health reasons Ind2p3 26.6% (14.3 to 38.9) 14.8% (5.9 to 23.7) 19.5% (10.5 to 28.5 19.1% (12.2 to 25.9) 20.1% (7.4 to 32.7) 30 Principal findings for attendance, promotion, continuity and drop out rates The results show very high percentages in the two regions for 6 to 12 year old children with good attendance according to the community survey (above 95 percent in all cases). Nevertheless, Table R2 shows that a higher percentage of children in the Q´anjobal area complied with the indicator for children attending class at least of 80% of the time in the 10 days prior to the baseline survey (98.4% compared to 94,1%, P<0.05, respectively). It is important to note that the attendance percentages for children from 5 to 12 years of age based on the “cohort or follow up group” methodology (Table R2a), were also very high but there were no differences between the students from the two regions. With regard to promotion, these results also showed very high values (all above 85 percent) and were similar for the two groups, including for the older children who were considered for this indicator (13 to 15 year olds). It can also be observed that the continuity indicator showed higher values for the 5 to 12 year old group than for the older children and that a higher proportion of the latter complied with this indicator in the Q´anjobal area than in the Mam area (79.1% vs. 56.7%, P<0.05). According to the community survey, dropout rates were very low for the study groups at 0.5% for the entire group interviewed. When asked their opinion, a large portion of the directors and teachers (above 85 percent) stated that student attendance was very good throughout 2013. In spite of this, a similar number (around 87 percent) reported that on the day of the survey at least one student was absent from their class with health reasons being the most predominant cause mentioned. The results show that a significant proportion of directors and teachers reported that at least one of their students were absent the previous week due to health reasons and the highest percentage corresponded to the respondents in the Mam area (44.2% compared to 22.3%, P <0.05). In the Mam region it was 2.8 times more probable (OR) that absences due to health reasons would be mentioned than in the Q´anjobal region. Additionally, around half of the directors and teachers interviewed in both linguistic regions considered that health reasons were one of the principal causes of student absences. Directors and teachers considered that the school breakfast is a very important incentive for student attendance and this was reflected in the values shown for this question in both regions (83.6% and 98.8%). The higher rate (P<0.05) corresponds to the respondents in the Q´anjobal region who had 15.8 times greater probability (OR) than their Mam region counterparts of considering the school breakfast as a determining factor for decreasing student absences. These results show that more than 90 percent of the directors and teachers felt that in 2013 there was a good rate of promotion of children to the next grade (75% or more). Nevertheless, a much smaller proportion in both groups thought that the dropout rate during the same year was 10% or lower. This is consistent with the number of schools where the majority of teaching staff shared this same opinion. A considerable percentage of students stated that they had attended classes the week prior to the survey with the students of the Q´anjobal region having a higher percentage than those of the Mam region (88.5% vs. 79.5%, P<0.05). The Q’anjobal students had two times more probability (OR) of having attended classes the week prior to the survey than those in the Mam region. It was also noted that approximately 15% of the children interviewed stated that some of their classmates had “not attended class today” for health reasons. Half of the children who were interviewed reported absences for this same reason during the previous week. 31 III Health and nutrition educational activities (Contribution to Feed the Future / Improved Knowledge of Health/Hygiene Practices) The information in this section reflects several perspectives. The first corresponds to the information that was obtained from the schools by reviewing their records, as well as from interviews with directors, teachers, students and the Parent Council. Information was also collected from interviews with parents in the community. Table R3a: Documentation of educational activities related to health and nutrition, and preventive programs. School records The confidence level of 95% is found in parenthesis. Indicator Pcilb9c.out Region Total (n=26 schools) Mam (n=13 schools) Q’anjobal (n=13 schools) Ind7e1a Percentage of schools that state that they have documentation of the implementation of at least one health and nutrition education activity in 2012 2.2% (0.0 to 6.7) 15.6% (0.0 to 44.3) 8.8% (0.0 to 23.8) Ind7e1b Percentage of schools that state that they have documentation of the implementation of at least one health and nutrition education activity in 2013 2.2% (0.0 to 6.7) 15.6% (0.0 to 44.3) 8.8% (0.0 to 23.8) Ind7e2a Percentage of schools that state that they have documentation of the implementation of at least one activity as part of a preventive program in 2012 0.0% (-) 0.0% (-) 0.0% (-) Ind7e2b Percentage of schools that state that they have documentation of the implementation of at least one activity as part of a preventive program in 2013 0.0% (-) 0.0% (-) 0.0% (-) I7e3a12 Percentage of schools that state that the students participated in at least one health and nutrition education activity or in a preventive program in 2012 74.9% (48.8 to 100) 80.8% (58.3 to 100) 77.8% (60.7 to 94.9) I7e3a12M Mean of the number of students who reported participating in at least one health and nutrition education activity at the school during 2012 64 students (24 to 105) 184 students (45 to 322) 124 students (46 to 201) I7e3a13 Percentage of schools that state that students participated in at least one health and nutrition education activity or preventive program during 2013 84.7% (62.5 to 100) 90.6% (72.9 to 100) 87.6% (73.6 to 100) I7e3a13M Mean of the number of students who reported participating in at least one health and nutrition education activity at the school during 2013 75 students (41 to 109) 190 students (56 to 324) 132 students (59 to 205) I7e3M12 Percentage of schools where they stated that the teachers participated in at least one health and nutrition activity or preventive program during 2012 74.9% (48.8 to 100) 87.4% (69.7 to 100) 81.1% (65.7 to 96.4) I7e3M12M Mean of the number of teachers who reported participating in at least health and nutrition education activity at the school during 2012 4 teachers (2 to 6) 5 teachers (3 to 11) 5 teachers (3 to 8) I7e3M13 Percentage of schools that stated that teachers participated in at least one health and nutrition education activity or preventive program at the school during 2013 84.7% (62.5 to 100) 87.3% (69.7 to 100) 86.0% (71.9 to 100) I7e3M13M Mean of the number of teachers who report participating in at least one health and nutrition education activity at the school during 2013 5 teachers (3 to 6) 7 teachers (2 to 11) 6 teachers (3 to 8) I7f3M12 Percentage of schools where students’ mothers completed at least one training on nutrition, food preparation and hygiene during 2012 69.4% (41.6 to 97.3) 64.9% (43.2 to 86.5) 67.2% (49.4 to 85.0) I7f3M12M Mean of the number of students’ mothers who completed at least one training on nutrition, food preparation 13 mothers 61 mothers 36 mothers 32 Indicator Pcilb9c.out Region Total (n=26 schools) Mam (n=13 schools) Q’anjobal (n=13 schools) and hygiene during 2012 (5 to 22) (0 to 127) (1 to 71) I7f3M13 Percentage of schools that stated that students’ mothers completed at least one training on nutrition, food preparation and hygiene during 2013 69.4% (41.6 to 97.3) 87.3% (69.7 to 100) 78.4% (62.4 to 94.3) I7f3M13M Mean of the number of students’ mothers who completed at least training on nutrition, food preparation and hygiene during 2013 11 mothers (2 to 20) 69 mothers (0 to 137) 40 mothers (3 to 73) I7g3M12 Percentage of schools that stated that students’ fathers completed at least one training on nutrition, food preparation and hygiene during 2012 37.9% (2.5 to 73.2) 46.0% (17.4 to 74.7) 41.9% (19.7 to 64.1) I7g3M12M Mean of the number of students’ fathers who completed at least one training on nutrition, food preparation and hygiene during 2012 4 fathers (1 to 8) 11 fathers (0 to 25) 8 fathers (0 to 15) I7g3M13 Percentage of schools that stated that the students’ fathers completed at least one training on nutrition, food preparation and hygiene during 2013 37.9% (2.5 to 73.2) 46.0% (17.4 to 74.7) 41.9% (19.7 to 64.1) I7fg3M13M Mean of the number of students’ fathers who completed at least one training on nutrition, food preparation and hygiene during 2013 4 fathers (1 to 8) 13 fathers (0 to 30) 8 fathers (0 to 17) Table R3b: Education in hygiene, health and nutrition. Samples at the school level: Directors and teachers The confidence level of 95% is found in parenthesis. (The groups with statistical differences are shown in green.) Indicator pcilb5b Region Total n=73 Mam n=40 Q’anjobal n=33 Ind7 Percentage of directors and teachers who state that their school trained teachers on a hygiene, health, feeding and nutrition topic during 2013 and/or in 2014 (up to April) 63.2% (31.0 to 95.5) 62.6% (27.9 to 97.3) 62.9% (38.2 to 87.5) Topics in which teachers were trained Hygiene 46.1% (19.4 to 72.8) 62.6% (27.9 to 97.3) 56.3% (31.2 to 81.3) Sample sizes for those who stated they were trained n=19 n=18 n=37 Hand washing 34.0% 66.0% 53.5% Teeth brushing 6.1% 31.5% 21.6% Personal hygiene (daily bath, change of clothes, use of latrine) 65.3% 38.1% 48.7% Preventive health 46.5% (22.3 to 70.7) 58.8% (22.2 to 95.3) 54.1% (28.6 to 79.5) Sample sizes for those who stated they were trained n=19 n=18 n=37 Water disinfection for human consumption (boil, chlorinate, filter, SODIS) 5.2% 42.4% 27.9% Disease Prevention 36.8% 40.0% 38.8% Cleanliness in school 3.2% 29.9% 5.1% Trash management 27.4% 21.8% 24.0% Feeding 46.5% (22.3 to 70.7) 53.3% (29.6 to 76.9) 50.7% (33.1 to 68.3) 33 Indicator pcilb5b Region Total n=73 Mam n=40 Q’anjobal n=33 Sample sizes of those who stated that they were trained n=19 n=18 n=37 Importance of breakfast 70.2% 59.0% 63.3% Nutrition 47.0% (23.1 to 71.0) 60.5% (24.6 to 96.3) 55.3% (30.3 to 80.4) Sample sizes of those who stated that they were trained n=19 n=18 n=37 Feeding for school age children 71.0% 68.4% 69.4% Basic diet and nutrition guidelines “Olla familiar” 31.4% 2.0% 13.4% Who gave the trainings MINEDUC 12.2% 10.6% 11.2% Ministry of Health 47.6% 15.8% 27.9% Ministry of Agriculture 8.8% 0% 3.4% PCI 47.3% 41.2% 43.5% Other NGO 1.2% 12.6% 8.2% Other 2.5% 16.9% 11.4% Ind7al Percentage of directors and teachers who state that students in their school were trained in some hygiene, health and nutrition topics at the school during 2013 and/or 2014 (up to April) 100% (-) 87.5% (74.3 to 100) 92.3% (85.0 to 99.5) Topics in which students were trained Hygiene 100% 84.3% 90.3% Preventive health 92.8% 79.8% 84.8% Feeding 86.4% 71.1% 77.0% Nutrition 69.2% 81.5% 76.8% Who gave the trainings The director 0% 6.1% 3.8% Teachers 74.3% 27.0% 45.1% Someone outside the school 41.6% 74.0 61.6% Someone outside the school MINEDUC 8.8% 0% 3.4% Ministry of Health 31.2% 26.9% 28.5% Ministry of Agriculture 2.7% 0% 1.0% PCI 16.8% 37.6% 29.6% Other NGO 0% 15.3% 9.5% Other 2.5% 8.2% 7.8% 34 Table R3c: Hygiene, health and nutrition education Samples at the school level: Students from 4th, 5th, and 6th grades The confidence level of 95% is found in parenthesis. (The groups with statistical differences are shown in green.) Indicator pcilb4a1, pcilb4a2 Region Total n=358 Sex Mam n=170 Q’anjobal n=188 Boys n=196 Girls n=162 Ind7al Percentage of students who state that they were trained at school on one of the hygiene, health and nutrition topics during 2013 and/or 2014 (up to April) 97.2% (94.4 a 99.9) 95.5% (90.8 a 100) 96.2% (93.2 a 99.1) 95.9% (92.4 a 99.5) 96.5% (93.6 a 99.4) Topics in which students were trained Hygiene 93.3% 93.9% 93.7% 94.5% 92.6% Preventive health 88.2% 75.9% 80.8% 81.1% 80.5% Feeding 92.7% 87.9% 89.8% 90.4% 89.0% Nutrition 82.5% 64.9% 71.9% 74.2% 68.9% Who gave the training The director 7.4% 9.0% 8.4% 10.0% 6.2% Teachers 70.3% 71.5% 71.0% 70.3% 72.0% Someone outside the school 64.7% 23.6% 39.8% 43.1% 35.6% MINEDUC 4.7% 0.2% 2.0% 1.7% 2.4% Person outside the school Ministry of Health 15.6% 6.1% 9.9% 9.4% 10.4% Ministry of Agriculture 0.6% 0.0% 0.3% 0.0% 0.6% PCI 48.4% 15.3% 28.4% 32.4% 23.2% Other NGO 0.2% 0.0% 0.1% 0.2% 0.0% Other 4.7% 3.3% 3.8% 5.2% 2.0% Ind7con Percentage of students who can state the principal health and hygiene practices or habits everyone should have 92.3% (88.6 to 96.0) 94.7% (92.5 to 96.9) 93.8% (91.9 to 95.7) 92.0% (89.1 to 95.0) 96.0% (92.9 to 99.1) Percentage of students who know the principal health and hygiene practices or habits everyone should have (mention at least 4 of the 7) 37.6% (27.7 to 47.6) 41.2% (34.8 to 47.5) 39.8% (34.2 to 45.4) 30.6% (21.5 to 39.7) 51.7% (38.3 to 65.1) Percentage of students who know the principal health and hygiene practices or habits everyone should have (mention at least 3 of the 7) ind7cona1 75.6% (69.1 to 82.1) 68.6% (63.9 to 73.2) 71.4% (66.9 to 77.8) 60.4% (51.1 to 69.8) 85.5% (77.5 to 93.5) Practices mentioned Daily Bath 88.4% 72.1% 78.6% 73.0% 85.7% Hand washing 61.1% 66.0% 64.0% 59.6% 69.8% Brushing teeth 32.1% 36.4% 34.7% 27.3% 44.3% Comb hair 44.0% 60.0% 53.6% 44.5% 65.5% Change clothes 67.6% 57.9% 61.7% 57.5% 67.1% Use shoes 14.1% 15.3% 14.7% 15.0% 14.6% Use of latrine or toilet 8.7% 1.0% 4.1% 4.2% 3.8% Percentage of students who know where they can obtain information about health and hygiene practices that everyone should have (mention at least one) Ind7con8a 78.0% (65.7 to 90.3) 43.4% (37.1 to 49.7) 57.1% (47.7 to 66.5) 59.3% (49.8 to 68.8) 54.3% (42.4 to 66.1) 35 Indicator pcilb4a1, pcilb4a2 Region Total n=358 Sex Mam n=170 Q’anjobal n=188 Boys n=196 Girls n=162 Where they can obtain information about the health and hygiene habits everyone should have School 66.5% 31.7% 45.5% 45.0% 46.1% Health Services 48.8% 12.1% 26.7% 29.4% 23.1% Percentage of students who know where they can obtain information about nutrition practices and habits (mention at least one) Ind7con11a 77.9% (60.8 to 94.9) 38.4% (31.3 to 45.4) 54.0% (40.2 to 67.8) 56.2% (39.9 to 72.4) 51.2% (39.5 to 62.9) Where they can obtain information about the feeding and nutrition habits everyone should have School 66.8% 32.5% 46.1% 48.5% 42.9% Health Services 47.7% 6.1% 22.6% 23.2% 21.7% Table R3d: Education on hygiene, health and nutrition. Samples at the school level: Members of the Parent Council Health and nutrition The confidence level of 95% is found in parenthesis. (The groups with statistical differences are shown in green.) Indicator pcilb7a, pcilb7a1 Region Total n=59 Mam n=28 Q’anjobal n=31 Ind7con Percentage of parents who state that they know the principal health and hygiene habits that everyone should have 100% (-) 73.3% (58.2 to 88.4) 84.3% (74.2 to 94.4) Percentage of parents who know the principal health and hygiene habits that everyone should have (mention at least 4 of the 7) 34.8% (18.7 to 50.9) 25.9% (12.4 to 39.5) 29.6% (18.9 to 40.3) Percentage of parents who know the principal health and hygiene habits that everyone should have (mention at least 3 of the 7) 79.4% (63.6 to 95.3) 38.6% (15.5 to 61.7) 55.4% (37.1 to 73.7) Practices mentioned Daily bath 100% 47.0% 68.8% Hand washing 59.7% 53.0% 55.8% Brushing teeth 17.7% 32.7% 26.5% Comb hair 42.2% 32.9% 36.7% Change clothes 77.3% 62.6% 51.4% Use shoes 3.8% 16.2% 11.1% Use latrine or toilet 14.1% 14.3% 14.2% Percentage of parents who state that they know where they can learn about or obtain information about personal health and hygiene habits (mention at least one) Ind7con8 68.8% (57.0 to 80.6) 80.6% (66.6 to 94.5) 75.7% (65.3 to 86.1) Places mentioned Schools /institutes /private schools 43.5% 36.0% 39.1% Health services 0.0% 0.0% 0.0% Churches 1.7% 7.6% 3.7% Libraries 0.0% 0.0% 0.0% Percentage of parents who state that they know where they can learn about or obtain information about 65.0% 55.5% 59.4% 36 Indicator pcilb7a, pcilb7a1 Region Total n=59 Mam n=28 Q’anjobal n=31 feeding, diet and nutrition (mention at least one) Ind7con10 (53.0 to 76.9) (42.9 to 68.0) (50.6 to 68.2) Places mentioned Schools /institutes /private schools 43.9% 23.2% 31.7% Health services 0.0% 0.0% 0.0% Churches 0.0% 0.0% 0.0% Libraries 0.0% 0.0% 0.0% Ind7al Percentage of parents who state that in school students are taught about hygiene, health, feeding and nutrition 96.6% (90.1 to 100) 86.2% (74.1 to 98.3) 90.5% (83.4 to 97.5) Topics which students are taught Hygiene 96.6% 79.8% 86.7% Preventive health 96.6% 86.2% 90.5% Feeding 93.2% 79.8% 85.3% Nutrition 76.3% 71.3% 73.4% Ind7conG Percentage of parents who mention at least three recommendations or important aspects about diet and nutrition of large groups of children 43.3% (23.4 to 63.2) 13.1% (0.0 to 27.7) 25.6% (14.7 to 36.4) Recommendations or aspects mentioned That the foods have a good calorie level 76.8% 24.3% 47.6% That the foods have protein with high biological value 21.5% 14.9% 17.8% That the foods have a good level of vitamins and minerals (vitamin A, iron, zinc) 92.9% 67.2% 78.6% That the foods are adequate based on the requirements for age and/or sex 0.0% 0.0% 0.0% That the foods are compatible with local food preparation styles 5.7% 0.0% 2.5% That the foods are served with good hygiene and at the appropriate temperature 19.6% 12.4% 15.6% That the foods have locally acceptable flavor, texture and/or color 28.7% 33.0% 31.1% Table R3e: Education in hygiene, health and nutrition Samples at the community level: Parents Theconfidence level of 95% is found in parenthesis. (The groups with statistical differences are shown in green.) Indicator pcilb3b, pcilb3b1 Region Total n=477 Mam n=222 Q’anjobal n=255 Ind7con Percentage of parents who state that they know the principal health and hygiene practices that everyone should have 93.5% (90.8 to 96.2) 83.4% (79.9 to 86.9 87.6% (85.4 to 89.8) Percentage of parents who know the principal health and hygiene practices that everyone should have (mention at least 4 of the 7) 27.3% (18.2 to 36.5) 22.9% (11.1 to 34.6) 24.7% (16.9 to 32.5) Percentage of parents who know the principal health and hygiene practices or habits that everyone should have (mention at least 3 of the 7) 69.6% (59.5 to 79.7) 50.8% (42.1 to 59.4) 58.7% (52.2 to 65.1) 37 Indicator pcilb3b, pcilb3b1 Region Total n=477 Mam n=222 Q’anjobal n=255 Practices mentioned Daily bath 91.4% 74.1% 81.3% Hand washing 64.2% 59.8% 61.6% Teeth brushing 22.3% 23.2% 22.8% Combing hair 29.9% 29.2% 29.5% Change clothes 74.0% 45.0% 57.1% Use shoes 2.1% 3.5% 2.9% Use latrine or toilet 5.6% 1.6% 3.3% Percentage of parents who state that they know where they can learn or obtain information about personal health and hygiene practices or habits (mention at least one) Ind7con8 76.1% (67.1 to 85.2) 59.6% (53.6 to 65.6) 66.5% (60.6 to 72.5) Places mentioned Schools /institutes /Private schools 44.2% 27.4% 34.5% Health services 0.0% 0.0% 0.0% Churches 1.3% 1.8% 1.6% Libraries 0.0% 0.0% 0.0% Percentage of parents who state that they know where they can learn or obtain information about feeding, diet and nutrition practices or habits (mention at least one) Ind7con9 71.4% (61.9 to 80.9) 48.5% (42.8 to 54.2) 58.1% (51.3 to 64.8) Places mentioned Schools /institutes /private schools 40.5% 27.0% 32.7% Health services 0.0% 0.0% 0.0% Churches 1.0% 0.6% 0.8% Libraries 0.0% 0.0% 0.0% Ind7conG Percentage of parents who mention at least three recommendations or aspects that are important regarding diet and nutrition for large groups of children 31.0% (22.4 to 39.5) 0.3% (0.0 to 0.8) 13.2% (7.7 to 18.6) Recommendations or aspects mentioned That the foods have a good calorie level 62.3% 15.1% 34.9% That the foods have protein with high biological value 13.0% 0% 5.5% That the foods have a good level of vitamins and minerals (vitamin A, iron, zinc) 72.5% 33.9% 50.1% That the foods are adequate based on the requirements for age and/or sex 1.7% 0.0% 0.7% That the foods are compatible with local food preparation styles 7.8% 0.3% 3.5% That the foods are served with good hygiene and at the appropriate temperature 11.4% 0.2% 4.9% That the foods locally acceptable flavor, texture and/or color 22.3% 0.5% 9.6% 38 Principal findings for health and nutrition education activities Based on school records: The results show that a small percentage of schools (less than one fifth of them) have documentation that shows the implementation of at least one health and nutrition education activity in 2012 and 2013, and that none of them can demonstrate the development of at least one preventive activity during these same years. Nevertheless, the results show, that there is a high percentage of schools where they claim that the students participated in at least one health and nutrition activity or an activity in a preventive program in both regions (around 80 percent). A considerable mean also reported a number of students who participated in the first of these components during 2012 and 2013 (124 and 132 students for the total population during these years, respectively). Similar behavior was seen for the teachers who participated in this kind of activity during these same two years - that is, more than 75 percent for all cases. With regard to the participation of the students’ parents, a considerable number of mothers (67.2% and 78.4% for 2012 and 2013, respectively), completed at least one training in nutrition, food preparation and hygiene. The numbers that corresponded to fathers with regard to this same activity are lower at only 41.9 % of the total population. Based on the samples of directors and teachers: A considerable proportion of directors and teachers (62.9% for the total population) stated that in their schools they trained on the topics of hygiene, health, feeding and nutrition in 2013 and 2014. These results showed that the reported values for each of these four areas were above 50 percent and that statistically significant differences were found within these areas of knowledge. Higher values (P<0.05) were reported for the following topics in the Q´anjobal region when compared to the Mam region: a) hand washing, b) water disinfection for human consumption and c) cleanliness at school (66.0% vs. 34.05, 42.4% vs. 5.2%, and 29.9% vs. 32.2%, respectively). On the other hand, the highest value was found for the teachers of the Mam region compared to the other region with regard to the topic of the basic diet and nutrition guidelines, “Olla familiar” (31.4% vs. 2.0%, P <0.05). The majority of the directors and teachers interviewed (43.5%) stated that they had received these trainings from PCI with the Ministry of Health mentioned in second place (27.9%). It is important to mention that although this is the baseline, PCI is mentioned due to the fact that the target population (as well as the sample) included schools that participated in Phase I of the Program. The majority of these same directors and teachers also stated that the students in their schools were trained in these four areas during 2013 and 2014, with the area of nutrition being mentioned most frequently (76.82% for the total population). According to this report, it can be seen that the trainings were primarily carried out by staff from outside the schools, followed by teachers (61.6%, and 45.1%, respectively). The results also show that the value reported for the teachers was higher in the Mam region (74.3% vs. 27.0%, P<0.05), and that the percentage that corresponded to staff from outside the school was greater in the Q´anjobal area (74.0% vs. 41.6%, P<0.05). It was also observed that the staff from outside the school was most frequently from PCI, and that this was reported more in the Q´anjobal area (37.6% compared to 16.8%, P < 0.05). The teachers of Q´anjobal region were three times more likely (OR) to attribute these trainings to PCI, than the teachers of the Mam region. 39 Based on student samples: It was observed that a very high proportion of the children interviewed (above 80 percent) stated that they were trained in these four areas of knowledge; and that of new nutrition was the area most mentioned (68.9%). It was observed that the values for the trainings in preventive health and feeding were higher in the Mam region compared to the Q´anjobal region: (88.2% vs. 75.9%, and 82.5% vs. 64.9%, P <0.05). The Mam respondents had 2.4 and 2.5 times greater probability (OR) of reporting this than the Q´anjobal, respectively. The children who were interviewed stated that they mainly received this training from their teachers, followed by staff from outside the school (71.0% and 39.8%, p<0.05, respectively). The latter report, contrary to the teachers, showed a higher value for the children in the Mam region compared to their counterparts in the Q´anjobal region (64.7% vs. 23.6%, P<0.05). Once again, PCI was identified, in this case by the children, as the most significant external staff to give these trainings (28.4% in the total population). The value in the Mam region was higher than that of the Q´anjobal (48.4% compared to 15.3%, P<0.05). The Mam had 5.2 times greater probability (OR) of stating this than those in the Q´anjobal area. It is interesting to note that this conclusion was higher for boys than for girls (32.4% compared to 23.2, P<0.05). A very high percentage of children (greater than 90 percent) stated that they know the principal health and hygiene practices or habits that everyone should have; nevertheless, these results show that 37.6% Mam children and 41.2% of Q´anjobal children did in fact know this information (know four of seven practices), and the highest value was for girls (P<0.05). These results also show that the daily bath was the most widely known habit among the Mam children, and that of hair combing was the most reported among the Q´anjobal. And as is to be expected, this was reported by a higher proportion of girls than boys (P<0.05). A little more than half of the students (57.1%) know where they can obtain information about health and hygiene practices, and a similar number (54%) know where they can obtain information about nutrition practices and habits. The percentage was higher in the children of the Mam region (P<0.05). Among these, less than half of the children stated that the school is the principal place in the community where they can obtain information about health and hygiene practices as well as about feeding and nutrition habits that everyone should have. The second most common place mentioned for this was the health services. As can be observed, higher values for both indicators were found among the Mam children compared to the Q´anjobal students (66.5% vs. 31.7%, and 66.8% vs. 32.5$, P<0.05, respectively). Based on members of the Parent Council: The results show that approximately 30 percent of the members of the Parent Council know the health and hygiene habits than everyone should have, and this knowledge is similar in both linguistic regions. Specifically it can be observed that members of the Parent Councils in the Mam region have greater knowledge about the practice of daily baths than those in the Q´anjobal region (100% compared to 47.0%, P<0.05). A similar situation was observed with regard to knowledge about changing clothes on a daily basis, where the values correspond to 77.3% and 62.4%, for each of these groups, respectively. In general it can be noted that a high percentage of these same parents stated that they knew where (at the local level) to obtain information about personal health and hygiene practices; nevertheless, for half of these, the only place they knew of was in schools/institutes/private schools, and a very few mentioned churches. It is interesting to observe that contrary to answers by the 40 children, none of the parents interviewed in these groups mentioned the health services as a source for this information. The knowledge of local sources for information on the feeding, diet and nutrition practices or habits was even more limited, given that they only mentioned schools/institutes/private schools. Again the members of this group (parents) were totally unaware of the education and preventive health activities carried out by the health services (convergence centers and/or health posts) in their own communities. In these results it can also be noted that the majority of the members of these councils know that the school is teaching their students about hygiene, health and nutrition. Based on the community survey Only a quarter of the parents interviewed know the health and hygiene habits that everyone should have (four of seven practices). Knowledge about daily baths and changing clothes was greater among the parents in the Mam region compared to those in the Q´anjobal region (91.4% vs. 74.1, P<0.05 and 74.0% vs. 45.0%, P<0.05, respectively). Of these parents, 66.5% and 58.1% stated that they knew where to obtain information about personal health and hygiene practices and good feeding, diet and nutrition habits. Their responses mostly focused on the schools/institutes/private schools, and to a lesser degree on the churches. This finding is consistent with the responses give by the members of the Parent Council as described in the previous paragraph. It is important to note that there was greater awareness of the schools as a local source of information for both indictors on the part of parents in the Mam region compared to those in the Q´anjobal region (44.2% vs. 27.4%, and 40.5% vs. 27.0%, P<0.05). 41 IV Improved Attentiveness One of the aspects measured in the school was whether children were attentive while receiving classes. This provided the levels for the improved attentiveness indicator (students in classrooms identified as inattentive). Table R4a: Students who are inattentive in class. Classroom observations CATHY The confidence level of 95% is found in parenthesis. (The groups with statistical differences are shown in green.) Indicator pcilb6b Region Total (n=75 periods of observation) Mam (n=36 periods of observation) Q’anjobal (n=39 periods of observation) Ind17 Mean of the percentage of inattentive students (presented at least one of the five categories) 36.6% (29.0 to 44.2) 77.4% (64.6 to 90.2) 60.4% (50.1 to 70.8) Categories (means of percentages) (ind17a) Distracted 11.4% (8.1 to 14.8) 29.5% (15.4 to 43.7) 22.0% (12.9 to 31.1) Paying attention to something else 8.9% (6.6 to 11.3) 19.2% (5.4 to 33.0) 14.9% (6.6 to 23.2) Doing something distracting 6.0% (4.1 to 7.8) 29.3% (20.9 to 37.7) 19.6% (13.0 to 26.3) Unmotivated 3.2% (0.6 to 5.8) 7.6% (3.1 to 12.0) 5.7% (2.8 to 8.6) Distracting with others 7.1% (4.7 to 9.5) 17.4% (7.1 to 27.7) 13.1% (6.7 to 19.5) Table 4b: Characteristics observed in the classroom during the evaluation of children’s attentiveness Classroom observations The confidence level of 95% is found in parenthesis. Indicator pcilb6b Region Total (n=75 observation periods) Mam (n=36 observation periods) Q’anjobal (n=39 observation periods) Ind17b Number of grades taught in the same classroom Only one grade 66.2% (48.2 to 84.1) 47.1% (12.8 to 81.5) 55.6% (35.1 to 76.2) Two grades 23.5% (6.4 to 40.7) 35.1% (2.4 to 67.8) 29.9% (10.1 to 49.8) Three grades 10.3% (0.0 to 23.0) 17.7% (0.0 to 39.7) 14.4% (1.2 to 27.7) Ind17c Number of students in the classroom Fewer than 20 41.9% (18.6 to 65.2) 28.8% (0.0 to 61.9) 34.3% (12.9 to 55.7) Between 20 and 30 43.3% (23.4 to 63.1) 31.4% (8.2 s 54.6) 36.3% (20.1 s 52.5) 31 or more 14.9% (3.4 to 26.4) 39.7% (4.2 to 75.3) 29.4% (8.0 to 50.8) 42 Table 4c: Attentiveness in class: 4th to 6th grade students’ opinions. Survey of students in the school The confidence level of 95% is found in parenthesis. (The groups with statistical differences are shown in green.) Indicator pcilb4b Region Total (n=358 students) Sex Mam (n=170 students) Q’anjobal (n=188 students) Boys (n=196 students) Girls (n=162 students) Ind17d Percentage of students who think their attention in the last class period was good 85.7% (77.0 to 94.4) 89.2% (83.5 to 94.9) 87.8% (82.9 to 92.8) 83.8% (77.8 to 89.7) 93.1% (87.6 to 98.7) Reasons why they think their attention wasn’t good in class (for those who stated that their attention was not good in the last class period) n=31 n=27 n=58 n=31 n=31 Not interested in the class topic 47.2% 35.1% 40.7% 40.7% 40.7% The teaching style was very boring 29.0% 9.2% 18.4% 24.4% 0.0% They needed educational materials 7.9% 0.0% 3.7% 2.7% 6.7% They didn’t understand or had a hard time understanding what was being taught 46.1% 46.4% 46.3% 34.9% 81.1% They can’t see or hear well 0.0% 0.0% 0.0% 0.0% 0.0% The classroom and/or the seats are very uncomfortable 0.0% 3.3% 1.7% 2.3% 0.0% They were/are hungry or very hungry 0.0% 0.0% 0.0% 0.0% 0.0% They were sleepy 4.5% 0.0% 2.1% 2.8% 0.0% They were tired 3.5% 0.0% 1.6% 0.9% 3.8% They were sick 11.0% 0.0% 4.1% 4.0% 8.5% They are very restless/don’t like to sit still 6.5% 37.4% 23.0% 24.4% 18.9% They have problems at home 0.0% 2.2% 1.2% 1.5% 0.0% Suggestions: What can be done to improve attention Improve quantity/quality of educational materials 56.0% 25.1% 37.3% 39.8% 34.1% Change/improve the teaching style 81.0% 60.1% 68.4% 67.3% 69.8% Improve conditions so students are able to see and hear better 7.1% 10.8% 9.4% 9.5% 9.1% Improve the school’s physical conditions 2.8% 0.0% 1.1% 0.8% 1.5% Eat before coming to school/receive timely breakfast 11.0% 0.9% 4.9% 5.4% 4.1% Sleep well at home 10.5% 5.0% 7.2% 5.8% 9.0% Come rested 3.9% 2.8% 3.3% 5.6% 0.3% If they are healthy 14.1% 6.2% 9.3% 7.3% 12.0% Improving conditions at home 0.6% 0.2% 0.3% 0.4% 0.3% 43 Principal Findings for Improved Attentiveness The results on Table R4a show various interesting aspects with regard to inattentiveness in the children who were observed in the classrooms in both regions. There was a gradual decrease of the means of the percentages of children from the category of “distracted” to the category “inattentive due to physical issues”, with the children of the Mam region being those who are most consistent with this pattern. At the category level, it can also be observed that the highest values for inattentiveness were seen in the children of the Q´anjobal region, and that two of these were statistically significant: distracted children and children doing some distracting activity (77.4% compared to 11.4%, and 29.3% compared to 6.05, P<0.05). We could also observe that the indicator for inattentiveness in children (children who presented at least one of the five categories measured) was higher for the children observed in the Q´anjobal region (77.4% compared to 36.6%, P<0.05), which shows a gap of 40.8% between the two groups. The general characteristics of the classrooms that were observed (Table R4b) did not present statistically significant differences between the two regions; nevertheless, a tendency towards better educational conditions in the classrooms of the Mam region was noted. That is, there was a greater tendency towards higher percentages of classrooms with fewer children, and with fewer grades in one classroom. These conditions seem to propitiate “lower levels of inattentiveness” in the students and would thus facilitate learning. Table R4c presents the results from the interviews which were carried out with students from fourth, fifth and sixth grades. A high proportion of these students (above 85 percent) considered that they had good attention in the class they had received or were receiving. This claim was higher in the case of girls who were interviewed compared to boys (93.1% compared to 83.8%, P<0.05). The values for the reasons why students were not paying attention right before the interview are also presented for those who thought their attention was just ok or bad. These results show that three principal causes were expressed by students for their inattentiveness: a) they didn’t understand or had a hard time understanding the class material (46.3%), b) the class topic wasn’t interesting to them (40.7%), and c) that they are very restless and don’t like to sit still (23.0%). Other causes were also mentioned such as “the way the class was presented was very boring”, they didn’t feel well, and the lack of educational materials (18.4%, 4.0% and 3.7%, respectively). It is interesting to observe that the causes of being hungry or very hungry and difficulties seeing or hearing were not mentioned by some of the students, and there were only statistically significant differences in three of these causes: lack of educational materials, feeling sick, and being very restless and not liking to sit still. The first two were higher in the Mam region and the third was higher in the Q´anjobal region (P<0.05). When asked what could be done to improve their attentiveness in class, the majority of these children indicated two possible solutions: a) Changing or improving the way the classes are taught (68.4%) and b) improving the quality and quantity of educational materials (37.3%). Other suggestions given by the students included: improving conditions for seeing and hearing, and having slept well at home the night before. It can be noted that there are four suggestions whose values were significantly different based on the region (P<0.05): a) improving the quantity and quality of the educational materials, b) having breakfast before coming to school/receiving a timely school breakfast, c) having slept well, and d) having good health. It is interesting to note that the percentages of the four suggestions mentioned above were greater in the children of the Mam region. Only one of these suggestions was higher based on the sex of those interviewed (being well rested) with a higher value for boys (P<0.05). 44 V Improved Reading ability of school-age children It is well known that reading is a complex process in which a variety of elements are at work between the reader and the text. The reader simultaneously fields several individual factors that are interrelated as he/she reads in order to allow adequate comprehension of what is being read. These factors include, for example, level of perceptive, intellectual, and social development, language, and previous learning. Table R5a. Student reading ability Community sample of those who completed 6th grade in 2013 in the target schools The confidence level of 95% is found in parenthesis. (The groups with statistical differences are shown in green.) Indicator pcilb11a Region Total (n=104 students) Sex Mam (n=56 students) Q’anjobal (n=48 students) Boys (n=59 students) Girls (n=45 students) Ind18a Mean of the percentage of reading ability-1 in students (0 to 100 point scale (Equal weighting for the three skills) 64.7 points (59.8 to 69.6) 64.7 points (60.9 to 68.5) 64.7 points (61.6 to 67.8) 63.7 points (60.9 to 66.4) 66.0 points (61. 1 to 70.9) Categories according to percentage obtained % of students in categories (ind18a1) Standard (81 points or more) 6.0% 5.4% 5.7% 2.5% 9.7% Medium (61 to 80 points) 56.1% 56.8% 56.4% 59.0% 53.2% Low (41 to 60 points) 36.7% 37.8% 37.3% 37.3% 37.1% Very Low (0 to 40 points) 1.2% 0.0% 0.6% 1.1% 0.0% Ind18b Mean of the percentage of reading ability-2 in students (0 to 100 point scale) (Weighting: speed 10%, accuracy 20%, comprehension 70%) 57.2 points (52.0 to 62.4) 54.6 points (49.0 to 60.2) 55.9 points (52.1 to 59.7) 54.3 points (51.1 to 57.4) 58.0 points (51.7 to 64.3) Categories according to percentage obtained % of students in categories (ind18b1) Standard (81 points or more) 5.2% 5.4% 5.3% 2.5% 8.8% Medium (61 to 80 points) 34.1% 19.2% 26.6% 22.3% 32.0% Low (41 to 60 points) 53.1% 57.0% 55.0% 61.1% 47.4% Very Low (0 to 40 points) 7.6% 18.4% 13.0% 14.1% 11.7% Ind18vel Mean of the percentage of reading speed in students (0 to 100 point scale) 61.5 points (54.1 to 68.8) 62.6 points (56.4 to 68.9) 62.1 points (57.2 to 66.9) 62.2 points (56.2 to 68.2) 61.8 points (54.7 to 69.0) Categories according to percentage obtained (ind18vel1) Standard (81 points or more) 15.1% 13.4% 14.2% 13.8% 14.8% Medium (61 to 80 points) 33.2% 23.0% 28.1% 31.1% 24.4% Low (41 to 60 points) 44.1% 60.0% 52.0% 48.5% 56.4% Very Low (0 to 40 points) 7.6% 3.6% 5.6% 6.6% 4.4% Ind18exa Mean of the percentage of reading accuracy in students (0 to 100 point scale) 83.8 points (80.9 to 86.8) 87.5 points (85.0 to 90.0) 85.7 points (83.5 to 87.8) 84.4 points (82.1 to 86.8) 87.2 points (84.4 to 89.9) Categories according to Standard (81 points or more) 14.2% 28.5% 21.3% 13.7% 30.9% 45 Indicator pcilb11a Region Total (n=104 students) Sex Mam (n=56 students) Q’anjobal (n=48 students) Boys (n=59 students) Girls (n=45 students) percentage obtained (ind18exa1) Medium (61 to 80 points) 57.2% 63.2% 60.2% 60.6% 59.7% Low (41 to 60 points) 24.8% 8.3% 16.5% 24.6% 6.4% Very Low (0 to 40 points) 3.9% 0.0% 2.0% 1.1% 3.0% Percentage of students who committed an error Sample size of sub sample n=27 n=23 n=50 n=28 n=22 Substitutions 100% 100% 100% 100% 100% Inversions 23.2% 47.7% 35.1% 28.9% 42.3% Guessing 98.4% 76.2% 87.6% 94.1% 80.0% Additions 100% 99.4% 99.7% 99.5% 100% Omissions 100% 100% 100% 100% 100% Intonation 100% 100% 100% 100% 100% Regressions 100% 100% 100% 100% 100% Percentage of students who made various reading errors (with sub sample) Less than 5 0% 0% 0% 0% 0% 5 1.6% 10.7% 6.0% 0.5% 12.5% 6 75.2% 55..3% 65.5% 76.5% 52.7% 7 23.2% 34.0% 28.4% 23.0% 34.8% Ind18com Mean of the percentage of reading comprehension in students (0 to 100 point scale) 49.0 points (42.9 to 55.2) 44.1 points (36.3 to 51.8) 46.6 points (41.6 to 51.5) 44.5 points (40.2 to 48.8) 49.1 points (40.9 to 57.3) Categories according to percentage obtained (ind18com1) Standard (81 points or more) 5.2% 0.7% 2.9% 2.3% 3.8% Medium (61 to 80 points) 16.9% 19.3% 18.1% 13.8% 23.5% Low (41 to 60 points) 31.0% 24.6% 27.8% 31.7% 23.0% Very Low (0 to 40 points) 46.9% 55.4% 51.2% 52.2% 49.8% 46 Table R5b: Opinion about students’ reading ability. Samples at school level: Directors and teachers The confidence level of 95% is found in parenthesis. (The groups with statistical differences are shown in green.) Indicator pcilb5c Region Total n=73 Mam n=40 Q’anjobal n=33 Ind18a Percentage of directors and teachers who state that the reading ability of students from their school who completed sixth grade meets standards 37.4% (7.6 to 67.2) 16.5% (0.0 to 35.4) 24.5% (4.9 to 44.1) Which of the skills need to be improved Speed 1.2% 5.7% 4.3% Comprehension 94.6% 92.0% 93.0% Accuracy 3.4% 2.3% 2.7% Suggestions for improving students’ skills Incorporating new teaching methods 52.4% 61.3% 57.9% Providing specialized reading texts 89.9% 54.8% 68.3% Training teachers in special techniques 5.6% 12.1% 9.6% Implementing monitoring and/or supervision systems 6.1% 21..9% 15.8% Emphasizing reading beginning in the earliest grades 11.3% 48.7% 34.4% Incorporating parents into their children’s education 8.4% 22.6% 17.2% Examining students sensory organs 3.4% 3.2% 3.3% Improving children’s vision and hearing 19.2% 12.7% 15.2% Findings for reading ability Based on the indicators for reading ability 1 and 2, a greater percentage of girls than boys scored in the standard category (P<0.05), and when measuring reading ability-2 it was clear that, in general, girls did better than boys. Similarly, girls displayed greater accuracy than boys (P<0.05), given that the percentage for them was greater (30.9%) in the standard category and lower in the low category (6.4%), while boys’ scores were 13.7% and 24.6%, respectively. This may be associated with the fact that girls of this age tend to be more mature than boys. Social roles and programs that have been implemented to improve girls’ education at an earlier age are other aspects that should be taken into account. In terms of where children live, reading ability-2 was greater in the Mam region than in the Q’anjobal area (P<0.05). It is important to highlight that in terms of reading ability-2 based on the average number of points (55.9 points), the level reached by all the children was generally low. In some cases it could be seen that speed was high with very low comprehension. Speed is directly related to practice. Among the accuracy errors, those of intonation and regression impact speed. In terms of accuracy, all the children made at least five of the seven types of errors, and the majority had six types of errors (65.5%). It was noted that in the Mam region the problem of guessing was higher than in the Q’anjobal region (P<0.05). With regard to guessing, systematic errors were found such as reading ministerio for ministro, promete for prominente, which suggests that the words that were invented are ones that are part of their vocabulary and thus they associate them. Within the systematic errors were also found words with decoding errors such as, for example, reading predilección as predilesion - errors based on teaching errors from earlier grades. Of the sub-sample, 100% had errors of substitution, addition, omission, intonation and regression which can be associated with an inadequate teaching methodology or poor reading learning practices throughout primary school. It would also be 47 important to evaluate the level of difficulty in the reading of recreational books that the MINEDUC competency mentions and the accuracy of reading processes in general. Comprehension was low in both groups (Mam and Q’anjobal) with the following percentages in the low (31.0% and 24.6%, respectively) and very low (46.9% and 55.4%) categories which demonstrates the inefficiency of the reading process. Teachers in the Mam as well as Q’anjobal areas consider that emphasis should be given to reading comprehension above the other skills. Perhaps they are not aware of the relationship which exists with the other skills in order to achieve an efficient reading process which is necessary to develop the competency expected by the end of sixth grade. This was most pronounced in the Mam group (89.9%) where they thought that specialized texts for reading were a priority. Nevertheless, the Q’anjobal group also mentioned this although by a lower percentage (54.8%). 48 VI School Feeding (Reduced short term hunger, increased knowledge of safe food prep and storage practices, increased access to requisite food prep and storage tools and equipment) This section presents information related to the school breakfast, such as: students who are hungry in school, knowledge of safe food preparation and storage practices, and access. Information was collected from parents, students and teachers. Table R6a: School Breakfast. Sample of students from 4th to 6th grades at school The confidence level of 95% is found in parenthesis. (The groups with statistical differences are shown in green.) Indicator Pcilb4c Region Total (n=358) Sex Mam (n=170) Q’anjobal (n=188) Boys (n=196) Girls (n=162) Ind19c Students with good eating habits at home (ate three meals/day at home last week and eats breakfast at home every day before coming to school) 87.5% (83.4 to 91.6) 86.2% (78.2 to 94.1) 86.7% (81.8 to 91.6) 83.3% (77.8 to 88.8) 91.1% (81.8 to 91.6) Iind19a Students who ate breakfast at home (today) 98.7% (97.0 to 100) 94.6% (89.4 to 99.9) 96.3% (93.3 to 99.1) 97.1% (94.4 to 99.8) 95.1% (91.2 to 98.9) Ind19b Students who brought some kind of snack to school (today) 26.5% (20.2 to 33.0) 24.1% (13.8 to 34.3) 25.1% (18.1 to 32.0) 22.3% (13.8 to 30.7) 28.7% (20.3 to 37.1) What they bring for snack Prepared food 24.1% 22.0% 22.8% 20.7% 25.5% Junk food 2.5% 2.1% 2.2% 1.5% 3.2% Ind19b3 Students who brought money to school to buy some kind of snack (today) 29.5% (21.3 to 37.8) 53.6% (42.2 to 65.0) 44.1% (33.9 to 54.3) 43.5% 44.8% Ind19d Time of school breakfast Before 10:00 a.m. 9.5% 17.0% 14.0% 16.1% 11.4% Between 10:00 and 11:00 a.m. 90.0% 47.1% 64.1% 62.1% 66.6% After 11:00 a.m. Note: This includes the afternoon school session 0.5% 35.8% 21.9% 21.8% 22.0% Ind19e1 Time at which they would like to receive their school breakfast Early: around 7:30 a.m. 19.1% 14.3% 16.2% 18.8% 12.8% Around 10 a.m. 76.3% 75.8% 76.0% 73.9% 78.7% At mid day Note: This includes the afternoon school session 4.6% 9.9% 7.8% 7.3% 8.4% 49 Ind19e Opinion of students who had had a snack at school (school breakfast): n=33 n=37 n=70 n=45 n=25 Did you like the school breakfast? A lot 98.5% 93.8% 96.5% 95.5% 98.6% A little 0.8% 6.2% 3.1% 3.9% 1.4% Didn’t like it 0.7% 0.0% 0.4% 0.6% 0.0% Ind19e2 Are hungry or very hungry at the time of the interview 9.7%% (4.9 to 14.5) 9.9% (0.0 to 19.6) 9.3% (2.5 to 16.0) 9.5% (1.7 to 17.3) 8.9% (1.8 to 16.0) Are hungry or very hungry at the time of the interview, among those who had eaten breakfast or a snack today n=166 n=180 n=346 n=190 n=156 8.7% (5.0 to 12.5) 8.1% (0.0 to 17.5) 8.4% (2.5 to 14.2) 9.2% (1.5 to 17.0) 7.2% (1.9 to 12.5) Are hungry or very hungry at the time of the interview, among those who had NOT had breakfast or a snack today n=4 n=8 n=12 n=6 n=6 83.1% 38.4% 49.3% 28.1% 60.8% Ind19e3 They consider that after snacking or having breakfast they feel energized to receive classes, learn and/or pay attention in class 44.8% (35.0 to 54.6) 22.6% (12.0 to 33.2) 31.4% (24.5 to 38.3) 26.4% (14.1 to 38.7) 37.9% (21.7 to 54.0) Table R6b: School Breakfast. Sample of directors and teachers The confidence level of 95% is found in parenthesis. (The groups with statistical differences are shown in green.) Indicator Piclb5b Directors and teachers Region Total (n=73) Mam (n=40) Q’anjobal (n=33) Ind19d Time at which school breakfast is served Before 10 a.m. 14.0% 15.6% 15.0% Between 10 a.m.and 11 a.m. 84.8% 47.0% 61.4% After 11 a.m. Note: includes afternoon session 1.2% 37.4% 23.5% Ind19e2 Think that there are students at school who are hungry or very hungry at the time of the interview 81.2% (57.6 to 100) 32.9% (21.9 to 44.0) 51.4% (33.6 to 69.2) Think that there are students at school today who are hungry or very hungry at the time of the interview, among those who indicated that the students had already had a snack at the time of the interview n=10 n=13 n=23 80.8% 20.5% 38.1% Think that there are students at school today who are hungry or very hungry at the time of the interview, among those who indicated that the students had NOT already had a snack at the time of the interview n=30 n=20 n=50 81.4% 45.7% 62.0% Ind19g Think that the school breakfast compensates nutritionally for the lack of breakfast for those students who do NOT eat at home 64.3% (38.4 to 90.2) 75.8% (66.8 to 84.8) 71.4% (61.3 to 81.5) Ind19f State that during 2013 and so far in 2014 the Parent Council members have been trained in some of the 68.2% 71.8% 70.5% 50 Indicator Piclb5b Directors and teachers Region Total (n=73) Mam (n=40) Q’anjobal (n=33) following topics: (38.9 to 97.5) (62.8 to 80.9) (58.2 to 82.7) Training topics Food transportation 29.0% 67.5% 52.8% Food storage 60.7% 65.8% 63.9% Food preparation 64.3% 66.5% 65.6% Food hygiene 39.1% 69.7% 58.0% Ind19h Think that the Parent Council played a key role in the school feeding program last year 94.0% (83.1 to 100) 90.2% (79.9 to 100) 91.7% (84.1 to 99.2) Table 6c: School Breakfast. Sample of parents in the school’s Parent Council The confidence level of 95% is found in parenthesis. (The groups with statistical differences are shown in green.) Indicator Piclb7b, pcilb7b1 Region Total (n=59) Mam (n=28) Q’anjobal (n=31) Ind19a State that at school students are given something to eat (snack or breakfast) 100% (-) 88.1% (75.5 to 100) 93.0% (86.1 to 99.9) State that MINEDUC has provided the food that is given to the students 98.3% 83.1% 89.3% State that PCI has provided the food that is given to the students 49.3% 77.3% 65.7% Ind19i Were trained in 2013, in at least one of the following topics: 76.6% (56.8 to 96.5) 77.1% (59.5 to 94.7) 76.9% (63.7 to 90.1) Training topics Management of school feeding program 66.6% 30.8% 45.6% Management and organization of food supplies in the school storage room 60.3% 41.4% 49.2% Order and cleanliness in food preparation 44.2% 48.4% 46.7% Hand washing 15.5% 25.7% 21.5% School feeding 23.5% 42.3% 34.6% When food is prepared for large groups (children in this case) Ind19j They know at least 3 of the 15 conditions that were evaluated to comply with food safety (Indicator of food safety knowledge) 91.4% (81.0 to 100) 62.7% (49.4 to 76.1) 74.6% (64.4 to 84.7) They know at least 6 of the 15 conditions that were evaluated to comply with food safety (Recommendation of cut off point for the Food safety indicator) 19.8% (6.5 to 33.2) 25.2% (13.9 to 36.5) 23.0% (14.0 to 32.0) Once prepared, the food should be covered 37.9% 18.7% 27.2% Use of tap water 32.4% 15.8% 23.1% Those who are cooking must wash their hands before cooking 63.4% 59.2% 61.1% Hand washing (wash with soap and running water for at least 20 seconds 52.3% 23.5% 36.2% 51 Indicator Piclb7b, pcilb7b1 Region Total (n=59) Mam (n=28) Q’anjobal (n=31) and dry hands well with clean towels) Those who are cooking should use hair nets 39.6% 64.0% 53.2% Those who are cooking should use aprons 53.3% 57.4% 56.6% Those who are cooking should not use rings or have polish on nails 8.6% 2.2% 5.0% Those who are cooking should not be ill 2.2% 1.4% 1.8% The work area must be clean 77.3% 83.3% 80.6% They should use a covered waste bin 26.7% 15.6% 20.5% There should be a refrigerator (to store left over foods or fruits and vegetables as well as meats and other foods that have not been cooked) 0.0% 0.0% 0.0% There should be no food or trash on the floor 18.9% 15.1% 16.7% There should not be any loose domestic animals in the kitchen 3.4% 40.5% 24.1% The water should have a microbiological test at least once a month 0.0% 3.7% 2.1% Ind19k Know at least 4 of the 8 basic storage guidelines: 29.0% (6.9 to 51.2) 26.5% (15.9 to 37.1) 27.5% (16.6 to 38.4) Basic guidelines mentioned Food should be on separate shelves 79.1% 49.3% 62.5% Meat, fruits and vegetables should be stored in the refrigerator 1.9% 0.0% 0.9% The store room Should be large enough to be functional and should be organized 16.3% 0.0% 7.2% Should be exclusively for food storage 48.8% 12.4% 28.5% Should be clean 72.4% 62.3% 66.7% Should be ventilated 1.9% 32.0% 18.7% Should have walls and roof that avoid water leakage and/or humidity 27.6% 46.0% 37.9% No evidence of rodents 71.8% 40.3% 54.3% 52 Table 6d: School Breakfast. Sample of parents at community level The confidence level of 95% is found in parenthesis. (The groups with statistical differences are shown in green.) Indicator Piclb3c, pcilb3c1 Directors and teachers Region Total (n=477) Mam (n=222) Q’anjobal (n=255) Ind19a State that at the school the students are given something to eat (snack or breakfast) 58.4% (41.1 to 75.8) 96.9% (93.2 to 100) 80.7% (69.3 to 92.2) State that MINEDUC has given the food that is provided to the students 73.7% 39.3% 53.7% State that PCI has provided the food that is given to the students 55.3% 54.0% 54.5% Cuando se hacen preparaciones o comidas para groups grandes of people Ind19j Know at least 3 of the 15 conditions that were evaluated to comply with food safety (Indicator for knowledge of food safety): 22.6% (17.9 to 27.3) 77.4% (72.7 to 82.1) 68.9% (63.2 to 74.7) Know at least 6 of the 15 conditions evaluated to comply with food safety (Recommendation for cut off point for the Food Safety indicator 21.5% (14.9 to 28.0) 17.2% (12.6 to 21.7) 19.0% (14.7 to 23.3) Conditions mentioned Foods (meat, vegetables or fruit) are processed on separate tables or boards 16.1% 15.3% 15.6% Once prepared, the food should be covered 35.9% 17.7% 25.3% Use of tap water 23.1% 6.4% 13.4% Those who cook have to wash their hands before cooking 63.7% 33.3% 46.0% Hand washing (wash with soap and running water for at least 20 seconds and dry hands well with clean towels) 59.6% 43.9% 50.5% Those who are cooking should use hair nets 34.5% 24.5% 44.0% Those who are cooking should use aprons 39.5% 50.3% 45.8% Those who are cooking should not use rings or have polish on nails 4.6% 1.8% 3.0% Those who are cooking should not be ill 1.2% 4.4% 3.1% The work area must be clean 72.4% 62.8% 66.8% They should use a covered waste bin 24.7% 15.3% 19.3% There should be a refrigerator (to store left over foods or fruits and vegetables as well as means and other foods that have not been cooked) 1.8% 0.0% 0.8% There should be no food or trash on the floor 10.0% 16.5% 13.8% There should not be any loose domestic animals in the kitchen 5.1% 23.8% 16.0% The water should have a microbiological test at least once a month 0.4% 2.1% 1.4% Ind19k Know at least 4 of the 8 basic storage guidelines: 21.0% (13.2 to 28.9) 4.8% (0.2 to 9.3 11.6% (7.2 to 15.9) Basic guidelines mentioned Food should be on separate shelves 42.1% 50.8% 47.1% Meat, fruits and vegetables should be stored in the refrigerator 9.7% 4.9% 6.9% The storage room Should be large enough to be functional and should be organized 7.4% 8.6% 8.1% 53 Indicator Piclb3c, pcilb3c1 Directors and teachers Region Total (n=477) Mam (n=222) Q’anjobal (n=255) Should be exclusively for food storage 19.8% 9.3% 13.7% Should be clean 73.0% 49.7% 59.5% Should be ventilated 5.2% 10.6% 8.3% Should have walls and roof that avoid water leakage and/or humidity 32.6% 1.1% 14.3% No evidence of rodents 58.7% 12.8% 32.1% Table R6e: Food safety. School Sample (observation) The confidence level of 95% is found in parenthesis. (The groups with statistical differences are shown in green.) (Reporting only on what could be observed) Indicator Pcilb8a Directors and teachers Region Total (n=22 schools) Mam (n=10 schools) Q’anjobal (n=12 schools) Ind20a Schools with good food management practices (snack/school breakfast) Comply with at least 4 of the following 5 practices: 32.2% (0.0 to 80.1) 43.1% (13.1 to 73.1) 38.1% (11.7 to 64.5) Food is covered 68.7% 93.3% 82.1% There is running water and if not, water is stored in recipients which you can’t put your hand in 100% 96.7% 98.3% Food is processed on separate tables (or different boards) for meats/chicken, fruits/vegetables (to avoid cross contamination) 51.3% 68.9% 60.9% There is a garbage bin in the kitchen and it is covered 0.0% 29.6% 16.1% There is no food on the floors and tables 44.4% 67.4% 56.9% Ind20b Schools with good food preparation practices (snack/school breakfast) Comply with at least 5 of the following 9 practices: 19.0% (0.0 to 47.4) 52.4% (20.0 to 84.9) 37.2% (13.6 to 60.8) The people responsible for food preparation No rings on hands 48.7% 82.5% 67.7% No fingernail polish 62.4% 100% 82.9% Use hair nets 9.5% 65.5% 40.0% Using clean aprons (only soiled from immeante use) 19.0% 52.4% 37.2% Washed hands before beginning food preparation 75.5% 6.7% 38.1% Used soap and running water to wash hands 0.0% 6.7% 3.7% Dried hands with clean cloth or towel 0.0% 3.4% 1.8% Are not ill (no one who is sick among workers) 72.9% 100% 87.7% No loose animals in the kitchen (cats, dogs, chickens, etc.) 58.3% 100% 81.0% Ind20e1 Schools with good organization and cleanliness of cooking utensils (Complied with at least three of the following: 1.dry, 2. clean, 3.covered and 4. in their specific place) 12.1% 100% 60.7% 54 Indicator Pcilb8a Directors and teachers Region Total (n=22 schools) Mam (n=10 schools) Q’anjobal (n=12 schools) Observations were made around 8a.m., except in the afternoon school where the observation was made around 1 p.m.. Ind20e2 Schools that had good order and cleanliness of the dishes used for eating (Complied with at least three of the following: 1.dry, 2. clean, 3.covered and 4. in their specific place) Observations were made around 8a.m., except in the afternoon school where the observation was made around 1 p.m.. 68.8% 100% 86.0% Ind20e3 Schools where a good food distribution process was observed. (Comply with at least 4 of the following 6 criteria: 1. Done in a clean place, 2. By staff with appropriate hygiene, 3.In clean utensils, 4. Asked the children to wash their hands, 5 Observed that children washed their hands, and 6. Supervised children while eating) 0% 100% 54.4% Ind20c Schools that stated that they had had laboratory analysis of water used for cooking and drinking at school n=12 n=13 n=25 24.0% 96.7% 61.0% Ind20c1 Schools that stated that the lab results were good for the water they used for cooking and drinking (of those who reported having water tested) n=3 n=12 n=15 100% 100% 100% Ind20d Schools that stated that they had a refrigerator n=12 n=13 n=25 7.8% 0.0% 3.9% Principal findings for school feeding The results showed that the majority of children interviewed (all above 80 percent) stated that they had good eating habits, that is, they had eaten three meals a day all last week and had eaten breakfast at home the day of the interview. A greater number of girls stated this compared to boys (91.1% compared to 83.3%, P<0.05). Almost all of the children interviewed stated that they had had breakfast the day of the interview and around a quarter of these said they took food to school for snack. For the majority these were foods prepared at home. A higher percentage of the children in the Q’anjobal region than in the Mam area, indicated that they took money to school to buy a snack (53.6% compared to 39.5%, P<0.05). The majority of these children like the school snack or breakfast to be served around 10:00 a.m. or after 11:00 a.m. A higher percentage of Mam children indicated that the best hour for the snack/breakfast is between 10:00 and 11:00 a.m. (90.0% compared to 47.1%, P<0.05). A large majority of students from both regions (94 percent or more) indicated that they liked the snack or breakfast served at the school. Of the children interviewed in both regions, 12 stated that they were hungry or very hungry at the time of the survey (all were interviewed first thing in the morning.) 55 The majority of the directors and teachers interviewed (61.4%) indicated that the school snack or breakfast is served between 10:00 and 11:00 a.m. A little more than half of these directors and teachers in both regions’ schools stated that there were children in their classes who were hungry or very hungry at the time of the interview. The majority of those who shared this opinion worked in schools in the Mam region (81.2% compared to 32.9%, P<0.05). The teachers of this linguistic region had 8.8 times more probability (OR) of having this opinion than their counterparts in the Q´anjobal region. A notable proportion of this same group (directors and teachers) indicated that during 2013 and part of 2014, members of the school’s Parent Council were trained in topics such as: food transportation, storage, preparation and hygiene. Higher percentages of directors and teachers in the Q´anjobal region gave information about these Parent Council trainings with regard to food transportation and hygiene (67.5% vs. 29.0% and 69.75 vs. 39.1%, P<0.05, respectively). These results also show that a large majority of the teachers claim that the Parent Councils played a key role in the school feeding program last year. The majority of Parent Council members know that, in the schools which were visited, school snacks or breakfasts are given to the students. Of these, around 90 percent stated that the MINEDUC had supported their school with the food that was given to the students. A lower proportion of this same group indicated that PCI had also provided this kind of support (it is important to remember that more than 50% of the schools in the second phase of the program participated in the first phase and thus have had PCI interventions). Those who acknowledged the support from the MINEDUC were mainly in the Mam region, while those who mentioned PCI were mostly in the Q´anjobal region (P<0.05). Less than half of the members of these Parent Councils stated that they were trained in 2013 in at least one of these topics: management of the school feeding program, management and organization of the food supplies in the school storage room, organization and cleanliness in food preparation, hand washing and school feeding. Percentages were greater in the Mam region for the first of these topics (66.6% vs. 30.8%, P<0.05). These results show that less than a quarter of the members of the Parent Councils interviewed (23.0 % of the total population), know at least 6 of the 15 basic aspects for compliance with safe food preparation. It can be seen that the four aspects most often mentioned were: cleanliness of the work area, hand washing before cooking, use of aprons, and the use of hair nets for those who cook or prepare foods. It is interesting to note that the absence of illness in those who are preparing food, the presence of a refrigerator for good food conservation and the micro biological analysis of water, were rarely or never mentioned by the people interviewed. The absence of loose domestic animals in food preparation areas was mentioned by the majority of the members of Parent Councils in the Q’anjobal region (P<0.05). [When the cut off point is changed for this indicator (at least 3 of the 15 aspects considered) it can be noted that a higher percentage of the Parent Councils of the Mam region complied with this indicator (91.4 compared to 62.7%, P<0.05]; where the Parent Council members of the Mam region were 6.3 times more likely (OR) to have this knowledge than their counterparts in the Q’anjobal region]. It should also be noted that a very low percentage of the Parent Council members (27.5% in the total population) know the basic concepts of food storage with the concept of refrigerating food (meats, dairy products and fruits) the weakest area among those interviewed. It was also observed that the aspects of cleanliness and the condition of the shelves in the storage room were most mentioned by this group of parents. The results of the community survey show that at this level knowledge of the existence of school snacks or breakfasts was also very high, as well as recognition that the MINEDUC and PCI provided support for this activity (a little more than 50 percent in both regions have this knowledge). Nevertheless, a higher proportion of those interviewed in the Mam region know more about the support that the MINEDUC provides (73.7% vs. 39.3%, P<0.05). On the other hand, a little less than a fifth of those interviewed in this group know at least 6 of the 15 basic aspects considered 56 in order to comply with safe food preparation. The aspects of cleanliness in the work area, hand washing and use of aprons and hair nets for those responsible for food preparation were most frequently mentioned. Again the ones those interviewed were least aware of were: presence of refrigeration in the kitchens, the micro-biological analysis of water and the absence of illness in those who prepare food for groups. Only 11.6% of those interviewed know the basic concepts of food storage. The aspects that were least known were the presence of refrigeration in food preparation areas and the size and organization that are needed for the store rooms. When the cut off point was changed for this indicator (at least 3 of the 15 aspects considered) a higher percentage of parents interviewed in the Q’anjobal region complied with this indicator, reaching relatively high values for subsequently measuring change. And even with this cut off point, the differences between the Q’anjobal and Mam regions (91.4% compared to 62.7%, respectively, P<0.05] are notable. Parents in the Q’anjobal region were 2.0 times more likely (OR) to have this knowledge than their counterparts in the Mam region. VII School Infrastructure (Increased use of health and dietary practices/Improved school infrastructure) The information in this section was obtained by means of observation at the schools and interviews with parents, at the community level and at the school (members of the Parent Council). This section describes how the schools are doing in terms of infrastructure needed to maintain good hygiene and food preparation. Table R7a: Observations at the school The confidence level of 95% is found in parenthesis. (The groups with statistical differences are shown in green.) Indicator Pcilb8b, pcilb8b1 Region Total (n=25 schools) Mam (n=12 schools) Q’anjobal (n=13 schools) Water and electricity services Ind21a Percentage of schools that have running water 100% (-) 74.3% (48.3 to 100) 86.9% (73.8 to 100) Ind21a1 Percentage of schools that state they have a system of running water that was built or rehabilitated by the PCI/USDA FFE Program (Of the schools with running water) n=12 n=9 n=21 35.1% (0.0 to 73.7) 21.0% (0.0 to 59.4) 28.9% (0.7 to 57.2) Ind21a2 Percentage of schools that have water at the time of the visit 71.6% (45.3 to 97.9) 61.6% (34.8 to 88.4) 66.5% (47.5 85.5) Ind21a3 Percentage of schools that have running water in the area where school foods are prepared 73.8% (46.0 to 100) 39.5% (9.6 to 69.4) 56.3% (34.4 to 78.3) Ind21a4 Percentage of schools that do NOT have running water, but have sufficient water stored to cover food preparation and hygiene needs - 33.8% (0.0 to 88) 33.8% (0.0 to 88) Ind21b Percentage of schools that have electricity 87.5% (68.3 to 100) 84.4% (55.7 to 100) 85.9% (68.4 to 100) Ind21b1 Percentage of schools that have electricity functioning at the time of the visit 63.5% (29.6 to 97.4) 81.2% (51.9 to 100) 72.5% (51.2 to 93.8) 57 Indicator Pcilb8b, pcilb8b1 Region Total (n=25 schools) Mam (n=12 schools) Q’anjobal (n=13 schools) Hygiene services Ind21c Percentage of schools that have latrines and/or flush toilets 100% (-) 100% (-) 100% (-) With Latrines 43.0% (2.2 to 83.8) 60.5% (28.1 to 92.9) 51.9% (27.1 to 76.7) With flush toilets 100% (-) 39.5% (7.1 to 71.9) 69.2% (51.1 to 87.2) Mean of the number students enrolled in 2014 per number of latrines and/or toilets (it was considered that all the services were clean at the beginning of class) 36 students per service (21 to 51) 34 students per service (22 to 46) 35 students per service (25 to 41) Ind21c1 Percentage of schools that have clean latrines and/or flush toilets 77.3% (53.2 to 100) 31.5% (0.0 to 64.2) 54.0% (32.3 to 75.7) Ind21c2 Percentage of schools that have a hand washing station near the latrines and/or flush toilets 84.0% (60.4 to 100) 55.8% (27.2 to 84.4) 69.6% (49.5 to 89.8) Ind21c2a Percentage of schools that have a hand washing station near the latrines and/or flush toilets, with water and soap available 8.0% (0.0 to 23.7) 26.4% (0.0 to 57.7) 17.4% (0.0 to 35.7) Food storage room Ind21d Percentage of schools that have a designated place for storage of foods used for school feeding (storage room) 92.0% (76.3 to 100) 71.0% (49.5 to 92.6) 81.3% (68.0 to 94.7) Ind21d1 Percentage of schools with a storage room that complies with at least 7 of the following 13 conditions: (of those that have a storage room) n=11 n=7 n=18 69.1% (47.6 to 90.7) 77.8% (55.9 to 99.2) 72.9% (57.6 to 88.2) Conditions observed Have improved equipment for food storage 34.3% 86.7% 57.6% Have adequate physical space related to what needs to be stored 51.7% 21.9% 38.5% Has good quality walls [Impermeable material, no presence of moisture] 100% 90.8% 95.9% Has a good quality roof [Complete and with no leaks] 80.2% 90.8% 84.9% Has sufficient ventilation 45.9% 40.3% 43.4% Is used exclusively for food storage 60.5% 86.7% 72.1% Is clean and orderly 71.6% 100% 84.2% Has shelves for product storage 62.9% 100% 79.4% No signs of rodents or other animals 88.9% 0% 49.4% The products being stored are separated 91.3% 100% 95.2% The food distribution control is up to date 22.3% 31.1% 26.2% There is a record of the inventory of the stored products 69.0% 9.2% 42.4% The storage area is constantly supervised by the Parent Council 62.9% 86.7% 73.5% 58 Indicator Pcilb8b, pcilb8b1 Region Total (n=25 schools) Mam (n=12 schools) Q’anjobal (n=13 schools) Place where food is prepared Ind21e Percentage of schools that have a designated place for food preparation (kitchen) 100% (-) 93.5% (80.1 to 100) 96.7% (90.0 to 100) Ind21e1 Percentage of schools that state that the kitchen was built by the PCI/USDA FFE Program 8.0% (0.0 to 24.7) 30.6% (0.0 to 61.8) 19.1% (1.1 to 37.2) Ind21e2 Percentage of schools with a kitchen that comply with at least 4 of the following 6 basic conditions: (of those that have a kitchen) n=12 n=12 n=24 65.8% (36.1 to 95.5) 55.8% (21.4 to 90.2) 60.9% (37.8 to 84.0) Conditions observed Have appropriate utensils for food preparation 88.9% 87.3% 88.2% Have sufficient physical space 62.9% 0.0% 34.9% Have acceptable security conditions [Prevention of accidents for those who work there] 62.9% 82.7% 72.9% Has good lighting 55.5% 70.2% 61.9% Has good ventilation 81.8% 73.6% 78.0% Has good hygiene 17.4% 100% 59.8% Table 7B: School Infrastructure. Samples at the school level: Members of Parent Council The confidence level of 95% is found in parenthesis. (The groups with statistical differences are shown in green.) Indicator pcilb7c Region Total n=59 Mam n=28 Q’anjobal n=31 Ind21a Percentage of parents who state that the school has a space for preparation of the school breakfast (kitchen) 100% (-) 88.1% (75.5 to 100) 93.0% (86.1 to 99.9) Percentage of parents who state that the kitchen complies with at least 6 of the following 10 basic conditions (of those who said there was a kitchen) n=28 n=27 n=55 63.1% (41.4 to 84.8) 58.5% (35.7 to 81.3) 60.6% (44.3 to 76.8) Criteria mentioned Sufficient space for those who cook 30.2% 44.8% 39.4% Has full walls and roof 86.6% 55.6% 69.3% Has a cement floor or one that can be swept and mopped 79.1% 78.2% 78.6% Has sufficient utensils for food preparation 30.2% 38.7% 34.9% Has a stove in good condition 80.6% 77.2% 78.7% Has good lighting 58.2% 81.8% 71.4% Has adequate ventilation 63.3% 72.3% 68.3% Has cabinets or closed spaces for storing dishes 17.8% 22.6% 20.5% 59 Indicator pcilb7c Region Total n=59 Mam n=28 Q’anjobal n=31 Has running water or a water tap nearby 73.6% 93.4% 84.6% Is free of smoke 67.6% 85.5% 77.6% Ind21b Percentage of parents who know that the school has had improvements in at least one of the following areas: 91.0% (77.9 to 100) 76.1% (58.2 to 94.0) 82.2% (70.8 to 93.7) Improvements in Latrines and/or toilets 89.2% 76.1% 81.5% Water system 48.4% 31.1% 38.2% Who helped with improvements (from those who stated that there had been improvements) MINEDUC n=24 n=23 n=47 34.0% 56.1% 46.0% Ministry of Health 3.8% 0.0% 1.7% Ministry of Agriculture 0.0% 0.0% 0.0% PCI 11.2% 10.2% 10.6% Other NGO 7.8% 8.9% 8.4% Table R7c: School Infrastructure. Community survey: Parents The confidence level of 95% is found in parenthesis. (The groups with statistical differences are shown in green.) Indicator pcilb3d Region Total n=477 Mam n=222 Q’anjobal n=255 Ind21a Percentage of parents who state that the school has a space for preparation of the school breakfast (kitchen) 99.2% (98.2 to 100) 92.9% (88.5 to 97.3) 95.6% (93.1 to 98.0) Percentage of parents who state that the kitchen complies with at least 6 of the following 10 basic conditions (among those who mentioned there was a kitchen) n=220 n=235 n=455 60.3% (45.2 to 75.3) 57.0% (40.0 to 74.1) 58.4% (46.9 to 69.9) Criteria mentioned Sufficient space for those who cook 57.1% 57.7% 57.4% Has full walls and roof 86.1% 80.1% 82.7% Has a cement floor or one that can be swept and mopped 82.0% 74.0% 77.5% Has sufficient utensils for food preparation 39.7% 36.5% 37.9% Has a stove in good condition 67.8% 61.8% 64.4% Has good lighting 66.7% 75.5% 71.7% Has adequate ventilation 41.8% 73.6% 59.7% Has cabinets or closed spaces for storing dishes 22.5% 28.0% 25.6% Has running water or a water tap nearby 80.0% 45.2% 60.4% Is free of smoke 59.7% 59.8% 59.7% Who has helped with MINEDUC 10.2% 8.7% 9.3% 60 Indicator pcilb3d Region Total n=477 Mam n=222 Q’anjobal n=255 improvements PCI 23.5% 20.8% 22.0% Other NGO 4.4% 1.2% 2.6% Percentage of parents who state that the school has made improvements in at least one of the following: 81.5% (73.0 to 90.0) 76.9% (65.6 to 88.2) 78.8% (71.6 to 86.0) Improvements in Latrines and/or toilets 79.2% 74.1% 76.2% Water system 48.2% 36.6% 41.5% Who helped with improvements MINEDUC 8.7% 7.9% 8.2% PCI 10.9% 14.1% 12.7% Other NGO 1.8% 4.1% 3.1% Principal findings for School Infrastructure Observation in the school The results in Table R7a show that the majority of the schools visited have running water (in the Mam region all did). As was expected, a low percentage (28.9%) of these schools reported that the water system was built or rehabilitated by the PCI/USDA FFE Program (remember that more than half the schools had participated in Phase I of the Program), and in a little more than half of these it was observed that they had a water tap where they prepared the foods at the school. It was also observed that the majority of the schools visited had electricity and that this basic service was functional. When compared to the basic hygiene conditions, these results show that all the sample schools had latrines or flush toilets with a slightly higher percentage of the latter (69.2% in the total population); the mean of the number of students for this type of service (latrine and/or toilet) is 35. It can be seen that a little more than half of the schools (54.0%) had clean latrines and/or flush toilets13, and around 70 percent of these have hand washing stations near the latrines and toilets; nevertheless, only in 17.4% of these places were water and soap available for hand washing. The majority of these schools (81.3%) have a storage room for storage of the foods to be used in the school feeding program, with aspects such as adequate separation of the products and foods as well as roof quality and they were clean and orderly. The major weaknesses found in these storage areas were related to lack of updated product delivery records, insufficient space for the foods that are stored there and lack of appropriate ventilation. 13 The determination of the degree of cleanliness of these latrines and/or flush toilets was obtained by means of observations carried out around 7:30 a.m. (upon arrival at the schools). 61 Almost all the schools (96.7%) have a kitchen that is used for preparation of the school snack or breakfast, and only 19.1% of these were reported to have been built or improved with help from the PCI/USDA FFE Program. On the other hand only 60.6% of the kitchens that were observed complied with at least four of the following six conditions: a) had appropriate utensils for food preparation, b) had sufficient physical space, c) had good security conditions for those working there, d) had good lighting, e) had good ventilation, and f) had good hygiene (at the time of the visit). The first, fifth and third of these conditions were the most common with values of 88.2%, 78.0% and 72.9%, respectively. A higher percentage of schools had kitchens with “sufficient physical space” in the Mam region than in the Q’anjobal region where none met this criteria (P<0.05). All the kitchens of the schools observed in the Q’anjobal region, however, had “good hygiene conditions”, with this having a higher value than in the kitchens of the Mam area (P<0.05). These results show that 60.6% of the Parent Councils of the schools visited stated that the kitchens at their schools complied with at least 6 of the 10 basic conditions (see Table R7B). The conditions that were mentioned most frequently were running water, adequate floors and the necessary ventilation to avoid accumulation of smoke in the kitchen. The least mentioned conditions were: the presence of cupboards or closed storage spaces for dishes, as well as sufficient utensils for food preparation. The majority of the members of the Parent Councils (82.2%) stated that improvements had been made in their schools. These were mainly related to latrines/toilets and not related to water systems. Of the Council members, 46.0% attributed the support for these improvements mainly to the MINEDUC. A large majority of parents interviewed at the community level knew that their school had a designated place for preparation of school snacks or breakfasts (kitchen). Similarly to the members of the Parent Councils, these parents (58.4%) stated that the kitchens complied with at least six of the basic conditions required for a good qualification. The majority of the parents interviewed at the community level stated that these kitchens had full walls and roofs, adequate floors that could be swept and mopped, and good lighting. From their responses it was evident that the kitchens lacked cupboards or closed storage spaces for dishes, as well as sufficient utensils which was also consistent with the information given by the Parent Council members. The parents interviewed knew that the schools of their community had carried out improvements (kitchens, latrines/toilets, water systems), and recognized PCI as the principal contributor to these improvements. V. CONCLUSIONS The goal of this baseline survey is to present details of the current situation for all project indicators. Given this, the most important conclusion is that the achievement of these impacts and effects will be a significant challenge for the program. Thus it would be wise to review the commitments and goals so that they can be realistic without putting to one side the challenge implied in achieving change. In particular, we want to share a conclusion with regard to the impact indicator which we consider to be most important and the most difficult to achieve – Literacy level (reading ability). The expected competency standards established by the MINEDUC are far from being achieved by the children evaluated in both the Mam and Q’anjobal groups. Based on the general results, it is clear that when they complete sixth grade, the children in these populations are not prepared to use the reading process as a tool to build deeper knowledge. The systematic errors show weak areas in the educational system that limit children in their development as literate readers. This process is most pronounced in the fourth through sixth grades although the teaching process in the earlier grades should also be strengthened. 62 VI. RECOMMENDATION Given that the end line study of the first phase was recently carried out, and the same kinds of weakness in terms of documenting processes and activities in the schools were found in this study, we once again recommend that the program promote good documentation systems in the schools where it works and so that they can be available for future evaluation activities. 63 ANNEXES Annexes contained in this document: Annex 1: Field work report Annex 2: Sample design and List of selected schools Annex 4: Text used for the evaluation of reading ability of children who had completed sixth grade Annex 5: Summary table of indicators for FFE II Program Annexes in separate files: Annex 3: PCI Guatemala USDA FFE ME Plan revised 12.31.13.docx 64 ANNEX 1 - FIELD REPORT FOR PCI BASELINE SURVEY IN COMMUNITIES OF HUEHUETENANGO (Study for PCI---March /April 2014) I. INTRODUCTION The data collection stage of an evaluation study is extremely important in order to be able to determine the impacts and effects of a development intervention in an objective manner. For this reason CIENSA gives special emphasis to the quality and scope of the activities of this stage. This report provides a summary description of the main activities and/or tasks that were carried out as part of the Baseline Survey for the extension of the PCI Food for Education program the Mam and Q´anjobal regions of the department of Huehuetenango. II. COMPOSITION OF THE FIELD TEAMS Two field teams (Mam and Q´anjobal) were staffed for this study, based on experience obtained during the end line survey carried out for PCI in these same geographic areas during February and March of this year. The training and standardization of staff with regard to the instruments and field work activities was carried out in the city of Huehuetenango, from April 9 to 12. Both teams were organized in the same manner14 as for the previous survey, that is, with emphasis on specific roles for each team member in order to maximize work efficiency and to guarantee the quality of data collection activities in each of the sample schools and communities. Details are presented in Table 1 of this report. The names and relevant characteristics of the members of each of these field teams are presented in Annex 1 of this report. III. METHODOLOGY A. Work in the Schools The arrival at each of the schools (see Annex 2) considered in the sample was programmed in advance through contact with the respective point persons15. The supervisor of each field team was responsible for coordinating the visits with the director of each school16. The field work was programmed for one day in each community even though there were various field activities to be carried out in each, especially in terms of the reading tests for children who had completed sixth grade the previous year. The activities carried out in the schools took place in the mornings and the community surveys were conducted in the afternoon. In each of the schools data collection activities began early (7:30 a.m.). The interviews with the fourth, fifth and sixth grade students were carried out first. In the morning the team also localized students who had completed sixth grade the year before (2013), in order to set up testing of reading quality and comprehension with them. 14 Each of these teams was composed of the following members: a supervisor, an editor and four interviewers 15 These contacts were carried out by the supervisors of each of the groups using cell phones. In some cases the visits had to be reprogrammed or adjusted based on the facilities and or difficulties expressed by the directors of the schools included in the sample. 16 These names were provided by PCI staff. 65 In most cases, the interviews with the director, the teachers, and members of the Parent Council were carried out mid-morning. Some of them had to be located in the community in the afternoons (when they hadn’t been present at school in the mornings). It is important to note that in the majority of cases these interviews were carried out in the local language (Mam o Q´anjobal), given that most of these Councils are composed of people from the community who only speak the local language. In Table 1 of this section the work plan used in the schools for data collection is described and Annex 3 has a complete list of the data collection instruments (for the school and community). Table 1 Work plan in schools Activities Responsible on team Target population Questionnaires used 1. Interviews with boys and girls (Carried out in the first 2 or 3 hours of the morning) Editor and interviewer Students from 4th, 5th and 6th grades 4 2. Review of attendance records and PRIM tables (Carried out throughout the morning) Supervisor and interviewer Directors of each school 9 and 10 3. Observation of hygiene/ school infrastructure/ preparation of school foods (Carried out in the first 3 hours of the morning) Interviewers School Director/teacher Member of Parent Council + 8 4. Interviews with directors and teachers (Carried out mid morning) Editor and interviewers Director and teachers + 5 5. Classroom observations (To detect inattentiveness in students) Interviewers Boys and girls who receive a standard class 6 6. Interviews with the Parent Council (Carried out from early until mid morning) Interviewers Members of the Parent Council + 7 + A key condition for the interviews with directors and teachers was that they would have been in their positions during 2013. This condition also applied to the members of the Parent Council. B. Work in communities Once the activities at the school were concluded, the entire field team in each region embarked on data collection in the communities (community survey). Based on their observations and/or a walk through of the sample communities, and with support from the authorities or local leaders17, each of the communities was divided into several parts or sectors and then two or three sectors were randomly selected for the survey. The data collection in each of the selected sections was carried out through home visits to all of the households in each of the selected sectors. As a second step, the interviewers knocked on each door and depending on the presence or absence of the target populations, the corresponding survey instrument was used via face to face interviews. At each house, the complete interview process (when questionnaires 0, 1, 2 and 3 were used) took between 25 and 35 minutes. 17 In most cases, these authorities or local leaders were members of the COCODES and/or members of the Parent Council. The latter were contacted at the time they were interviewed in the schools. 66 The names of these questionnaires, the means of application and the mechanics of the community survey are summarized in Table 2. A person trained and standardized in each of the groups (Mam and Q´anjobal) was responsible for conducting the reading quality and comprehension test with a maximum of five students per community who had completed sixth grade in 2013. This test was given individually for each of these students who were located and called to an appropriate location in each community18. The first part of this test was carried out at the local level and the second (which required greater depth and detail of analysis), was carried out in the CIENSA offices by means of a specific analysis based on the recordings that were made in the community during the field work. All the activities for application of this test were supervised by a psychology and education professional who is part of the CIENSA group. Table 2 Work plan in communities Activities Responsible on team Target population Questionnaires used 1. Community survey (Normally carried out in the afternoon in each community) Supervisor, editor, and interviewers Households with adolescents and school age children who had been in school in 2013 0, 1, 2 and 3 2. Test of reading quality and comprehension 1. Interviewer (Group A) 2. Editor (Group B) 3. Psychology and education professional Students who completed sixth grade at the community school in 2013. 11 Names and characteristics of the questionnaires for field work 1. Questionnaire 0 Filter for characterization and coverage: for characterization of the population and coverage of the school in each of the communities 2. Questionnaire 1 Questionnaire for adolescents: for attendance records and promotion of adolescents - 13 to 15 years old 3. Questionnaire 2 Questionnaire for school age children: for attendance records and promotion of school age children - 5 to 12 years old 4. Questionnaire 3 Questionnaire for community survey: for parents with regard to their children’s school participation (adolescents and school age) during 2013 and 2014 5. Questionnaire 11 Instrument for reading comprehension: For boys and girls (youth) who completed sixth grade at the target school, in each of the sample communities. Given that the visits to the schools could only be done during the week (Monday to Friday) and that the field work began on a Tuesday19, the field teams took advantage of the two weekends to carry out community surveys, which meant that some were carried out in the mornings on Saturdays based on prior coordination with the respective local contacts. Quality control mechanisms In order to guarantee the quantity and quality of data collected in the schools and communities, several quality control mechanisms were utilized during this study, principally supervision and editing. The first of these was 18 The reading quality and comprehension tests were carried out in locales such as schools, community halls and parents’ homes. 19 Due to difficulties in contacting schools the first day, the field work began on Tuesday, April 22 and ended on May 7. Each of the field teams stayed in their respective region during the entire data collection period. 67 carried out by the supervisor for each of the groups as well as by CIENSA’s Psychology Professional. The editing and in-depth review of the data was carried out in the schools and communities themselves by the editor of each group. This mechanism was extremely useful for timely detection and opportune correction of errors and inconsistencies which might have otherwise been missed, negatively impacting data collection. By means of the quality control we also increased the degree of standardization acquired by the interviewers during the training stage, which also was beneficial for the data collection process. The editors were responsible for overseeing the questionnaires that were applied in each school and community, as well as for filling out a summary sheet which detailed the products of each day’s work. IV. PRODUCTS OBTAINED Field team A (Mam Area) ---- Baseline ---- No. Community (Municipality) Data collection instruments applied (ID) RT Community School 0 1 2 3 T 4 5 6 7 8 9 10 T 1 El Palmar (San Pedo Necta) 3 26 2 12 13 53 11 3 1 3 1 1 1 21 2 Nimá (San Pedro Necta) 5 18 7 12 13 50 15 4 1 0 1 1 1 23 3 Ojo of Agua of las P (San Pedro Necta) 6 26 7 15 19 67 13 3 1 2 1 1 1 22 4 Huixoc (San Pedro Necta) 5 26 3 15 16 60 15 4 1 3 1 1 1 26 5 Los Alisos (San Pedro Necta) 5 25 3 19 20 67 15 4 1 3 1 1 0 25 6 Hoja Blanca (San Pedro Necta) 4 25 2 15 15 57 15 3 1 1 1 1 0 22 7 Chacpantzé (San Gaspar Ixchil) 5 29 11 20 20 80 15 4 1 3 1 1 0 25 8 Monte Rico (Cuilco) 5 22 5 14 14 55 14 3 1 2 1 1 0 22 9 Nueva Esperanza (Cuilco) 1 26 5 16 17 64 15 1 1 3 1 1 0 22 10 Yerba Buena (Cuilco) 5 28 5 19 20 72 15 4 1 3 1 1 0 25 11 Buena Vista (Cuilco) 5 26 8 21 21 76 15 4 1 1 1 1 0 23 12 Cholivá (Cuilco) 5 26 2 15 15 58 3 3 13 Coxtón (Cuilco) 4 25 5 19 19 68 1 1 2 RT = Reading Test; T = Total 68 Field team B: (Q´anjobal Area) ---- Baseline --- No. Community (Municipality) RT Data collection instruments applied (ID) Community School 0 1 2 3 T 4 5 6 7 8 9 10 T 1 Seclecán (Sta. Cruz Barillas) 3 25 6 18 19 68 14 3 1 3 1 1 1 24 2 El Quetzal (Sta, Cruz Barillas) 5 28 9 19 23 79 15 4 1 3 1 1 1 26 3 Jolomtaj (Sta Cruz Barillas) 4 26 7 19 21 73 10 3 1 3 1 1 1 20 4 Monte Bello M: (Sta Cruz Barillas) 3 20 8 17 19 64 14 1 1 3 1 1 1 22 5 San Juan La Ceiba (Sta Cruz Barillas) 3 23 8 18 19 68 15 1 1 3 1 1 1 23 6 La Campana I (Sta Cruz barillas) 5 22 8 16 18 64 15 2 1 3 1 1 1 24 7 Tziquiná Grande (Sta. Eulalia) 3 20 9 17 19 65 15 1 1 3 1 1 1 23 8 Santa Eulalia (Sta. Eulalia) 5 23 8 17 19 67 15 4 1 3 1 1 1 26 9 Pueblo Nuevo (San P. Soloma) 4 26 13 19 20 78 15 3 1 3 1 1 1 25 10 Sjolom Ku (San P. Soloma) 4 24 9 18 20 71 15 3 1 1 1 1 1 23 11 Yulchen (San P. Soloma) 2 27 5 18 19 69 15 3 1 0 1 1 1 22 12 Gelná (San P. Soloma) 2 25 11 18 20 74 15 2 1 0 1 1 1 21 13 Yac Xap (San P. Soloma) 5 23 9 17 19 68 15 3 1 0 1 1 1 22 RT = Literacy Test, T = Total V. LESSONS LEARNED 1. In order to carry out an evaluation that meets all the requirements of a field study, it is essential to set up timely contacts with all relevant people involved at the local level. In the past, this was extremely complicated due to the distances and geographic characteristics of the communities. Today this complexity has been minimized due to the availability of cell phones which are present almost everywhere. 2. Appropriate use of instruments in local languages (Mam and Q´anjibal) greatly facilitated the application of the majority of the data collection tools for this baseline study (community survey, questionnaire for the Parent Council and questionnaires for the students), and also guaranteed understanding of the questions that were asked of the different target groups in the study. 3. The experience gleaned from the field work for the End Line study carried out earlier this year in these regions of Huehuetenango, facilitated the organization and development of the Baseline activities. Additionally, the majority of the staff charged with data collection had participated in the End Line study which contributed to the smooth and successful completion of this study. 4. Although a field calendar was designed, several of the visits had to be reprogrammed due to competing activities that several of the schools or their staff were involved in. In spite of this the data collection process was carried out within the established time frame without sacrificing the expected quality of the work. It is important to acknowledge the contributions of PCI’s technical staff especially in facilitating contacts and additional communication that was required along the way. 69 ANNEX 2 - SAMPLING DESIGN FOR PCI BASELINE STUDY 2014 List of schools selected 20 March 2014 Sampling Design The second phase of the Program is being carried out with a total of 279 schools distributed in six municipalities of the department of Huehuetenango. The objective of the quantitative study was to determine the current status of the effect and impact indicators established for the Program. Given that 174 of the 279 Phase II schools participated in Phase I, the sampling design is stratified. The strata are defined as a combination between whether or not the school participated in Phase 1 and the region where the school is located (Mam/Q’anjob’al). Thus there are four strata: Number of schools and students per stratum: Strata Municipality Number of new schools for 2014* Total number of students in the stratum* Number of schools in the # sample Presence in Phase I Region I YES Mam Cuilco 36 12,085 6 San Gaspar Ixchil 12 San Pedro Necta 32 Total 80 II YES Q’anjob’al San Pedro Soloma 15 17,187 7 Santa Cruz Barillas 36 Santa Eulalia 43 Total 94 III NO Mam Cuilco 53 San Pedro Necta 6 4,215 7 Total 59 IV NO Q’anjob’al San Pedro Soloma 18 6,158 6 Santa Cruz Barillas 26 Santa Eulalia 2 Total 46 * Information provided by PCI: FFE II Schools with MINEDUC code 2014-03-04.xlsx The number of schools to be sampled responded to the size of the sample proposed in the proposal document for this study. As can be seen in the above table, the total number of schools in the sample is 26, 13 per region and 13 which did/or did not participate in Phase I. The selection of schools was carried out by strata taking into account the probability of selection based on the size of the schools, according to the total number of students (PPS Sampling). 70 Selected Schools Strat a Municipali ty School Name Primary School Code Pre-school Code Number of students I San Gaspar Ixchil 1 EORM Caserío Chacpantzé 13-29-5402-43 13-29-4847-41 189 San Pedro Necta 2 EORM Los Alisos 13-06-0362-43 13-06-0338-41 231 3 EORM Huixoc 13-06-0360-43 13-06-0333-41 118 4 EORM Aldea Nimá 13-06-0363-43 13-06-0339-41 230 Cuilco 5 EORM Aldea Horno of Cal 13-04-0253-43 13-04-5708-42 171 6 EORM Yerba Buena Alta 13-04-0235-43 13-04-5710-42 181 Replacement Schools San Gaspar Ixchil 2 7 EORM Aldea Coxtón 13-29-0003-43 13-29-0006-41 59 Cuilco 2 8 EORM El Chilcal 13-04-0228-43 13-04-5711-42 314 II Santa Eulalia 7 EORM Tziquina Grande 13-17-4731-43 13-17-5012-41 127 8 EOUM JV Santa Eulalia 13-17-5728-43 13-17-6599-41 552 San Pedro Soloma 9 EORM Aldea Pueblo Nuevo 13-08-0012-43 13-08-0003-42 297 Santa Cruz Barillas 1 0 EORM Aldea El Quetzal 13-26-1199-43 13-26-1129-41 324 1 1 EORM Aldea Jolomtaj, Nueva America 13-26-1217-43 13-26-1146-41 137 1 2 EORM Caserío Saclecan, El Quetzal 13-26-2058-43 13-26-4948-41 63 1 3 EORM Monte Bello Momonlac 13-26-0033-43 13-26-0153-41 81 Replacement Schools Santa Cruz Barillas 2 9 EOUM Oscar Gonzalez Recinos JM 13-26-1181-43 13-26-1117-41 1179 3 0 EORM Yulconop 13-26-1190-43 13-26-6295-41 247 III San Pedro Necta 1 4 EORM, Cantón Ojo of Agua, Aldea Agua of las Palomas 13-06-0014-43 73 1 5 EORM Caserío San Juan, aldea El Palmar JM 13-06-0374-43 13-06-0344-41 41 Cuilco 1 6 EORM, Caserío Campamento Parte Baja, Hoja Blanca 13-04-5933-43 13-04-6989-42 80 1 7 EORM Caserío Buena Vista, Aldea Yerba Buena 13-04-0055-43 130 1 8 EORM Nueva Esperanza Posonicapa Chiquito. 13-04-0031-43 58 1 9 EORM Choliva 13-04-0017-43 13-04-0006-42 74 2 0 EORM Caserío Monte Rico, La Frontera, Aldea La Laguna 13-04-0023-43 13-04-0008-42 57 Replacement Schools Cuilco 3 1 EORM Caserío San Antonio, Aldea Paviltzaj 13-04-0034-43 66 3 2 EORM Caserío El Herrador, Aldea Joví 13-04-0223-43 13-04-4690-42 157 IV San Pedro Soloma 2 1 EORM Aldea Sjolom Ku JM 13-08-0028-43 13-08-0024-41 144 2 2 EORM Aldea Yulchén JM 13-08-1462-43 13-08-6336-41 211 2 3 EORM Aldea Gelná 13-08-0499-43 13-08-0435-41 71 71 Strat a Municipali ty School Name Primary School Code Pre-school Code Number of students 2 4 EORM Aldea Yacxap JM 13-08-5715-43 13-08-0446-41 575 Santa Cruz Barillas 2 5 EORM Aldea San Juan La Ceiba JM 13-26-0031-43 75 2 6 EORM La Campana Uno 13-26-5899-43 85 Replacement Schools San Pedro Soloma 3 3 EORM Aldea Chuchjil 13-08-4760-43 13-08-4791-41 122 Santa Cruz Barillas 3 4 EORM Caserío Esperancita Yulmacap 13-26-1666-43 103 72 ANNEX 4 READING USED FOR THE EVALUATION The text was given to the children in a two page format with ARIAL 12 font. UNA POMADA PARA LA NARIZ Jan Terlow: El rey de Katoren Los habitantes de la ciudad de Pituita sufren desde hace años una enfermedad. Sus narices, cuando les pica algún mosquito, se hinchan y se dilatan hasta convertirse en una especie de horrible protuberancia. Las únicas personas capaces de curar esta enfermedad son los Taras, que, según la tradición, parecen ser los sabios más eminentes de esa ciudad. y para curar este mal han elaborado una pomada de NP, de Nariz Prominente, que así es como llamaban a esta enfermedad. Su fórmula secreta ha ido transmitiéndose de padres a hijos y hoy en día es todavía un misterio. Sin embargo, existe un Problema: tanto la medicina que recetan como las tarifas de sus consultas son demasiado caras. La esposa del Presidente Municipal de Pituita sufre desde hace unos días NP. Aunque confía en los Taras, llamó en secreto a un médico de la ciudad vecina para que viniera a conocer su Problema. – Mire Doctor Stach – Explica la señora –. El origen de nuestra enfermedad está en unos mosquitos que se desarrollan en nuestras tierras y parece que tienen predilección por picar en la nariz de las personas. y aunque la NP se puede curar, el remedio resulta tan costoso… Cada vez que alguno la padecemos nos arruinamos… Como mi marido es un músico de talento, llevamos ahorrando quince años para comprar un piano, y cuando tenemos el dinero necesario, uno de los dos siente los primeros síntomas de una NP… y el dinero desaparece en pomada… y como escuché muchos elogios de usted… – ¿Cuál es la pomada? – Replicó el Doctor Stach. Mire, aquí la tengo. La acabamos de comprar en la mansión Tara. La mujer abrió el frasco cuidadosamente y se frotó el apéndice nasal con la famosa pomada que despedía un fuerte olor a vinagre. El Doctor Stach quedó muy interesado por el tema de la misteriosa enfermedad NP y su carísima pomada curativa, de manera que decidió estudiar de cerca otros casos parecidos. Tras conocer a otros vecinos, y sospechar que algo no encajaba en todo aquel asunto, decidió asistir disfrazado a la asamblea de los Taras. Consiguió una toga y, como un Tara más, se presentó a ella, convenciendo al resto de los asistentes, que por supuesto no lo conocían, de que era un Tara de una aldea vecina. La asamblea fue sorprendente para el Doctor. Nadie hablaba de ciencia ni de medicina. La ocupación obsesiva de todos los Taras era ganar dinero. El doctor persuadió a un Tara para que preparara un poco de pomada y así enterarse de su composición, era como aderezar una ensalada: un poquito de aceite, otro de mostaza, un chorrito de vinagre y una pizca de sal. Stach se dirigió a casa del Presidente Municipal pero éste no podía creer en lo que oía, no podía pensar que los honorables y ancestrales Taras pudieran estar engañando al pueblo con un remedio tan sencillo y económico y que además se enriquecieran a costa de la confianza de los habitantes de Pituita. Sin embargo, la demostración del Doctor Stach fue contundente. Delante de todo el pueblo se dejó picar por un mosquito, luego preparó la pomada y se la aplicó curándose al poco tiempo. Pituita conoció varios días de agitación. Una comisión de vecinos al mando del Presidente Municipal tomó la siguiente resolución: Los Taras deberán trabajar para la comunidad de Pituita haciendo diferentes tareas… de este modo devolverán a sus habitantes lo pagado por la pomada. 73 ANNEX 5 SUMMARY TABLE OF INDICATORS FOR FFE II PROGRAM The following table is presented to facilitate follow up on the indicators established for this Program Result Indicator BL Value Target PCI Description according to Baseline Page BL/ Executive Summary Contribution to Feed The Future Number of public/private partnerships formed as a result of USDA assistance 0 4 Indicator not measured in BL Number of beneficiaries participating in productive safety nets as a result of USDA assistance 0 47,579 Indicator not measured in BL Number of people trained in child health and nutrition, as a result of USDA assistance 0 24,150 Indicator not measured in BL Improved literacy of school age children Percentage of girls who, upon completing 6th grade, demonstrate reading comprehension equivalent to that defined by the national standards 58.0 80 Mean of the Percentage of reading ability -2 in students (girls) Weighting: speed %, accuracy 20%, comprehension 70% 44 / ix Percentage of boys who, upon completing 6th grade, demonstrate reading comprehension equivalent to that defined by the national standards 54.3 80 Mean of the percentage of reading ability -2 in students (boys) Weighting: speed 10%, accuracy 20%, comprehension 70% 44 / ix Total number of individuals benefitting directly from USDA-funded interventions 0 47,579 Indicator not measured in BL Total number of individuals benefitting indirectly from USDA-funded interventions 0 198,070 Indicator not measured in BL Increase in governmental support Value of the public resources assigned to primary education (include list of teachers, infrastructure, materials and supplies, etc.) 0 50,000 Indicator not measured in BL Increased in participation of local organizations and community groups Number of Parent Councils contributing to the schools a result of USDA assistance 0 230 Indicator not measured in BL 74 Result Indicator BL Value Target PCI Description according to Baseline Page BL/ Executive Summary Improved attentiveness in class Percentage of children who are inattentive in class 60.4 15 Mean of the percentage of students with inattentiveness (presented at least one of the five categories) (Classroom observations) 44 / ix Reduced short term hunger Percentage of students who indicate that they are hungry or very hungry 9.3 15 Percentage of students (boys and girls) who are hungry or very hungry at the time of the interview (sample of 4th to 6th grade students) 49 / x Increased access to food Percentage of students eating daily meal at school 0 100 Indicator not measured in BL Improved school attendance Percentage of boys with at least 80% attendance 85.1 80 Percentage of students (boys) who state that they attended class every day of the past week (sample of students in 4th to 6th grades) 26 / vii Percentage of girls with at least 80% attendance 84.9 80 Percentage of students (girls) who state that they attended class every day of the past week (sample of students in 4th to 6th grades) 26 / vii Reduced health related absences Percent of students who state that their absence is usually health related 19.5 10 Percentage of students (boys, girls) who state that in the last four weeks they have missed class for health reasons (sample of students in 4th to 6th grades) 29 / viii Increased use of health and dietary practices Percent of schools that clean their latrines (at least once a day) 54.0 50 Percentage of schools that have clean latrines and/or flush toilets 66-67 / xi Percent of schools with soap and water in hand washing stations 17.4 50 Percentage of schools that have hand washing stations near the latrines and/or flush toilets, with soap and water present 66-67 / xi Percent of schools with clean equipment for cooking and feeding students (consistent with accepted standards) 60.7 80 Schools where good order and cleanliness were observed in relation to cooking utensils (comply with at least three of the following: 1. are dry, 2. are clean, 3. are covered and 4. stored in an appropriate place) 53-54 Improved knowledge of health and hygiene practices Percent of parents who can identify at least three important health and hygiene practices 58.7 75 Percentage of parent who know the principal health and hygiene habits or practices that everyone should have. Mention at least three of the seven (community sample) 35 / viii 75 Result Indicator BL Value Target PCI Description according to Baseline Page BL/ Executive Summary Percentage of students/parents who can identify at least one local source of information on good health practices 66.5 70 Percentage of parents who state that they know where they can learn more or obtain information about personal health and hygiene habits or practices (mention at least one) 37 57.1% Percentage of students who state that they know where they can learn more or obtain information about personal health and hygiene habits or practices (mention at least one) 35 Increased knowledge of food safety practices Percentage of people with knowledge of at least three food safety practices 13.2 80 Percentage of parents who mention at least three recommendations or important aspects of diet and nutrition for large groups of children 37 Increased knowledge of nutrition Percentage of students/parents who can identify at least one local source of information on nutrition practices 58.1 70 Percentage of parents who state that they know where they can learn or obtain information about feeding, diet and nutrition practices or habits (mention at least one) 37 Percentage of parents who identify at least three important nutrition practices 13.2 80 Percentage of parents who mention at at least three recommendations or important aspects about diet and nutrition for large groups of children 37 Increased access to utensils and equipment for food preparation and storage Percentage of schools with improved food prep and storage equipment 0 29 Indicator not measured in BL Improved school infrastructure Number of students benefitting from the (rehabilitated or constructed) kitchens as a result of USDA assistance 0 6,390 Indicator not measured in BL Number of students benefitting from the (rehabilitated or constructed) water stations or systems as a result of USDA assistance 0 8,520 Indicator not measured in BL Increased student enrollment Percentage increase in boys enrolled in school as a result of USDA assistance 0 1 Indicator not measured in BL Percentage increase in girls enrolled in school as a 0 1 Indicator not measured in BL 76 Result Indicator BL Value Target PCI Description according to Baseline Page BL/ Executive Summary result of USDA assistance Increased community understanding of benefits of education Number of parents in the target communities who are members of the Parent Councils (or a similar group in the school community) 0 1,150 Indicator not measured in BL Percent of parents who can name at least three benefits of sending their children to school 55.8 60 Fathers/mothers who value the education their children receive in school: mention at least three benefits or reasons (community sample) 22-23 / vii