This publication was produced at the request of the United States Department of Agriculture. It 
was prepared independently by the external evaluator ADOC 
Foreign Agricultural 
Service, United States 
Department of Agriculture
McGovern-Dole Food for 
Education Project, “Learning 
for Life” in Guatemala
Midterm Evaluation
January 2020
DISCLAIMER: The author’s views expressed in this publication do not necessarily reflect the 
views of the United States Department of Agriculture or the United States Government.
Midterm Evaluation of the McGovern-Dole Food for 
Education Project “Learning for Life” in Guatemala
Learning for Life is a McGovern-Dole Food for Education project financed by the United States 
Department of Agriculture (USDA) under agreement FFE-520-2016/010-00, implemented in the 
department of Totonicapán in the western highlands of Guatemala by Catholic Relief Services – United 
States Conference of Catholic Bishops (CRS) and its local partners Proyecto de Desarrollo Santiago 
(PRODESSA) and Pastoral Social Caritas Arquidiócesis de Los Altos (PSC). 
The project promotes a school feeding program with commodities donated by USDA to 
complement those purchased with funds from the Ministry of Education of Guatemala. Also, the project 
promotes the implementation of two literacy methodologies: Jardín de Letras (Kotz'i'j tz’ib' in K'iche' 
language) to teach reading and writing in both K'iche' and Spanish languages, and Kemom Ch'ab'al, 
applied throughout primary school (first through sixth grades) to strengthen reading comprehension 
skills in both languages. 
The purpose of this midterm evaluation is to assess the progress towards the project goals and 
to establish a midterm point of comparison between the baseline evaluation (FY 2018) and the final 
evaluation (FY 2021), covering the first three years of the implementation of phase two of the project. 
This report answers 12 research questions related to the relevance, efficiency, effectiveness, impact and 
sustainability of the project, and measures 11 outcome indicators, comparing them to the baseline 
measurements and midterm goals.
Agreement Number: FFE-520-2016/010-00
Project Duration: 2016-2021 
Implemented by: CRS
Evaluation Authored by: Friné Paz (ADOC)
3
s
Midterm Evaluation of the 
McGovern-Dole Food for Education 
Project “Learning for Life” in 
Guatemala
January 2020
1
Contact information
CRS HEADQUARTERS 
Courtney Plummer 
courtney.plummer@crs.org 
Project Support Officer 
USDA Donor Engagement 
CRS GUATEMALA 
Nicole Kast 
nicole.kast@crs.org 
Head of Programming 
CRS GUATEMALA (FIELD OFFICE) 
Emily Drummer 
emily.drummer@crs.org 
Chief of Party MGD - FFE 
ADOC GUATEMALA 
Friné Paz
adocguatemala@gmail.com
Main Researcher
This publication was made possible by financial support of the United States Department of 
Agriculture, under cooperative agreement FFE-520-2016/010-00. Any opinions, findings, 
conclusions, or recommendations expressed in this publication are those of the author(s) and do 
not necessarily reflect the views of the United States Department of Agriculture or the United 
States Government.
 Guatemala, January 2020
2
Acronyms
ADOC Organizational and Community Development Association (external evaluator)
Asociación de Desarrollo Organizacional y Comunitario
AIR American Institutes for Research
CRS Catholic Relief Services – United States Conference of Catholic Bishops
DC District Coordinator of the Departmental Ministry of Education
DDS Dietary Diversity Score
DIGEDUCA General Directorate for Educational Evaluation and Research
Dirección General de Evaluación e Investigación Educativa
EGRA Early Grade Reading Assessment 
ELGI Early Grade Reading Test
Evaluación de Lectura para Grados Iniciales
INTECAP Technical Institute for Training and Productivity
Instituto Técnico en Capacitación y Productividad
MDES Minimum detectable effect size
MEAL Monitoring, Evaluation, Accountability and Learning
MGD McGovern-Dole Food for Education project
MINEDUC Ministry of Education of Guatemala
Ministerio de Educación de Guatemala
PA Pedagogical Assistant of the Departmental Ministry of Education
PRODESSA Proyecto de Desarrollo Santiago (implementing partner)
PMP Performance monitoring plan
PSC Pastoral Social Cáritas Arquidiócesis de Los Altos (implementing partner)
SINAE National Educational Accompaniment System
Sistema Nacional de Acompañamiento Educativo
TAC Technical-Administrative Coordinator of the Departmental Ministry of 
Education
USDA United States Department of Agriculture
3
Contents
1 Executive summary................................................................................................................. 7
1.1 Project background........................................................................................................... 7
1.2 Midterm evaluation purpose and methodology................................................................ 7
1.3 Midterm evaluation conclusions ...................................................................................... 8
1.4 Midterm evaluation recommendations............................................................................. 9
2 Project background ............................................................................................................... 10
3 Geographic area .................................................................................................................... 10
4 Target population.................................................................................................................. 11
5 Midterm evaluation purpose ................................................................................................. 11
6 Midterm evaluation methodology......................................................................................... 12
6.1 External evaluation team................................................................................................ 13
6.2 Evaluation ethics............................................................................................................ 13
6.3 Quantitative sample........................................................................................................ 14
6.4 Qualitative sample.......................................................................................................... 16
6.5 Limitations of the midterm evaluation........................................................................... 16
7 Midterm evaluation questions and answers.......................................................................... 17
8 Midterm evaluation findings................................................................................................. 26
8.1 Literacy indicators.......................................................................................................... 26
8.1.1 (Indicator 4) Percent of students who, by the end of one grade of primary 
schooling, demonstrate that they can read and understand the meaning of grade level text .30
8.1.2 (Indicator 5) Percent of students who, by the end of two grades of primary 
schooling, demonstrate that they can read and understand the meaning of grade level text .32
8.1.3 (Indicator 6) Percent of students who, by the end of three and six grades of primary 
schooling, demonstrate that they can read and understand the meaning of grade level text .34
8.2 School personnel techniques indicators ......................................................................... 36
8.2.1 (Indicator 10) Number of teachers/educators/teaching assistants in target schools 
who demonstrate use of new and quality teaching techniques or tools as a result of USDA 
assistance................................................................................................................................36
8.2.2 (Indicator 12) Number of school administrators and officials in target schools who 
demonstrate use of new techniques or tools as a result of USDA assistance ........................39
8.2.3 (Indicator 24) Percentage of DCs, TACs and PAs from the Totonicapán 
Department of Education that have increased their capacity to fulfill their roles and 
responsibilities.......................................................................................................................40
4
8.3 Student attention span, hunger scale and attendance rate .............................................. 41
8.3.1 (Indicator 14) Percentage of students in the classrooms defined as "very attentive" 
using a scale that defines established criteria ........................................................................41
8.3.2 (Indicator 15) Percentage of students in participating primary schools who indicate 
not being hungry during the school day.................................................................................43
8.3.3 (Indicator 21) Average student attendance rate in USDA supported 
classrooms/schools.................................................................................................................45
8.4 Health and nutrition indicators....................................................................................... 45
8.4.1 (Indicator 7) Number of individuals who demonstrate use of new child health and 
nutrition practices as a result of USDA assistance ................................................................45
8.4.2 (Indicator 8) Number of individuals who demonstrate use of new safe food 
preparation and storage practices as a result of USDA assistance.........................................48
9 Midterm evaluation conclusions........................................................................................... 51
9.1 Conclusions on the literacy indicators ........................................................................... 51
9.2 Conclusions on the school personnel techniques indicators .......................................... 52
9.3 Conclusions on the student attention span, hunger scale and attendance rate ............... 52
9.4 Conclusions on the health and nutrition indicators........................................................ 53
10 Midterm evaluation recommendations ................................................................................. 53
10.1 Recommendations on the literacy indicators.............................................................. 53
10.2 Recommendations on the school personnel techniques indicators............................. 54
10.3 Recommendations on the student attention span, hunger scale and attendance rate.. 54
10.4 Recommendations on the health and nutrition indicators .......................................... 54
10.5 Recommendations on sustainability ........................................................................... 55
11 References............................................................................................................................. 56
12 Annexes................................................................................................................................. 57
12.1 Annex 1: Learning for Life project’s results framework............................................ 58
12.2 Annex 2: Learning for Life project’s literacy methodologies.................................... 60
12.3 Annex 3: Reading skills of children assessed with ELGI .......................................... 61
12.4 Annex 4: Learning for Life project’s evaluation indicators....................................... 64
12.5 Annex 5: Quantitative sampling plan ......................................................................... 66
12.6 Annex 6: Midterm evaluation consultants.................................................................. 68
5
Tables
Table 1 Number of participants in the Learning for Life project in the 2019 fiscal year............ 11
Table 2 Number of sampled schools by intervention group ......................................................... 15
Table 3 Number of sampled schools by municipality .................................................................. 15
Table 4 Percentage of schools by type.......................................................................................... 16
Table 5 Historical values of indicator 4: Percentage of first grade students who demonstrate 
reading skills at grade level........................................................................................................... 30
Table 6 Percentage of students in the cohort 1st grade 2018 / 2nd grade 2019 who demonstrate 
reading skills at grade level........................................................................................................... 32
Table 7 Historical values of indicator 6: Percentage of third and sixth grade students who 
demonstrate reading skills at grade level...................................................................................... 34
Table 8 Historical percentage values of indicator 6: Percentage of third and sixth grade students 
who demonstrate reading skills at grade level, by grade and sex ................................................. 34
Table 9 Historical values of indicator 10: percentage and number of teachers who demonstrate 
use of new techniques or tools...................................................................................................... 36
Table 10 Number of sampled classrooms, by grade and literacy intervention............................. 38
Table 11 Historical values of indicator 12: Percentage and number of school administrators who 
demonstrate use of new techniques or tools ................................................................................. 39
Table 12 Historical values of indicator 24: Percentage of DCs and TACs from the Totonicapán 
Department of Education that have increased their capacity to fulfill their roles and 
responsibilities.............................................................................................................................. 40
Table 13 Historical values of indicator 14: Percentage of students defined as “very attentive” .. 41
Table 14 Percentage of students defined as “very attentive”, by informant................................. 42
Table 15 Percentage of students indicating a certain attention level in class, by grade and by sex
....................................................................................................................................................... 42
Table 16 Percentage of students indicating different feelings at school, by grade and sex.......... 42
Table 17 Definition of the hunger scale levels ............................................................................. 44
Table 18 Historical values of indicator 15: Percentage of students who indicate not being hungry 
during the school day.................................................................................................................... 44
Table 19 Percentage of students in each level of the hunger scale, by grade ............................... 44
Table 20 Historical values of indicator 7: Percentage and number of individuals who demonstrate 
use of new child health and nutrition practices............................................................................. 46
Table 21 Percentage of schools with the following health and nutrition practices observed ....... 46
Table 22 Percentage of school principals who report health care practices in their schools during 
the last year ................................................................................................................................... 47
6
Table 23 Percentage of schools including each food group in the school meal served the day of 
the evaluation................................................................................................................................ 47
Table 24 Historical values of indicator 8: Percentage and number of individuals who demonstrate 
use of new safe preparation and storage practices........................................................................ 48
Table 25 Percentage of schools with the following safe food preparation and storage practices 
observed ........................................................................................................................................ 49
Table 26 Constructs of the ELGI test ........................................................................................... 61
Table 27 Comparison of early grade reading skills for first and second grade students assessed 
with ELGI ..................................................................................................................................... 62
Table 28 Evaluation indicators measurements ............................................................................. 64
Graphs
Graph 1 Percentage of first grade students in each of the five stages of learning to read ........... 31
Graph 2 Percentage of second grade students in each of the five stages of learning to read ...... 33
Graph 3 Comparison between first and second grade 2019 literacy results................................. 33
Graph 4 Percentage of third grade students in each of the four reading performance levels ....... 35
Graph 5 Percentage of sixth grade students in each of the four reading performance levels....... 35
Graph 6 Percentage of teachers in each level of implementation of Kemom Ch’ab’al and Jardín 
de Letras methodologies............................................................................................................... 37
Graph 7 Percentage of teachers using new or improved techniques (levels 3 or 4 of 
implementation) by grade and literacy intervention ..................................................................... 37
Graph 8 Comparative of indicator 10: percentage of women and men using new or improved 
teaching techniques (levels 3 or 4 of implementation)................................................................. 38
Graph 9 Variables used to measure use of new techniques or tools for school administration.... 40
Graph 10 Percentage of girls and boys indicating different feelings at school ............................ 43
Graph 11 Percentage of schools using each type of water purification........................................ 50
Figures
Figure 1 Learning for Life project’s geographic location............................................................. 10
Figure 2 Stages in the model to explain and predict how students learn to read.......................... 28
Figure 3 Student’s performance levels, according to the standardized reading tests results........ 29
Figure 4 Student’s achievement levels, according to the standardized reading tests results........ 30
7
1 Executive summary
Project background
Learning for Life is a McGovern-Dole Food for Education project financed by the United 
States Department of Agriculture (USDA) under agreement FFE-520-2016/010-00, implemented 
in the department of Totonicapán in the western highlands of Guatemala by Catholic Relief 
Services – United States Conference of Catholic Bishops (CRS) and its local partners Proyecto 
de Desarrollo Santiago (PRODESSA) and Pastoral Social Caritas Arquidiócesis de Los Altos 
(PSC). 
Currently, the project is halfway through its second phase (implemented from FY 2017 
through FY 2021). Phase one of the project was implemented from FY 2014 through early 
2018.1 The strategic objectives of the project aim to improve literacy of school age children and 
increase the use of health and dietary practices in 337 schools of six (out of eight) municipalities 
of Totonicapán. The project promotes a school feeding program with food donated by USDA to 
complement those purchased with funds from the Ministry of Education of Guatemala. Also 
promotes the implementation of two literacy methodologies: Jardín de Letras (Kotz'i'j tz’ib' in 
K'iche' language) to teach reading and writing in both K'iche' and Spanish to early grade 
students, and Kemom Ch'ab'al, which is applied throughout primary school to strengthen reading 
comprehension skills in both languages. 
The project's participants are more than 69,000 individuals, including 8,139 pre-primary 
students, 39,831 primary students, 1,795 teachers, 337 school principals and 19,521 volunteer 
parents who contribute to the preparation of school meals.
Midterm evaluation purpose and methodology
The purpose of this midterm evaluation is to assess the progress towards the project goals 
and to establish a midterm point of comparison between the baseline evaluation (FY 2018) and 
the final evaluation (FY 2021), covering the first three years of the implementation of phase two 
of the project. This report answers 12 research questions related to the relevance, efficiency, 
effectiveness, impact and sustainability of the project, measures 11 outcome indicators and 
compares them to the baseline measurements and midterm goals.
The information may be useful for project staff in making strategic decisions during the 
last two years of implementation of the phase two. It may be used by the Ministry of Education 
for supervision and accompaniment of teachers and school principals within the target area and 
to compare results with other similar contexts. Various stakeholders involved in literacy 
instruction, hygiene, school health and nutrition, and school feeding could replicate successful 
practices and collaborate on the sustainability of results. The project will disseminate the 
information via learning events (departmental level), meetings and briefs.
This midterm evaluation was performed by the consulting firm ADOC, who hired a team 
of 63 enumerators to support the field work. ADOC also conducted the baseline evaluation and a 
1 Phase one was award number: FFE-520-2013/029-00.
8
special study on reading scores within the target area in 2018 and has evaluated other MGD 
projects in the country.
The external evaluation team followed the same methodology as in the baseline 
evaluation. The same tools were applied, and others were added according to the specific needs 
of this midterm evaluation. The same criteria were followed for the design and validation of data 
collection instruments, enumerator training and in-field supervision. 
ADOC managed the review of the ethics committee of the local University of “San Pablo 
de Guatemala” on the research protocol, data collection instruments and consent forms. 
Throughout the process of this midterm evaluation, efforts were made to ensure respect for and 
security of informants, confidentiality, cultural relevance and appropriateness to the context of 
the target area.
The quantitative sample randomly selected 135 schools, following the sample design 
created by the American Institutes for Research (AIR) included in the project's evaluation plan. 
In each school, the principal and a representative sample of students, teachers and volunteer
parents were interviewed/surveyed. Guided observation measured the use of educational 
materials distributed by the project, students’ attention in the classroom, infrastructure, health 
and nutrition practices, food preparation and storage practices.
A representative sample of 1,261 first graders, 1,245 second graders, 1,839 third graders, 
and 1,649 sixth graders were assessed with the Ministry of Education's standardized reading 
tests, provided to the Learning for Life project under a confidentiality agreement by the General 
Directorate for Educational Evaluation and Research (DIGEDUCA - the branch of the Ministry 
of Education in charge of research and the national standardized tests). DIGEDUCA trained 
external evaluator’s key staff to strengthen their capacity to administer the tests and standardize 
enumerators. The ELGI test, a Spanish-language adaptation of EGRA (Early Grade Reading 
Assessment) for Guatemalan context, was applied in the early grades (first and second grades) 
using digital forms specially developed for the project and validated by the DIGEDUCA. 
Reading comprehension skills tests for third and sixth grades were applied using printed forms 
that were then double digitized to minimize the likelihood of data entry errors. The test scoring 
process was carried out by the Ministry of Education to ensure compliance with their standard 
procedures and data analysis was based on models created by them.
The collected data were triangulated with qualitative information provided by informants 
during interviews and focus groups. Diversity was promoted in the focus groups of project 
participants and in addition 53 internal and external actors were interviewed. The qualitative 
data were processed in the same way and with the same software as in the baseline.
In formulating the findings, conclusions and recommendations, the external evaluation 
team took into account the limitations of the study and the contributions of the project's 
management and technical team captured during workshops reflecting on the preliminary results.
Midterm evaluation conclusions
The external evaluator finds that the actions of the Learning for Life project are 
adequately directed towards achieving the two strategic objectives, the financial management of
the project is efficient and effective, performing actions beyond those planned. Important
progress is observed in improving school attendance, student’ attentiveness and hygiene, health, 
9
nutrition, food preparation and food storage practices, while improvement in reading instruction 
seems to be occurring at a slower pace due to various challenges in the national education system 
that are also recognized by the Ministry of Education in various studies cited in this document.
Informants recognize that the Learning for Life project has contributed to the 
improvement of students' reading skills within the target area, which is evidenced by an increase 
in midterm indicator measurements compared to the baseline. To better support this progress, it 
is important that the project focuses on increasing the fidelity of the implementation of its 
literacy methodologies over the next two years by seeking opportunities to work closely with the 
local structure of the Ministry of Education.
There is also evidence of a significant increase in hygiene, health, and food preparation 
and storage practices. All of the qualitative informants stated that the project has strongly 
contributed to this improvement through training and technical accompaniment in the schools. 
The Learning for Life project, together with other stakeholders, has been influencing and 
building capacity since its first phase of implementation, resulting in important advances in the 
formulation and implementation of policies and regulations and in the creation of official 
programs. Examples of the project's support in this area are the implementation of the National 
School Feeding Law, the creation and implementation of the Municipal Intercultural Education 
Policy in Totonicapán, the first drafts of the Ministry of Education’s National Educational 
Materials and Texts Program, and the creation of the national program "Committed to First 
Grade.”
In general, the project is recognized and accepted by participants and has achieved 
strategic alignment at national, regional and community levels, maintaining excellent 
relationships with government structures, allies and other stakeholders, which undoubtedly 
increases the likelihood of sustainability of results.
Midterm evaluation recommendations
The external evaluation team presents a set of 19 recommendations so that they can be 
analyzed by the project and the donor to implement an action plan to support the adequate 
achievement of the goals towards the end line.
Some recommendations on the project’s literacy interventions relate to raising awareness 
of teachers’ key roles in helping students acquire literacy skills and how those skills are used as 
building blocks for the rest of their academic journeys. Another important point is to strengthen 
the fidelity of implementation of the project’s literacy methodologies. 
Regarding health and nutrition, the external evaluator recommends reinforcing some 
hygiene practices during trainings, establish in what ways the project will support schools to 
maintain and improve the dietary diversity, and increase schools’ local purchasing capacities, 
among other aspects.
10
2 Project background
Learning for Life is a McGovern-Dole Food for Education project financed by the United 
States Department of Agriculture (USDA) under agreement FFE-520-2016/010-00, implemented 
in Guatemala by Catholic Relief Services – United States Conference of Catholic Bishops (CRS) 
and its local partners: Proyecto de Desarrollo Santiago (PRODESSA) and Pastoral Social Caritas 
Arquidiócesis de Los Altos (PSC). 
The project is halfway through its second phase (implemented from FY 2017 through FY 
2021). Phase one of the project was implemented from FY 2014 through early 2018.2
The overall theory of change postulates that children’s literacy will improve if they attend 
school more regularly, if they eat school meals with adequate nutritional value to ensure 
classroom attentiveness, and if teachers improve literacy instruction skills. 
The two strategic objectives of Learning for Life aim to: 1) Improve literacy of school 
age children, and, 2) increase the use of health and dietary practices. Under these two strategic 
objectives, the Learning for Life project implements 18 different activities, which lead to 
immediate and intermediate results (see Annex 1 of project framework for reference).
3 Geographic area
New studies of the socio-economic and educational conditions of the department of 
Totonicapán have not been released since the baseline evaluation of the project. The only 
updated data is the total population of 418,569 according to the national census in 2018 (INE, 
2019).
Figure 1 Learning for Life project’s geographic location
Source: Learning for Life data system
The Learning for Life project’s coverage area is mostly rural, located in the department of 
Totonicapán in the western highlands of Guatemala. In phase two, the project supports 337
public primary schools in six municipalities: Momostenango, Santa Lucía La Reforma, San 
2 Phase one was award number: FFE-520-2013/029-00.
11
Andrés Xecul, San Bartolo Aguas Calientes, Santa María Chiquimula and Totonicapán 
(northern rural area only for Totonicapán municipality). The municipalities of Santa María 
Chiquimula and Totonicapán did not participate in phase one and began receiving project 
interventions in phase two of the project. 
4 Target population
The project targets preschool and primary school students, teachers, school administrators 
and volunteer parents in 337 schools in six municipalities (225 schools that participated in the 
first phase of the project and 112 new schools). Each year, the project evaluates with the 
Ministry of Education the possibility of serving new public schools that open during the project 
within the target area.
Table 1 shows the number of each type of participant registered by the project in the 2019 
fiscal year.
Table 1 Number of participants in the Learning for Life project in the 2019 fiscal year
Type of participant Number of participants
Preschool students 8,139
Primary school students 39,831
Teachers 1,795
School principals 337
Volunteers 19,521
Total 69,623
Source: Learning for Life monitoring system data
5 Midterm evaluation purpose
According to the terms of reference, the purpose of this external midterm evaluation is to 
assess progress toward achieving project goals and indicators to establish a midpoint comparison 
between the baseline evaluation (second quarter of the 2018 fiscal year) and the final evaluation 
(end of the 2021 fiscal year), specifically:
i) To critically and objectively review the implementing experience and the implementing 
environment,
ii) to assess progress in implementation,
iii) to assess the relevance of the intervention,
iv) to provide an early signal of the effectiveness of the interventions,
v) to document lessons learned,
vi) to assess sustainability efforts to date, and,
vii) to discuss and recommend mid-course corrections, if necessary. 
The midterm evaluation considers the first three years of implementation of the second 
phase of the Learning for Life project (from October 2017 to September 2019). 
12
The midterm evaluation process met the established purpose and this report is intended to 
provide integrated information to the donor, the project’s internal staff and stakeholders. The 
Ministry of Education and other organizations interested in literacy, school feeding, health and 
hygiene practices in school can use this information to extend project activities or replicate 
successful practices. The results of the official standardized reading tests applied to a large 
sample of students during this midterm evaluation may be used by the Ministry at regional and
national levels to compare with similar bilingual contexts and make decisions about 
accompaniment to teachers and school principals in the department of Totonicapán.
After receiving feedback and approval from USDA for this report, the external evaluator 
and the project team will meet to analyze the final results and recommendations, to build an 
action plan for the last two years of the second phase of the project.
6 Midterm evaluation methodology 
The midterm evaluation follows the baseline evaluation methodology, maintaining the 
tools and calculation procedures for the indicators, adding some items or tools to obtain data and 
information to answer the specific midterm evaluation questions. The same criteria for data 
collection tool design and validation, enumerator training and in-field supervision were applied. 
Quantitative and qualitative data collection occurred at the end of the 2019 school year 
from beginning of September to the second half of October. 
The quantitative sample keeps the original design created by the American Institutes for 
Research (AIR) included in the approved evaluation plan for Learning for Life (see annex 5 for 
details). This design includes a two-level cluster random assignment of schools with treatment 
assignment at level 2 (level 2: schools, level 1: individuals), with a confidence interval of 95% 
(alpha = 5%), statistical power of 80% (beta = 20%) and minimum detectable effect sizes 
(MDES) from 0.20 to 0.31 standard deviations (depending on the variable being measured).
The quantitative sample size calculated to maintain that power of inference during the
midterm evaluation is 135 schools, which was achieved during midterm data collection and 
represents 40% of total of schools participating in the Learning for Life project.
The midterm evaluation also included qualitative data collection to complement and 
explain quantitative findings. Qualitative data were collected and triangulated through focus 
groups and key informant interviews, using the same software (Atlas.ti), coding and analysis 
procedures as in the baseline. The rigor and quality criteria used to order the qualitative data 
were dependence, credibility, transfer and confirmation, allowing open codification at first level 
to create categories, axial at second level to compare categories, and selective at the last level for 
narratives and explanations.
After data analysis, the main researcher and a coordinator from the external evaluator 
team met with management and technical staff of the project in order to present and consider 
preliminary findings, clarify specific details of implementation and identify further analysis 
possibilities. The conclusions and recommendations were formulated taking their contributions 
into account. 
13
6.1 External evaluation team
ADOC (Organizational and Community Development Association, for its acronym in 
Spanish) is a non-profit association with operations in Guatemala since 2002 with experience in 
social research, design, monitoring, evaluation and implementation of humanitarian projects. 
They have worked with various NGOs and donors in Guatemala and Central America (including 
evaluation of other MGD projects in the country), also with the private sector in the framework 
of corporate-social responsibility.
ADOC performed the Learning for Life project baseline evaluation and a special 
longitudinal study on literacy outcomes in 2018 (post-test).
The ADOC midterm evaluation team consisted of one main researcher, three advisors, 
three in field coordinators, nine team supervisors and 56 enumerators. Most of the midterm 
evaluation team also participated in the baseline evaluation. During data collection the staff were 
distributed into nine teams to be able to complete data collection in a timely fashion. 
6.2 Evaluation ethics
In order to ensure that national and international standards related to research ethics were 
applicable and in compliance with the Code of Federal Regulations, Title 45, Part 46 (Protection 
of Human Subjects, 2018), the external evaluator managed an ethics committee review from the 
local university “Universidad San Pablo de Guatemala” on the research protocol, data collection 
tools and informants’ consent and assent forms.
Application of the national standardized reading tests, provided by the Ministry of 
Education, required a signed confidentiality agreement from all individuals who had access to 
them (typography, data collectors, data entry and supervisors) and personnel involved in the 
study. 
Data collection methods and tools were validated for cultural relevance and contextual 
appropriateness before the baseline evaluation. The majority of the same tools were used for
midterm data collection and new tools/methods were evaluated and field tested for cultural 
relevance and appropriateness in participant schools outside the selected sample.
The great majority of enumerators in the midterm evaluation are from the project’s 
geographical coverage area, to ensure linguistic and cultural appropriateness. There was at least 
one bilingual (K'iche’/Spanish) person in each enumerator team to communicate with 
monolingual K’iche’ informants.
The enumerators were hired according to their knowledge of the local culture and region, 
their experience applying the national standardized reading tests and their data collection skills. 
The supervisors were selected for their capacities and previous experience working with ADOC. 
Students assessed in first, third and sixth grade were selected using random numbers 
generating software, in line with the project’s USDA-approved evaluation plan. For longitudinal 
purposes, enumerators intentionally looked for the same second grade students assessed in 2018 
(when they were in first grade). 
Each supervisor found out which school his/her team would evaluate one day before the 
visit, while each enumerator found out on the morning of the visit. This was to prevent schools 
from being notified of the visits too far in advance, ensuring school days were relatively 
14
“typical.” Enumerators and supervisors were not trained in the test scoring or data analysis 
procedures, as these were controlled by the DIGEDUCA and data cleaning personnel of ADOC 
to mitigate biases/manipulation. Data quality was monitored daily. No systematic biases or 
conflicts of interest were identified during the midterm evaluation. All information was handled 
under confidential terms, with proper data security and protocols. After data analysis, the 
external evaluator team delivered the tests and participant databases to the Learning for Life 
team in CRS’s office for safe keeping and audit requirements. Additionally, the external 
evaluator erased all information from their physical and electronic files.
6.3 Quantitative sample
According to the sampling plan for the midterm evaluation, a total of 135 schools (as 
observation units) were selected, out of which 135 (100.0%) were evaluated. 85.9% of the 135 
schools included in the midterm sample were also evaluated in the baseline. Nine schools (6.7%) 
in the original sample had to be replaced since they did not allow the evaluation team to enter 
because of special activities due to local festivities or because teachers refused. Replacement 
schools were randomly selected to reach the full 135 school sample, after grouping by cluster 
and size.
In each school, the school principal and a sample of key informants were 
surveyed/interviewed, including students of first, second, third and sixth grade, teachers, and 
volunteer parents. Furthermore, guided observation tools were used to study student attention in 
the classroom, literacy materials, infrastructure, child health and nutrition practices, food 
preparation and food storage practices.
Informants sample sizes and demographic information are described later in this 
document for each indicator.
To study the effects of literacy interventions, participant schools were distributed into 
different groups of packaged treatment interventions, in line with the project’s evaluation plan
approved by USDA. 
• Group 1-A: new schools incorporated in the second phase of the project that began to 
receive literacy interventions in the 2019 school year. This was the control group for 
2018.
• Group 1-B: new schools incorporated in the second phase of the project that began to 
receive literacy interventions in the 2018 school year.
• Group 2-A and 2-B: participant schools coming from the first phase of the project and
receiving literacy interventions since the 2014 school year.
The difference for groups with suffix -A and groups with suffix -B lies in the project’s 
literacy methodology for first grade, thus, schools within groups -A receive Kemom Ch’ab’al
and those with -B, Jardín de Letras. All groups receive Kemom Ch’ab’al from second to sixth 
grade (see Annex 2 for explanation of these methodologies).
After the 2018 school year, the Learning for Life project’s evaluation plan no longer 
considered a counterfactual scenario with a control group of schools due to ethical concerns 
about not providing project interventions in the target area. Data coming from control group 1-A 
were useful to compare results between students not receiving/receiving project’s bilingual 
literacy methodologies during a special longitudinal study that collected data at the beginning 
15
and end of the 2018 school year. As of 2019 school year, schools of group 1-A began to receive 
Kemom Ch’ab’al methodology.
Tables 2 and 3 show the number of sampled schools by intervention group and by 
municipality.
Table 2 Number of sampled schools by intervention group
Group Number of sampled schools Total project schools in group
1-A 27 35
1-B 40 53
2-A 34 160
2-B 34 65
TOTAL 135 3133
Source: Learning for Life midterm evaluation database
Table 3 Number of sampled schools by municipality
Municipality Number of sampled schools Total project schools in 
municipality
Momostenango 45 144
San Andrés Xecul 8 21
San Bartolo Aguas Calientes 8 30
Santa Lucía La Reforma 7 35
Santa María Chiquimula 39 66
Totonicapán 28 41
TOTAL 135 337
Source: Learning for Life midterm evaluation database
Most of participant schools are rural, however there are some differences in their 
structures. According to the classification of the Ministry of Education and depending on the 
number of students enrolled, a ‘gradada’ school (Spanish term with no English translation) is 
one with enough students so that there is just one grade assigned to each teacher; in a 
‘multigrade’ school, two or more grades share the same teacher and the school principal usually 
is also a teacher; and, a ‘one-room’ school has only one person serving as the school principal 
and the teacher for all primary school grades. Table 4 displays the proportions of each type of
school within the coverage area and the midterm sample.
3 The project covers 337 schools. The total project schools with an intervention group assigned are 313, because 7 
schools were created by the Ministry of Education after the random assignment of packaged interventions and 17 
schools received literacy materials from a previous edition of the Jardín de Letras methodology (taken out of the 
sample).
16
Table 4 Percentage of schools by type
Type Percentage in coverage 
area
Percentage in sample
Gradada school 32.0 39.3
Multigrade school 58.2 57.8
One-room school 9.8 3.0
Source: Learning for Life midterm evaluation database
6.4 Qualitative sample
A total of 10 focus groups were conducted in communities among the six municipalities 
covered by the Learning for Life project. The focus groups promoted participant diversity, with 
participants including: 49 volunteer fathers, 73 volunteer mothers, 18 male school principals and 
teachers, and 17 female school principals and teachers.
In addition, 53 different actors were interviewed, including USDA personnel in 
Guatemala, officials in the departmental office of the Ministry of Education, management and 
technical staff of CRS and its implementing partners for the project. 
6.5 Limitations of the midterm evaluation
As with any large study, there were limitations in carrying out this evaluation. 
• One limitation possibly biasing results of this midterm evaluation is that the visits of the 
enumerator teams were announced by external evaluator’s staff to the school principals 
several days in advance for logistical reasons. The advance notice to schools could have 
affected school cleanliness or attendance variables or other aspects evaluated the day of 
the visit. (A coordinator from ADOC contacted the principals by phone. As mentioned 
earlier, neither the supervisors nor the enumerators were aware of the visiting plan to 
mitigate conflict of interest.)
• Reading tests and tools related to hunger and attention variables were applied at different 
moments during school day, according to the characteristics of each school and the 
decisions of each principal, hence the reported perceptions of student hunger or tiredness 
could have been reported higher or lower due to possible differences in sensations of 
fatigue or hunger at the different times of day these tools were applied and the different 
times of day that schools serve school meals.
• The cluster sample design only permits generalizations for each intervention group and 
the whole target area. It is not possible to generalize findings at municipal level because
schools of each municipality were randomly distributed into different intervention groups
and the number of schools sampled per municipality may not be statistically 
representative of the total schools in the municipality. 
• Cross-contamination is a possibility through the exchange of first grade literacy materials 
(Kemom Ch’ab’al / Jardín de Letras) between teachers or by teachers transferred among 
schools of different intervention groups.
• Because of the high proportion of multigrade schools in the target area and also in the 
sample, many school principals are also teachers using the project’s reading 
17
methodologies in one or more grades simultaneously. For that reason, their perceptions 
may have been counted both in principals and teachers sample as they were surveyed 
with tools designed for principals as well as with tools designed for teachers.
• The Ministry of Education conducted testing with a national sample in 2019 using the 
same standardized reading tests as this midterm evaluation but the results were not yet 
available at the time of this report. However, the project has sufficient information to 
anchor and compare the midterm evaluation results with national results when they 
become available with the DIGEDUCA. The baseline and midterm results were 
statistically analyzed in the same way so that they are comparable over time within the 
target area.
• Currently, there is only a K’iche’ test available for early grade reading (called ESWUJ), 
used the baseline evaluation. However, in the approved evaluation terms of reference the 
Learning for Life project had to make a cost-benefit decision to keep midterm evaluation 
costs within budget while also measuring second grade reading levels to measure the 
reading levels for the corresponding indicator. In third and sixth grade, a validated 
reading comprehension test does not yet exist in K’iche’ language, limiting knowledge of 
student reading progress in this language. Additionally, the Ministry of Education has no 
published any data, nor studied K’iche’ language learning indicators, so the project would 
not be able to compare any project data obtained with national data.
7 Midterm evaluation questions and answers
The terms of reference for this midterm evaluation included a list of possible evaluation 
questions, from which the project and the external evaluator chose the most feasible in order to 
assess relevance, efficiency, effectiveness, impact and sustainability during the first three years 
of implementation. This section includes a concise analysis of the 12 evaluation questions agreed
upon between the external evaluator and the project.
All answers were obtained by triangulation through the below techniques and informant
groups. There is further discussion around the results of these questions in sections 8 and 9. The 
data collected to answer these questions, as well as to report on project indicators, were used to 
create the recommendations included in section 10.
1
Evaluation question 
To what extent does the project design align with the Ministry of 
Education’s current goals, objectives and strategies?
Criteria
• Relevance
Answer
The actions of the project are in compliance with the McGovern-Dole results 
framework and entirely aligned with the National School Feeding Law and the Ministry of 
Education’s strategic plan at the time of evaluation (priorities, actions, and strategy) 
(MINEDUC, 2016).
18
Method
• Qualitative 
Techniques
• Document and literature review
• Focus groups
• Interviews
Source / informants
• Ministry of Education’s strategic plan for 2016-2020 (MINEDUC, 2016)
• Ministry of Education departmental level staff
• CRS, PSC and PRODESSA project staff
2
Evaluation question
How appropriate are project interventions for Totonicapán’s local 
culture and context?
Criteria 
• Relevance
Answer
All informants report that school feeding interventions and bilingual educational 
interventions of the project are relevant to the social and cultural context of the target area.
The educational materials, training contents and human interactions provided and 
supported by the Learning for Life project respect and support the bilingualism, multiethnicity 
and cultural characteristics of Totonicapán department.
Method
• Qualitative 
Techniques
• Focus groups
• Interviews
Source / informants
• Volunteer parents
• School principals
• Teachers
• Ministry of Education departmental staff
• CRS, PSC and PRODESSA project staff
Evaluation question
19
3 What factors have inhibited the achievement of project goals, 
objectives and expected results?
Criteria
• Effectiveness
Answer
There were no project internal inhibitory factors detected during the midterm 
evaluation. Some external factors were identified, for instance, school dropouts, mainly due to 
economic conditions. 
Also, the evaluation found a lower than expected level of teacher commitment to 
implementing the literacy methodologies as prescribed, despite proper planning, training and 
accompaniment from the project. Some considerations for this low level of commitment have 
to do with linguistic challenges (regional variations of K’iche’ or Spanish monolingual 
teachers) and lack of classroom time due to other educational activities/courses.
Method
• Qualitative 
Techniques
• Focus groups
• Interviews
Source / informants
• USDA staff in Guatemala
• Ministry of Education departmental staff
• Volunteer parents
• School principals
• Teachers
• CRS, PSC and PRODESSA project staff
4
Evaluation question
What factors have facilitated the achievement of project goals, 
objectives and expected results?
Criteria
• Effectiveness
Answer
In general, the project is well accepted by the participants and has reached strategic 
alignment at different levels of intervention, has built processes and procedures to facilitate 
communications within and between executing organizations and other allies and 
20
stakeholders, sustaining a complex system of relationships in adherence to government 
structures and local context and leveraging capacities already installed.
Method
• Qualitative 
Techniques
• Focus groups
• Interviews
Source / informants
• USDA staff in Guatemala
• Ministry of Education departmental staff
• Volunteer parents
• School principals
• Teachers
• CRS, PSC and PRODESSA project staff
5
Evaluation question
How can project interventions be adjusted to achieve project targets?
Criteria
• Effectiveness
Answer
No need for specific adjustment of project activities was identified. It is important that 
the project keeps seizing opportunities for alliances with the private sector, at local and 
national levels, and with local government (municipalities). 
It is necessary to find a way to work with the local structure of the Ministry of 
Education to increase the fidelity of implementation of the project’s literacy methodologies.
Method
• Qualitative 
Techniques
• Focus groups
• Interviews
Source / informants
• USDA staff in Guatemala
• Ministry of Education departmental staff
• Volunteer parents
• School principals
• Teachers
21
• CRS, PSC and PRODESSA project staff
6
Evaluation question
What alternative interventions would be more cost-effective while still 
achieving the same results?
Criteria
• Efficiency
Answer
Consistent with the perceptions of USDA personnel in Guatemala, CRS and 
implementing partners staff, the financial management of the project is efficient and effective, 
performing actions and tasks beyond those planned. To achieve the same results beyond the 
life of the project, there are costs involved that cannot be eliminated and most likely cannot be 
reduced, therefore they should only be assumed by other actors (transferred).
The Learning for Life project has already found ways to reduce costs. For example, 
the educational materials of the literacy methodologies have been re-edited to be as compact 
as possible, including the essential contents and meeting the requests of the Ministry of 
Education. On the other hand, trainings and technical accompaniment to principals, teachers, 
PTA’s and volunteers must be conducted continuously due to turnover and other factors.
Method
• Qualitative 
Techniques
• Focus groups
• Interviews
Source / informants
• USDA staff in Guatemala
• Ministry of Education departmental staff
• CRS, PSC and PRODESSA project staff
7
Evaluation question
In what ways has the project had an impact, if any, on school-aged 
children’s literacy, enrollment and attendance?
Criteria
• Impact
Answer
22
The results of the national standardized tests applied during midterm data collection to 
a representative sample of approximately 6,000 students suggest an increase in the values of 
all reading skills indicators, compared to the baseline.
Monitoring data of the project show a slight decreasing trend in the overall number of 
school-aged children enrolling in school within the target area during the last three years, 
mirroring the same (but more marked) downward tendency at the national level registered by 
the Ministry of Education. However, the enrollment numbers for first and second grade 
students in the project coverage area are showing an increase in the past two years. 
The average attendance rates monitored by the project during the last school year is 
similar to the rate measured during midterm evaluation data collection. The average school 
attendance rates meet project targets and denote an increase from the baseline.
Method
• Quantitative and qualitative 
Techniques
• Official standardized reading tests for first, second, third and sixth grade
• Headcount in classroom
• Focus groups
• Interviews
Source / informants
• Project monitoring system
• Students
• Volunteer parents
• School principals
• Teachers 
8
Evaluation question
Is there a notable difference in first grade literacy scores between 
schools that have used the early grade bilingual literacy methodology 
Jardín de Letras for two or more years compared with schools that do 
not have this methodology?
Criteria
• Impact
Answer
No, in this midterm evaluation, test results do not show advantages for groups of 
schools with the early grade literacy methodology Jardín de Letras, compared to those without 
it. This evaluation was unable to measure whether students made gains (or not) in K’iche’ 
language because only Spanish language test results were compared. During this evaluation it 
23
was not possible to apply the early grade reading test in K'iche' due to time and budget 
constraints.
Method
• Quantitative
Techniques
• Official standardized early grade reading test
Source / informants
• Students
9
Evaluation question
Is there a notable difference in third and sixth grade literacy scores 
based on the number of years that schools have used the bilingual 
reading comprehension methodology Kemom Ch’ab’al?
Criteria
• Impact
Answer
No, in this midterm evaluation, test results do not show any difference in third and 
sixth grade reading scores in schools with more years of use of the Kemom Ch’ab’al
methodology. This evaluation was unable to measure whether students made gains (or not) in 
K’iche’ language because only Spanish language test results were compared. No reading 
comprehension test was administered in K'iche' because the Ministry of Education does not 
have a validated version available for third and sixth grade.
Method
• Quantitative 
Techniques
• Official standardized reading tests for third and sixth grade
Source / informants
• Students
10
Evaluation question
Are there notable longer-term effects on reading comprehension in 
third grade students that have received the early grade bilingual literacy 
methodology Jardín de Letras in 2017 compared to students who did 
not?
24
Criteria
• Impact
Answer
No, in this midterm evaluation, test results do not show any difference between third 
grade students who received Jardín de Letras in 2017 and those who did not. This evaluation 
was unable to measure whether students made gains (or not) in K’iche’ language because only 
Spanish language test results were compared. No reading comprehension test was 
administered in K'iche' because the Ministry of Education does not have a validated version 
available for third grade.
Method
• Quantitative 
Techniques
• Official standardized reading test for third grade
Source / informants
• Students
11
Evaluation question
What evidence is there to suggest whether the effects of the various 
interventions will be sustained beyond the life of the project?
Criteria
• Sustainability
Answer
The factors reported by the internal and external informants indicate a high probability 
of key project interventions and results being sustainable after project life because of influence 
and capacity building at different levels. 
At the national level, advances in the implementation of the National School Feeding 
Law, supported by Learning for Life and other MGD project implementers in Guatemala, and 
good relationships with the outgoing and incoming central ministerial authorities, strongly 
imply the sustainability of the results. Recently, as a result of the influence from the project 
and other stakeholders, the Ministry of Education has created the national program “Committed 
to First Grade,” which seeks to generate conditions to help students achieve the expected 
literacy competencies, specifically train first grade teachers and ensure that schools have 
culturally relevant educational materials. Furthermore, the Ministry of Education has stated 
interest in reproducing the educational materials of the Jardín de Letras methodology in 
K’iche’ and other Mayan languages both within and beyond the project’s coverage area.
25
At the time of this evaluation, the Ministry of Education was about to pass the National 
Educational Materials and Texts Program. The project collaborated during the construction of 
the first drafts.
At the regional level, CRS and its implementing partners are constantly boosting 
dialogue spaces with the elected municipal authorities and the departmental ministerial 
structure in Totonicapán. The accompaniment of the project to the development of the 
Municipal Intercultural Education Policy, passed by the Mayor of Totonicapán, is another 
promising factor that points towards project’s results sustainability since the policy helps 
designate funding and municipal efforts toward different areas of education in Totonicapán 
municipality, prioritizing reading comprehension as a fundamental educational area.
At local levels, the project has advanced significantly in capacity building for volunteer 
parents, teachers and school principals.
With these advances, it will be key to continue to strengthen the scale-up of the 
National School Feeding Law, particularly in regards to production planning, certification of 
local producers and streamlining processes between government ministries. To better sustain 
advances in bilingual literacy, the project should also continue to push for policies that support 
bilingual materials creation and distribution, as well as continue to train teachers and Ministry 
of Education officials on bilingual literacy methodologies and school administration skills. 
Method
• Qualitative 
Techniques
• Focus groups
• Interviews
Source / informants
• USDA staff in Guatemala
• Ministry of Education departmental staff
• Volunteer parents
• School principals
• Teachers
• CRS, PSC and PRODESSA project staff
12
Evaluation question
What factors were key to eliciting community support/involvement?
Criteria
• Sustainability
Answer
26
The Learning for Life project was able to harness the already existing culture of 
volunteerism in most communities of Totonicapán department, which has been a key factor for 
the involvement and support of community members.
Method
• Qualitative 
Techniques
• Focus groups
• Interviews
Source / informants
• Ministry of Education departmental staff
• Volunteer parents
• School principals
• Teachers
• CRS, PSC and PRODESSA project staff
8 Midterm evaluation findings
The midterm evaluation measures 11 outcome indicators as part of the Learning for Life 
project’s performance monitoring plan (PMP). This section presents the midterm values, 
compares them with the baseline values and describes the main quantitative and qualitative
findings. (Annex 4 includes a table integrating the values of all indicators.)
Literacy indicators
Since the baseline, the General Directorate for Educational Evaluation and Research 
(DIGEDUCA - the branch of the Ministry of Education in charge of research and the national 
standardized tests) granted permission to the Learning for Life project to use the national 
standardized reading tests in early grades (first and second), third and sixth grades of primary 
school, under a strict confidentiality agreement. Use of the national standardized tests in 
baseline, post-test, midterm, and final evaluations allows the project to compare student reading 
outcomes to the competencies that the Ministry of Education believes students of those grades 
should develop, according to the National Base Curriculum. 
Using a longitudinal approach, the same cohorts of students evaluated in the baseline at
the beginning of the 2018 school year were also assessed by the project at end of the school year 
with the same standardized reading tests. For this reason, the literacy indicators are presented 
with three datapoints, referred to as: baseline (March 2018) , post-test (September 2018) and 
midterm (September 2019); whilst the rest of indicators are presented with two datapoints
(baseline and midterm).
During the baseline, post-test and midterm evaluations, the external evaluator adhered to 
the Ministry’s methodology and protocols. Prior to enumerator training and midterm data 
collection, the main researcher, coordinators and team supervisors attended a workshop 
27
conducted by the DIGEDUCA, to reinforce their enumerator training skills, and ensure correct 
application and management of the standardized tests.
The external evaluator team developed digital forms using the software Tangerine, 
validated by the DIGEDUCA, for the one-on-one administration of the early grade reading tests. 
The tests for third and sixth grade were self-administered with printed forms distributed to 
students and double digitized in order to minimize data entry errors.
The scoring process was done by the DIGEDUCA to ensure compliance with their
standard procedures and validity of results. Data analyses were conducted by the external 
evaluator’s team, and the databases were shared with the DIGEDUCA to also support the 
government with a wider data sample to be used for national research.
The ELGI test (Spanish-language adaptation of the Early Grade Reading Assessment
created by the Ministry of Education of Guatemala) was applied to a sample of first and second 
grade students. According to the Ministry of Education, the ELGI test is applicable to “early 
grades,” which in Guatemala are considered first through third grade. The Ministry of Education 
also created a model to both explain and predict how students learn reading skills. Once the 
midterm evaluation tests results were scored by the DIGEDUCA, the Ministry of Education 
model was then applied to interpret results. The reading prediction model categorizes students
into one of five stages of learning. When developing the model, the researchers determined that 
some reading skills predict which reading skills will be learned next. For example, knowledge of 
letter names and sounds, and phonological awareness predict the ability to decode; decoding 
predicts fluidity; and the latter predicts reading comprehension. (Cotto & Del Valle, 2017)
Figure 2 provides a short description of each stage. Cotto & Del Valle (2017) present a 
more detailed description (p. 26-30).
28
Figure 2 Stages in the model to explain and predict how students learn to read
Source: own elaboration with data from Cotto & Del Valle, 2017
The national standardized reading comprehension tests at grade level were applied to 
third and sixth graders. Results were analyzed according to the levels of performance and levels 
of achievement developed by the Ministry of Education in 2006 with the Bookmark Method. 
The two-stage scoring process first goes through a traditional count of correct responses and then 
uses data anchoring based on individual question responses for data comparability. (Quim & 
Santos, 2015, p.12) The midterm evaluation results for third and sixth grades are compared to the 
prior two datapoints: baseline and post-test. 
Depending on the test results, each student is categorized into one of four performance 
levels and into one of two achievement levels (Quim & Santos, 2015, p.13). The figures below 
display an explanation on these levels
1. Emergent
Children who have not yet begun the formal process of learning.
Students who are attending school but show some difficulty in learning the letters or 
do not speak Spanish
2. Initial
Students who identify the vowels and can read some isolated words, know a few 
letters and have difficulty joining the sounds that form a word
3. Decoder
Students who identify more than five letters and can read words with the letters they 
already know. The reading process has not yet become automatic
4. Automated
Students who can read several words in an automated way but still do not have 
enough fluency or reading comprehension 
5. Fluent
Students who read fluently and understand what they read 
29
Figure 3 Student’s performance levels, according to the standardized reading tests results
Source: own elaboration with data from Quim & Santos, 2015
1. 
Unsatisfactory
Students who have not developed grade level skills, abilities or knowledge
2. Needs 
improvement
Students with less than expected proficiency 
3. Satisfactory
Students who demonstrate an adequate level of the competencies evaluated
4.Excellent
Students who demonstrate a higher level of the competencies evaluated
30
Figure 4 Student’s achievement levels, according to the standardized reading tests results
Source: own elaboration with data from Quim & Santos, 2015
Learning for Life is, through another external consultancy and in parallel with this 
evaluation, carrying out a special study on the fidelity of implementation of the project's literacy 
methodologies and the analysis will include the correlation with the results of the official 
standardized reading tests in order to find possible supplementary explanations.
8.1.1 (Indicator 4) Percent of students who, by the end of one grade of primary 
schooling, demonstrate that they can read and understand the meaning of 
grade level text
During the midterm evaluation, a sample of 1,261 first grade students (610 girls and 651 
boys) of 135 schools were assessed with ELGI (9 students on average per school). The indicator 
is measured adding the percentage of students in stages “4. Automated” and “5. Fluent” of the 
Ministry’s model. 
The midterm value for this indicator is 26.6%, surpassing the project’s goal of 21.0% for 
the midterm. The disaggregation by sex is 28.2% for girls and 25.0% for boys, showing a slight 
advantage for girls that is not maintained for the other grades assessed.
Table 5 Historical values of indicator 4: Percentage of first grade students who demonstrate reading 
skills at grade level
Baseline value
(March 2018)
Post-test value
(September 2018)
Midterm value
(September 2019)
1.7 21.9 26.6
Source: Learning for Life evaluation database
The post-test (21.9%) and the midterm values (26.6%) were collected at the end of the 
school year, consequently they are comparable, registering an increase of 4.7% in one calendar 
Students in 
"4. Excellent" 
level of 
performance 
Students in 
"3. 
Satisfactory" 
level of 
performance 
Achievement
Students in "2. 
Needs 
improvement" 
level of 
performance 
Students in "1. 
Unsatisfactory" 
level of 
performance 
Non 
achievement
31
year. The baseline value (1.7%) was collected at the beginning of the 2018 school year, when 
first grade students would still likely have low first grade reading competencies. 
Some teachers recognize the various contributions of the Learning for Life project, such 
as its bilingual literacy methodologies, reading materials distribution, teacher and principal 
training and accompaniment. A female teacher in San Andrés Xecul municipality mentioned, “I 
indeed realize that the tools are useful and that children like them and want to learn more. I feel 
motivated to apply techniques that I learned, books are practical tools that help children a lot, 
because now they can read and also understand what they read.” A male school principal in 
Totonicapán municipality expressed that teachers in his school, “are very happy because we are 
seeing the change. Now children learn to read faster. We are applying the techniques, guides and 
practical and dynamic exercises that help children ... in the past fewer children learned to read.”
Graph 1 shows the distribution of assessed students in the five stages of learning to read.
The frequency distribution is consistent within the four treatment groups. No statistical 
differences were detected in the results of the different intervention groups. This suggests that 
no advantages were detected for groups that had the Jardín de Letras treatment compared to 
groups that had the Kemom Ch’ab’al reading comprehension methodology. 
Graph 1 Percentage of first grade students in each of the five stages of learning to read
Source: Learning for Life midterm evaluation database
The distribution of children among the five stages in the midterm is very similar to the
post-test 2018 results and the results for the public sector at national level in 2014 (the most 
recent year that the Ministry of Education’s data is available on national standardized testing.), 
denoting that there is a trend for first grade students to remain in the first two stages of learning, 
even after completing their first year of primary schooling. According to the Ministry of 
Education, that situation could be closely related to teaching practices. (Cotto & Del Valle, 
2015, p.30-31) Similarly, the way teachers are assigned to first grade, often as a punishment or as 
a space for new teachers, may also influence first grade results. Del Valle & Mirón (2017) made 
a special study to characterize the first-grade teachers’ profile in the public sector and found that 
just 50% teachers surveyed had been assigned to first grade voluntarily. To address this issue, 
the new official program “Committed to First Grade” intends to eliminate non-standardized 
9.8
43.6
20.0
10.2
16.3
1. Emergent 2. Initial 3. Decoder 4. Automated 5. Fluent
1st grade
32
methods of assigning teachers in schools, providing a profile with the competencies that first 
grade teachers should meet. 
Annex 3 includes a comparative table of the average reading skills of children assessed 
with ELGI in each of the three datapoints of Learning for Life evaluation (baseline, post-test and 
midterm), according to the constructs described by the Ministry of Education to explain the 
structure of the test. The values of the reading skills variables included in that table are 
consistent with the stages of learning the students were in.
8.1.2 (Indicator 5) Percent of students who, by the end of two grades of primary 
schooling, demonstrate that they can read and understand the meaning of 
grade level text
A total of 1,245 second grade students (634 girls and 611 boys) of the same 135 schools 
were also evaluated with ELGI (9 students on average per school). Just like for first grade, the 
percent of second grade students in the last two learning stages were summed for this indicator’s 
measurement.
A representative sample of this cohort was also assessed during baseline and post-test in 
2018 when they were in first grade. This allows the project to better understand how students of 
early grades in target schools are learning to read and developing essential reading skills for the 
upcoming school years. Table 6 presents the longitudinal results.
Table 6 Percentage of students in the cohort 1st grade 2018 / 2
nd grade 2019 who demonstrate reading 
skills at grade level 
Beginning of 1st grade
(March 2018)
End of 1st grade
(September 2018)
End of 2nd grade
(September 2019)
1.7 21.9 52.3
Source: Learning for Life evaluation database
For second grade students, the midterm value for indicator 5 is 52.3%, surpassing the
midterm goal by 21.3% (the project’s midterm goal is 31.0%). The disaggregation by sex is 
52.7% for girls and 51.9% for boys, suggesting a balanced performance for both genders.
Graph 2 shows the distribution of students in the five stages of learning to read and graph 
3 compares first and second grade results for this midterm evaluation.
33
Graph 2 Percentage of second grade students in each of the five stages of learning to read
Source: Learning for Life midterm evaluation database
Graph 3 Comparison between first and second grade 2019 literacy results
Source: Learning for Life midterm evaluation database
Logically, students should follow a natural learning progression for reading skills, which 
is also explained by the gradual movement of the students toward the last stage (5. Fluent). But, 
almost a half of the second grade students are still not achieving an adequate level of fluency and 
nearly a third of them seem to be stagnated in the first two stages of learning, despite two years 
of primary schooling. Data suggest a kind of bottleneck in stage three which is consistent with a 
special study done by PRODESSA in ten participant schools in the 2018 school year. (Paz, 2018) 
This study suggests that most early grade teaching practices focus on decoding and mechanical 
writing, rather than on developing reading fluency, creative expression, or other important 
literacy building blocks. 
Nevertheless, various informants want to see a gradual improvement of reading skills. 
During focus groups, for example, some teachers indicate that there seems to be more interest in 
3.9
25.4
18.5 19.3
33.0
1. Emergent 2. Initial 3. Decoder 4. Automated 5. Fluent
2nd grade
9.8
43.6
20.0
10.2
16.3
1st grade
3.9
25.4
18.5 19.3
33.0
2nd grade
34
reading than before the project. They recognize the benefits of the training and the books 
distributed by the Learning for Life project and some materials distributed by the Ministry of 
Education.
8.1.3 (Indicator 6) Percent of students who, by the end of three and six grades of 
primary schooling, demonstrate that they can read and understand the 
meaning of grade level text
During midterm data collection, the official standardized reading tests for third and sixth 
grade were applied to 3,488 students (3
rd grade: 909 girls and 930 boys, 6
th grade: 809 girls and 
840 boys.) 
As mentioned earlier and according to the Ministry’s model, the value of this indicator 
was obtained from the percentage of children who attain achievement (levels three and four of 
the model). This indicator aggregates the percentages for third and sixth grades for a total of
19.9%, which is 3.9% higher than the midterm goal (16.0%), and 3.2% higher than one calendar 
year before.
Table 7 Historical values of indicator 6: Percentage of third and sixth grade students who demonstrate 
reading skills at grade level
Baseline value
(March 2018)
Post-test value
(September 2018)
Midterm value
(September 2019)
8.3 16.7 19.9
Source: Learning for Life evaluation database
Table 8 presents a disaggregation of the historical values of this indicator by grade and 
sex. There are no statistically significant differences between girls’ and boys’ performance. 
Data for sixth grade suggests a consistent trend of lower reading achievement at grade level
compared to the National Base Curriculum’s expected competencies. 
Table 8 Historical percentage values of indicator 6: Percentage of third and sixth grade students who 
demonstrate reading skills at grade level, by grade and sex
Grade Baseline value
(March 2018)
Post-test value
(September 2018)
Midterm value
(September 2019)
Girls Boys Girls Boys Girls Boys
3
rd 8.9 9.7 20.4 18.5 22.9 22.6
6
th 7.2 7.1 14.4 13.2 15.9 17.4
Source: Learning for Life evaluation database
The official reading tests for third and sixth grade assess diverse competences and skills 
at grade level and are therefore not comparable between grades. During test self-administration, 
students are invited to perceive, compare, analyze, relate, infer, hypothesize, generalize and 
understand grade level texts. Reading comprehension can occur at several levels and the 
instruments evaluate the students’ literal, inferential and analytical levels (Quim & Santos, 2015, 
35
p.11). Graph 4 and graph 5 display the percentage of children of each grade in each performance 
level. It can be noted that the shape of distribution of students across levels is very similar for 
both third and sixth grades.
Graph 4 Percentage of third grade students in each of the four reading performance levels
Source: Learning for Life midterm evaluation database
Graph 5 Percentage of sixth grade students in each of the four reading performance levels
Source: Learning for Life midterm evaluation database
In focus groups, some teachers recognized that the application of Kemom Ch’ab’al
methodology and tools have been very important in supporting students to improve their 
bilingual reading comprehension skills. A sixth grade teacher expressed, “I work with sixth 
graders and I have applied the Kemom Ch´ab´al reading program, with the book activities in both 
Spanish and K’iche’. Children are motivated and interested, they also have a workbook to solve 
the exercises of each lesson, now they are better prepared.” Regarding students’ reading 
performance and parental support, another teacher explained, “There are many factors that 
27.6
49.6
18.5
4.2
1. Unsatisfactory 2. Needs
improvement
3. Satisfactory 4. Excellent
3rd grade
26.3
57.0
12.1
4.5
1. Unsatisfactory 2. Needs
improvement
3. Satisfactory 4. Excellent
6th grade
36
influence children's performance in reading. Some children do not have the support of their 
parents because they are not motivated to read at home and that is reflected in the school. In 
many cases, parents have few economic resources.”
School personnel techniques indicators
8.2.1 (Indicator 10) Number of teachers/educators/teaching assistants in target 
schools who demonstrate use of new and quality teaching techniques or tools 
as a result of USDA assistance
This indicator value is measured in accordance with the results of the fidelity of 
implementation special study (carried out simultaneously during the midterm evaluation) which 
verified how closely a sample of teachers implemented Kemom Ch’ab’al and Jardín de Letras
methodologies as prescribed. The analysis model considered the dimensions of adherence, 
dose, quality, differentiation and acceptance. The results were obtained from self-administered 
questionnaires to teachers and principals, short interviews to a random sample of students and 
the guided observation of a random sample of students’ workbooks to check if exercises had 
been filled out by the student and corrected by the teacher. The model produced a measure for 
each classroom reflecting how closely the teacher follows the prescribed project methodologies. 
The spectrum for the measure were divided into four levels of implementation: “1. Not applied 
properly”, “2. Improvement required”, “3. Partial implementation” and “4. Proper 
implementaton.” 
The midterm value of this indicator (27.1%) was obtained summing the percentage of 
teachers in levels three and four. This value is below the midterm goal of approximately 38% of 
teachers demonstrating new techniques and tools as result of the project. Table 9 presents the
percentage obtained with the midterm sample and the extrapolation to the total population of 
teachers in the target area. The baseline value is only for reference as it was obtained with a 
simpler tool (the baseline tool items were also included in the midterm questionnaire for 
teachers).
Table 9 Historical values of indicator 10: percentage and number of teachers who demonstrate use of 
new techniques or tools
Baseline value
(March 2018)
Midterm value
(September 2019)
Percentage of teachers demonstrating new 
techniques or tools according to the sample
24.5 27.1
Extrapolation to the registered number of teachers 
in the target area
499
(294 women, 205 men)
535
(375 women, 160 men)
Source: Learning for Life evaluation database
A total of 487 classrooms of the sampled schools were considered to measure this 
indicator. 57.9% of teachers sampled were women and 42.1% were men, which is consistent 
with a greater proportion of female teachers in the project’s target area. Graph 6 shows the 
percentage of teachers in each level of implementation. Table 10 below displays the number of 
37
classrooms considered and graph 7 shows the percentage of teachers meeting indicator 10, 
dissagregated by grade and literacy methodology.
Graph 6 Percentage of teachers in each level of implementation of Kemom Ch’ab’al and Jardín de Letras 
methodologies
Source: Learning for Life midterm evaluation database
Graph 7 Percentage of teachers using new or improved techniques (levels 3 or 4 of implementation) by 
grade and literacy intervention
Source: Learning for Life midterm evaluation database
18.1
54.8
18.7
8.4
1. Not applied
properly
2. Improvement
required
3. Partial
implementation
4. Proper
implementaton
Teachers
52.2%
1st grade 
(Jardín de Letras)
36.2%
1st grade 
(Kemom Ch'ab'al)
17.7%
2nd grade 
(Kemom Ch'ab'al)
26.5%
3rd grade 
(Kemom Ch'ab'al)
18.6%
6th grade 
(Kemom Ch'ab'al)
38
Table 10 Number of sampled classrooms, by grade and literacy intervention
Grade Literacy intervention Number of classrooms
1 Jardín de Letras 69
1 Kemom Ch’ab’al 58
2 Kemom Ch’ab’al 130
3 Kemom Ch’ab’al 117
6 Kemom Ch’ab’al 113
TOTAL 487
Source: Learning for Life midterm evaluation database
It is interesting to note that data indicate that female teachers tend to use the 
methodologies as prescribed more than their male counterparts (see graph 8). No associated 
factors have yet been identified.
Graph 8 Comparative of indicator 10: percentage of women and men using new or improved teaching 
techniques (levels 3 or 4 of implementation)
Source: Learning for Life midterm evaluation database
Each teacher surveyed was asked to quantify her/his own level of reading comprehension 
in each language. Data shows an average of reading comprehension of 84.7% in Spanish and 
68.2% in K’iche’. Totonicapán’s context shows very balanced levels of student verbal 
bilingualism (K’iche’/Spanish) when students enter school (Mérida & Montúfar, 2016).
Teachers’ linguistic skills could be factors associated with teaching practices, and these in turn 
with the reading skills of their students.
All participant teachers in focus groups affirmed that they have applied techniques and 
tools supported by the project to improve student reading comprehension. A teacher in Santa 
María Chiquimula municipality highlited, “The program has been very useful for children, 
because each book is specialized for one grade, … stories in the books increase their interest.
Children understand the topics better because they are made for their age ...”. 
31.6
21.0
Women Men
Teachers
39
Most teachers believe that the application of Kemom Ch´ab´al tools/books is very 
feasible, because they contain real stories and encourage students to read. Referring to his 
students, a teacher in Momostenango municipality said, “We have seen that little by little they 
are adopting a culture of reading that is helping to develop themselves.” A teacher in Santa 
María Chiquimula also declared, “It has been very gratifying to see how children get excited and 
interested in reading. They are exploring, discovering and building their learning. We have 
changed from traditional ways of teaching to creative and dynamic methods.”
Teachers also noted some barriers to implementing the project’s methodologies, which 
include: time, language and lack of support at home. A female school principal in 
Momostengango municipality said, “I, as the school principal, can confirm that we all use the 
Learning for Life reading programs. They are very good. I also work with first graders. Each 
teacher has their own perspective regarding their experience in the application, although the 
worst enemy is time.” A group of teachers in San Bartolo Aguas Calientes municipality informed 
that, “According to our experience as teachers, we have observed that there are coworkers who 
do not master the K'iche´ language and its application is difficult for them. This generates a 
greater investment of time to understand and apply the methodologies directly with the children 
they serve”. Another teacher in Santa María Chiquimula expressed, “I have always wanted (and 
I hope it becomes a national level priority) to propose implementing a school for parents,
because parents are the starting point.”
8.2.2 (Indicator 12) Number of school administrators and officials in target 
schools who demonstrate use of new techniques or tools as a result of USDA 
assistance
Data collected suggest a significant increment from the baselinte (from 40.4% to 87.4% 
of sampled school principals), surpassing the midterm goal of 50%. This indicator was measured 
using the same tools as in the baseline evaluation. 
Table 11 Historical values of indicator 12: Percentage and number of school administrators who 
demonstrate use of new techniques or tools
Baseline value
(March 2018)
Midterm value
(September 2019)
Percentage of school administrators who 
demonstrate use of new techniques or tools
according to the sample
40.4 87.4
Extrapolation to the registered number of school 
administrators in the target area
134
(55 women, 79 men)
295
(127 women, 168 men)
Source: Learning for Life evaluation database
40
As shown in graph 9, there is an increment from the baseline to the midterm in the use of 
techniques and tools for school administration supported by the project. 
Graph 9 Variables used to measure use of new techniques or tools for school administration
Source: Learning for Life evaluation database
In focus groups, some school principals expressed that the educational tools supported by 
the project facilitate the teaching-learning process and foment students' interest in reading. A 
school principal in San Bartolo Aguas Calientes municipality said, “With the Learning for Life 
project it has been possible to motivate children to improve their fluency and reading 
comprehension with new techniques and tools.”
8.2.3 (Indicator 24) Percentage of DCs, TACs and PAs from the Totonicapán 
Department of Education that have increased their capacity to fulfill their 
roles and responsibilities
As of September 2019, Totonicapán Department of Education has 34 supervisors, who 
use the titles District Coordinator,Technical-Administrative Coordinator or Pedagogical Advisor. 
These individuals are charged by the Ministry of Education with accompanying and supervising
school principals in the Learning for Life target area. Nine of the DC and TAC supervisors were 
randomly selected and interviewed, and all of them reported an increase in their capacities and 
the teachers/principals they supervise because of the support of the Learning for Life project.
District Coordinators and Technical Administrative Coordinators were prioritized for interviews 
over Pedagogical Advisors due to their amount of time in their roles (Pedagogical Assistants 
were newly added in 2019 under new Ministry of Education structuring).
Table 12 Historical values of indicator 24: Percentage of DCs and TACs from the Totonicapán 
Department of Education that have increased their capacity to fulfill their roles and responsibilities
Baseline value
(March 2018)
Midterm value
(September 2019)
Reported as zero 100.0
Source: Learning for Life evaluation database
70.8
90.7 86.9
76.5
99.2 96.6
Implement project's
methodologies
With school
improvement plan
Monitoring the
improvement plan at
least monthly
Baseline Midline
41
Informants’ statements indicate a high level of acceptance and recognition of what the 
project is doing to push the educational quality in Totonicapán. The following are some textual 
declarations of the interviewed DCs and TACs:
- “Some activities with the APV project [APV is the Spanish acronym for Learning for 
Life], such as the different workshops and everything…I can say that they have certainly 
supported us to fulfill our roles.”
- “We appreciate the support given to the Technical Administrative Coordinator, school 
principals and teaching staff. We have seen favorably the activities and the processes [of 
the project], in favor of children. All teachers have accepted the project in a very good 
way and that contributes to improving the quality of education. We are also willing to 
support the project next year.”
- “… thanks to this project the number of students or the enrollment of this year has 
increased. It has been possible to increase first grade graduation, especially. The training 
sessions have been well received because of the topics covered.”
- “The institution develops training for district principals [administrators], this helps them 
to improve administrative and management processes in each establishment [school]. The 
same [principal training] facilitates coordination.”
- “… the training for teachers and Pedagogical Advisors improves accompaniment for
administrative tasks undertaken by teachers and students in each of the establishments of 
the district that I manage.” 
Student attention span, hunger scale and attendance rate
8.3.1 (Indicator 14) Percentage of students in the classrooms defined as "very 
attentive" using a scale that defines established criteria
Most teachers in focus groups gave similar opinions to those of a teacher in 
Momostenango municipality, “Now we have excellent tools and techniques, because they help 
and motivate children. Now there is more interest in reading. Children are attentive because 
classes are creative and dynamic. Also, the food given in school helps because children are well 
fed, pay attention and learn.”
Table 13 Historical values of indicator 14: Percentage of students defined as “very attentive”
Baseline value
(March 2018)
Midterm value
(September 2019)
67.1 75.2
Source: Learning for Life evaluation database
All students assessed with the reading tests were also surveyed regarding attentiveness 
and hunger. Quantitative data denote an increase in the students’ levels of attention in the 
classroom (75.2%), exceeding the midterm goal of 69.0%, an increasing 8.1 percentage points 
from baseline. 
Similarly to the baseline evaluation, data were triangulated using a scale with a set of 
criteria. Information came from students’ own perception of their attentiveness, teachers’
perception of their students’ attentiveness, and a guided classroom observation on the day of the 
42
study. Table 14 disaggregates data according to each type of informant. Students’ self￾perception of their attentiveness was lower than the other two measures. Furthermore, the 
students’ self-perception of their attentiveness seems to be very similar between boys and girls 
but decreases toward upper grades of primary school (see table 15). This could be due to better 
self-awareness or knowledge of their environments (time, school subjects, recess, family 
situations, etc.). Midterm data tools do not include items to further examine the differences 
between student and teacher perceptions, and observations.
Table 14 Percentage of students defined as “very attentive”, by informant
Informant Baseline value
(March 2018)
Midterm value
(September 2019)
Students 44.9 59.7
Teachers 75.2 82.8
Observers 81.1 83.0
Source: Learning for Life evaluation database
Table 15 Percentage of students indicating a certain attention level in class, by grade and by sex
Students’ perception of attention Grade Global
1
st 2
nd 3
rd 6
th Girls Boys Total
Pays attention all the time 81.0 79.1 50.3 42.8 61.4 58.0 59.7
Pays attention most of the time 5.3 4.2 15.6 28.6 15.2 15.0 15.1
Is easily distracted 8.2 10.1 12.2 12.6 10.3 12.0 11.1
Is always distracted 3.7 4.3 7.9 5.9 5.0 6.6 5.8
Does not know 1.9 2.4 14.0 10.2 8.1 8.5 8.3
Source: Learning for Life midterm evaluation database
Table 16 Percentage of students indicating different feelings at school, by grade and sex
Grade Percentage of students 
reporting they feel 
happy
Percentage of students 
reporting they want to 
play
Percentage of students 
reporting they want to 
participate in class
Girls Boys Total Girls Boys Total Girls Boys Total
1
st
 71.3 68.9 70.1 45.7 53.4 49.6 43.7 44.2 44.0
2
nd 68.4 68.7 68.6 46.5 48.4 47.4 52.0 46.6 49.4
3
rd 45.8 35.0 40.3 26.6 29.8 28.2 26.7 22.1 24.4
6
th 43.3 36.7 39.9 22.2 28.9 25.7 24.6 19.5 22.0
Source: Learning for Life midterm evaluation database
43
Interestingly, as students move through grade school, they seem to feel less happiness and 
playfulness. Similarly, as students move through the grades, fewer students express wanting to 
participate in class (see table 16). These findings may well be linked to decreasing attentiveness 
rates through the grades. Midterm data tools did not include items to further examine this.
As shown in table 16 and graph 10, more girls surveyed reported that they feel happy and 
willing to participate in class. Conversely, more boys indicated wanting to play in school.
Graph 10 Percentage of girls and boys indicating different feelings at school
Source: Learning for Life midterm evaluation database
8.3.2 (Indicator 15) Percentage of students in participating primary schools who 
indicate not being hungry during the school day
All parents in focus groups coincided that boys and girls are not hungry when they are at
school, because they are provided with food from the project and from the Ministry of 
Education. They said that school meals are varied and nutritious, and their children have told 
them that they like the food they receive at school. Some parents expressed that, “The children 
go to school happy because in addition to learning and playing, they are going to eat” and, “The 
food given in school has helped a lot, because now they want to go to school." Additionally, 
teachers said that school meals allow students to pay more attention during the school day. They 
indicated that, “Children no longer feel hungry when they start classes.”
Quantitative tools collected data from those who feel hunger firsthand: the students. Data 
were analyzed using the same scale as in the baseline (see table 17). Surveyed students were 
asked to report if during the school day they felt hungry but also if they experienced hunger 
related effects, such as sleepiness, illness, sadness or fatigue. The analysis included the students’ 
attentiveness and variables that indicate the moment of the day the students generally have
breakfast.
54.5
33.1 34.5
48.5
37.7
30.3
Feel happy Want to play Want to participate in
class
Girls Boys
44
Table 17 Definition of the hunger scale levels
Category Level 1 Level 2 Level 3 Level 4 
Access to breakfast
Score: 40
Had 
breakfast at 
home and at 
school
Had breakfast 
at home, but 
not at school 
Had breakfast 
at school, but 
not at home
Did not have 
breakfast at 
school, nor at 
home
Number of reported 
effects of hunger
(sleepiness, illness, 
sadness, fatigue, 
hunger)
Score: 30
1 effect 2 effects 3 effects 4 or 5 effects 
Student attention 
results 
Score: 30 
I pay 
attention all 
the time
I pay attention 
most of the 
time
I get easily 
distracted 
I am always 
distracted 
Level Low
Score 0-25
Regular
Score 26-50
Medium
Score 51-75
High
Score 76-100
Source: own elaboration
The results suggest sustained regular levels of hunger from the baseline, not reaching the 
midterm goal of 65.0% of students not being hungry (see table 18). The hunger scale levels in 
students by grade are presented in table 19.
Table 18 Historical values of indicator 15: Percentage of students who indicate not being hungry during 
the school day
Baseline value
(March 2018)
Midterm value
(September 2019)
54.4 52.1
Source: Learning for Life evaluation database
Table 19 Percentage of students in each level of the hunger scale, by grade
Hunger level Grade Global
1
st 2
nd 3
rd 6
th
1. High 0.8 0.4 7.1 5.5 4.5
2. Medium 5.0 5.2 31.1 25.2 19.3
3. Regular 18.1 18.2 24.9 32.8 24.5
4. Low 76.0 76.1 36.9 36.5 52.1
Source: Learning for Life midterm evaluation database
45
Third and sixth graders seem to be more hungry than younger children. Most of them are 
in levels two and three of the hunger scale. This could be related to their level of physical 
activity during the school day, their energy requirements and the school meal serving size. It 
should also be noted that the percentage of third (83.5%) and sixth graders (87.8%) who indicate 
to usually having breakfast at home are lower than percentages of first (96.2%) and second 
(96.6%) graders who usually have breakfast at home.
8.3.3 (Indicator 21) Average student attendance rate in USDA supported 
classrooms/schools
As mentioned in the baseline report, one of the challenges of the education system at the 
national level is obtaining daily school attendance records for each student. The Ministry of 
Education is the only body with the authority to request the records from teachers and many 
teachers are not willing to permit the project’s staff to access their records, therefore measuring a 
students’ attendance indicator had been difficult since the first phase of the Learning for Life
project. This fiscal year, the project had access to the updated version of the USDA’s indicators 
manual, which states that the MGD projects must collect data from a representative sample of 
schools at least twice per reporting period (in the case of Learning for Life, reporting to the 
donor is on a semi-annual basis).
Learning for Life project’s MEAL staff (Monitoring, Evaluation, Accountability and 
Learning) collected data from 36 schools during the first semester of the 2019 school year and 
from 29 schools in the second semester. When visiting each school, permission was sought 
from the principal and teachers to go into the classrooms to count the children present and the 
data was entered into a digital form for comparison with the number of students officially 
enrolled according to the Ministry of Education’s database (school staff have been very 
supportive of this procedure). The average attendance rate reported in the project's monitoring 
system for 2019 is 92.5% (92.6% for girls and 92.0% for boys).
The external evaluation team replicated the student count procedure on the day of the 
midterm evaluation data collection in each sampled school and obtained an average attendance 
rate of 89.3%.
The difference between the monitoring system’s value (92.5%) and the midterm 
evaluation’s value (89.3%) for this indicator could be due to the fact that the samples of schools 
were different and that during the evaluation each school was visited only once, while the project 
makes two or more visits to each school every semester. Yet, both values are very close to the 
midterm project’s goal for this indicator (90.0%).
Health and nutrition indicators
8.4.1 (Indicator 7) Number of individuals who demonstrate use of new child 
health and nutrition practices as a result of USDA assistance
The percentage of individuals demonstrating use of new child health and nutrition 
practices during the midterm evaluation doubles that measured in the baseline (25.9%). The 
midterm value of this indicator (51.9%) also surpasses the project’s midterm goal of 
approximately 32.8%.
46
Table 20 Historical values of indicator 7: Percentage and number of individuals who demonstrate use of 
new child health and nutrition practices
Baseline value
(March 2018)
Midterm value
(September 2019)
Percentage of individuals demonstrating new 
practices according to the sample
25.9 51.9
Extrapolation to the registered number of 
individuals trained by the project
4,380
(4,205 women, 175 men)
9,672
(8,953 women, 719 men)
Source: Learning for Life evaluation database
There is an important increase in most of the best practices reported or observed at 
schools the day of the study. The following tables present the percentages for both the baseline 
and midterm evaluations.
Table 21 Percentage of schools with the following health and nutrition practices observed
Practice Baseline value
(March 2018)
Midterm value
(September 
2019)
School with dietary diversity + 53.5 99.2
No trash in the classrooms 92.9 96.9
Clean classrooms 90.6 95.3
Weekly planning of menus + 78.2 93.8
Trash cans in the classrooms 87.6 93.8
Schools that distribute meals at the beginning or the 
middle of the school day +
91.2 93.8
No accumulated trash 88.8 92.2
Clean toilets/latrines 81.8 88.4
Classrooms with hygiene corner 60.0 73.6
Handwashing after using toilet/latrine + 54.7 63.6
Handwashing before eating+ 61.8 62.0
Toothbrushing + 45.3 48.8
(Practices marked with ‘+’ are positively weighted into the calculation of the indicator)
Source: Learning for Life evaluation database
47
Table 22 Percentage of school principals who report health care practices in their schools during the last 
year
Practice Baseline value
(March 2018)
Midterm value
(September 
2019)
Deworming 92.2 90.8
Vaccination 18.0 80.7
Delousing 12.5 20.2
Vitamins supplement 7.8 20.2
Vitamin A supplement 18.8 17.6
Basic free medical services 10.9 15.1
Source: Learning for Life evaluation database
The Learning for Life project works with the Ministry of Health and support them with 
logistics or transporting their staff to conduct municipal health fairs in which the Ministry 
provides the services shown in table 22.
Table 23 Percentage of schools including each food group in the school meal served the day of the 
evaluation
Group Baseline value
(March 2018)
Midterm value
(September 
2019)
Milk and milk products 79.4 95.3
Fruits 44.7 93.0
Sweets (sugar, honey) 88.2 93.0
Vegetables 44.1 90.7
Oil or cream 68.2 89.9
Eggs 34.1 89.1
Varied foods 52.4 84.5
Legumes and seeds 37.1 77.5
Roots and tubers 12.9 73.6
Meats and charcuterie 20.0 69.8
Cereals 62.4 61.2
Fish or seafood 0.0 24.8
Source: Learning for Life evaluation database
48
Regarding dietary diversity, the midterm data analysis uses the same food group
categories as in the baseline. An important increase in access to food in the sampled schools was 
observed, most likely due to the current implementation of the National School Feeding Law, 
supported by the project. The average dietary diversity score (DDS) of the school menus and 
meals served during data collection was 9.4 food groups (out of 12), showing an increment of 
four points over the baseline. Below is the disaggregation of foods found in school meals served 
the day of the evaluation (see table 23).
Volunteer parents expressed their opinions about the improvement of child health and 
nutrition practices in schools: "Before in school we didn't receive much training. There were 
mothers who didn't know how to prepare food properly and the children got sick. They didn't 
want the food anymore because their parents told them not to receive it because they got sick." 
"The trainers of the project tell us that we must put into practice in our homes what we have 
learned, we do it and now our children get sick less now." "As mothers we feel more secure, 
because now we prepare the food with a lot of love, because it is for our children, and no one 
wants them to get sick.” 
8.4.2 (Indicator 8) Number of individuals who demonstrate use of new safe food 
preparation and storage practices as a result of USDA assistance
The percentage of individuals demonstrating use of new safe food preparation and storage 
practices during the midterm evaluation (72.1%) surpasses the baseline value (67.6%) and the 
midterm goal (68.6%) (see table 24).
Table 24 Historical values of indicator 8: Percentage and number of individuals who demonstrate use of 
new safe preparation and storage practices
Baseline value
(March 2018)
Midterm value
(September 2019)
Percentage of individuals demonstrating 
new practices according to the sample
67.6 72.1
Extrapolation to the registered number of 
individuals trained by the project
11,433
(10,747 women, 686 men)
13,437
(12,438 women, 999 men)
Source: Learning for Life evaluation database
As with health and nutrition practices, there is a notable improvement in safe food 
preparation and storage practices, demonstrated in the table below.
49
Table 25 Percentage of schools with the following safe food preparation and storage practices observed
Practice Baseline value
(March 2018)
Midterm value
(September 
2019)
Washing food before it is cooked + 97.0 100.0
Washing utensils before using them to cook 95.3 100.0
Handwashing before cooking + 94.1 99.2
Wearing aprons to cook 82.9 95.3
Garbage-free kitchen 93.5 94.6
Animal-free kitchen 95.3 93.8
Wearing hairnets to cook 50.6 83.7
Commodity dispatch sheets visible + 48.8 79.1
Kardex (inventory) up to date + 45.2 75.2
Food stored on shelves or inside cabinets + 80.0 73.6
Table for estimating food calculations (per student 
population) +
45.8 72.9
Food storage facility used exclusively for food + 64.1 70.5
Evidence of rodents or insects - 52.3 24.0
(Practices marked with ‘+’ or ‘-’ are positively or negatively weighted into the calculation of the indicator)
Source: Learning for Life evaluation database
Concerning the access and type of water schools use, interviews with school principals
suggest that participants, one way or another, have access to a supply that is sufficient for school 
population. 91.5% of school principals declared that water supply meets the student populations’ 
needs. Despite these perceptions, however, at least 32.6% of schools do not have access to 
running water during the school day. Most schools continue boiling water as their main
purification method, followed by chlorination and filtration (see graph 11). Two sampled 
schools reported using other methods, of which they did not report what the other method was 
that they used.
50
Graph 11 Percentage of schools using each type of water purification
Source: Learning for Life midterm evaluation database
During focus groups, all volunteer parents confirmed that they have received training on 
how to prepare and store food, learning nutritious recipes to cook in schools that they can also 
prepare at home. Parents recognized the importance of hygiene when preparing food. Some 
mothers said that preparing food requires a lot of work, but that they are rewarded when they see 
that their children are well fed and nourished. 
Additionally, parents expressed their joy because training in health and nutrition practices 
has helped them a lot. They now pay attention to the expiration date of some foods, which they 
did not know before. A mother in Momostenango municipality said, “We have received various 
trainings, for example on how to safely store food, review the effective dates of products, water 
treatment, the importance of boiling water, hygiene in food preparation and how to avoid food 
contamination.”
Most parents expressed that children enjoy food and they are happy to see that their 
children are growing up healthy and learning. A volunteer father expressed, “Every day we make 
different meals, they are very nutritious and children like them very much. In the school there is 
a menu for each day, the weekly menu is posted on the wall; thanks to the training we receive 
now we can make many recipes and the children are not bored.” 
What the parents expressed was very consistent with the training they have received from 
the Learning for Life project on improving their knowledge and practices of food preparation,
hygiene, health and nutrition. All parents expressed that now they have knowledge about proper 
food handling, preparation and distribution according to the type of menu. 
Schools
Boiling (64.3%) Chlorination (15.5%) Filtration (18.6%) Other (1.6%)
51
9 Midterm evaluation conclusions
The external evaluator, ADOC, finds, through triangulation, that the Learning for Life 
project actions adequately support the achievement of intermediate results and strategic 
objectives. 
Regarding the key elements in the theory of change of the project: a) improved school 
attendance, b) improved child nutrition and c) improved reading instruction, the evidence 
suggests that important progress has been made towards these components. Particularly, there 
has been great progress made in improving school attendance and child nutrition, while 
improving reading instruction seems to be occurring at a slower pace.
The conclusions and recommendations below are organized in relation to the indicators 
analyzed in the findings section. This set of conclusions extends the answers for the evaluation 
questions presented earlier in this report.
Conclusions on the literacy indicators
• Data show progress in the literacy indicators for all assessed grades. In one calendar 
year, the literacy indicators rose: 4.7% in first grade, 3.3% in third grade and 2.9% in 
sixth grade (second grade data was not yet collected in baseline or post-test). All literacy 
indicator values surpassed the midterm goals.
• Teachers, principals, parents, and Ministry of Education officials alike recognize that the 
Learning for Life project has contributed to improving student literacy skills in target
schools.
• Data of where students fall in the reading prediction model (first and second grade) and 
reading performance model (third and sixth grade) show the same distribution
frequencies as the results of the Ministry of Education’s national sample from 2014 (the 
most recent year Ministry of Education data is available at the national level).
• There are higher percentages of students at the first three of the five stages of learning to 
read (1
st grade: 73.4%, 2
nd grade: 47.8%), therefore, by the end of the second grade of 
primary school almost half of the students have not yet achieved the fluency and 
comprehension skills expected by the Ministry of Education, which is a sign that 
educational practices need to be changed as several studies suggest that the initial grades 
define the future trajectory of students (Del Valle & Mirón, 2017).
• There are higher percentages of students at the two lowest of the four reading 
performance levels for the upper grades assessed (3
rd grade: 77.2%, 6
th grade: 83.3%).
Project data and Ministry of Education data both consistently show that sixth grade 
students struggle more to meet grade level reading competencies, compared with lower 
grades. This may be because the higher grades have a greater academic load and teachers 
tend to spend less time on the reading course, so it is important to promote that students 
receive more support and use more time at home and in school to practice reading.
52
Conclusions on the school personnel techniques indicators
• Teachers, school administrators, supervisors (DCs and TACs), and key personnel of the 
Departmental Ministry of Education of Totonicapán recognize and welcome the Learning 
for Life project. This has been a key factor for success to date and could be key to
achieving and sustaining results.
• The percentage of teachers who implement the project’s literacy methodologies as 
prescribed (27.1%) is lower than expected (the midterm goal is 759 teachers,
approximately 38% of total teachers in coverage area). Qualitative data indicate that 
teachers and school principals accept the methodologies and recognize their
effectiveness, though quantitative data from surveys and direct observation suggest a low 
fidelity of implementation. Various teachers expressed that some of the barriers that have 
prevented them from implementing the methodologies are lack of classroom time, 
linguistic challenges (regional variations of K’iche’ or Spanish monolingual teachers) and 
lack of support at home. Following the overall theory of change of the Learning for Life
project, it is inferred that achieving greater fidelity of implementation means improved 
literacy instruction skills that should result in children’s reading skills improvement;
therefore this is an important aspect that the project may consider for strategic decision 
making.
• Principals using new school administration techniques and tools increased by 47%
compared to the baseline, and all the educational supervisors (DCs and TACs)
interviewed stated that the project’s support has contributed to improving their skills in
fulfilling their roles and responsibilities, which is very positive as they are all key actors 
in the success and sustainability of the project’s results.
Conclusions on the student attention span, hunger scale and attendance rate
• Student attentiveness is perceived as better by teachers (82.8%) and external observers 
(83.0% - by ADOC team) than students self-perception of their attentiveness (59.7%). 
Student self-perceptions of attentiveness also fall as students move through primary 
school (1st grade: 81.0%, 2
nd grade: 79.1%, 3
rd grade: 50.3%, 6
th grade: 42.8%). The
situation could be related to emotional factors since third and sixth graders manifest in 
lower percentages of positive feelings at school, such as happiness (1st grade: 70.1%, 2
nd
grade: 68.6%, 3
rd grade: 40.3%, 6
th grade: 39.9%), desire to play (1
st grade: 49.6%, 2
nd
grade: 47.4%, 3
rd grade: 28.2%, 6
th grade: 25.7%) or desire to participate in class (1
st
grade: 44.0%, 2
nd grade: 49.4%, 3
rd grade: 24.4%, 6
th grade: 22.0%). 
• The levels of hunger sensation during the school day are similar to the baseline and 
indicate that most of students feels low (52.1%) and regular (24.5%) hunger levels. It is 
likely that the food donated by the Learning for Life project and food bought with 
Ministry of Education funds under the National School Feeding Law may help assuage 
high hunger rates. Between 3.2% and 16.5% of students do not eat breakfast at home, 
which may also suggest why student hunger sensations have not improved since baseline. 
Additionally, students received school meals at baseline, as well as at midterm, likely 
leading to similar percentages of hunger sensations. 
53
• The percentages of students feeling medium or high levels of hunger sensation during 
school day increase in third and sixth grades (1
st grade: 5.8%, 2
nd grade: 5.6%, 3
rd grade: 
38.2%, 6
th grade: 30.7%), which could be related to their activity levels, caloric needs, or 
the ration size being served.
• The headcount of students in the classroom on the day of the evaluation (89.3%) 
compared with the Ministry of Education’s enrollment data is similar to project’s 
monitoring data (92.5%) for the average school attendance rate. It is evident that current
attendance in participating schools is high.
Conclusions on the health and nutrition indicators
• There is a notable increase in health and nutrition practices applied in sampled schools. 
The most observed practices are high dietary diversity (99.2%), the absence of garbage in 
classrooms (96.9%) and clean classrooms (95.3%). While the least observed practices
(similar to the baseline) are children washing their hands after using the toilet/latrine 
(63.6%), washing their hands before eating (62.0%) and brushing their teeth after eating 
(48.8%).
• The midterm evaluation also demonstrates an increase in dietary diversity in schools 
(average DDS incremented from 5.4 in the baseline to 9.4 in the midterm – from a total 
of 12 food groups), closely related to the fact that the National School Feeding Law now 
mandates that schools cook nutritionally balanced menus (as of 2019 school year). 
• Access to water seems to be a solved problem for most schools. The majority of school 
principals (91.5%) are of the opinion that the water supply is sufficient for the school 
population. This may be a contributing factor in improved health and nutrition practices.
However, given that 32.6% of schools do not have regular access to running water during 
the school day, there may also still be a need to improve water access.
• There was an observed increase in safe food preparation and storage practices at midterm. 
The most observed practices are washing fruits and vegetables before cooking (100.0%), 
washing utensils before cooking (100.0%), and volunteer parents washing their hands 
before cooking (99.2%). The least observed practices are regular Kardex (inventory)
control (75.2%), food stored on shelves or inside cabinets (73.6%), the use of tables to 
estimate the proportions for food preparation according to the number of students 
(72.9%) and the existence of spaces used exclusively for food storage (70.5%).
10 Midterm evaluation recommendations
Based on the findings and conclusions discussed in this report, the external evaluator 
team suggests the following set of recommendations in order to plan and prioritize important 
actions that can be implemented in the remaining two years of the project.
Recommendations on the literacy indicators
1. Improve the awareness of early grade teachers through trainings and Ministry of 
Education accompaniment about early grade teachers’ key roles in helping students 
54
acquire their first literacy skills – and how those literacy skills are used as building blocks 
for the rest of their academic journeys. 
2. Improve effectiveness or increase pedagogical accompaniment for first and second grade 
teachers in the moments that they are working on phonological awareness and decoding
skills, as a first step towards improving achievement in early grade reading, without 
losing sight of activities that also support fluency and understanding.
3. If financially feasible, continue with the longitudinal study of the cohort 1st grade 2018 –
2
nd grade 2019 - 3
rd grade 2020 to contribute towards the USDA’s and DIGEDUCA’s 
body of research model that explains and predicts students reading acquisition skills.
4. Refine the project’s strategy for helping make better use of the methodologies, materials 
and support provided by the project, taking into account the identified barriers of time, 
linguistic skills and lack of support for reading at home.
5. If financially feasible, do a special study to assess why sixth grade students have lower 
reading performance, and use the findings to support a change in teacher training or 
accompaniment for the grade. 
Recommendations on the school personnel techniques indicators
6. Deepen the analysis of the variables from the fidelity of the implementation study to 
understand the causes of the low levels of implementation and construct an action plan to 
increase the level of teacher commitment. Leverage the local structure of the SINAE
(Spanish acronym for National Educational Accompaniment System) to help coach 
teachers and improve commitments.
7. Ensure that school principals and PTAs have access to resources (databases, maps) that 
can help them solicit support from local organizations or municipal structures. 
Recommendations on the student attention span, hunger scale and attendance 
rate
8. Better study the decrease in the positive feelings in third and sixth grades students, which 
could be related to factors such as their home situations, interest in their own education, 
family economic situation, emotional development or bullying.
9. Due to a greater feeling of hunger during the school day detected for older students, it is 
recommended to assess whether their nutritional needs are being covered with the school 
rations sizes being served. Based on the results, the project may consider working with 
the donor to reassess the end line goal for this indicator.
Recommendations on the health and nutrition indicators
10. According to the midterm evaluation results, it is important that the project focuses on 
increasing health practices as follows:
o During the training of participants, specifically reinforce the importance of proper 
handwashing before eating and after using the toilet/latrine.
o Supporting participant schools to maintain the practice of water purification for 
consumption and food preparation. 
55
o Seeking alliances with local actors to donate toothbrushes and toothpaste to 
increase this practice in target schools.
11. Establish in what ways the project will support schools to maintain and improve the 
dietary diversity they have already achieved, according to the guidelines of the Ministry 
of Education, and increasing schools’ local purchasing capacities, for example, through
the food fairs organized by the project.
12. Determine and implement practical ways to support standardization of food proportion 
practices during preparation in accordance to the number of students in each school.
13. Explore the possibility of certifying volunteers trained in safe food preparation. For this, 
certification or endorsement by the Ministry of Education, the INTECAP (Technical 
Institute for Training and Productivity in Guatemala) or by other related entity could be 
sought. This measure would support the empowerment and dignification of the service of 
volunteers and could help them to be considered by other organizations working on the 
same topic, in support of sustainability.
Recommendations on sustainability
14. Meet with the Ministry of Education authorities of the incoming government to explain 
the details of the project implementation and to align with their priorities.
15. Strengthen the actions of the Municipal Education Office and promote the formulation 
and implementation of municipal education policies for the other five municipalities 
covered by the project, in addition to Totonicapán municipality.
16. Explore the local geopolitical environment to determine opportunities for influence and 
sustainability with the organization of the “48 Cantones4
,” due to the strong adherence 
from communities in the target area of the project to that organization.
17. Set the deadline to begin with the implementation of the project’s sustainability plan.
18. Produce documentation that establishes what concrete actions of the project are aligned 
with government policies at different levels, what government materials have been 
adapted or could be adapted by the project and what materials the project has transferred 
or plans to transfer to the government or other organizations.
19. Strengthen the capacities of TACs and PAs under the Ministry of Education’s SINAE 
initiative to closely monitor the implementation of the project’s methodologies.
4
"48 Cantones" is an organization of community authorities of Maya-K'iche’ populations in Totonicapán 
department that promotes compliance with ancestral and community principles in public administration. The 
board of directors is integrated by community mayors from all municipalities of the department. 
(http://48cantones.org/)
56
11 References
Cotto, E. & Del Valle, M. (2017). Modelo para explicar y predecir el aprendizaje de la lectura 
para Guatemala. Guatemala: Dirección General de Evaluación e Investigación 
Educativa, Ministerio de Educación. Consulted in: 
http://www.mineduc.gob.gt/digeduca/documents/investigaciones/2018/MODELO￾ELI.pdf
Del Valle, M. & Mirón, R. (2017). Perfil del docente de primero primaria y la enseñanza de la 
lectura. Guatemala: Dirección General de Evaluación e Investigación Educativa, 
Ministerio de Educación. Consulted in: 
http://www.mineduc.gob.gt/digeduca/documents/investigaciones/2018/Perfil_docente_pri
mero_primaria.pdf
INE. (2019). PRINCIPALES RESULTADOS CENSO 2018. Guatemala: Instituto Nacional de 
Estadística. Retrieved from: https://www.censopoblacion.gt/documentacion
Mérida, V. & Montúfar, F. (2016). ¿Qué pasa en primer grado? Investigación sobre la 
lectoescritura inicial en idioma k’iche’ y español en cuatro municipios de Totonicapán.
Guatemala: PRODESSA. Consulted in:
https://www.prodessa.net/sites/default/files/Investigacion%20Primer%20Grado%20Toto
nicapan%20PRODESSA%202016.pdf
MINEDUC. (2016). PLAN Estratégico de EDUCACIÓN 2016-2020. Guatemala: Ministerio de 
Educación. Consulted in: 
https://www.mineduc.gob.gt/portal/contenido/menu_lateral/quienes_somos/politicas_edu
cativas/pdf/PLAN-EDUCACION.pdf
Paz, G. (2018). “Prácticas docentes para el desarrollo de la lectoescritura inicial”. Estudio 
descriptivo realizado en diez escuelas de aplicación del Programa Kotz’i’j tzi’b’ / Jardín 
de letras bilingüe en cinco municipios del departamento de Totonicapán durante el ciclo 
escolar 2018. Guatemala: PRODESSA. Consulted in: 
https://www.prodessa.net/sites/default/files/informe_de_investigacion_practicas_docente
s_2018.pdf
Protection of Human Subjects, 45 C.F.R. §46 (2018). Consulted in https://www.ecfr.gov/cgi￾bin/text-idx?tpl=/ecfrbrowse/Title45/45cfr46_main_02.tpl
Quim, M. y Santos, J. (2015). Informe departamental y municipal de primaria 2014. Guatemala: 
Dirección General de Evaluación e Investigación Educativa, Ministerio de Educación.
Consulted in: 
http://www.mineduc.gob.gt/digeduca/documents/informes/2014/Informe_Dep_primaria2
014.pdf
57
12 Annexes
12.1. Annex 1: Learning for Life project – Results framework
12.2. Annex 2: Learning for Life Project’s Literacy Methodologies
12.3. Annex 3: Reading skills of children assessed with ELGI
12.4. Annex 4: Evaluation indicators of the Learning for Life project
12.5. Annex 5: Quantitative sampling plan
12.6. Annex 6: Midterm evaluation consultants
58
Annex 1: Learning for Life project’s results framework
59
60
Annex 2: Learning for Life project’s literacy methodologies 
Kemom Ch'ab'al
Kemom Ch'ab'al is a reading program for the entire primary school level that promotes integral 
development of reading skills and learning of values. It implements four priority elements of the 
Educational Reform in Guatemala: 
1. Culture of peace and human rights
2. Gender equity
3. Logical thinking 
4. Bilingualism and interculturalism
The Learning for Life project distributes a reading book and a workbook for each student of the 
primary schools in the coverage area. The project also distributes a pedagogical guide to each teacher. 
Every lesson in the student book is related to the priority elements and the workbook includes 
comprehension and analysis activities. All educational materials are specially designed for each grade, 
are bilingual Spanish/K'iche', and have been carefully designed to be relevant to the Maya-K'iche' 
cultural context.
The project also conducts several workshops each year so that all teachers are trained to properly 
apply the methodologies, providing them with tools for three important moments in the reading class: 
before, during and after reading.
In addition, one of the most important elements that strengthen Kemom Ch'ab'al is a program of 
periodic personalized visits that the project's technical staff carry out to accompany the teachers, resolve 
their doubts and support them so that the methodology can be applied in the particular circumstances of 
each classroom.
Jardín de Letras (Kotz'i'j tz'ib' in K'iche')
Jardín de Letras is a bilingual program for first grade students to learn to read and write in 
K'iche' and Spanish with relevant materials and meaningful activities.
The Learning for Life project distributes a workbook to all first grade students in the groups of 
schools that receive this methodology5
. The workbook begins with line and shapes tracing to support 
fine motor skills, then introduces vowels and consonants from both languages to achieve decoding, 
forming words and sentences to support fluency and writing with special emphasis on children's analysis 
and creativity. 
Similar to what is done with Kemom Ch'ab'al, the first grade teachers implementing Jardín de 
Letras also receive several training workshops per year and regular visits for pedagogical 
accompaniment in the classroom.
5 Those schools do not receive Kemom Ch'ab'al program materials for first grade, only for second through sixth grade.
61
Annex 3: Reading skills of children assessed with ELGI
Table 26 Constructs of the ELGI test 
Construct Skills assessed
1. Oral language Listen and understand oral language
2. Knowledge of the alphabet Identify the letters of the alphabet by name 
and sound
3. Phonological awareness Identify and employ the sounds forming 
words
4. Automatic reading speed Name familiar letters quickly
5. Decoding Read whole words
6. Fluency Read with speed and accuracy
7. Reading comprehension Comprehend a read text
8. Writing Encode sounds of letters forming words 
which mean something
Source: Own elaboration with data from Cotto & Del Valle, 2017
62
Table 27 Comparison of early grade reading skills for first and second grade students assessed with ELGI
Construct Sub-test6
1
st grade 1
st grade 1st grade 2nd grade
Baseline Unit of measurement
(Mar 2018)
Post-test
(Sep 2018)
Midterm
(Sep 2019)
Midterm
(Sep 2019)
1. Oral 
language
1. Proper understanding of 
oral instructions
78.4 88.7 87.1 95.4 Percentage of students with 
proper understanding level
7.Understanding an oral 
passage
1.1 2.2 4.1 7.9 Percentage of students who 
answered all understanding 
questions correctly after 
listening a passage
2. Knowledge 
of the alphabet
2.1 Knowledge of letters 
names
9.7 23.3 26.3 38.3 Mean of the number of letter 
names that students identify
3.1 Knowledge of letters 
sounds
15.1 42.3 45.5 51.2 Mean of the number of letter 
sounds that students identify
3. 
Phonological 
awareness
4. 1 Recognition of initial 
phonemes 
34.8 74.6 80.3 91.6 Percentage of students with 
advanced level of phoneme 
knowledge
4.2 Separation of 
phonemes
5.0 32.9 44.3 64.4 Percentage of students with 
advanced level of phonemes 
separation
4. Automatic 
reading speed
2.2 Speed to name letters 17.3 33.2 33.1 52.0 Mean of the number of letter 
names correctly read per 
minute
3.2 Speed to emit the 
letters sounds
23.9 48.7 51.3 60.3 Mean of the number of letter 
sounds correctly mentioned 
per minute
5. Decoding
5.1 Reading short words 17.9 52.3 56.9 79.1 Percentage of students who 
have read all the short words 
correctly
6 The number of each subtest is mapped to a specific ELGI section. ELGI includes 14 sub-tests: 1, 2.1, 2.2, 3.1, 3.2, 4.1, 4.2, 5.1, 5.2, 6, 7, 8, 9 and 10. 
63
Construct Sub-test6
1
st grade 1
st grade 1st grade 2nd grade
Baseline Unit of measurement
(Mar 2018)
Post-test
(Sep 2018)
Midterm
(Sep 2019)
Midterm
(Sep 2019)
6. Speed to read non sense 
words
12.3 19.6 15.3 28.6 Mean of the number of non￾sense words read per minute
6. Fluency
5.2 Speed to read familiar 
words
12.4 20.4 23.0 37.2 Mean of the number of 
familiar words read per 
minute
8 Reading a passage 
fluently
17.4 31.8 32.5 54.9 Mean of the number of words 
within a passage read 
correctly per minute 
7. Reading 
comprehension
9. Reading comprehension 2.6 26.6 33.3 60.4 Mean of percentage of correct 
comprehension answers after 
reading a passage
8. Writing 10. Writing 2.2 45.1 51.4 79.1 Mean of percentage of correct 
written words
Source: Learning for Life evaluation database
64
Annex 4: Learning for Life project’s evaluation indicators 
Table 28 Evaluation indicators measurements
#
Indicator
Baseline 
value
(March 
2018)
Post-test 
value
(September 
2018)
Midterm 
value
(September 
2019)
Midterm 
goal
4 Percent of students who, by the end of one 
grade of primary schooling, demonstrate 
that they can read and understand the 
meaning of grade level text
1.7 21.9 26.6 21.0
5 Percent of students who, by the end of two 
grades of primary schooling, demonstrate 
that they can read and understand the 
meaning of grade level text
- - 52.3 31.0
6 Percent of students who, by the end of 
three and six grades of primary schooling, 
demonstrate that they can read and 
understand the meaning of grade level text
8.3 16.7 19.9 16.0
7 Number of individuals who demonstrate 
use of new child health and nutrition 
practices as a result of USDA assistance
4,380 - 9,672 6,105
8 Number of individuals who demonstrate 
use of new safe food preparation and 
storage practices as a result of USDA 
assistance
11,433 - 13,437 12,776
10 Number of teachers/educators/teaching 
assistants in target schools who 
demonstrate use of new and quality 
teaching techniques or tools as a result of 
USDA assistance
499 - 535 759
12 Number of school administrators and 
officials in target schools who demonstrate 
use of new techniques or tools as a result 
of USDA assistance
134 - 295 167
14 Percentage of students in the classrooms 
defined as "very attentive" using a scale 
that defines established criteria
67.1 - 75.2 69.0
15 Percentage of students in participating 
primary schools who indicate not being 
hungry during the school day
54.4 - 52.1 65.0
65
#
Indicator
Baseline 
value
(March 
2018)
Post-test 
value
(September 
2018)
Midterm 
value
(September 
2019)
Midterm 
goal
21 Average student attendance rate in USDA 
supported classrooms/schools
87.8 - 89.3 90.0
24 Percentage of DCs, TACs and PAs from 
the Totonicapán Department of Education 
that have increased their capacity to fulfill 
their roles and responsibilities
0 - 100.0 60.0
Source: Learning for Life evaluation database
66
Annex 5: Quantitative sampling plan
The evaluation plan for the MGD Learning for Life project includes a quantitative sample 
designed by AIR (American Institutes for Research) from the quasi-experimental model shown in the 
diagram.
The model considers an initial total of 331 schools in phase two of the project and the possible 
effects over time for 225 schools (160 + 65) that were also covered in phase one. The quasi￾experimental design was planned for the 2017 and 2018 school years (in Guatemala the school year runs 
from January to October) as part of a pilot to show which package of literacy interventions for early 
grades would work best, using data from a control group (1-A) and four intervention groups (1-B), (2-
A), (2-B) and (2-C), with two data points, the first in February 2018 (baseline) and the second in 
September 2018 (post-test).
The sample size and the initial random selection were made by AIR, the calculations used were 
based on the formula7
:
𝑀𝐷𝐸𝑆 = 𝑀√
𝜌
𝑃(1 − 𝑃) 𝐽
+
1 − 𝜌
𝑃(1 − 𝑃) 𝑛𝐽
Where:
 is the intra-cluster correlation (without covariates), that is the proportion of the overall 
variance due to variation between the clusters.
P is the proportion of clusters (for example, schools) allocated to the treatment group by the 
randomization procedure.
n is the number of individuals (for example, pupils) in each cluster.
7 More information can be found in Chapter 6 of the publication "A guide to running randomized controlled trials for 
educational researchers", available at https://www.nfer.ac.uk/media/2114/rct01.pdf
67
J is the total number of clusters randomized.
M is a multiplier based on the t-distribution.
AIR used the PowerUp!8
tool to make the calculations of the number of schools (J in the 
formula) for the baseline (February 2018) and post-test (September 2018) sample. The chosen model was 
a two-level cluster random assignment of schools with treatment assignment at level 2 (level 2: schools, level 
1: individuals) with a confidence interval of 95% (alpha = 5%), statistical power of 80% (beta = 20%) and
intra-cluster correlation of  = 0.125 (based on previous studies from the Ministry of Education in similar 
rural bilingual contexts and corroborated after data collection).
AIR also determined that comparisons between groups to detect effects should be 1-A versus 1-
B, 2-A versus 2-B and 2-B versus 2-C. After entering the data in the PowerUp! tool for each 
comparison, the result for the total baseline sample size was 179 schools.
Following the 2018 evaluation results and because the Ministry of Education distributed its own 
teacher literacy toolkit, the project decided to reduce one intervention by 2019, keeping only the Kemom 
Ch'ab'al and Jardín de Letras methodologies. Thus, groups 2-B and 2-C were merged into one, now 
called only 2-B. In addition, group 1-A started receiving Jardín de Letras, since according to the 
original evaluation plan, the schools in the control group would start receiving the successful reading 
interventions after 2018.
For the midterm evaluation in 2019, the external evaluation consultant team used the same 
procedures and tools as AIR. Due to the merger of the 2-B and 2-C group, only two comparisons are 
necessary in early grades for this evaluation: 1-A versus 1-B and 2-A versus 2-B. By entering the data 
into the PowerUp! tool with the same parameters as the baseline sample, the total sample size is 135 
schools. Please note that the sample size for the midterm evaluation is less than the sample size for the 
baseline evaluation due to the decrease in the number of comparisons between groups.
The following table shows the MDES (minimum detectable effect sizes) for the variables used to 
calculate literacy indicators in the midterm evaluation9
and to run the hypothesis testing to answer the 
research questions about literacy interventions:
Grade Comparison between groups MDES
1
st and 2
nd 1-A vs 1-B 0.305 SD
1
st and 2nd 2-A vs 2-B 0.306 SD
3
rd 1-A vs 1-B 0.296 SD
3
rd (1-A + 1-B) vs (2-A + 2-B) 0.203 SD
6
th 1-A vs 1-B 0.296 SD
6
th (1-A + 1-B) vs (2-A + 2-B) 0.203 SD
8 More information can be found at https://cran.r-project.org/web/packages/PowerUpR/readme/README.html
9
Like in the baseline evaluation, the cluster effect was taken into account only for indicators with data coming from 
students. The rest of the indicators, by their nature, were analyzed with simple random sampling procedures because they 
originated from informants such as principals, teachers and volunteers.
68
Annex 6: Midterm evaluation consultants
• Friné Paz, MBA. Research team leader. More than 10 years of experience in community 
organization, quantitative and qualitative program evaluation. Experience leading Title II 
program evaluations. Knowledge and experience in standardized reading tests application and 
analysis of results.
• María André Solares, MBA. Field team manager. Experience in leading and coordinating field 
teams for quality data collection, including logistical oversight and supervision. 
• Edgar Ruano, BA. Field coordinator. Experience in community development, team 
coordination for data collection. Responsible for enumerator training on qualitative and 
quantitative data collection tools. 
• Eber Barrera. Field coordinator. Technical supervisor of data collection teams, enumerator 
trainer for standardized reading tests.