i   FINAL  EVALUATION  REPORT   FOOD  FOR  EDUCATION  “LEARNING  FOR  LIFE”   GUATEMALA   With  the  support  of:   Catholic  Relief  Services  (CRS)  Guatemala  and  USDA,  through  an  evaluation  grant  to:   Boston  College  School  of  Social  Work   November 2017 2 Final Evaluation REPORT - STRENGTHENING READING AND SCHOOL FEEDING IN FY 2013 MCGOVERN-DOLE “LEARNING FOR LIFE” Project Program: McGovern-Dole International Food for Education and Child Nutrition Agreement Number: FFE-520-2013/029-00 Project Duration: 2013 – 2016 Implemented by: Catholic Relief Services Submitted by: Thomas M. Crea, PhD Antonia Diaz---Valdes, MSW Kerri Evans, MSW Paola Campos, MSW DISCLAIMER: This evaluation was conducted by an independent third party firm. The views expressed in this report do not necessarily reflect the views of the United States Department of Agriculture or the United States Government. Accessibility Note: An accessible version of this report can be made available upon request by emailing FAS.monitoring.evaluation@usda.gov. i   TABLE  OF  CONTENTS   ACRONYMS ..................................................................................................................................................iv LIST  OF  TABLES .............................................................................................................................................  v   LIST  OF  FIGURES ...........................................................................................................................................  v   EXECUTIVE  SUMMARY.................................................................................................................................  vi A. RESULTS ..........................................................................................................................................vii B. RECOMMENDATIONS .................................................................................................................... viii FINAL  EVALUATION  REPORT........................................................................................................................  1   A. Introduction  and  Background..........................................................................................................  1   i. Introduction.................................................................................................................................  1   ii. Background..................................................................................................................................  1   iii. Key  Evaluation  Questions ............................................................................................................  1   B. Methods ..........................................................................................................................................  1   i. Study  Sites  and Target Population...............................................................................................  1   ii. Sample .........................................................................................................................................  2   C. Measures.........................................................................................................................................  2   i. Reading  Comprehension  Test......................................................................................................  3   ii. School  Environment  Measure......................................................................................................  3   iii. Focus  Groups  for  Parent  Volunteers............................................................................................  3   iv. Interviews  with  Principals,  Teachers,  School  Administrators,  PTA  members,  parents,  CTAs,  and project  staff .........................................................................................................................................  3   v. Structured  Classroom  Observations  and  Child Attentiveness.....................................................  3   RESULTS .......................................................................................................................................................  3   A. CHILD  LITERACY  (STRATEGIC  OBJECTIVE) ........................................................................................  3   B. QUALITY  OF  LITERACY  INSTRUCTION  (RESULT  1.1) .........................................................................  6   i. Teachers’  Use  of  New  Techniques  and  Tools  (Result  1.1) ...........................................................  6   ii. Teacher  Attendance  (Result  1.1.1) ..............................................................................................  7   iii. Establishing  School  Improvement  Plans  (Result  1.1.1)................................................................  8   iv. School  Materials  (Result  1.1.3b)..................................................................................................  8   v. Kemom  Ch’ab’al  Methodology  (Result  1.1.4)..............................................................................  9   vi. Reading  Techniques  (Result  1.1.3)............................................................................................. 11 vii. Administrator’s  Use  of  New/Quality  Techniques  (Result  1.1.5) ............................................ 12 viii. Trainings  (Result  1.1.5) .......................................................................................................... 12 ii C. STUDENT  ATTENTIVENESS  (RESULT  1.2)........................................................................................ 13 i. Student  Attentiveness  (Result  1.2). ........................................................................................... 13 ii. Student  Hunger  and  Food  Intake  (Result  1.2.1) ........................................................................ 15 D. STUDENT  ATTENDANCE  (RESULT  1.3)............................................................................................ 16 iii. Food  Preparation ....................................................................................................................... 19 iv. Student  Participation  in  Extra-­‐Curricular  Activities  (Result  1.3.1) ............................................. 20 v. Number  of  School  Government  Established  (Result  1.3.1a) ..................................................... 21 vi. School  Gardens.......................................................................................................................... 21 vii. School  Days  Missed  by  Students  Due  to  Illness  (Result  1.3.2)............................................... 22 viii. Improvements  or  construction  of  the  school  (Result  1.3.3). ................................................. 23 ix. School  Enrollment  (Result  1.3.4) ............................................................................................... 24 x. Importance  of  Education  (Result  1.3.5)..................................................................................... 24 E. GENERAL  OVERVIEW  OF  RESULTS  (RESULT  1.4)............................................................................ 25 i. Perception  and  Sustainability  of  the  Program  (Result  1.4)........................................................ 25 ii. Increased  Capacity  of  Government  Institutions  (Result  1.4.1).................................................. 27 iii. Increased  Government  Support  (Result  1.4.3) .......................................................................... 28 iv. Increased  Engagement  of  Local  and  Community  Members  (Result  1.4.4)................................ 29 DISCUSSION ............................................................................................................................................... 33 A. Relevance....................................................................................................................................... 33 B. Effectiveness.................................................................................................................................. 33 C. Efficiency........................................................................................................................................ 34 D. Impact............................................................................................................................................ 34 E. Sustainability .................................................................................................................................  35   RECOMMENDATIONS ................................................................................................................................ 36 1) Conduct  a  barriers  analysis  to  explore  why  literacy  for 6th  graders  has  plateaued...................... 36 2) Conduct  a  barriers  analysis  to  examine  reasons  behind  decreasing  school  enrollment  and attendance.  Also  examine  why  teachers’  reports  of  attentiveness,  and  observed  attentiveness,  are significantly  different............................................................................................................................. 36 3) Conduct  a  barriers  analysis  to  examine  reasons  behind  low  teacher  attendance  rates. .............. 36 4) Vary  the  food  provided  by  the  program,  and  ensure  that  food  is  culturally  appropriate. ............ 37 5) Reinforce  the  benefits  of  education  with  parents  and  improve  future  measurement  of  this indicator................................................................................................................................................. 37 6) Strengthen  and  coordinate  training  on  bilingual  education  for  teachers ..................................... 37 iii 7) Consider  expanding  the  Spaces  to Grow  program,  and  highlight  potential  benefits  to  directors and  teachers.......................................................................................................................................... 38 8) Articulate  what  sustainability  looks  like  for  the  FFE  program,  communicate  this  with beneficiaries,  and  enhance  beneficiaries’  capacities  to  operate  independently. .................................  38 REFERENCES............................................................................................................................................... 39 APPENDICES............................................................................................................................................... 40 iv ACRONYMS   APCC   Asociación  Proyecto  Conrado  de  la  Cruz CRS   Catholic  Relief  Services  -­‐  United  States  Conference  of  Catholic  Bishops CTA   Coordinador  Técnico  Administrativo  /  Technical  and  Administrative  Coordinator  for  the Ministry  of  Education DIDEDUC   Dirección  Departamental  de  Educación  de  Guatemala  /  Departmental  Office  for  the Ministry  of  Education DIGEFOCE   Dirección  General  de  Fortalecimiento  de  la  Comunidad  Educativa  /  Department  of Educational  Capacity  Building  for  the  Ministry  of  Education EBI Educación  Bilingüe  Intercultural  /  Intercultural  Bilingual  Education   DIGEBI   Dirección  General  de  Educación  Bilingüe  Intercultural  /  Bilingual  Education  Office FFE   Food  for  Education IRB   Institutional  Review  Board MINEDUC   Ministerio  de  Educación  /  Ministry  of  Education PRODESSA   Proyecto  de  Desarrollo  Santiago PTA   Parent  Teacher  Association USDA   United  States  Department  of  Agriculture WFP World  Food  Program v   LIST  OF  TABLES   Table  1.  Sample ...........................................................................................................................................vi Table  2.  Simplified  Indicator  Matrix  Comparing  Baseline,  Midterm,  and  Final  Evaluation  Results.............ix Table  3.  Steps  Encouraged  by  Staff  Before  and  After  Reading....................................................................  6   Table  4.  Type  of  Materials  Received  (n=112) ..............................................................................................  8   Table  5.  Observation  of  Classroom  Conditions............................................................................................  9   Table  6.  Steps  of  the  Kemom  Ch'ab'al  Methodology  Remembered  (n=110) ............................................ 10 Table  7.  Strengths  identified by  teachers  regarding  the  Kemom  Ch’ab’al  Methodology.......................... 11 Table  8.  Weaknesses  identified  by  teachers  regarding  the  Kemom  Ch’ab’al  Methodology ..................... 11 Table  9.  Amount  of  Principal  Oversight  Given  to  Teachers....................................................................... 12 Table  10.  Principals  and  Teachers’  training  participation.......................................................................... 12 Table  11.  Principals’  and  Teachers’  training  participation......................................................................... 13 Table  12.  Observation  of  Children  Attentiveness...................................................................................... 14 Table  13.  Factors  Influencing  Attentiveness  (n=116) ................................................................................ 15 Table  14.  Food  Given  by  Grade  Level ........................................................................................................ 15 Table  15.  Food  children  ate  for  breakfast.................................................................................................. 15 Table  16.  Percentage  of  Children  Attending  School  that  had  no  Breakfast .............................................. 16 Table  17:  Factors  that  Helped  Increase  Student  Attendance.................................................................... 17 Table  18.  Reasons  of  Why  Students  Missed  Classes.................................................................................. 18 Table  19.  People  who  participated  in  Food  Preparation  according  to  principals  (n=113) ........................ 19 Table  20.  Problems  with  Food  Preparation ............................................................................................... 19 Table  21.  Students’  Participation  in  Extracurricular  Activities................................................................... 20 Table  22.  Kinds  of  Illness  that  caused  Students  to  Miss  Classes................................................................ 22 Table  23.  Parents  Responses  to  Why  Education  is  Important................................................................... 24 Table  24.  DIDEDUC  Training  Topics  Attended  by  CTAs  (n=8  CTAs)............................................................ 27 Table  25.  DIGEFOCE  Training  Topics  Attended  by  CTAs ............................................................................ 28 Table  26.  Trainings  of  School  Boards/Council  (n=110) .............................................................................. 28 Table  27.  Issues  and  Recommendations  Related  to  Bilingual  Education .................................................. 37 LIST  OF  FIGURES   Figure  1.  Sixth  Grade  Reading  Assessment  Scores  at  Baseline,  Midterm  and  Final ....................................  4   Figure  2. Sixth  Grade  Reading  Assessment  Scores  (Boys  vs.  Girls)  at  Final  Evaluation................................  4   Figure  3. Third  Grade  Reading  Assessment  Scores  at  Baseline,  Midterm  and  Final....................................  5   Figure  4. Third  Grade  Reading  Assessment  Scores  (Boys  vs.  Girls)  at  Final  Evaluation ...............................  5   Figure  5.  Average  Teacher  Attendance  in  Number  of  Days  by  Grade  Level ................................................  7   Figure  6.  Month  in  which  the  schools  received  Kemom  Ch’ab’al  materials.............................................. 10 vi EXECUTIVE  SUMMARY   From  2013-­‐2016,  Catholic  Relief  Services-­‐United  States  Conference  of  Catholic  Bishops  (CRS)  in   Guatemala  implemented  Phase I  of  the  Food  for  Education  (FFE)  project,  using  donated  commodities  and   funds  provided  by  the  United  States  Department  of  Agriculture  (USDA)  to  accomplish  the  following:  (1)   Improve  quality  of  literacy  instruction;  (2)  Improve  student  attentiveness;  and  (3)  Improve  student   attendance.  The  overall  project  is  focused  on  the  strategic  objective  to  improve  the literacy  of  school-­‐age   children  in  four  municipalities  in  the  department  of  Totonicapán. The  purpose  of  the  final  evaluation  is  to  assess  and  analyze  the  performance  of  the  project  by comparing   Final  Evaluation  findings  with  the  results  of  the  baseline  and  midterm  evaluation  studies  in order  to   provide  lessons  learned  and  recommendations  for  USDA,  program  participants  and  other  key stakeholders  for  future  food  assistance  and  capacity  building  programs. The  guiding questions  for  the  Final  Evaluation  are  the  following: 1. Do  project  activities  have  a  correlation  with  improved  student  performance? 2. Has  the  quality  of  literacy  instruction  improved? 3. Has  student  attentiveness  improved? 4. Have  students  improved  their  attendance? 5. Are  attendance  and  educational  performance  correlated? The  sample  was  drawn  from  the  population  of  N=221  schools  in  the  Department  of  Totonicapán,  using  a   simple  random  sample  of  n=120  schools (see  Table  1).  Within  selected  schools,  all  3rd and  6th grade   students  were  given  an  assessment  of  reading  comprehension  and  perceptions  of  the  school   environment.  All  directors,  teachers  (3rd grade,  6th grade,  or  multigrade),  and  parent  volunteers  were   interviewed about  their  perceptions  of  the  effectiveness  and  implementation  of  FFE.  Parent  focus  groups   (n=20)  were  conducted  at  randomly  selected  schools  across  municipalities. All  CTAs  (n=9)  in  the   Department  of  Totonicapán  were  interviewed,  as  were  project  staff  with  CRS  and  implementing  partners   (n=13).  Two  representatives  of  USDA  were  also  interviewed  by  phone  on  their  perspectives  of  FFE   effectiveness  and  implementation  in  relation  to  the  CRS FFE project in  Guatemala.   Table  1.  Sample Municipality Schools Students Principals Teachers Parents Parent   Focus   Groups   CTAs Project   Staff   3 rd Gr. 6 th Gr. Momostenango 70 1,356 1,083 68 139 265 14 6   13 San  Andres   Xecul 9   249 186 9   26 37 0   1   Santa  Lucia  la   Reforma 24 264 157 22 36 94 4   1   San  Bartolo   Aguas   Calientes 17 165 164 17 29 82 2   1   Total   120   2,034   1,590   116   230   478   20   9   13   vii A. RESULTS Results  of  the  Strategic  Objective  and  Results  Indicators  are  outlined  below.    The  Results  Indicator  matrix   can  be  located  in  Appendix  A.  Analysis  of  findings  related  to  the  cross-­‐cutting  questions  of  Relevance,   Effectiveness,  Impact,  Efficiency,  and  Sustainability, can be  found  in  the  Discussion  section  of  the  report. Reading  Performance.  Results  were  positive  for  children’s  reading  performance (Strategic  Objective),  but   6th graders  failed  to  meet  their  final  targets  for  increased  literacy  (+10  points  above  baseline).  At  final   evaluation,  girls  scored  33.87%  correct  on  the  reading  comprehension  assessment,  compared  with   25.19%  at  baseline  and  31.20%  at  midterm.  Boys  scored  32.98%  at  final  evaluation,  compared  with   24.31%  at  baseline  and  31.19%  at  midterm. By  contrast,  3rd grade  students  demonstrated  great   improvement  in  literacy  at  final  evaluation,  compared  with  baseline  and  midterm  scores.  At  final   evaluation,  boys  scored  31.30%  on  reading  comprehension,  compared  to  8.88%  at  baseline  and  18.87%   at  midterm.  Girls  scored  29.76%  at  final  evaluation,  compared  to  9.19%  at  baseline  and  17.59%  at   midterm.  These  scores  are  well  above  the  target  of  10+  points  above  baseline. Quality  of  Literacy  Instruction.  Almost  all  teachers (98.70%)  used  one  or  more  new  and  innovative teaching  techniques  (Indicators  1.1,  1.1.4),  an increase  from  the  midterm  evaluation  and  above  the  final   target  of  80%.  Teacher  attendance  at  final  evaluation  (83.3%)  increased  slightly  from  the  midterm  (81.7%)   but  fell  short  of  the  target  of  90%  (Indicator  1.1.1).  All  schools  (100%)  have  been  provided  literacy   instructional  materials,  meeting  the  final  target. Nearly  all  principals  have  used  new  and  quality  teaching   techniques  and  tools  (93.10%),  well  above  the  final  target  of  75%  (Indicator  1.1.5).     Student  Attentiveness.  Standardized  assessments  of  classroom  observations  resulted  in  an  average  score   of  52.07%  attentiveness.  In  order  from  highest  to  lowest,  the  attentiveness  score  per  municipality  ranged   as  follows:  San  Bartolo  Aguas  Calientes  (57.88%),  Santa  Lucia  la  Reforma  (55.18%),  Momostenango   (51.61%),  and  San  Andres  Xecul  (44.25%). However,  the  majority  of  teachers  (85.0%)  identified  their   students  as  being  attentive  (Indicator  1.2),  exceeding  the  final  target  of  80%.  Most  principals  rated   students  as  being  less  hungry  during  the  school  day  because  of  the  program  (92.59%)  (Indicators  1.2.1,   1.2.1a). Student  Attendance.  Student  attendance  increased  significantly  for  girls  but  dropped  substantially  for   boys.  For  3rd grade  girls,  90.6%  attended  classes  at  least  80%  of  the  school  year,  an  increase  from  75.1%   at  midterm  and  well  above  the  final  target  of  85%  (Indicator  1.3a).  For  6th grade  girls,  87.10%  attended   classes  at  least  80%  of  the  school  year,  an  increase  from  73.0%  at  midterm  and  above  the  final  target  of   85%  (Indicator  1.3a).  By  contrast,  3rd grade  boys  attendance  dropped  to  68.52%  from  80.8%  at  midterm,   well  below  the  target  of  85%  (Indicator  1.3).  Sixth  grade  boys’  attendance  dropped  to  65.21%  from  76.8%   at  midterm,  also  well  below  the  target  of  85%  (Indicator  1.3).  The  number  of  children  who  participated  in   extra-­‐curricular  activities  by  the  final  evaluation  totaled  3,594,  meeting the  final  target  of  3,505. The  average  number  of  days  missed  by  students  because  of  illness  dropped  to  less  than  one  day  (M=0.57   day),  a  significant  decrease  from  4.1  days  at  midterm  and  exceeding  the  final  target  of  2  days  lost  to   illness  (Indicator  1.3.2).  The  number  of  educational  facilities  constructed  or  rehabilitated  totaled  416   schools,  exceeding  the  final  target  of  312  schools  (Indicator  1.3.3).  Boys  saw  a  significant  decline  in   enrollment,  with  an  8.9%  drop  in  enrollment  in  2016  (16,790)  compared  to  baseline  in  2014  (18,437).  This   pattern  held  true  for  girls  as  well,  with  a  7.7%  drop  in  enrollment  in  2016  (16,394)  compared  to  baseline   in  2014  (17,765).   viii Few  parents  were  able  to  identify  three  benefits  of  primary  education  for  their  children.  At  final   evaluation,  only  16.81%  identified  three  reasons,  compared  to  35.9%  at  midterm  and  well  below  the   target  of  80.0% (Indicator  1.3.5).  We  speculate  that  these  results  reflect  a  problem  in  measurement  of   this  indicator,  rather  than  a  reflection  of  parents’  commitment  to  their  children’s  education.   Most  schools  in  the  sample  (115  out  of  n=120;  95.83%)  had  a  school  government;  thus  it  is  estimated  that   212  schools  out  of  the  population  of  221  FFE  schools  have  developed  a  school  government.  While  this   number  is  short  of  the  final  target  of  221  schools,  not  all  221  schools  have  cooperated  with  CRS  to   implement  FFE  fully,  because  they  did  not  participate  in  the  school  feeding  program;  thus,  the  estimate  of   212  schools  with  a  school  government  likely  represents  all  schools  fully  implementing  FFE.  A  majority  of   principals  (74.55%)  stated  that  school  governments  have  been  strengthened,  and  increase  from  67.5%  at   midterm. Most  schools  (107  of  115  schools;  93.0%)  have  a  school  improvement  plan;  thus,  we  estimate   that  216  of  221  schools  have  a  school  improvement  plan,  an  increase  from  142  at  midterm,  but  short  of   the  final  target  of 221. Increased  Capacity  of  Government  Officials. About  half  (50.43%)  of  disbursements  arrived  on  time   (Indicator  1.4.3),  an  increase  since  midterm  (21.4%)  and  greatly  exceeding  the expectations  of  the final   target  of  10.0%.  215  schools  are  estimated  to  have  a  PTA  that  contributes  to  their  schools  (Indicator   1.4.4),  the  same  as  the  midterm,  and  likely  reflecting  all  participating  FFE  schools.  At  final  evaluation,  26   MINEDUC  staff  have  been  trained  (Indicator  1.4.1),  an  increase  from  14  at  midterm  and  exceeding  the   final  target  of  20.   B. RECOMMENDATIONS     Brief  recommendations  are  provided  below.  Please  see  the  Recommendations  section  of  the  report  for   the  full  explanation  of  each  recommendation. (1) Conduct  a  barriers  analysis  to  explore  why  literacy  for  6th  graders  has  plateaued. (2) Conduct  a  barriers  analysis  to  examine  reasons  behind  decreasing  school  enrollment  and attendance. Also  examine  why  teachers’  reports  of  attentiveness,  and  observed  attentiveness,   are  significantly  different. (3) Conduct  a  barriers  analysis  to examine  reasons  behind  low  teacher  attendance  rates.   (4) Vary  the  food  provided  by  the  program,  and  ensure  that  food  is  culturally  appropriate. (5) Reinforce  the  benefits  of  education  with  parents  and  improve  future  measurement  of  this   indicator. (6) Strengthen  and  coordinate  training  on  bilingual  education for  teachers (7) Consider  expanding  the  Spaces  to  Grow  program,  and  highlight  potential  benefits  to  directors   and  teachers (8) Articulate  what  sustainability  looks  like  for  the  FFE  program,  communicate  this  with  beneficiaries,   and  enhance  beneficiaries’  capacities  to  operate  independently. Below  is  a  simplified  Results  Indicator  Matrix  comparing  baseline,  midterm,  and  final  evaluation   indicators.  Please  see  Appendix  A  for  the  full  set  of  information  on  indicators. ix Table  2.  Simplified  Indicator  Matrix  Comparing  Baseline,  Midterm,  and  Final  Evaluation  Results Result   Indicator   Baseline   Midterm   Final   Target   Final   Evaluation   Level  of  Student  Reading  Performance   Improved  literacy   of  school-­‐aged   children Percent  of  girls  literate  (%  correct) 25.19% 31.20% +10% 33.87% Percent  of  boys  literate  (%  correct) 24.31% 31.19% +10% 32.98% Has  the  Quality  of  Literacy  Instruction  Improved?   Improved  quality  of   literacy  instruction 1.1  Percentage  of  teachers  in  target   schools  who  demonstrate  use  of  new   and  quality  teaching  techniques  or  tools1 64.9% 87.5% 80% 98.70% More  consistent   teacher  attendance 1.1.1  Percentage  of  teachers  at  target   schools  who  attend  and  teach  school  at   least  90%  of  scheduled  school  days 88.8% 81.7%   90% 83.30% Improved  literacy   instructional   materials 1.1.3b  Percentage  of  classrooms  in   target  schools  with  literacy  instructional   materials  (textbooks,  workbooks,  etc.)   sufficient  for  effective  instruction 75.4% 100% 100% 100.00% Increased    skills   and  knowledge  of teachers 1.1.4  Percent  of  teachers  in  target   schools  who  demonstrate  use  of  new   and  quality  teaching  techniques  or  tools 0.0% 87.5% 70% 98.70% Increased  skills  and   knowledge  of   school   administrators 1.1.5  Percent  of  school  administrators  in   target  schools  who  demonstrate  use  of   new  and  quality  teaching  techniques  or   tools 0.0% 87.0%   75% 93.10% Has  Student  Attentiveness  Improved?   Improved  students´   attentiveness 1.2  Percent  of  students  in  classrooms   identified  as  attentive  by  their  teachers 27.0% 82.2% 80% 85.15% Reduced  short   term  hunger 1.2.1  Percent  of  3rd  grade  students  in   target  schools  who  indicate  they  are  not   hungry  during  the  school  day 15.8% 86.9% 60% -­‐-­‐ 1.2.1a  Percent  of  6th  grade  students  in   target  schools  who  indicate  they  are  not   hungry during  the  school  day 8.2% 73.2% 60% -­‐-­‐ Have  students  improved  their  attendance?   Improved  student   attendance 1.3  Percent  of  boys  regularly  (80%)   attending  USDA  supported   classrooms/schools 60.0% 80.8% 85% 68.5%   76.8% 85% 65.2% 1.3a  Percent  of  girls  regularly  (80%)   attending  USDA  supported   classrooms/schools 71.7% 75.1% 85% 90.6%   73.0% 85% 87.10%   1 This  indicator  was  calculated  as  the  percentage  of  teachers  who  used  one  re  more  of  the  following  new  and   innovative  tools  and/or  techniques:  read  the  title,  look  the  images,  look  at  the  structure/type  of  reading,  exercises   to  identify  the  theme,  exercises  to  identify  the  main  ideas,  exercises  to  identify  the  main  events,  exercises  to  identify   the  main  characters,  students  take  turns  to  read  aloud,  and  choral  reading. x Result   Indicator   Baseline   Midterm     Final   Target   Final   Evaluation     Level  of  Student  Reading  Performance  (Continuation)   Increased   economic  and   cultural  incentives   (or  decreased   disincentives) 1.3.1  Number  of  school  children  who   participate  in  extra-­‐curricular  school   activities 1,820 2,400 3,505 3,594 Reduced  health   related  absences 1.3.2  Average  number  of  school  days   missed  by  students  due  to  illness 2.5  days     67.6% from  1-­‐5   days 4.1  days 66.3%  of   students   missed   school   from  1-­‐5   days  per   year 2 days 0.57 days 63.06% of   students   missed   school  from   0-­‐5  days  per   year Improved  school   infrastructure 1.3.3  Number  of  educational  facilities   (school  buildings,  classrooms,  and   latrines)  rehabilitated  /  constructed  as   a  result  of  USDA  assistance 0 244   312 416   Increased  student   enrollment 1.3.4  Percent  increase  in  boys  enrolled   in  school  as  a  result  of  USDA  assistance 0% -­‐  1.0%   18,245   (2015) 4% 18,4372 -­‐8.9%   16,790 (2016) 1.3.4a  Percentage  increase  in  girls   enrolled  in  school  as  a  result  of  USDA   assistance 0% -­‐  2.6%   17,307   (2015) 4%   17,7653 -­‐7.7%   16,394 (2016) Increased   community   understanding  of   the  benefits  of   education 1.3.5  Percentage  of  parents  in  target   communities  who  can  name  at  least   three  benefits  of  primary  education 96.4% 35.9% 80% 16.81%   School   governments   Number  of  school  governments   established 213 162 221 212 Number  of  school  governments   strengthened 67.5% 74.55% Improvement   plans Number  of  existing  school   improvement  plans 142 142 221 216                                                                                                                           2 This  number  was  obtained  from  the  2014  baseline  report. 3 This  number  was  obtained  from  the  2014  baseline  report. xi Result   Indicator   Baseline   Midterm     Final   Target   Final   Evaluation     Increased  Capacity  of  Government  Officials   Increased   government   support 1.4.3  Percentage  of  MINEDUC   disbursements  (financial  or  material)   which  arrive  on  time  to  the  appropriate   institution. 13.1% 21.4%   10% 50.4%   Increased   engagement  of   local  organizations   and  community   groups 1.4.4  Number  of  PTAs  contributing  to   their  school  as  a  result  of  USDA   assistance 69 215 221 215   1.4.4a  Number  of  PTAs  that  have   developed  a  school  improvement  plan 18 62 133 212 Increased  capacity   of  government   organizations 1.4.1  Number  of  DIDEDUC  and   DIGEFOCE  staff  who  have  increased  their   capacity  to  implement  their  roles  and   responsibilities 0 14 20 26   1 FINAL  EVALUATION  REPORT FOOD  FOR  EDUCATION  “LEARNING  FOR  LIFE”  GUATEMALA   A. Introduction  and  Background i. Introduction   Food  for  Education  (FFE)  programs  are  developed  to  promote  nutrition,  and  health  of  school  age  children,   and  indirectly  to promote  economic  growth  and  social  equity  (Alderman  &  Bundy,  2012;  Martorell,  1999).   Evidence  suggests  that  FFE  programs  alleviate  immediate  hunger,  increase  enrollment  in  schools,  and   attendance  among  school  aged  children  (Alderman  &  Bundy,  2012;  Cheung  &  Perrota,  2010;  Alderman,   Gilligand,  &  Lehrer,  2012;  WFP,  2007).    In  addition,  FFE  programs  promote  the  investment  in  human   capital,  which  in  long-­‐term  help  to  reduce  poverty,  and  promote  community  and  economic  growth   (Alderman  &  Bundy,  2012;  Martorell,  1999).  The  primary  purpose  of  the  McGovern-­‐Dole  Food  for   Education  initiative,  one  of  the  most  prominent  school  feeding  programs  implemented  globally,  is  to   increase  childhood  literacy  (USDA,  2016). ii. Background   From  FY2013-­‐2016,  funded  by  USDA,  CRS  implemented  Food  for  Education in  Guatemala  focused  on  the   strategic  objective  to  improve  the  literacy  of  school-­‐age  children  in  four  municipalities  in  the  department   of  Totonicapán.   The  purpose  of  the  final  evaluation  is  to  assess  and  analyze  the  performance  of the  project  by  comparing   Final   Evaluation   findings   with   the   results   of   the   baseline   and   midterm   evaluation   studies   in   order   to   provide   lessons   learned   and   recommendations   for   USDA,   program   participants   and   other   key   stakeholders  for  future  food  assistance and  capacity  building  programs. iii. Key  Evaluation  Questions   The  questions  guiding  this  evaluation  are  as  follows: 1. Do  project  activities  have  a  correlation  with  improved  student  performance? 2. Has  the  quality  of  literacy  instruction  improved? 3. Has  student  attentiveness  improved? 4. Have  students  improved  their  attendance?   5. Are  attendance  and  educational  performance  correlated?     B. Methods i. Study  Sites  and  Target  Population The  four  Municipalities  included  in  this  project  are  the  following:   2 1. Momostenango; 2. San  Andrés  Xecul; 3. Santa  Lucía  de  la  Reforma;   4. San  Bartolo  Aguas  Calientes. The   project   focused   on   several   key   actors,   including   elementary   students,   teachers,   administrators,   volunteer   parents,   and   government   officials.   The   target   population   comprises   those   actors   in   the   120   school  across  4  municipalities.   ii.  Sample         From  the  population  of  schools (N=221),  the  study  drew  a  simple  random  sample  of  n=120  schools.  This   number  gives  95%  confidence  in  the  representativeness  of  responses,  with  a  margin  of  error  of  ±6.1.  This   number  of   schools  is   the   same  as  used  in   the  midterm  evaluation,  and   substantially  more   than   schools   used  in  the  baseline  evaluation  (n=69).  In  each  school,  all  available  3rd and  6th grade  students  were  given  a   reading  comprehension  assessment.  Sixth  graders  also  completed  a  measure  of  their  perceptions  of  the   school   environment (see   measures   section).   Each   school   principal   was   interviewed   using   a   structured   protocol.   All   available   teachers (3rd grade,   6th grade,   and   multigrade) and   parents  were   also   invited   to   participate   in   structured   interviews   and   all   available   parent   volunteers were invited   to   participate in a focus group. The sample   numbers are provided in Table 1.   Among   parents, 80%   (n=379)   were   female,   and   98%   (n=465)  identified  as  being Maya  K’iche’.  The  mean  of  K’iche’  language  skills  are  as  follows:   the  level  of   speaking,   and   level   of   understanding   are   about   84.00   (SD=25.16),   and   82.41   (SD=28.24),   whereas   the   reading   skills,   and   writing   skills   are   much   lower,   about   20.43   (SD=28.31),   and   19.80   (SD=28.96)   respectively.   In   addition,   52.40%   (n=120)   of   teachers were   females.   On   average   teachers   had   been teachers   in   that   school for   about   11   years   (SD=6.51,   and   4.68   months   (SD=3.06).   91.70%   (n=210)   of   teachers  were  Maya  K’iche’.  The  mean  of  K’iche’  language  skills  are  as  follows:  the  level  of  speaking,  and   level   of   understanding   are   about   64.48  (SD=32.13),   and   75.76  (SD=28.62),   compared   with   the   reading   skills,  and  writing  skills  at  about  58.49  (SD=31.89),  and  56.39  (SD=31.08)   respectively.  57.76%  (n=67)  of   principals are   male.   Principals   had   worked   at   the   school   an   average   of   10   years   (SD=6.59)   and   4.71   months  (SD=3.27).   93.10%   self-­‐identify   as   Maya   K’iche’,   6.03%   as   Ladino   and   0.86%   as   other.     Finally,   55.56%  (n=5)  of  CTAs are  male  and  the  mean  length  of  time  that  CTAs  had  worked  with  the  school  was  7   (SD=3.20) years  and  3.33  (SD=3.32)  months.   All  participants  underwent  a  process  of  informed  consent  prior  to  their  participation.  Parents  who  were   present  on  the  day  of  data  collection  provided  parental  consent  for  their  children’s  participation;  parental   consent  was  waived  for   parents  who  were   not   physically   present.  Data   collection   supervisors   read   the   child  assent  form   to  all   children   prior   to   reading   comprehension  assessments,  and   children  were  given   the   opportunity   to   ask   questions.   All   protocols   were   reviewed   and   approved   by   the   Boston   College   Institutional  Review  Board  (IRB). C. Measures   Multiple  data  collection  tools  were  employed  for  data  collection (see  Appendix  D).  These  include the   following:   3 i.  Reading  Comprehension  Test   The  Reading  Comprehension  Test  is  a  30-­‐question  reading  assessment  test  developed  in  Spanish  by   PRODESSA.  This  test  was  administered  to  all  3rd and  6th grade  students  present  on  the  day  of  data   collection  in  the  120  schools  in  the  sample. ii. School  Environment  Measure   A  School  Environment  Measure  was  given  to  all  6th grade  students  to  assess  their  perceptions  of  school  in   terms  of  relationships  with  teachers  and  peers.  This  measure  is  a  combination  of  standardized  measures,   including  the  Negative  School  Environment  Index  (Moon  &  Alarid,  2015);  the  School  Connectedness   Measure  (Zullig  et  al.,  2015);  and  the  School  Climate  Measure  -­‐ Revised  (Zullig  et  al.,  2015).  Results  of  this   measure  are  presented  in  Appendix  B. iii. Focus  Groups  for  Parent  Volunteers     Parent  Volunteers  at  each  school  were  invited  to  participate  in  a  focus  group.  These  focus  groups   followed  a  semi-­‐structured  protocol,  and  assessed  volunteers’  perceptions  of  the  program’s  relevance,   effectiveness,  and  adequacy.   iv. Interviews  with  Principals,  Teachers,  School  Administrators,  PTA   members,  parents,  CTAs,  and  project  staff   Structured  interviews  were  completed  at  schools  with  all  3rd and  6th grade  teachers  (or  multigrade   teachers  in  unitary  school),  and  parent  volunteers,  related  to  their  perspectives  on  the  multiple  aspects  of   FFE  implementation.  Directors  were  also  interviewed  and  asked  similar  questions,  as  well  as  about   student  and  teacher  attendance.  In  addition,  all  CTAs  were  interviewed  in  their  respective  offices  across   the  Department,  and  project  staff  were  also  interviewed  about  their  perspectives  on  the  strengths  and   challenges  of  FFE  implementation.  Two  USDA  staff  were  also  contacted  by  phone  to  discuss  their   perspectives. v. Structured  Classroom  Observations  and  Child  Attentiveness     A  protocol  was  developed  to  assist  enumerators  to  observe  teachers  in  the  classroom  context,  and   establish  the  extent  to  which  teachers  have  improved  their  teaching  skills  using  the  Kemom  Ch´ab´al   methodology,  as  well  as  the  extent  to  which  children  were  paying  attention  in  class  during  the  period  of   observation.         RESULTS     A. CHILD  LITERACY  (STRATEGIC  OBJECTIVE) For  6th  grade  boys  and  girls,  literacy  rates  increased  substantially  compared  with  baseline  results,  but   only  marginally  compared  with  midterm  results  (see  Figure  1).  At  final  evaluation,  girls  scored  33.87%   correct  on  the  reading  comprehension  assessment,  compared  with  25.19%  at  baseline  and  31.20%  at   midterm.  Boys  scored  32.98%  at  final  evaluation,  compared  with  24.31%  at  baseline  and  31.19%  at   midterm.  This  indicator  thus  falls  short  of  the  goal  of  +10  points  added  to  literacy  scores  since  baseline.   Of  note,  these  rates  are  calculated  as  the  average percentage  of correct  responses  out  of  100.0%  for  each   4 subgroup  (see  Figure  1),  and  so  represent the  amount  of  answers  correctly  identified  by  children  in  the   assessment.    This  methodology  was  also  used  in  the  baseline  and  midterm  evaluations.  These  numbers   do  not  reflect  the  actual  literacy  rates  of  children,  which  should  be  calculated  as  the  percent  of  children   who  score  60%  or  higher  on  reading  comprehension assessments.  Using  this  standard,  n=80  children  in   3rd grade  achieved  literacy  (3.9% of  n=2,034),  and  n=46 6th grade  children  achieved  literacy  (2.9%  of   n=1,590).   Figure  1. Sixth  Grade  Reading  Assessment  Scores  at  Baseline,  Midterm  and  Final Some   variation   emerged   on   the   percentage   of   correct   answers   on   the   reading   comprehension   assessment   subscales.  Overall,   6th   grade   students   performed   highest   on   the   “Identifying   Themes”   and   “Cause  and  Effect”  subscales,  and  lowest  on  the  “Main  Idea”,  “Prediction”,  and  “Interpretation”  subscales   (see   Figure   2).   Girls   scored   higher   than   boys   to   a   statistically   significant   degree   on   the   “Comparison”   (p<.05)  and  “Fact  vs.  Opinion”  (p<.01)  subscales. Figure  2. Sixth  Grade  Reading  Assessment  Scores  (Boys  vs.  Girls)  at  Final  Evaluation *  p<.05; **  p<.01 25.19%   31.20%   33.87%   24.31%   31.19%   32.98%   0.00%   5.00%   10.00%   15.00%   20.00%   25.00%   30.00%   35.00%   40.00%   BASELINE   MIDTERM   FINAL    Girls    Boys   5 Third  grade  students  demonstrated  great  improvement  in  literacy  at  final  evaluation,  compared  with   baseline  and  midterm  scores  (see  Figure  3).  At  final  evaluation,  boys  scored  31.30%  on  reading   comprehension,  compared  to  8.88%  at  baseline  and  18.87%  at  midterm.  Girls  scored  29.76%  at  final   evaluation,  compared  to  9.19%  at  baseline  and  17.59%  at  midterm.  These  scores  are  well  above  the   target  of  10+  points  above  baseline. Figure  3.  Third  Grade  Reading  Assessment  Scores  at  Baseline,  Midterm  and  Final As  with  6th  graders,  subscales  for  reading  comprehension  also  varied  for  3rd  graders  (see  Figure  4).  Third   graders  scored  highest  on  the  “Prediction”,  “Character  and  Environment”,  and  “Logic”  subscales,  and   lowest  on  the  “Main  Ideas”  and  “New  Words”  subscales.  Boys  scored  higher  than  girls  to  a  statistically   significant  degree  on  the  “New  Words”  subscale  (p<.01;  see  Figure  4). Figure  4. Third  Grade  Reading  Assessment  Scores  (Boys  vs.  Girls)  at  Final  Evaluation **  p<.01   9.19%   17.59%   29.76%   8.88%   18.87%   31.30%   0.00%   5.00%   10.00%   15.00%   20.00%   25.00%   30.00%   35.00%   BASELINE   MIDTERM   FINAL    Girls    Boys   6 Parents  believed  that   food  provision  at   school  helped  their   children   academically:   43.82%  (n=209)  of   parents  stated  they   strongly  agreed  that   their  children  have   learned  more  due   the  provision  of  food   at  school,  and   51.57%  (n=246)  of   parents  stated  they   agreed  that  school   food  helped  their   children  to  learn   more. B. QUALITY  OF  LITERACY  INSTRUCTION  (RESULT  1.1) i. Teachers’  Use  of  New  Techniques  and  Tools  (Result  1.1)   The  Kemom  Ch’ab’al  Methodology  encourages  reading  the  title;  looking  at  images;  and  looking  at  the   structure  before  reading.  According  to  principals  and  teachers,  children  are  encouraged  to  do  a  series  of   steps  both  before  and  after  they  read.  Oral  exercises  designed  to  identify  the  theme;  main  ideas;  main   events;  and  main  characters  are  common  after  reading.  Table  3 below  shows  how  often  these  are   incorporated  principals  and  teachers.   Table  3.  Steps  Encouraged  by  Staff  Before  and  After  Reading   Steps   Principals   (n=112)   Teachers   (n=230)   Number  (%) Number  (%) Before  Reading Read  the  Title 93  (83.04%) 183  (71.91%) Looking  at  images 101  (90.18%) 213  (93.01%) Looking  at  the  structure/type  of  reading 84  (75.00%) 174  (75.98%) After  the  reading Exercises  to  identify  the  theme 86  (76.79%) 160  (69.87%) Exercises  to  identify  the  main  ideas 87  (77.68%) 161  (70.31%) Exercises  to  identify  the  main  events 75  (66.96%) 138  (60.26%) Exercises  to  identify  the  main  characters 92  (82.14%) 155  (67.69%) Almost  all  teachers  (98.70%)  stated  they  used  one  or  more  new  and  innovative  teaching  techniques,  such   as  teaching  techniques  other than  silent  individual  reading,  or  one  or  more  activities  encourage  through   the  Kemom  Ch’ab’al  methodology.   7 Principals  were  asked  how  many  of  the  teachers  in  the  school  lead  activities  in  K’iche’  and  Spanish.  Only   11.21%  of  principals  said  that  no  teachers  within  the  school  lead  activities  in  both  languages.  Across  all   schools,  the  number  of  teachers  doing  activities  in  both  K’iche’  and  Spanish  ranged  from  0  to  16  with  an   average  of  4.06  teachers  (SD=3.39).  The  principals  said  that  on  average,  schools  were  spending  145   minutes  (SD=136.85)  dedicated  to  reading  in  Castilian/Spanish  (mode  150  minutes,  range  =  [30,  1200].   On  the  other  hand,  principals  reported  that  teachers  spent  an  average  of  101  minutes  per  week   dedicated  to  reading  K'iché  (SD=  91.63,  mode  30,  range  =  [0,675]).   Teachers  stated  they  read  weekly  in   Spanish  on  average  130.98  minutes  (SD=89.67,  range= [5,750]),  and  they  read  weekly  in  K’iche’  on   average  91.47  minutes  (SD=73.11,  range=  [0,  450]).  About  half,  54.35%  (n=125)  of  teachers  said  they   have  enough  training  about  bilingual  teaching.     ii. Teacher  Attendance  (Result  1.1.1)   On  average,  principals  reported  that  teachers  across  all  grades  attended  school  155  days  during  the  2016   school  year (see  Figure  5).  When  looking  at  teachers  by  grade  level,  multigrade  level  teachers  attended   the  least  frequently  with  an  average  of  126  days,  and  the  attendance  rates  improved  with  each  year  in   school  so  that  the  sixth grade  teachers  attended  an  average  of  161  days  in  the  2016  school  year.   However,  it  should  be  noted  that  of  the  116  Principals,  42.24%  (n=36)  had  incomplete  records  of  teacher   attendance,  and  31.03%  (n=36)  had  no  record.  According  to  the  principals,  87  of  116  (75.00%)  felt  that   the  school  improvement  plans  have  influenced  teacher  attendance,  and  75%  (n=87)  of  principals  felt  that   the  intervention  had  helped  to  improve  teacher  attendance, whereas  25%  said  the  intervention  did  not   help  in  this  regard. The  chart  below  shows  the  average of  teachers’  attendance  in  number  of  days  per   grade,  during  2016.   Following  the  program  intervention,  there  was  an  average  of  5.65  (83.30%)  teachers   per  school  with  an  attendance  rate  of  at  least  90%.  In  addition,  75%  (n=315)  of  parents  stated  the   teachers  attended4 at  least  90%  of  the  school  days.  According  to  the  parents,  the  teachers  attended  on   average  87.02  (SD=19.59)  school  days,  whereas  principals  reported  that  teachers  on  average  attended   156.02  (SD=33.14)  days  of  school.   Figure  5.  Average  Teacher  Attendance in  Number  of  Days by  Grade  Level                                                                                                                         4 Parents’  perception  of  teachers’  attendance.  School  registers  of  teachers’  attendance  were  not  available. 8 iii. Establishing  School  Improvement  Plans  (Result  1.1.1)   According  to  principals,  most  schools  (107  of  115   schools;  93.0%)  have  a  school  improvement plan.   CRS   monitoring  data  that  show  216  schools  have  a  signed   agreement  to  implement  school  improvement  plans.   Most  principals  (96%) said  that  the  Educational   Council/Board  of  Education  participated  in  the   development  of  the  school  improvement  plan.  A   slightly  smaller  number,  91%  of  principals  (n=97)  said   that  they  can  explain  the  improvement  plan,  and  91%   (n=95)  of  principals  said  the  plan  has  been   implemented  within  their  school.  Of  the  107   principals  who  responded, 40.19%  (n=43)  said  the   plan  was  very  useful.  The  majority,  57.01%  (n=61)  said   the  plan  was  useful.  Only  a  few  felt  it  was  somewhat   useful  or  not  useful  (n=1,  0.93%; and  n=2,  1.87%, respectively).               iv. School  Materials  (Result  1.1.3b)   CRS  monitoring  data  show  that  school  materials  have  been  disseminated  to  100%  of  target  schools,   meeting  the  final  target  indicator.  According  to  the  principals,  a  variety  of  tangible  materials  were   received  by  the  schools.  The  number  of  classrooms  per  school  which  had  materials  ranged  from  0  to  20   classrooms.  The  average  number  of  classrooms  per  school  was  5.06  (SD=  4.05).  The  table  below  explains   the  types  of  materials  received.   Table  4.  Type  of  Materials  Received  (N=112) Type  of  Material Number  (%) MINEDUC  books 108  (96.43%) Work  books 67  (59.82%) Pencils,  crayons,  markers,  watercolors,  erasers,  paint 86  (76.79%) Glue,  scissors,  masking  tape 88  (78.57%) Poster,  cardboard,  construction  paper 88  (78.57%) Books  about  Kemom  Ch’ab’al  methodology 103  (91.96%) Teaching  Portfolios 55  (49.11%) The  materials  are  being  actively  used  by  110  (98.21%)  of  principals.  The  quality  of  the  materials  was  rated   as  very  good  by  19.64%  (n=22),  good  by  62%  (n=55.36),  and  regular  by  25%  (n=28).  96.52%  of  teachers   (n=222)  stated  the  material  was  useful,  and  90.43%  (n=208)  of  teachers  stated  the  material  was  written   in  K’iche’  and  Spanish.  In  addition,  97.39%  (n=224)  of  teachers  stated  the  students  were  interested  in  the   9 material,  and  32.45%  (n=73)  of  teachers  stated  the  material  was adequate  or  more  than  enough  for  all   children.  On  the  other  hand,  only  50%  of  principals  felt  the  amount  of  materials  was  adequate  or  more   than  enough  for  all  children. Data  collectors  completed  structured  observations  of  randomly  selected  classrooms  (see  Table  5).  The   items  were  ranked  from  1  (strongly  disagree)  to  5  (strongly  agree).  The  items  that  scored  highest  were   Lighting  and  Ventilation  are  Adequate  with  a  score  of  4.06  (SD=0.87),  and  the  Blackboard  is  in  Good   Condition  with  a  score  of  4.02  (SD=0.81).  The  item  with  the  lowest  score  was  the  Desks  are  Cluttered  with   a  score  of  2.10  (SD=1.01).  Overall  classroom  condition  scores  indicate  a  mean  of  60.47%  (SD=16.67%).  For   the  calculation  of  the  overall  score,  included  the  reverse  coding  of  the  item  referred  to  “the  desk  are   cluttered”.  Therefore,  higher  scores  indicated  better  classroom  conditions  (“(R)”  refers  to  reverse-­‐scored   items). Table  5.  Observation  of  Classroom  Conditions Item   N   Range   Mean  (SD)   %  Agree  or   strongly  agree   The  desks  are  placed  in  a  circle  or  U 227 1-­‐5 2.80  (1.35) 39.65% The  desks  are  cluttered (R) 226 1-­‐5 2.10  (1.01) 13.12% The  desks  are  in  good  condition 226 1-­‐5 3.70  (1.06) 73.07% The  classroom  is  clean  and  tidy 227 1-­‐5 3.87  (1.00) 79.74% Lighting  and  ventilation  are  adequate 225 1-­‐5 4.06  (0.87) 87.56% The  blackboard  is  in  good  condition 223 1-­‐5 4.02  (0.81) 88.34% The  teacher  has  classroom  materials  for   the  children 226 1-­‐5 3.95  (0.88) 84.95% Overall  Classroom  Conditions   (Standardized)   218   0-­‐100%   60.47%  (16.67%)       v. Kemom  Ch’ab’al  Methodology  (Result  1.1.4) Almost  all  principals  (96.55%,  n=112)  have  implemented  the  Kemon  Ch'ab'al  Methodology  into  the   schools.  Across  112  schools,  Principals  reported  that  only  one  teacher  in  the  school  was  applying  the   Kemom  Ch'ab'al  Methodology.  Almost  all  (97.38%)  teachers  (n=223)  stated  they  used  the  Kemom   Ch’ab’al  methodology.  In  addition,  teachers  stated  that  they  have  attended  on  average  to  3.34  (SD=2.01,   range=[0,15])  trainings  of  the  Kemom  Ch’ab’al  methodology.  The  total  number  of  teachers  in  a  school   varied  widely  but  there  was  a  mean  of  4.98  teachers  (range=[1,22],  SD=  3.94).  Most  commonly,  schools   had  3  teachers  (n=23,  20.35%),  and  over  78%  of  schools  had  6  or  fewer teachers. According  to  the  principals,  34.48%  (n=40)  felt  that  the  Kemom  Ch’ab’al  Methodology  was  very  effective   in  improving  students’  reading  comprehension.  Another  59.48%  (n=69)  felt  it  was  effective,  and  only   6.03%  (n=7)  felt  it  was  somewhat  effective.  Similarly,  34.48%  (n=40)  of  principals  felt  the  school  feeding   program  was  very  effective  in  improving  students’  reading  comprehension.  Another  61.21%  (n=71)  felt   school  feeding  was  effective,  and  4.31%  (n=5)  said  somewhat  effective  in  improving  student’s  reading   comprehension. The  Kemom  Ch’ab’al  materials  were  distributed  to  schools  throughout  2016.  Below  is  a  chart  detailing   when  materials  were  received,  the  highest  months  were  March  with  47%  and  February  with  another  29%   of  school  receiving  materials  at  that  time. 10 Figure  6.  Month  in  which  the  schools  received  Kemom  Ch’ab’al  materials Various  study  participants  were  asked  about  the  steps  in  the  Kemom  Ch'ab'al  methodology (see  Table  6).   Below  is  a  table  detailing  their  roles,  and  how  many  could  name  each  step  of  the  methodology.  For   principals,  almost  all  could  remember  to  scan  (89%,  n=98)  and  the  least  could  remember  exercises  about   the  reading  (68%,  n=75).  For  teachers,  the  most  people  could  remember  to  scan  (86.05%,  n=186)  and  the   least  could  remember  review  the  exercise  (69.30%,  n=149). Table  6.  Steps  of  the  Kemom Ch'ab'al  Methodology  Remembered  (N=110) Steps   Principals   Teachers   Number   (%)   Number  (%)   Scan the  reading 98  (89.09%) 186  (86.05%) Read the  text 90  (81.82%) 183  (85.12%) Exercises  about  the  reading 75  (68.18%) 167  (77.67%) Review  the  exercises 76  (69.09%) 149  (69.30%) Reflect  on  the  content  of  the  reading 81  (73.64%) 151  (70.23%) Teachers  were  asked  to  identify  strengths  and  weaknesses  of  the  Kemom  Ch’ab’al  methodology.  311   codes  were  extracted  from  225  comments  made  regarding  strengths,  and  241  codes  were  extracted  from   230  comments  made  regarding  weaknesses.  As  shown  in  table  7,  the  main  strengths  were related  to  the   learning  process  with  30.55%  (95)  of  comments,  referred  to  the  development  of  critical  thinking,   improvement  of  reading  comprehension,  improvement  of  writing,  among  others.   11 Table 7.  Strengths  identified  by  teachers  regarding  the  Kemom  Ch’ab’al  Methodology Cluster   n  (%)   Learning  (critical  thinking,  improve  reading  comprehension,  increase  vocabulary,   improve  writing,  among  other) 95 (30.55%) General  benefits  and  strengths  of  the  methodology  (socialization,  improve   communication  skills,  it's  interesting  for  children,  the  structure  of  the  methodology,   promotes  child  participation,  it  can  be  applied  to  other  courses,  among  others) 58  (18.65%) Content  (is  adequate  to  the  context  and  ages,  it's  understandable,  the  topics  of  the   readings,  among  others) 67  (21.54%) Language  (is  bilingual,  children  learn  Spanish,  promotes  K'iche') 49  (15.76%) Tools  and  techniques  (there  are  several  types, the  material  is  given  in  time,  enough  for   all  children,  and  pretty) 42  (13.50%) As  shown  in  table  8,  there  were  two  main  weaknesses/suggestions.  First,  61.83%  (149)  of  comments   were  related  to  the  variation  in  the  K’iche’  language  of  the  books  and  the  K’iche’  people  in  the  community   spoke.  Second,  13.69%  (33)  of  comments  were  related  to  the  length of  the  readings.  In  addition,  about   5%  of  comments  referred  to  no weaknesses.  Other  weaknesses  mentioned  by  teachers (19.48%  of   comments)  were:  teachers did  not  speak  K’iche’  and  did  not  felt  comfortable  teaching  it (7.88%),  the   content  or  instructions  were  hard  to  understand  (3.73%),  among  others  (see  Table  8).   Table  8. Weaknesses  identified  by  teachers  regarding  the  Kemom  Ch’ab’al  Methodology Cluster   n  (%)   Variation  in  K’iche’  language  between  the  methodology  material  and  the  community 149  (61.83%) Length  of  the  readings 33  (13.69%) The  teachers  did  not  speak K’iche’ and  did  not  felt  comfortable  teaching  it 19  (7.88%) No weaknesses 12  (4.98%) The  methodology  is  not  adequate  to  the  community/school  context   10  (4.15%) Content  and  instructions  are  hard  to  understand 9  (3.73%) The  material  is  insufficient 3  (1.24%) Other  (such  as  too  much  K’iche’  or  too  much  Spanish,  or  students  are  not  interested,   Michelle  lectures,  among  others) 6  (2.49%) Most  teachers  agreed  68.30%  (n=153)  or  strongly  agreed  21.80%  (n=49)  that  the  content  of  the  Kemom   Ch’ab’al  methodology  is  relevant  to  the  cultural  context. vi. Reading  Techniques  (Result  1.1.3)   Teaching  techniques  other  than  silent  individual  reading  are  included  in  many  classrooms.  According  to   principals,  76.19  %  of  teachers  (n=80)  apply  techniques  of  choral  reading  in  the  classroom  and  70.48%   (n=74)  of  teacher  utilize  taking  turns  to  read  aloud.  Similarly,  76.42%  (n=175)  of  teachers  stated  they   applied  techniques  of  choral  reading  in  the  classroom,  and  71.05%  (SD=162)  of  teachers  stated  they   applied  the  technique  of  taking  turns  to  read  aloud.   12 vii. Administrator’s  Use  of  New/Quality  Techniques  (Result  1.1.5)   93.10%   (n=108)   of   the   principals   themselves   have   utilized   new   and   innovative   teaching   techniques.   According   to   the  principals,  all   teachers  have  improved   their   teaching  skills  because  of   the  Learning  for   Life   Program (see   Table  9).   16.38%  (n=19)   of   principals   said   their   skills   improved   some,   14.66%  (n=17)   said   they   improved   a   lot,   and   67.24%   (n=78)   said   they   improved   very   much.   Principals   supervise   the   teachers   in   the   areas   of   Kemom   Ch’ab’al   Methodology,   Classroom   Management   and   New   Teaching   Techniques.  Below  is  a table detailing  the  amount  of  time  spent  supervising  teachers  in  each  area.  Across   all  topical  areas,  most principals  only  gave  some  or  a  little  supervision  to  teachers.     Table  9. Amount  of  Principal  Oversight  Given  to  Teachers Topic  of  Supervision   n   Very   much   A  lot   Some   A  little   Very  little   Number   (%)   Number   (%)   Number   (%)   Number   (%)   Number   (%)   Kemom  Ch’ab’al 112 11  (9.82%) 15  (13.29%) 23  (20.54%) 46  (41.07%) 17  (15.18%) Classroom  Management 111 10  (9.01%) 22  (19.82%) 29  (26.13%) 34  (30.63%) 16  (14.41%) New  Teaching   Techniques 111 11  (9.91%) 23  (20.72%) 26  (23.42%) 36  (32.43%) 15  (13.51%) viii. Trainings  (Result  1.1.5)   Almost  all  96%  of  principals  were  trained  in  the  Kemom  Ch’ab’al  Methodology.  Very  few  principals  (46%)   in  comparison  to  school  boards  (79%)  were  trained  in  Funds  Management.  99.13%  of  teachers  (n=228)   were  trained.  The  most  common  training  attended  was  about  the  Kemom  Ch’ab’al  methodology  (97.39%   of  teachers  were  trained),  followed  by  school  improvement  plan’s  trainings  (67.69%  of  teachers),  and  the   topic  in  which  fewer  teachers  were  trained  was  health  and  nutrition  (46.29%  of  teachers)  (see  Table  10). Table  10. Principals  and  Teachers’  training  participation   Training  topics   Principals   Teachers   Number  (%)   Number  (%)   Kemom  Ch’ab’al  methodology 111  (95.69%) 224  (97.39%) Good  Practices  in  Health  and  Nutrition 83  (71.55%) 106  (46.29%) Importance  of  Education 92  (79.31%) 151  (65.94%) Funds  Management/  Resources  Management 53  (45.69%) 107  (49.77%) School  Improvement  Plan 98  (84.48%) 155  (67.69%) Teachers  identified  eleven  main  areas  in  which  they  wanted  to  be  trained.  Most  comments  were  related   to  Reading  Techniques  and  Strategies,  K’iche’,  and  Mathematics,  among  other  less  mentioned  topics (see   Table  11).   13 Table  11.  Principals’ and  Teachers’  training  participation   Area/Topic   Number  (%)   Reading  techniques  and  strategies 41  (13.49%) K’iche’ 39  (12.83%) Mathematics 27  (8.88%) Administration  and  planning 19  (6.25%) Learning  and  teaching  techniques  and  strategies 19  (6.25%) Kemom  Ch'ab'al'  methodology 18  (6.00%) Health  and  Nutrition 17  (5.60%) Use  and  creation  of  didactic  material 15  (5.00%) Educational  Psychology 14  (4.60%) Civic  Education 11  (3.62%) L1-­‐L3 11  (3.62%) Other  (social  sciences,  physical  education,  arts  and  music,   evaluation  techniques,  first  aid,  etc.) 73  (24.00%) Total   304  (100%)   C. STUDENT  ATTENTIVENESS  (RESULT  1.2)   i. Student  Attentiveness  (Result  1.2).   Teachers  stated  that  on  average  85.15%  (SD=15.40,  range=[0,  100])  of  children  were  more  attentive  due   to  the  program  implementation.  In  addition,  96.09%  (n=221)  of  teachers  observed  specific  behavioral   changes  due  to  the  program  implementation.   Data  collection  supervisors  were  instructed  to  conduct  classroom  observations  for  randomly  selected   teachers  in  schools.  A  series  of  5-­‐point  Likert-­‐scale  items  were  used  to  assess  aspects  of  students’   attentiveness  in  the  classroom  (see  Table  12). The  items  were  ranked  from  1  (strongly  disagree)  to  5   (strongly  agree). The  average  attentiveness  score  was  52.07%  (SD=18.39%),  the  highest  score  was   obtained  by  the  municipality  of  San  Bartolo  Aguas  Calientes  (Mean=57.88%,  SD=18.11%),  followed  by   Santa  Lucía  De  La  Reforma  (Mean=55.18%,  SD=18.50%).  The  lowest  score  was  obtained  by  the   Municipality  of  San  Andrés  Xecul  (Mean=44.25%,  SD=14.17%) (see  Table  12). Reversed  scored  items  are   denoted  with  “(R)”.  Thus,  higher  scores  indicate better  attentiveness.   Most  97.49%  (n=465)  parents  stated  that  receiving  food  at  school  helped  their  children  to  be  more   attentive.  47.80%  of  parents  (n=228)  stated  they  agree  that  school  food  helped  increase  attention,  and   49.69%  (n=237)  of  parents  stated  they  strongly  agreed  with  that  sentence.  The  factors  that  helped   increase  attention  that  most  teachers  identified  was  school  food  (80.79%  of  teachers),  and  the  factor  that   least  teachers  identified  was  teachers’  attendance  (58.95%  of  teachers).  According  to  focus  groups,   parents  believe  that  the  food  provided  at  the  school  has  helped  children  to  be  more  attentive  and   focused  while  in  class  (14  comments,  45%  of  schools),  and  not  worried  about  being  hungry.  [“The   advantage  is  that  the  children  are  not  hungry,  they  always  have  something  to  eat  so  when  they  come  to   school,  they  do  not  feel  sleepy  and  come  to  focus  on  learning”  – “La  ventaja  es  que  los  niños no  tienen   hambre,  siempre  tienen  algo  que  comer  para  que  tambien  cuando  vengan  a  la  escuela  no  se  vengan  a   dormir,  siempre  vienen  a  dedicarse…  atentos,  a  aprender  a  leer,  a  escribir”]. 14 Table  12.  Observation  of  Children  Attentiveness Item   N   Range   Mean   (SD)   %  Agree  or   strongly  agree   Children  are  attentive  for  long  periods  of  time  (about   15  minutes) 226 1-­‐5 3.67  (0.91) 73.45% The   children   seem   distracted   during   class   activities (R) 223 1-­‐5 2.54  (1.03) 22.87% The  children  follow  the  teacher's  instructions 225 1-­‐5 4.02  (0.72) 88.44% The  children  are  sleepy  in  the  classroom (R) 225 1-­‐5 2.09  (0.80) 7.11% The  children  have  difficulty  concentrating (R) 226 1-­‐5 3.82  (0.83) 78.32% The  children  focus  on  a  task  until  they  complete  it 224 1-­‐5 3.97  (0.68) 86.16% The   children   work   independently   (take   materials,   work  on  your  own,  and  ask  for  help  if  necessary) 225 1-­‐5 3.68  (0.98) 73.33% The  children  listen  attentively 226 1-­‐5 3.96  (0.76) 84.52% The  children  are  fighting  or  restless  in  the  classroom (R) 226 1-­‐5 2.04 (0.88) 9.29% The  boys  participate  well  in  the  classroom 225 1-­‐5 4.01  (0.74) 88.89% The  girls  participate  well  in  the  classroom. 226 1-­‐5 3.95  (0.72) 85.84% The  children  are  reading  books  in  Spanish. 225 1-­‐5 3.68  (1.15) 75.11% The  children  are  reading  books  in  K’iche’ 224 1-­‐5 2.91  (1.28) 43.75% The   teachers   give   the   same   opportunities   for   participation   to   boys   and   girls   (with   questions,   allowing   them   to   speak,   calling   them   by   their   name,   etc.) 225 1-­‐5 4.11  (0.74) 90.22% There   working   notebooks   or   solved   exercises   that   demonstrate   that   children   perform   exercises   to   strengthen  their  understanding  of  written  texts 225 1-­‐5 4.11  (0.72) 88.44% Overall  Classroom  Attentiveness  (Standardized)   213   0-­‐100%   52.07%  (18.39%)   Momostenango 128 0-­‐91% 51.61%  (18.74%) San  Andres  Xecul 25 9-­‐66% 44.25%  (14.17%) Santa  Lucia  La  Reforma 35 3-­‐100% 55.18% (18.50%) San  Bartolo  Aguas  Calientes 25 19-­‐91% 57.88% (18.11%) Additionally,  according  to  principals,  children  in  94.44%  of  the  schools  (n=102)  are  paying  more  attention   in  the  classroom  during  the  2016  school  year.  Specifically,  Table  13 shows  which  factors  helped  to   increase  attentiveness.  43.86%  of  teachers  (n=100)  identified  the  provision  of  food  as  the  most  important   factor  that  helped  increase  attention,  followed  by  New  Tools  and  Techniques  of  Kemom  Ch’ab’al   Methodology  (14.47%),  and  Kemom  Ch’ab’al  Textbooks  (14.04%).  On  the  other  hand,  58.62%  (n=68)  of   principals  identified  School  Food  as  the  most  important  factor,  followed  by  Kemom  Ch’ab’al  Textbooks   (18.97%),  and  the  use  of  New  Tools  and  Techniques  (12.93%). 15 Grade Number  (%) Preschool 98  (85.22%) First  Grade 114  (99.13%) Second  Grade 114  (99.13%) Third  Grade 114  (99.13%) Fourth  Grade 114  (99.13%) Fifth  Grade 114  (99.13%) Sixth  Grade 113  (98.26%) Table  13.  Factors  Influencing  Attentiveness (N=116) Factors  influencing  Attention   Principals   Teachers   Number  (%) Number  (%) Kemon  Ch’ab’al  Textbooks 104  (89.66%) 181  (79.04%) Kemon  Ch’ab’al  Workbooks 103  (88.79%) 163  (71.18%) School  Food 112  (96.55%) 185  (80.79%) New  Techniques  and  Instruments 91  (78.45%) 151  (65.94%) Teacher’s  Attendance 100  (86.21%) 135  (58.95%) Material  was  appealing  to  students N/A 148  (68.84%) ii. Student  Hunger  and  Food  Intake  (Result  1.2.1)   Nearly  all  the  principals  (92.59%)  stated  that  students  are  less  hungry  because  of  the  FFE  program.   Unfortunately,  in  the  drafting  of  the  measurement  instruments  for  the  final  evaluation,  the  question  to   children  of  whether  they  were  hungry during  the  school  day  was  inadvertently  omitted.  Thus,  we  are  not   able  to  capture  students’  perspectives  on  hunger  and  can  only  report  principals’ perceptions  of  student   hunger. According  to  the  principals,  71.93%  (n=82)  of  schools   serve   food   every   day, 6.14%   (n=7)   serve   food   four   times   a   week,   10.53%   (n=12)   serve   it   three   times   a   week  and  11.4%  (n=13)  served  it  only  twice  a  week.   Table  14.  Food  Given  by  Grade  Level5 In  94.59%  (n=105)  of  schools  the  principals  said  some   or  all  of  the  funds  that  pay  for  the  food  came  from   MINEDUC,  96.40%  (n=107)  received  funds  from   Catholic  Relief  Services,  and  6  schools  (5.41%)  said   they  received  funds  from  APCC for  the  food.   Principals  reported  which  grades  are  given  food   during  the  school  day.                                                                                                                         5 The  percentage  represents  how  many  schools   give  food  to  students,  by  grade  level 16 Most  teachers  (91.40%;  n=202)  stated  students  liked  the  food  given  at  school.  20  teachers  made   comments about  why  students  did  not  like  the  food.  14  comments  (70%)  referred  to  the  monotonous   food  given  at  school.  All  other  comments  (30%)  were  related  to  food  preparation,  specifically  around  the   use  of pinto  beans.  In  addition,  68.04%  (n=149)  of  teachers  stated  there  were  no  problems  with  the   quality  of  food.   Parents  said  that  their  children  had  breakfast  at  school  every  day.  The  most  consumed  food  groups  at   breakfast reported  by  parents  were  tortillas  (49.79%),  coffee  (47.07%),  eggs  (46.86%),  beans  (42.68%),   tamalitos  (40.38%),  and  atol  (32.22%). The  least  reported  groups  were:  Cream  (0.21%),  Fish  (0.21%),   sausage  (0.84%),  and  Meat  (0.84%).  In  addition,  8  parents  (1.67%)  stated  their  children  did  not  have   breakfast  at  home. Table  15.  Food  children  ate  for breakfast6 In  principals’ opinions,  on  average  34.73%  of  first  graders,   34.59%  of  second  graders,  33.94%  of  third  graders,  33.32%   of  fourth  graders,  32.68%  of  fifth  graders,  and  31.91%  of   sixth  graders  attending  school  had  no  breakfast. Table  16.  Percentage  of  Children  Attending  School  that  had   no  Breakfast D. STUDENT  ATTENDANCE  (RESULT  1.3)                                                                                   52.61%  (n=121)  of  teachers  stated  they  have  attendance  records  for  male  students,  while  50.87%  (n=117)   stated  they  have  attendance  records  for  female  students.  The  record  showed  that  the  average   percentage  of  male  students’  attendance  for  third  grade  was  56.46%  (SD=44.09%,  range=[0,100]),  and  for                                                                                                                           6 The percentage  represent  number  of  parents   who  stated  their  children  ate  that  food  group  at   breakfast. Food  Group Number  (%) Nothing  (no-­‐ breakfast) 8  (1.67%) Beans 204  (42.68%) Eggs   224  (46.86%) Tortillas 238  (49.79%) Tamalitos 193  (40.38%) Bread 74  (15.48%) Atol 154  (32.22%) Cream 1  (0.21%) Cheese 26  (5.44%) Chile  or  Chirmol 56  (11.72%) Coffee 225  (47.07%) Milk 35  (7.32%) Meat 4  (0.84%) Chicken 8  (1.67%) Herbs 59  (12.34%) Fruits  &  Vegetables 59  (12.34%) Rice  &  Pasta 85  (17.78%) Cereals 37 (7.74%)   Sausage 4  (0.84%) Fish   1  (0.21%) Grade Mean  (SD) Preschool 34.73  (35.70) First  Grade 34.59  (35.82) Second  Grade 33.94  (35.93) Third  Grade 33.32  (36.11) Fourth  Grade 32.68  (36.03) Fifth  Grade 31.91  (36.07) Sixth  Grade 34.73  (35.70) 17 sixth  grade  was  50.59%  (SD=46.87%,  range=[0,100]).  For  female  students,  the  average  attendance  was   84.72%  (SD=24.61%,  range=[0,100])  for  third  grade,  and  85.04%  (SD=26.46%,  range=[0,100])  for  sixth   grade.  Those  teachers  who  did  not  have  records  estimated  that  on  average  male  students  attended   81.79%  (SD=26.45,  range=[0,100])  of  the  school  days,  and  female  students  attended  on  average  87.54%   (SD=17.12%,  range=[0,100])  of  school  days.   73.50%  (n=147)  of  teachers  stated  that  students’  attendance  increased with  program  implementation.   91.41%  (n=149)  of  teachers  stated  that  the  School  Food  helped  increase  attendance,  followed  by  the  use   of  New  Teaching  Techniques,  which  was  positively  identified  by  75.46%  of  teachers  (n=123).  In  addition,   79.04%  of  teachers  (n=181)  stated  the  program  helped  increase  student  attendance.  91.38%  (n=106)  of   principals  feel  the  program  helped  increase  student  attendance.   Table  17:  Factors  that  Helped  Increase  Student  Attendance Factors  influencing  Attendance   Principals   Teachers   Number  (%) Number  (%) The  used  of  new  teaching  techniques 123  (75.46%) 83  (71.55%) Food  provision 149  (91.41%) 110  (94.83%) Kemon  Ch’ab’al  Textbooks 137  (87.05%) 101  (87.07%) Enrollment  campaigns 79  (53.02%) N/A Kemon  Ch’ab’al  Workbooks N/A 98  (84.48%) Teacher  Attendance N/A 89  (76.72%) The  used  of  new  teaching  techniques 123  (75.46%) 83  (71.55%) Findings  from  focus  groups  show  that  parents  believe children’s  attendance  improved  (48  comments,   65%  of  schools).  They  state  that  this  improvement  was  due  to  parents’  support  encouraging children  to   attend  school  (42  comments,  55%  of  schools)  [“Parents  have  to  go  to  school  and  ask  how  their  children   are  doing,  for  example,  in  grades,  performance,  so  that  children  know  they  have  their  parent’s  support,   and  send  them  every day  to  school”-­‐ “Papá  y  mamá  tiene  que  venir  a  preguntar  cómo  está  su  hijo,  cómo   va  de...sus  calificaciones,...qué  rendimiento,  para  que  los  niños  se  dan  cuenta  que  tienen  el  apoyo  de   papá  y  mamá,  y  mandarlos  todos  los  días”],  and  being  motivated  by  the  food  provided  at  the  school  (6   comments,  20%  of  schools)  [“Children  know  that  if  they  do  not  eat  at  home,  they  will  eat  at  school”  -­‐ “Ya   los  niños  saben  que  si  no  comen  en  la  casa  ya  aquí  (en  la  escuela)  hay  comida  para  ellos”].   Additionally,  parents  stated  that  the  benefits  of  children  attending  school  were  learning  in  general  (14   comments,  45%  of  schools)  [“(Mothers  say)  get  up,  so  you  can  go  to  school  and  learn,  to  write,  to  add”  -­‐ “(Las  madres  dicen)  levántate  mamita,  para  que  te  vas  a  ir  a  la  escuela,  para  que  aprendés,  vas  a  escribir,   vas  a  hacer  suma”],  and  eating  in  the  school  (22  comments,  60%  of  schools)  stating  that  “children happily attend  because  they  know  they  will  eat  there”  [(“...vienen  felices  (a  la  escuela)...,  ellos  van  a  comer)]. Parents  recognize  that  both  the  Kemom  Ch’ab’al  books  and  food  provided  at  school  has  helped  children   to  be  motivated  to  attend  [“Now  children  have  their  books,  food,  and  I  believe  that  is  providing  a  good   way  of  life  to  our  children”  – “Les  ha  venido  libro  …Kemon  Ch’ab’al,  en  cambio  antes  no  había,  …,  viene   alimentación  y  creo  que  es  una  buena  vida  que  les  han  dado  a  los  niños”]. 17.16%  of  parents  (n=82)  listed  three  reasons  why  children  did  not  attend  school,  while  36.40%  (n=174)   listed  two  reasons,  and  40.37%  (n=193)  mentioned  only  one  reason.  There  were  a  total  of  791  codes,  the   most  common  reason  is  sickness/illness  (49.05%  of  comments),  followed  by  lack  of  interest  from  children   (13.78%  of  comments),  and  lack  of  support  from  parents  (10.37%  of  comments).  On  the  other hand,   18 32.88%  (n=73)  of  teachers  listed  three  reasons  why  children  did  not  attend  school,  39.64%  (n=88)  listed   two  reasons,  and  27.48%  (n=61)  mentioned  only  one  reason.  There  were  a  total  of  484  codes,  the  most   common  reason  is  sickness/illness  (26.24%  of  comments), followed  by  children  working (20.04%  of   comments),  and  lack  of  support  from  parents  (16.32%  of  comments).  Finally,  38.39%  of  principals  (n=43)   listed  three  reasons  why  children  did  not  attend  school,  while  38.39%  (n=43)  listed  two  reasons,  23.22%   (n=26)  mentioned  one  reason.  There  were  a  total  of  484  codes,  the  most  common  reason  was   sickness/illness  (24.37%  of  comments),  followed  by  children working (18.49%  of  comments),  and  lack  of   support  from  parents  (12.18%  of  comments).  See  Table  18 below  for  a  complete  list. Table  18. Reasons  of  Why  Students  Missed  Classes Reasons   Parents   Teachers   Principals   Number  (%)   Number  (%)   Number  (%)   Sickness/Illness 338  (49.05%) 127  (26.24%) 58  (24.37%) Lack  of  Interest  from  children 109  (13.78%) 38  (7.85%) 13  (5.46%) Lack  of  Support  from  Parents 82  (10.37%) 79  (16.32%) 29  (12.18%) Children Working 54  (6.83%) 97  (20.04%) 44  (18.49%) Other  Activities 44  (5.56%) 41  (8.47%) 24  (10.08%) Family  issues 31  (3.92%) 32  (6.61%) 21  (8.82%) Weather  and  school  distance 23  (2.91%) 9  (1.86%) 5  (2.10%) Lack  of  resources  (poverty) 27  (3.41%) 34  (7.02%) 21  (8.82%) Class  suspension  and  teachers  related   issues  (maltreatment,  delay,  etc) 14  (1.77%) 6  (1.24%) 3  (1.26%) School  environment 8  (1.01%) 2  (0.41%) 0  (0.00%) Other  reasons 11  (1.39%) 19  (3.93%) 20  (8.40%) Total   791  (100%)   484  (100%) 238  (100%) In  focus  groups,  reasons  provided  by  parents  for  children  not  attending  school  were  diseases  in  general   (23  comments,  70%  of  schools);  children helping  parent’s  in the  field  or  other work  tasks  (13  comments,   40%  of  schools)  [“We  (parents)  send  children  to  work,...carry  wood,  harvest  crops”-­‐“Los  mandamos  los   niños  al  trabajo,  va  a…,  cargar  leña,  cortar  y  cultivar  tierra”];  children  are  not  willing  to  attend  (12   comments,  35%  of  schools)  [“Children  sometimes  do  not  want  to  come  (to  school)”-­‐ “Lo  que  pasa  a  veces   con  los  niños  es  que  a  veces  les  agarra  pereza,  y...no  quieren  venir”];  parents’  lack  of  responsibility  (6   comments,  30%  of  schools)  [“Sometimes  we  (parents)  are  irresponsible  and  wake  up  late  and  tell  our  kids   not  to  go  to  school”  -­‐ “porque  a  veces  como  padres  de  familia  tenemos  una  irresponsabilidad  con   nuestros  hijos  porque  a  veces  nosotros  no  nos  levantamos  a  temprana  hora...,  nos  agarra  la  tarde  y   después  les  decimos  a  nuestros  hijos  que  no  se  van  (a  la  escuela)”];  family  activities  (3  comments,  10%  of   schools);  rain  or  weather  conditions  (3  comments,  15%  of  schools);  and  school  remoteness/distance (2,   10%  of  schools)  [“It  rains  a  lot  and  that  is  why  children  miss  school  because  they  get  wet  and  they  are   afraid,  they  live  far  away  from  the  school”  -­‐ “Llueve  mucho  por  eso  es  que  hacen  falta  los  niños  que   vienen  porque  se  mojan  y  tienen  miedo  pues,  vienen  unos  lejos”]. From  the  information  gathered,  it   seems  that  one  of  the  concerns raised  by  parent  committees is  children  not  attending  school.  Some   parents  commented  that  they  have  opened  a  channel  of  communication  with  teachers  to  learn  which 19 Problems N Number  (%) Food  Preparation   114 34  (29.82%) Food  Organization 34 12  (35.29%) Types  of  Food   34 26  (74.29%) Food  Storage   34 4  (11.76%) Freshness   34 8  (23.53%) Cleanliness  of  the  Kitchen   34 2  (5.88%) Hygiene   34 5  (14.71%) children  are  not  attending.  Once  they  have  the  information,  they  might  inquire  with  other parents  the   reason  for  non-­‐attendance.  This  is  interesting  because  the  committee  is  working  on  behalf  of  the   communities’  development  expressing  that  the  path  to  development  is  through  school  education.   iii. Food  Preparation   Overall,  87.77%  (n=409)  of  parents  stated  they  took  turns  to  prepare  the  food  that  was  given  to  the  kids   at school. Disaggregating by  gender,  80.61%  (n=79)  of  fathers,  and  89.67%  of  mothers, stated  that   parents  took  turns  to  prepare  the  food. Overall,  75.75%  (n=353)  of  interviewed  parents  participated  as   volunteers  in  the  kitchen.  However,  only  50%  of  fathers  stated  they  were  volunteer,  while  82.61%  of   mothers  stated  they  were  volunteers.  Overall,  parents  volunteer 15.29  (SD=29.79)  days. Fathers   volunteer  on  average  12.94  (SD=26.28)  days,  while  mothers  volunteer  15.62  (SD=30.34)  days. 42.68%   (n=204)  of  parents  strongly  agreed  that  food  preparation  was  well  organized,  and  52.51%  (n=251)  of   parents  stated  they  agreed  that  food  preparation  was  well  organized.   Table  19.  People  who  participated  in  Food  Preparation according  to  principals (n=113) Who  prepared  the   food Number  (%) School  Board 26  (23.01%) Mothers 103  (91.15%) Fathers 13  (11.50%) Hired  Cook 20  (17.70%) Other 3  (2.65%) According  to  the  focus  groups  participants,  parents  received  training  in  food  preparation (24  comments,   65%  of  schools).  They  mentioned  that  their  role  as  the  parents’  committee  was  to  receive,  store,  and   supervise  food  preparation  for  children  in  the  school  (35  comments,  75%  of  schools)  [“We  are  here  at  the school  supervising  food,  we  work  in  shifts,  starting  monday  through  friday”  -­‐ “Nosotros  estamos  aquí  en   la  escuela,  velando  por  la  refacción,  estar  al  día,  estamos  por  turnos,  unos  empiezan  de  lunes,  martes,   miércoles,  jueves,  viernes..”].   Table  20.  Problems  with  Food  Preparation   29.82%  of  principals  (n=34)  said   there  were  problems  with  the  food   preparation,  such  as  preparation,   organization,  type  of  food,  storage,   freshness,  cleanliness  in  the   kitchen  and  hygiene (see  Table  20). Findings  from  parent focus  groups  also  showed  that  there  were  problems  with  food  preparation  (130   comments,  80%  of  schools).  Of  these,  the  most  relevant  problem  is  with  beans  (73  comments,  75%  of   schools),  either  being  hard  to  cook  (51  comments,  70%  of  schools),  unfamiliar  type  of  beans  (14   Principals  reported  that  multiple  people  helped   to  prepare  the  food.  91.15%  (n=103)  of   principals  stated  that  the  students’  mothers   prepared  the  food,  followed  by  School  Board member  (23.01%) 20 comments,  25%  of  schools),  uncooked  beans  causing  stomach  problems  (5  comments,  10%  of  schools),   or  extra  expense  due  to  the  amount  of  wood  needed  to  cook  the  beans  (3  comments,  10%  of  schools).   [“The  problem  we  have  most  is  with  beans  because  they  do  not  cook,  it  takes  a  lot  of  wood  and  instead  of   helping  children,  they have  stomach  discomfort  because  they  do  not  cook”  -­‐ “El  problema  que  tenemos   más  es  con  el  frijol  porque  no  se  coce  y  lleva  mucha  leña,  tarda  para  cocer  y,  en  vez  de  ayudar a  los  niños,   nos  empachamos  con  el  frijol  porque  no  se  coce”;  “Children  do  not  want  to  eat  brown  (pintos)  beans,   they  are  more  familiar  with  white,  black,  or  red  beans”  -­‐ “...es  que  son  frijoles  pintos  entonces  (los  niños)   no  quieren  eso,  lo  que  más  conocen  los  niños  son  frijoles  blancos,  negros,  colorado…”].   Other  problems  related  with  food  preparation  were  monotonous  foods  (6  comments,  10%  of  schools)   stating that  it  would  be  good  to  have  a  variety  of products  such  as  other  kinds  of  legumes  so  that  children   do  not  get  bored  from  the  same  food;  lack  of  potable  water  (3  comments,  15%  of  schools);  lack  of   appropriate  cookware  (2  comments,  10%  of  schools);  and  lack  of  seeds  to  grow  food  at  the  school  garden   (1  comment).  Another  problem  identified  with  food  aid  and  preparation,  was  food  delivery  being  late.   Findings  show  4  comments  (all  from  one  school)  in  which  parents  report  to  have  had  late  food  deliveries   [“...the  institution  has  been  late,  not  only  school  supplies  but  also  food  arrives  very  late”  -­‐ “…la  institución   ha  tardado  mucho.  No  solo  los  útiles,  la  refacción…,  se  tarda  demasiado;  lo  que  a  mi  me  gustaba  bastante   de  Conrado  es  que  eran  muy  puntuales”]. iv. Student  Participation  in  Extra-­‐Curricular  Activities  (Result  1.3.1)   CRS  monitoring  data  showed  that  3,594  children  participated  in  extra-­‐curricular  school  activities,  meeting the  final   target   of   3,505   children.   Yet,  when   asked,  most   parents  (87.66%;  n=419)   did   not   know   about   after  school  activities,  and  12.34%  (n=59)  were  aware  of  the  activities  offered.  75.93%  (n=41)  of  parents   mentioned  there  was  school  reinforcement.  44.07%  (n=26) of  parents  stated  there  were  other  activities,   such  as  cultural  activities,   sports,  and  others. About  26%  of   schools had  a  “Space   to  Grow”  program  in   place.  Thus,  about  61%  of  parents  who  stated   they  did  not  know  about  extracurricular  activities,  did  so   due  to  lack  of  awareness.   Table  21.  Students’  Participation  in  Extracurricular  Activities Activities after school hours Clusters Number (%) Codes Number (%) Celebrations 42.30% (11) Cultural activities 26.92% (7) family celebrations 15.38% (4) Recreation 26.93% (7) Sports 19.23% (5) Music 3.85% (1) Games/Playing 3.85% (1) Reinforcement 23.08% (6) Reading 15.38% (4) Homework 3.85% (1) team work 3.85% (1) Food meetings 7.69% (2) Food meetings 7.69% (2) Total 100% (26) Many  parents  (71.13%;  n=340)  stated  they  would  like  their  children  to  participate  in  after  school  activities   to  help  them  with  their  studies.  28.87%  (n=138)  of  parents  were  not  interested in  activities  after  school.   Their  reasons  were  coded  in  140  codes.  The  lack  of  interest  is  mainly  due  to  weather  conditions  and   21 school  distance  (16.43%  of  the  comments),  security  issues  (4.48%  of  the  comments),  because  children   have  to  work  or  help  at  home  (29.29%  of  the  comments),  or  because  they  do  not  have  time  to  spend   more  hours  at  school  (22.86%  of  the  comments).   Almost  half  of  the  principals  (43.97%,  n=51)  were  aware  of  what  the  Spaces  to  Grow  program  entailed.   One  quarter  (25.66%,  n=29)  had  a  Spaces  to  Grow  program  in  place.  The  project  was  designed  with  a  goal   of  40  schools  (1/3)  implementing  a  Spaces  to  Grow  Program.  89.66% of  these  programs  included   homework  reinforcement,  82.76%  included  recreation,  and  82.76%  included  personal  growth  and  values.   While  the  implementation  rate  is  low  thus  far,  91.51%  (n=97)  of  the  principals  said  there  is  an  interest  in   having  the  Spaces  to  Grow  program  at  their  school.   v. Number  of  School  Government  Established  (Result  1.3.1a)   96.02%  of  teachers  (n=217)  indicated  that  the  school  has  a  school  government.  Those  217  teachers   worked  in  115  schools,  meaning  that,  95.83% of  120  sampled  schools  have  a  school  government   established.  Therefore,  it  was  estimated  that  212  (95.83%)  schools  have  established  a  school  government   out  of  the  population  of  221  schools.  This  estimate  suggests  that  all  schools  actively  participating  in  all   aspects  of  FFE  implementation  have  a  school  government  established. According  to  the  principals,  the  numbers  were  only  slightly  higher  showing that  98.28%  of  schools   (n=114)  had  a  school  government.    98.25%  of  principals  and  95.31%  of  teachers  (n=203)  said  that  school   governments  work  on  committees.  All  but  one  principal  reported  that  the  school  government  is  co-­‐ed   (99.12%,  n=113),  and 99.07%  of teachers  (n=213)  agreed  that  the school  government  is  composed  of   both  boys  and  girls. More  specifically,  principals  reported  that  51.35%  of  Presidents  are  female,  42.34%  of  Vice  Presidents  are   female,  67.57%  of  Secretaries  are  female,  63.96%  of  Treasurers  are  female,  and  66.67%  of  Vocals   (members  who  can  vote  in  the  absence  of  other  elected  members)  are  female.  Similarly,  46.57%  (n=95)   of  teachers  stated  that  girls  were  President,  49.51%  (n=101)  of  teachers  stated  girls  were  Vice  Presidents,   70.59%  (n=144)  of  teachers  stated  girls  were  Secretary,  54.41%  (n=111)  of  teachers  indicated  girls  were   Treasurer,  and  66.18%  (n=135)  of  teachers  indicated  girls  were  Vocals.  Overall,  74.55%  (n=82)  principals   feel  that  the  school  government  was  stronger  in  2015-­‐2016  as  compared  with  2014-­‐2015  school  year.   96.55%  (n=112)  of  the  principals  believe  that  the  teachers  are  interested  in  a  school  government  that  is   formed  in  participation  with  the  children. vi. School  Gardens   54.15%  of  teachers  (n=124)  and  53.59%  of  principals (n=61)  stated  there  is  a  garden  at  their  school.   70.40%  of  teachers  (n=88)  stated  that  the  garden  helped  to  increase  student’s  attendance,  72.00%  (n=90)   of  teachers  stated  the  garden  helped  increase  students’  attention,  and  71.20%  (n=89)  of  teachers  stated   the  garden  helped  increase  the  quality  of  teaching.  75.41%  (n=46)  of  principals  feel  the  garden  has   contributed  to  school  attendance,  77.05%  (n=47)  feel  it  has  contributed  to  attentiveness,  and  91.80%  feel   it  has  contributed  to  quality  of  teacher’s  instruction.  28.60%  (n=129)  of  parents  volunteer  in  the  school   garden,  on  average  8.41  days  (SD=28.69).  Overall,  in  the  parent’s  focus  groups,  school  gardening  was  not   mentioned  except  in  3  schools.  Comments  were  made  around  the  school  garden  being  an  improvement   to  the  school  (7  comments,  15%  of  schools)  stating  that  it  yields  food  to  be  given  to  children  [“The  school   22 garden  has  improved  as  well...,  the  institution  gave  us  basic  seeds,  onion,  lettuce  which  is  already  planted   and  growing.  When  ready,  we  will  harvest  and  prepare  for  children”  -­‐“El  huerto  ha  mejorado  también...,   la  institución  nos  dio  semillas  de  lo  básico,  cebolla,  lechuga  y  eso  ya  está  sembrado  en  el  huerto  ahorita,   ya  está  creciendo.  Cuando  ya  está,  ya  se  va  a  cosechar  se  lo  vamos  a  dar  a  los  niños”];  and  receiving   trainingin  school  gardening  (5  comments,  15%  of  schools)  [“(we)  have  received  training  in  school   gardening,  trainers  tell  us  that  we  can  use  diverse  materials  to  plant”  -­‐ “(los  padres)  hemos  recibido   capacitaciones  sobre  el  huerto…,(los  instructores)  advierten  a  uno  si  quiere  hacer  un  su  huerto  no  es   necesario  que  tenga  un  terreno,  sino  que  se  puede  hacer  en  llantas  o  en  esos  desechables,  es  eso  lo  que   nos  han  venido  a  decir”].   vii. School  Days  Missed  by  Students  Due  to  Illness  (Result  1.3.2)   According  to  the  principals,  on  average,  3rd graders  missed  0.55  days  (SD=  1.28),  and  6th graders  missed   an  average  of  0.54  days  (SD=1.27) during  the  2016  school  year.  Overall,  the  average  number  of  days   missed  for  students  in  first  through  sixth  grade  was  0.57  days  (SD=1.31),  better  than  the  final  target  of  2   days  missed  because  of  illness.   Most  parents  (70.38%;  n=221)  said  the  number  of  children  that  missed  school  due  to  illness  did  not   increase.  The  most  common  diseases  reported  by  parents  were  respiratory  issues  and  colds (43.98%  of   the  reported  illness/diseases),  followed  by  fever  or  headache  (28.80%  of  the  reported  illness/diseases),   and  stomach/intestinal  infections  (18.33%  of  the  reported  diseases).  The  table  below  shows  the  specifics   mentioned  by  parents. Table  22.  Kinds  of  Illness  that  caused  Students  to  Miss  Classes Clusters   Number  (%)   Codes   Number  (%)   Eruptive  Diseases   9  (4.71%) Measles 3  (1.57%) Chickenpot 6  (3.14%) Respiratory  and  cold  diseases   84  (43.98%) Tonsillitis 2  (1.05%) Bronchitis 1  (0.52%) Respiratory  diseases 8  (4.19%) Flu 33  (17.28%) Cough 40  (20.94%) Fever  and  headache   55  (28.80%) Headache 14  (7.33%) Fever 41  (21.47%) Stomach/Intestinal  Infections   35  (18.33%) Diarrhea 14  (7.33%) Intestinal  diseases 5  (2.62%) Stomach  ache 15  (7.85%) Parasites 1  (0.52%) Other  (not  specified)   5  (2.62%) Not  specified 3  (1.57%) Other 2  (1.05%) Nose  Bleeding   3  (1.57%) Nose  Bleeding 3  (1.57%) Total   100%  (191)   23 viii. Improvements  or  construction  of  the  school  (Result  1.3.3).   According  to  CRS  monitoring  data,  416  facilities  were  built  during  Phase  I  of  FFE.    This  number  exceeds   the  final  target  of  312  educational  facilities  rehabilitated  or  constructed  as  a  result  of  USDA  assistance.   Improvements  included  kitchens  built  or  rehabilitated  (n=159  schools),  latrines  built  or  rehabilitated   (n=100),  and  wells  and  water  systems  built  or  rehabilitated  (n=157). Over  half  (57.24%;  n=261)  of  interviewed  parents  participated  as  volunteers  in  the  improvement  or   construction  for  an  average  of  7.44  days  (SD=  18.60)  at  the  school.  Schools  had  between  1  and  22   classrooms,  with  the  mean  number  of  classrooms  per  school  being  6.12  (SD  4.26).     According  to  parent’s  focus  groups,  school  improvement  was  categorized  into  two  themes:   improvements  in  infrastructure  (60  comments,  75%  of  schools)  and  other  non-­‐infrastructure   enhancements  (111  comments,  90%  of  schools).  The  most  frequent  infrastructure  topic  mentioned  was   improved  school  kitchens  (35  comments,  60%  of  schools),  followed  by  improved  bathrooms  (16   comments,  25%  of  schools),  obtaining  water  deposits  (6  comments,  20%  of  schools),  improved  food   storage  (2  comments,  10%  of  schools),  and  obtaining  a  sports  field  (1  comment).  The  kitchen   enhancements  mainly  referred  to  having  new  stoves  which  are  used  to  prepare  food  in  the  school  [“We   had  a  very  small  stove,  now  we  have  a  bigger  one  which  makes  it  easier  to  cook”  -­‐ “...teníamos  la  estufa,   o  sea  la  plancha  muy  pequeña,  ahora  ya  tenemos  una  estufa  grande,  más  facilidades  para  cocinar”].  As   per  bathrooms,  comments  revolve  around  increasing  the  number  of  bathrooms  they  had  [“We  have   more  bathrooms  which  is  very  useful”  -­‐ “….  se  amplió  más  los  baños,  eso  es  de  mucha  utilidad”]. However,  there  were  some  issues  with  school  infrastructure:  lack  of a  room  solely  dedicated  to  food   storage  (13  comments,  15%  of  schools),  kitchen  or  food  storage  being  small  or  inadequate  (22  comments,   50%  of  schools),  lack  of  sewage  (3  comments,  10%  of  schools),  and  lack  of  water  deposit  (1  comment).   The  theme  of  non-­‐infrastructure  school  improvements  mainly  referred  to  overall  cleanliness.  Personal   hygiene  (38  comments,  70%  of  schools),  and  keeping  food  storage  (29  comments,  75%  of  schools),   kitchen  (25  comments,  45%  of  schools)  and  bathrooms  (7  comments,  25%  of  schools)  clean.  Parents   mostly  mentioned  improved  personal  hygiene  in  children  attending  school  and  in  mothers  who  prepare   food  [“...the  benefit  is  that  we  do  things  clean,  ...clean  and  prepare  food,  well  cooked,  and  as  my  peer   says,  prepare  our  children  so  that  they  come  to  school  clean,  change  their  clothes,  give  them  a  bath,   comb  their  hair”  -­‐ “…nos  ha  beneficiado  de  que  hacemos  las  cosas  así  aseados,…la  limpieza  y  preparados   bien  los  alimentos,  bien  cocidos  y  como  dice  la  compañera  bien  preparar  a  nuestros  hijos,  que  vengan  a   estudiar  limpios,  cambiarles  la  ropa,  bañarlos,  peinarlos…”].  Parents  also  mentioned  improvements  in  the   cookware  they  received  to  prepare  food  (5  comments,  20%  of  schools)  being  an  adequate  size  [“...(we)   are  better  because  now  we  have  big  cookware  to  prepare  food”  -­‐ “…  (estamos)  mejor  porque  ahora  ya   contamos  con  materiales  ya  grandes  para  poder  cocer  la  refacción”]. 24 ix. School  Enrollment  (Result  1.3.4)   According  to  CRS  monitoring  data,  school  enrollment  for  both  boys  and  girls  has  dropped  significantly   since  baseline  and  midterm  (see  Appendix  A).  In  2014,  18,437  boys  were  enrolled.  This  figure  dropped  to   18,245  in  2015  (-­‐1.0%)  and  dropped  again  to  16,790  in  2016  (-­‐8.9%  decrease  compared  to  midterm).  For   girls,  17,765  were  enrolled  in  2014,  dropping  to  17,307  in  2015  (-­‐2.6%  decrease)  and  to  16,394  in  2016  (-­‐ 7.7%  decrease  compared  to  baseline).  In  addition,  about  half  of  principals surveyed (n=62,  56.36%)  do   not  feel  that  the  program  has  helped  to  improve  student  enrollment. Conversations  with  at  least  one  CTA,  and  with  USDA  representatives,  confirmed  that  declining  school   enrollment  is  a  widespread  problem  in  Guatemala  more  generally.  Given  limited  economic  opportunities,   many  children  are  opting  to  leave  school  and  migrate  to  urban  areas, or  north  to  Mexico  and  the  U.S.,  in   search  of  employment.   x. Importance  of  Education  (Result  1.3.5)   99.79%  of  parents  (n=472)  think  education  is  important  for  their  children,  and  n=464  parents  provided  an   explanation  of  why  education  is  important (see  Table  23).  16.81%  of  parents  (n=78)  provided  three  or   more  explanations,  while  35.78%  of  parents  (n=166)  provided  two  explanations,  and  47.41%  of  parents   (n=220)  provided  just  one  explanation.  Parents’  comments  were  coded  into  806  codes,  and  6  clusters  of   codes.  The  main  benefit  identified  for  parents  was  Learning  (44.67%  of  comments),  followed  by  Having  a   Better  Future  (39.08%  of  comments),  and  Personal  Development  (12.66%  of  comments). Table  23.  Parents’ Responses  to  Why  Education  is  Important Clusters Number (%) Learning (to learn how to write, read, count, among others) 360 (44.67%) Having a better future (getting better jobs, better opportunities, among others) 315 (39.08%) Personal Development (Development of the child, socialization, children can perform what they are asked) 102 (12.66%) Family and community development (fighting poverty, becoming active members of the community, among others) 26 (3.23%) Decrease hunger 2 (0.25%) Children are given materials to study 1 (0.12%) Total 100% (806) Most  of  the  sampled  schools  (97.41%,  n=113)  have  PTAs.  Therefore,  it's  estimated  that  215  schools   participating  in  the  project  have  PTAs.  90.27%  of  principals  indicated  that  the  school  board/council  works   on  commission.  97.35%  of  principals  said  the  school  board  was  trained  in  2016. As  expressed  in  focus  groups,  education  is  very  important  for  parents.  They  think  that  through  education   and  literacy,  their  children  may  overcome  problems  (14  comments,  45%  of  schools)  [“…education  is   important  so  that  suffering  no  longer  prevails.  Through  educations  there  is  change,  here  we  now  know   how  to  read  and  write,  that  is  going  to  be  helpful  in  life”-­‐ “la  educación  es  importante  para  no  seguir   sufriendo  como  nuestros  abuelos  o  antepasados.  Siempre  hay  un  cambio,  aquí  ya sabemos  leer  y  escribir,   por  algo  más  les  va  a  servir  en  la  vida”].   25 E. GENERAL  OVERVIEW  OF  RESULTS  (RESULT  1.4)   According  to  parents´  focus  groups,  benefits  of  the  program  include  having  a  better  community  economy   (18  comments,  65%  of  schools)  due  to  savings  in  food  (17  comments,  60%  of  schools)  and  wood  (1   comment).  [“...we  (the  committee  and  a  teacher)  are  conducting  a  simple  analysis  of  how  much  we   receive  from  CRS  and  it  is  a  very  interesting  amount,  helping  our  community  a  lot…not  only  supporting   family  economy  but  also  community  development...”  -­‐“…(el  consejo  de  padres  junto  con  un  maestro)   estamos  haciendo  un  pequeño  análisis  así,  de  cuánto  recibe  la  comunidad  del  apoyo  de  CRS,  y  pues  es  un   dinero  muy  interesante…,  ayuda  mucho  a  nuestra  comunidad…No  sólo  apoya  la  economía  familiar,   también  el  desarrollo  de  nuestra  comunidad...”]. i. Perception  and  Sustainability  of  the  Program  (Result  1.4)   Regarding  to the  overall  project,  five  CTAs  (55.55%)  feel  it  is  very  sustainable  and  the  other  44.44%  feel  it   is  somewhat  sustainable.  37.16%  of  teachers  (n=84)  think  the  program  is  sustainable  or  very  sustainable,   27.88%  of  teachers  (n=63)  think  the  program  is  somewhat  sustainable,  and  34.96%  (n=79)  of  teachers   stated  the  program  is  not  sustainable  without  CRS/USDA  help.  Only  4.39%  (n=5)  of  the  principals  feel  that   the  FFE  program  is  very  sustainable  without  USDA  funding.  Just  less  than  1/3  (28.07%)  of  principals  feel   that  the  program  is  sustainable,  28.95%  feel  it  is  somewhat  sustainable,  but  the  majority  (38.60%)  feel  it   is  not  sustainable.   When  asked  about  sustainability  plans,  none  of  the  parents  in  focus  groups  (20  schools)  mentioned  to   have  one  in  place.  Consequently,  program  sustainability  arose  as  a  problem  (45  comments,  80%  of   schools).  Parents  stated  that they  needed  more  support  for  their  communities’  development.  During  the   conversation,  parents  stated  that  they  would  not  be  able,  or  have  not  figured  out  yet  how  to  provide  the   foods  that  the  program  delivers  [“We  (parents)  earn  little  for  work,  very  little.  Imagine  if  we  have  to  buy   rice  for  example,  we  pay  around  three  hundred (Quetzales)7 for  100  pounds of  rice,  we  will  not  to  be   able  to  do  so” -­‐    “...nosotros  ganamos  poco  por  el  trabajo,  muy  poco.  Imagínese,  compramos  un  quintal   de  arroz  por  ejemplo,  esté  entre  trescientos  y  algo (Quetzales),  es  cuestión  que  no  vamos  a  poder   comprar…;  “Without  their  help  (CRS)  we  cannot  do  anything  and  the  children  will  not  have  food,  no   tortillas,  no  beans,  no  rice,  no  atol”  -­‐ “Sin  la  ayuda  de  ellos  (CRS)  no  podemos  hacer  nada  y  los  niños  se   quedan  sin  comida  y  sin  comer,  sin  tortillas,  sin  frijol,  sin  arroz,  sin  atol”].  Additionally,  parents  mentioned (3  comments  in  3  different  schools) that  in  case  CRS  no  longer  supported  their  communities  with  food   aid, government  support  (at  least  what  they  claim  to  receive  at  the  time  of  the  focus  group)  would  not  be   enough.  They  describe  government  support  as  scant  and  sometimes  being  delivered  late.  Therefore,   compared  to  the  aid  they  receive  from  CRS,  they  consider  government  support  as  insufficient  to  feed   their  children  at  schools  on  a  daily  basis.  Thirty  comments,  in  65%  of  the  schools,  were  found  stating  that   communities  need  more  help,  either  from  government  or  other  institutions  such  as  CRS, in  order  to  thrive   [“We  need  more  things  in  the  school,  like  I  said,  we  need  more  help,  for  example  to  improve  the  food   storage  and  the  kitchen”  -­‐ “…necesitamos  más  cosas  aquí  en  la  escuela,…  como  le  vuelvo  a  repetir,   nosotros  necesitamos  más  ayuda…  como  en  la  bodega  y  más  mejoramiento  de  nuestra  cocina”].   However,  although  none  of  the  parents  at  the  focus  groups  mentioned  that  they  had  a  plan  for   sustainability,  they  were  able  to  mention  important  achievements  due  to  team  work  in  their   communities. Community  achievements  and  team  work  accounted for  68  comments in  95%  of  schools                                                                                                                           7 Quetzal  or  GTQ  (Guatemala  Quetzal)  is  the  national  currency  in  Guatemala.  In  August  2017,  300  GTQ  were   equivalent  to  approximately  41  US  dollars  (USD). 26 [“If  there  is  work  to  be  done,  all  the  rural  community  works  on  it.  We  are  united,  whatever  the  project,   we  do  it  together,  we  prevail”  –    “Si  hay  algún  trabajo,  lo  hace  toda  la  comunidad  rural.  Estamos  unidos,   cualquier  proyecto  o  cosa  que  hacemos,  lo  hacemos  juntos,  tenemos  logro”].  It  may  be  that  through  this   achievement  empowerment  and  team  work, they  may  figure  out  and  build  sustainability plans.   Only  31.90%  (n=37)  of  the  principals  have  thought  about  a  sustainability  plan  for  the  future.  More  than   half  (52.83%,  n=56)  of  the  principal’s  felt  that  there  is  another  modality  for  reducing  hunger  in  schools   which  could  be  more  efficient  and  more  cost-­‐effective.   However,  the  majority  of  the  follow-­‐up  responses  to  this  question  spoke  to  enhancements  of  the  FFE   program  rather  than  alternative  programs.  From  the  116  principals  interviewed  there  were  24  codes   grouped  into  the  four  themes  of  Sustainability  (43  comments),  food  (34  comments),  other  programming   options  (6  comments)  and  funding  (2  comments).  For  example,  under  the  sustainability  code,  19   principals  mentioned  the  role  of  school  gardens  and  specifically  6  said  their  school  needed  to  create  a   school  garden,  and  3  said  the  gardens  needed  improvements.  Another  7  principals  mentioned  that   creating  gardens  at  home  would  be  an  effective  manner  to  reducing  hunger  outside  of  school  time.  Six   principals  mentioned  that  an  increased  effort  was  needed  to  buy  local  as  it  could  support  local  businesses   and  economy  as  well  as provide  foods  with  which  the  children  are  familiar.  Three  principals  mentioned   each  of  the  following:  using  existing  community  resources  and  increased  parent  involvement. In  regards  to  the  food  served,  7  principals  emphasized  the  need  for  nutritious  and healthy  food,  6   mentioned  changing  the  menu  at  school,  3  mentioned  creating  a  rotating  menu  with  a  variety  of  food   options  and  2  mentioned  adding  school  breakfast  as  an  option.  Many  principals  mentioned  specific  foods   that  should  be  included  such  as  fruits  and  vegetables  (8),  vitamins  (3),  different  beans  (3),  milk  (1),   protein  (1)  and  eliminating  sweets  (1).  In  regards  to  alternative  programming  from  the  FFE  program,  only   a  few  principals  were  mentioned  such  as  financial literacy  training  (1),  Programs  run  by  the  Ministry  of   Health  (1),  the  Healthy  Schools  program  (1),  Parent  education  on  local  resources  (2)  and  Parent  education   on  healthy  cooking  (1).  In  order  to  make  everything  successful  one  principal  mentioned  that  more  funds   were  needed  and  another  suggested  asking  the  municipality  for  help. 77%  of  CTAs  (n=7)  said  they  had  seen  an  increase  in  the  capacities  of  the  government  institution  during   the  application  of  the  project.  100%  of  CTAs  (n=9)  reported  that  the  have  observed  improvements   regarding  the policy  and  regulatory  frameworks.  Additionally,  the  parent  volunteer  focus  groups   mentioned  that  they  do  not  have  a  sustainability  plan  [“If  the  program  is  discontinued,  we  would  only   have  the  government  aid  which  is  not  enough,  I  repeat,  we  do  not  have  a plan  in  case  aid  (from  CRS)   stops  coming”-­‐ “…si  algún  día  dejara  de  existir  (la  ayuda  de  CRS),  quedaríamos  sólo  con  la  ayuda  que   manda  el  gobierno…  y  es  poco  no  alcanza..,  le  repito  verdad,  no  hemos  tenido  algún  plan  o  algo  por  si   dejara  de  venir  esa  ayuda”].   Regarding  sustainability,  staff  mentioned  that  they  believe  the  program  may  be  sustainable  through   collaborative  work  (9  comments)  with  diverse  public  and  private  institutions,  and  community  members. There  are  two  communalities  across  all  staff  sustainability  solution  path.  The  first  one  involves  community participation.  As  mentioned  by  all  staff  interviewees,  infrastructure  and  community  members  will  remain.   It  is  through  trained  human  resources  who  are  part  of  the  communities  that  sustainability  may  be   achieved.  Community  members  need  to  be  aware  and  empowered  to  be  able  to  foresee that  through   them,  the  community  may  keep  developing  strategies  to  support  their  communities.  The  second  solution   strategy  that  all  staff  mentioned  has  to  do  with  community  members  management  abilities.  These  are expressed  as  community  members  being  able  to  assess  their  own  needs,  know  who  to  contact  (i.e.,   27 individuals,  local  government  institutions,  non-­‐government  organizations),  ask  for  what  is  needed,  follow   up,  manage  the  resources,  and  report  on  results  making  this  process  iterative  and  collaborative.  Several   strategies  are  mentioned  as  to  how  to  achieve  sustainability:  MINEDUC  increasing  food  aid  for  children  at   schools  (4 comments);  community  participation  and  involvement  (9  comments);  MINEDUC  reinforcing   bilingual  education  (4  comments);  MINEDUC  modifying  policies  to  be  able  to  buy  local  produce  or  other   goods  that  may  benefit  local  economy  (2 comment);  continue  to  train  communities  in  self-­‐advocacy  and   management  capabilities  (5  comments);  in  the  trainings,  incorporate  actual  community  examples  of   achievements  attained  through  their  own  collaborative  work  that  may  help  portray  possible  paths  to   sustainability  though  helping  develop  short  and  long  term  plans (8  comments);  continue  to  reinforce   school  gardens  (8  comments);    involve  private  institutions  that  may  be  interested  in  investing  in  schools   (1  comment);  .  What  may  not  be  sustainable,  as  stated  by  staff  members, is  the  non-­‐food  aid  that  CRS   provides  such  as  shoes or  other  goods  (2 comments). Interviews  with  USDA  staff  provided  helpful  feedback  on  perspectives  around  program  sustainability.  The   pursuit  of  sustainability  should  occur  at  multiple  levels,  from  advocacy  at  the  policy  level,  government   budgets  to  support  improved  policy,  and  strengthened  collaborations  with  local  organizations  and   communities  to  improve  capacity  for  implementation.  USDA  representatives  indicated  that  CRS  has  been   active  at  each  of  these  levels,  and  there  is  evidence  to  suggest  that  the  Government  of  Guatemala  may   pass  a  bill  that  requires  school  feeding  for  all  children  in  the  country. Both  USDA  representatives  mentioned  the  previous  relationship  between  CRS  and  APCC,  and  the   problems  introduced  when  it  was  discovered  that  APCC  had  problems  with  transparency,  accountability,   and  in  meeting  the  challenges  of  FFE  implementation. A  local  USDA  representative  stated  that  CRS  might   operate  more  effectively  in  the  future  by  not  subcontracting  to  a  local  partner,  but  rather  by  becoming  an   implementing  agency  instead  of  a  contracting  agency.  In  this  case,  CRS  would  be  directly  involved  in   implementing  the  program  and  coordinating  with  communities  and  schools,  rather  than  working  through   the  intermediary  of  a  partner  institution.   ii. Increased  Capacity  of  Government  Institutions  (Result  1.4.1)   Of  the  CTA’s  interviewed,  88.88%  (n=8)  participated  in  the  DIDEDUC  trainings.  The  table below  details  the   number  of  people  who  attended  each  training.   Table  24.  DIDEDUC  Training  Topics  Attended  by  CTAs (n=8  CTAs) Training Number (%) Your  Roles  and  Responsibilities  (1.1.3) 4  (50%) The  Use  of  the  Reading  Materials  (1.1.3) 7  (88%) School  Supervision  (1.1.5) 5  (63%) The  Use  of  Tools  to  Test  Reading  (1.1.5) 7  (88%) Application  of  the  Kemom  Ch’ab’al  Methodology  (1.1.5) 8  (100%) Supervising  the  Application  of  the  Kemom  Ch’ab’al  Methodology  (1.1.5) 7  (88%) Data  Logging  (1.1.3) 3  (38%) Data  Analysis  (reports)  (1.1.3) 6  (75%) 28 Only  8  of  the  9  CTA’s  responded  about  which  courses  they  had  taken.  All  8  took  the  Application  of  the   Kemom  Ch’ab’al  Methodology  Training.  88%  of  the  CTA’s  (n=7)  took  The  Use  of  Reading  Materials;  The   Use  of  Tools  to  Test  Reading;  and  Supervision  of  the  Application  of  the  Kemom  Ch’ab’al  Methodology.   The  training  on  Roles  and  Responsibilities  and  Data  Logging  were  not  well  attended  with  half  or  less  of  the   CTAs.  Other  trainings  mentioned  by  CTAs  included:  program  plans  for  the  school;  leadership;  and   Intercultural  Bilingual  Education  (Educación  Bilingüe  Intercultural;  EBI). Of  the  CTA’s  interviewed,  44.44%  (n=4)  participated  in  the  DIGEFOCE  staff  trainings,  and  3  chose  not  to   respond.  Training  topics  included  Your  Roles  and  Responsibilities;  Identification  of  School  Needs;   Coordination  with  Educational  Council;  Home  Gardens;  and  Home Gardens.  100%  of  CTA’s  attended   Identification  of  School  Needs  and  Exchange  of  Knowledge  and  Practices.  Almost  all,  75%  attended  Your   Roles  and  Responsibilities,  Coordination  with  Educational  Council  and  Exchange  of  Knowledge  and   Practices. Table  25. DIGEFOCE  Training  Topics  Attended  by  CTAs Training Number (%) Your  Role  and  Responsibilities 3  (75%) Identification  of  School  Needs 4  (100%) Coordination  with  Educational  Council 3  (75%) School  Gardens 4  (100%) Exchange  of  Knowledge  and  Practices 3  (75%) The   Educational   Council/School   Board   were   trained   on   topics   such   as   Kemom   Ch’ab’al   Methodology;   Good  Health  and  Nutrition;  The  Importance  of  Education;  Funds  Management;  School  Improvement  Plan.   Overall,  very  few  (19%)  of  the  school  boards  were  trained in  the  Kemom  Ch’ab’al  Methodology.  Over  80%   of  School  Boards  were  trained  in  Good  Health  and  Nutrition. Table  26.  Trainings  of  School  Boards/Council  (n=110) Training Number (%) Kemom  Ch’ab’al  Methodology 21  (19.09%) Good  Health  and  Nutrition 91  (82.73%) Importance  of  Education 73  (66.36%) Funds  Management/  Resources  Management 87  (79.09%) School  Improvement  Plan 78  (70.91%) iii. Increased  Government  Support  (Result  1.4.3)   CTAs  reported  that  on  average,  only  41%  of  the  disbursements  made  by  the  Ministry  of  Education  came   promptly  to  the  school  in  2016  (Range  =  [0,100%],  SD=39.03). The  principals  reported  that  half  (50.43%,   n=58)  of  the  schools  received  funds  from  the  MINEDUC  arrived  to  the  school  on  time.  However,  the   principals  noted  that  only  42.98%  of  the  time  (n=49)  the amount  of  the  funds  allocated  were  sufficient  to   meet  the  needs  of  the  children.   29 According  to  parent  volunteer  focus  groups,  3  comments  were  made  (all  from  one  school)  on  MINEDUC   disbursements  being  late.  Parents  stated  that  it  affects  their  personal  financial  situation  when  they  have   to  buy  school  supplies  upfront  before  actually  receiving  the  aid.  [“…a  veces  los  desembolsos  se  han   tardado  demasiado  y  comenzamos  a  comprar  útiles  escolares  para  nuestros  niños  verdad,  para  que  ellos   estudien”].   iv. Increased  Engagement  of  Local  and  Community  Members  (Result  1.4.4)   42.68%  (n=204)  of  parents  strongly  agreed,  and  46.86%  of  parents  agreed  that  the  food  is  prepared  by   community  members.  During  the  focus  groups,  56  comments  in  90%  of  the  schools,  support  the     engagement  of  diverse  community  members  working    together  to  achieve  goals  [“...here  we  work   together,  when  something  is  needed,  we  tell  the  community  “we  need  this”  and  they  provide  their   support,  that  is  how  we  achieved  school  enhancements”  -­‐    “...aquí  se  trabaja  en  conjunto,  cuando  hay  un   trabajo  pues,  se  le  dice  a  la  comunidad  “nos  hace  falta  esto”  y  sí  apoyan  por  esta  manera  pues,  que  si,  la   escuelita  aquí  si  tiene  su  cambio  (mejora)”;  “We  are  organized,  we  all  come  to  clean  the  school,  teachers,   the  committee,  parents  also”  -­‐ “Nos  organizamos  también,  venimos  a  lavar  la  escuela,  todos,  los   maestros,  el  consejo  y  padres  de  familia  también,  así  nos  organizamos”].   The  focus  group  participants  mentioned  collaborative  work  with  diverse members  of  the  community  and   believe  collaborations  are  an  important  achievement  that  helps  support  the  community’s  development   (103  comments,  100%  of  schools).  [“There  are  children  that  get  hungry  rapidly  and  this  (food)  helps   them,  so  this  not  only  supports  family  economy,  it  also  supports  the  development  of  our  community,   which  has  benefited  from  this  program”  -­‐ “…hay  niños  que  agarran  hambre  rápido  y  esto  (la  refacción)  los   apoya  a  ellos,  entonces  no  solo  apoya  la  economía  familiar,  también  el  desarrollo  de  nuestra  comunidad,   porque  este  programa  ha  levantado  mucho  a  la  comunidad”]. According  to  project  staff  interviews,  diverse  activities  were  executed  during  phase  I  of  FFE  which  were   aimed  to  ultimately  support  education  in  the  target  communities.  Mentioned  activities  ranged  from  staff   organization  and  implementation  activities  to  get  the  program  started  (3  comments)  to  providing  food  for   schools  to  enhance  children’s  nutrition  (9  comments),  help  develop  reading  and  writing  capabilities  in   children  in  a  bilingual  context  (Spanish  and  K’iche’)  (1  comment),  and  help  develop  community   management  capacity  (4  comments). One  of  the  main  lessons  learned  at  this  project  stage  were  that  in  order  to  achieve  positive  results,   teamwork  is  crucial  (4  comments),  not  only  collaborating  with  community  members,  but  including  key   partners  such  as  MINEDUC  or  PRODESSA.  This  learning  has  led  staff  members  to  continue  planning  and   executing  interinstitutional  meetings  on  a  regular  basis  to  discuss  project-­‐related  successes and  issues. Another  lesson learned  was that  when  communities  actually  receive  the  tangible  goods  such  as  food  or   cookware,  they  feel  motivated  and  start  working  as  teams  (3  comments).  It  was  also  mentioned  as  a   learning  (4  comments)  that  bilingual  instruction  in  schools  has  to  be  reinforced  and  that  teachers  should   be  better  trained  and  supervised  in  the  use  of  Kemom  Ch’ab’al  books  (3  comments).                   The  main  challenge  was  to  motivate  and  organize  parents  and  community  members  to  participate  in  the   program  (9  comments)  and  its  related  activities  (e.g.,  food  preparation,  trainings,  bilingual  instruction).   Other  challenges  mentioned  were  to  avoid  duplicity  with  other  institutions  trying  to  be  clear  on  FFE   objectives  (1  comment)  so  that  programs  do  not  overlap. 30 Positive  practices  mentioned  were  that  staff  perceives  that  communities  achieved  to  build  management   capabilities  in  order  to  advocate  for  their  needs  with  local  authorities  (4  comments).  Interinstitutional   regular  meetings  were  also  perceived  as  a  positive  practice  (9  comments)  since  all  stakeholders  may  solve   issues  together.   Plans  and  recommendations  made  by  the  staff  included  contextualizing  the  program  to  the  local   communities  in  which  it  is  implemented  (3  comments),  vary  food  provided  through  the  program  (1   comment),  continue  providing  help  since  outcomes  from  this  type  of  programs  will  show  on  the  long   term  (3  comments),  enhance  collaboration  between  key  partners  (9  comments),  review  if  program   should run  on  conditional  terms  (i.e.,  schools not  complying  with  provision  of  food  to  children  5  days  a   week  should  probably  not  receive  aid  anymore)  (2  comments),  strengthen  key  community  players  such  as   CTAs  (3  comment),  extend  program  to  a  public  policy  perspective  to  catch  attention  not  only  of  local   authorities  (2  comments).     All  staff  interviewees  assessed  that  children,  parents,  teachers  and  principals   view  FFE  as  a  positive  program  (9  comments)  which  helps  not  only  children  but  other  community   members.  They  believe  that  children  perceive  the  program  as  friendly  because  they  receive  food  at   school  (9  comments)  and  because  they  are  motivated  to  attend  school  due  to  the  new  techniques  that   teachers  are  implementing  (3  comments).  Regarding  teachers,  staff  believe  that  they  perceive  the   program  as  a  contributor  to  their  own  continuous  education  and  development  or  reinforcement  of  their   own  capabilities  (3  comments).  Staff  believe  that  parents  view  FFE  as  an  ally  because  through  trainings   they  learned  how  to  access  local  authorities,  advocate  for  their  communities’  needs,  and  achieve  results   such  as  infrastructure  improvements  at  the  schools.             Lessons  Learned  and  Best  Practices Lessons  learned  at  the  final  evaluation  reflect  much  of  the  information  reported  in  the  midterm   evaluation,  while  new  strategies  and  issues  also  emerged.  Below  we  enumerate  the  lessons  learned  and   the  best  practices  associated  with  each. (1) Favorable  ratings  of  the  Kemom  Ch’ab’al  methodology   Nearly  all  stakeholders  rated  the  Kemom  Ch’ab’al  methodology  highly  in  terms  of  its  relevance  and   effectiveness,  and  in  terms  of  its  innovative  use  of  a  variety  of  teaching  techniques  in  the  classroom.  Yet,   11.21%  of  principals  said  that  no  teachers  within  the  school  lead  activities  in  both  languages,  and   principals  also  said  that,  on  average,  teachers  spend  more  time  teaching  in  Spanish  than  K’iche’.  Some   teachers  indicated  they  do  not  speak  K’iche’,  which  may  account  for  the  difference  in  time  spent  overall   across  schools  teaching  Spanish  vs.  K’iche’.  The  Kemom  Ch’ab’al  methodology  is  therefore  emerging  as  a   best  practice  for  bilingual  education,  but  further  support  is  needed  for  teachers  not  comfortable  teaching   in  K’iche’. Information  from  parent  focus  groups  related  to  best  practices  suggested  that  PRODESSA  was   instrumental  in  supporting  improved  teaching.  Onsite  and  immediate  feedback  and  coaching  provided  to   teachers  by  PRODESSA  technicians  during  supervision  sessions  at  schools  allowed  teachers  to  enhance   their  teaching  techniques  or  apply  new  techniques  to help  improve  children’s  attentiveness. 31 (2) Cultural  relevance  of  food  and  dietary  diversity  for  schoolchildren   The  issues  around  pinto  beans  continue  to  pose  a  problem  for  program  implementation.  As  in  the   midterm  evaluation,  most  stakeholders  identified  pinto  beans  as  being  difficult  to  cook  and  requiring  a  lot   of  fuel,  and  that  children  did  not  like  the  beans.  This  issue  is  also  of  importance  because  pinto  beans  are   culturally  foreign  in  the  context  of  Totonicapán,  where  black  beans  are  commonly  used  instead.  The  final   evaluation  also  revealed  that  schools  receive  supplemental  budgets  from  MINEDUC  for  the  purchase  of   additional  foods  to  supplement  FFE  provisions.  Commonly,  PTAs  purchase  fresh  fruits  and  vegetables,   eggs,  and  occasionally  chicken.  The  combination  of  FFE  provisions  with  MINEDUC  supplements  thus   constitutes  a  best  practice  in  terms  of  dietary  diversity,  with  the  caveat  that  pinto  beans  should  be   replaced  by  black  beans  in  future  phases  of  FFE  implementation. (3) The  possible  impact  of  larger  societal  changes  and  migration  patterns  on  school   enrollment  and  attendance   One  of  the  most  significant  lessons  learned  in  the  final  evaluation  is  related  to  school  attendance  and   enrollment.  Attendance  for  boys  dropped  substantially  since  the  midterm  evaluation,  although  girls   continued  to  attend  at  target  rates.  Enrollment  also  dropped  significantly,  such  that  a  consistent  and   rather  drastic  downward  trend  in  enrollment  can  be  seen  from  baseline  to  final  evaluations.  While  a  best   practice  is  not  yet  connected  with  this  lesson  learned,  a  recommendation  in  this  report  is  conduct  a   barriers  analysis  to  enrollment  in  Totonicapán  (see  Recommendations  section).  Anecdotal  reports  suggest   that  drops  in  student  enrollment  are  a  countrywide  phenomenon  in  Guatemala,  and  that  these  decreases   may  be  related  to  migration  in  search  of  greater  economic  opportunities. (4) The  importance  of  communication  and  collaboration  at  multiple  levels  of   intervention   One  of  the  main  lessons  learned  according  to  stakeholders  is  that  teamwork  is  crucial  to  achieve  positive   results.  Communication  and  collaboration  is  essential,  not  only  with  community  members,  but  also  with   key  partners  such  as  MINEDUC  or  PRODESSA.  CRS  staff  have  taken  leadership  in  conducting   interinstitutional  meetings  on  a  regular  basis  to  discuss  project-­‐related  successes  and  issues among  a   variety  of  stakeholders.  Some  of  the  challenges  that  have  emerged  through  this  process  include   motivating  and  organizing  parents  and  community  members  to  participate  in  program  activities  such  as   food  preparation,  trainings,  and  bilingual  education.  Other  challenge is  related  to  duplication  of  efforts   related  to  bilingual  education,  such  as  the  different  methodologies  and  instructions  given  by  USAID   funded  programs  vs.  the  USDA  FFE  programs  implemented  in  the  same  schools.    For  example,  the  USAID   education  program  states  that  bilingual  education  should  begin  in  2nd grade  while  FFE  states  1st grade.   These  differences  understandably  introduce  confusion  among  stakeholders and  have  illustrated  the  need   to  communicate  and  collaborate  more  closely  with  other  US  Government  entities.   CRS  has  taken  a  leadership  role  in  addressing  the  issues  at  local,  municipal,  and  national  levels,  through   interorganizational  meetings  and  advocacy  to  advance  school  feeding  and  literacy  across  the  country. Information  on  best  practices,  gleaned  from  parent  focus  groups,  suggest  that  regular  interorganizational   meetings  allowed  stakeholders  to discuss  and  solve  issues  on-­‐site  and  were  also useful  for  identifying  key   contacts  that  were  helpful  when  encountering  problems  in  the  field. In  addition,  the  CRS  monitoring   system  allowed  stakeholders  to  identify  delays  or  complications  in  the  process  in  a  timely  manner  and   facilitated the  development  of  strategies  to  overcome  the  issues. 32 (5) The  central  role  played  by  parents  and  the  importance  of  parent  engagement   and  empowerment   The  ultimate  success  of  FFE  relies  on  the  commitment  of  parents  to  implement and  be  invested  in the   program.  Parent  focus  group  data  suggested  that  parents  felt  empowered  within committees  in  uniting school  authorities  and  community  members  around  shared  goals  and  objectives,  specifically  around  the   food  chain  process (reception,  storage,  preparation,  and  consumption). Parents  themselves  have also experienced  the  benefits  of  education,  through  ongoing  training  in  topics  like  sanitation.  Parents  have   been  able  to  translate  this  knowledge  beyond  the  school  environment  into  their  homes  and  communities,   for  example,  by  promoting  hygienic  practices  and  clean  environments  related  to  food  preparation. This   type  of  capacity  building  for  parents,  in  terms  of  increased  knowledge  and  skills,  thus  may  provide  a   diffusion  effect  beyond  the  school  into  communities,  and  also  serves  as  an  example  of  how  empowering   and  building  parents’  capacities  also  supports  long-­‐term sustainability of  program  activities. (6) The  importance  of  defining  and  emphasizing  sustainability,  and  communicating   what  has  already  been  accomplished  and  what  needs  to  be  accomplished   As  identified  in  the  midterm  evaluation,  the  issue  of  sustainability  remains  unclear  for  most  stakeholders,   and  particularly  for  parents.  Yet,  interviews  with  USDA  staff  revealed  a  vision  for  sustainability  that  does   not  focus  on  increased  parent  involvement  alone,  but  on  advocacy  and  collaboration  at  the  national  level,   and  among  governmental  and  private  donors.  As  mentioned  above,  CRS  has  exerted  significant  leadership   in  these  efforts  to  promote  sustainability  through  MINEDUC  and  governmental  support,  outside  of  which   the  sustainability  of  current  project  activities  (particularly  food  provision)  is  unlikely.  These  efforts  are   crucial  and  commendable,  and  should  be  communicated  more  to  project  stakeholders  in  the  local  and   municipal  levels.  The  result  of  this  communication  will  be  a  definition  of  sustainability  that  does  not   expect  parents  to  produce  food  provisions  on  their  own  to  match  current  USDA  support  – a  goal  that  is   highly  unlikely  if  not  impossible.  Having  an  understanding  of  what  CRS  and  USDA  mean  by  sustainability   could  empower  stakeholders  to  understand  where  they  can  best  contribute  to  long-­‐term  sustainability  of   the  project,  without  the  expectation  that  they  must  shoulder  the  program  in  the  absence  of  future   support. 33 DISCUSSION The  discussion  of  results  is  organized  by  the  crosscutting  questions guiding  the  evaluation.  We  will  discuss   implications  for  findings  in  each  of  the  following  areas:  (1)  Relevance;  (2)  Effectiveness;  (3)  Efficiency;  (4)   Impact;  and  (5)  Sustainability. A. Relevance   Evidence  suggests  that  project  activities  are  leading  to  the  intended  goals.  Particularly  for  3rd graders,   literacy  scores  increased  substantially  from  baseline  and  midterm  score.  However,  6th graders  fell  short  of   their  literacy  targets.  This  shortfall  is  not  likely  explained  by  teacher  attendance  which  was  highest  for  6th grade  compared  to  other  grades.  Yet,  according  to  teachers,  attendance  for  6th grade  boys  was  estimated   to  be  quite  low  (50.59%)  compared  to  girls  (85.04%).  Furthermore,  only  half  of  school  directors  believed   that  FFE  has  improved  school  enrollment.  At  least  one  CTA  stated  that  enrollment  is  declining  because   children  are  migrating  to  urban  areas  and  to  the  United  States  in  search  of  economic  opportunities.  It  is   therefore  possible  that,  for  6th graders,  the  demand  to  provide  economically  for  themselves  and  their   households  outweighs  the  importance  of  continuing  to  focus  on  education.  A  USDA  representative  noted   that  decreasing  school  enrollment  and  attendance  is  a  nationwide  problem  in  Guatemala.  This   representative  cited  another  organization  implementing  FFE  in  nearby  Huehuetenango  that  conducted  a   study  about  decreasing  enrollment.  This  study  found  that  children  were  abandoning  school  to  migrate  in   search  of  economic  opportunities  elsewhere.  Parents  in  the  current  study,  however,  did  not  identify   migration  or  economic  need  as  reasons  why  children  do  not  attend  school. Problems  with  food: About  1/3rd of  principals  identified  some  problems  with  food,  and  the  most  common   problem  was  the  type  of  food  (74.29%).    Qualitative  information  from  parent  focus  groups  (75%  of   schools)  revealed  the  type  of  beans  provided  by  FFE  (pinto  beans)  was  problematic  for  a  variety  of   reasons:  being  hard  to  cook, culturally  unfamiliar,  uncooked  beans  causing  stomach  problems,  and   expensive  given  the  amount  of  wood  needed to  cook  the  beans.  These  themes  are  similar  to  those   expressed  by  parents  in  the  midterm  evaluation  of  FFE. Benefits  of  curriculum:  Opinions  of  the  relevance  of  the  Kemom  Ch’ab’al  methodology  were  virtually   uniformly  positive.  Nearly  all  teachers  and  principals  reported  having  been  trained  in  the  Kemom  Ch’ab’al   methodology,  and  a  majority  of  teachers  and  principals  could  name  each  step  in  the  methodology.   Almost  all  principals  believed  that  Kemom  Ch’ab’al  was  effective  in  helping  improve  children’s  reading   comprehension. B. Effectiveness   Most results  indicators  increased  substantially  compared  to  baseline  and  midterm  findings.  However,  a   number  of  results  were  just  short  of  the  indicator  target.  These  included  literacy  for  6th grade  girls   (33.87%,  final  target=35.19%)  and  6th grade  boys  (32.98%,  final  target=34.31%),  teacher  attendance   (83.3%,  final  target=90.0%),  attendance  for  3rd grade  boys  (68.52%,  final  target=85.0%) and  attendance   for  6th grade  boys  (65.21%,  final  target=85.0%).    The  potential  reasons driving  these  lower  numbers  are   explored  in  the  Relevance  section  above. 34 Parents’  ability  to  identify  three  reasons  for  education  dropped  precipitously  to  16.81%,  compared  to   35.9%  at  the  midterm  evaluation,  and  well  below  the  final  target  of  80.0%.  One reason  might  be  the   manner  in  which  the  question  was  asked  of  parents:  “In  your  opinion,  why  is  education  important  for   children?”  Enumerators  were  also  instructed  not  to  prompt  parents  for  answers,  but  simply  to  record   what  parents  stated.    The  question  in  the  baseline  and  midterm  evaluations  was  as  follows:  “How  does   primary  education  help your  children?”  and  the  results  were  widely  divergent  in  baseline  (96.4%)  and   midterm  (35.9%)  evaluations.  In  the  future,  to  capture  this  indicator  more  effectively,  a better  question   might  be  as  follows:  “What  are  the  three  main  reasons  education  is  important  for  your  child?”    It  would   then  be  clear  to  parents  that  three  reasons  were  expected.  Otherwise,  this  indicator  may  be  measuring  a   parents’  reticence  to  engage  with  the  enumerator,  and  not  their  commitment  to  education.     Most  principals,  parents  and  teachers  believed  that  school  feeding  was  effective  in  helping  improve   students’  reading  comprehension  and  attentiveness.  In  fact,  principals  and  teachers  alike  identified   school  feeding  as  the  most  important  factor  in  influencing  attention,  above  teaching  materials,   techniques,  and  teachers’  attendance  rates.  As  noted  in  the  Relevance  section  above,  some  issues   emerged  in  midterm  and  final  evaluations  about  the  type  of  beans  provided. Parents  agreed  that  collaboration  is  essential  in  preparing  food  effectively  at  schools,  and  with  engaging   the  larger  community  to  support  its  development.  Project  staff  strongly  believed  that  teamwork  and   collaboration  among  project  actors  are  key  to  the  success  of  the  project.  Staff  pointed  to  working  closely   not  just  with  community  members,  but  with  government  agencies  like  MINEDUC  and  implementing   partners  such  as  PRODESSA  and  APCC.  The  unfortunate  events  that  unfolded  with  APCC  during  Phase  I   reinforced  the  need  to  work  in  close  collaboration  with  implementing  partners.  Regular   interorganizational  meetings  were  identified  as  a  key  strategy  to  support  the  project’s  success,   strengthen  collaboration,  and  avoid  duplication  of  efforts. C. Efficiency   More  than  ½ of  principals  indicated  that  another  modality  for  reducing  hunger  would  be  more  efficient   and  cost-­‐effective.  The  alternatives  provided  by  the  principals,  however,  focused  more  on  modifications   to  the  existing  program  rather  than  on  different  programming  models.  For  example,  principals  pointed  to   the  creation  or  expansion  of  school  gardens,  as  well  as  promoting  home  gardens  to  reduce  student   hunger  outside  of  the  classroom.  Some  reinforced  the  need  to  support  local  businesses  in  purchasing   food  and  to  provide  food  that  was  more  culturally  appropriate  for  children.  Other  issues  related  to  this   theme  were  drawing  from  existing  community  resources,  and  increasing  parent  involvement  in  the   program. D. Impact   In  terms  of  results  achieved  at  the  final  evaluation,  the  program  had  the  greatest  impact  on  increased   literacy  for  3rd graders;  teachers’  increased  use  of  new  and  innovative  teaching  techniques;  greater   attention  of  students  according  to  teachers;  school  attendance  (for  girls  only);  reduction  in  missed  classes   because  of  illness;  strengthened  school  governments;  more  disbursements  arriving  on  time;  greater   number  of  PTAs  contributing  to  school  improvement  plans;  and  greater  training  and  increased  capacity  of   MINEDUC  staff. 35 The  program  fell  short  of  its  target  in  increasing  literacy  for  6th graders.  In  addition,  the  number  of  parents   who  identified  three  reasons  why  education  is  important  dropped  considerably.    Attendance  for  3rd grade   and  6th grade  boys  has  also  dropped.  The  possible  reasons  for  these  findings  have  been  explored  above.   Teacher  attendance  has  declined  slightly  since  the  midterm  evaluation.   The  mechanisms  leading  to  improvement  of  outcomes  appear  related  to  school  feeding,  improved   teaching  methodologies,  strengthened  bilingual  education,  and  improved  community  involvement  in   school  feeding  and  governance.  The  missed  targets  may  be  the  result  of  larger  societal  trends  in   Guatemala  around  migration  and  economic  need  (in  the  case  of  literacy  and  attendance  for  6th graders),   or  problematic  measurement  (in  the  case  of  parents’  identification  of  reasons  why  education  is   important). E. Sustainability   CTAs tended  to  believe  the  program  is  sustainable  without  CRS  and  USDA  support,  but  fewer  teachers   agreed  with  this  assessment.  Less  than  1/3rd of  principals  believe  the  program  would  be  sustainable and   only  about  1/3rd of  principals  had  considered  a  plan  for  sustainability  after  USDA  and  CRS  support  ends.   Parent  focus  groups  in  80%  of  the  schools  identified  sustainability  as  a  problem,  and  expressed  that  as  a   community,  they  would  not  be  able  to  support  school  feeding  in  the  absence  of  CRS  and  USDA  support.     A  conversation  about  sustainability  with  USDA  representatives  was  helpful  in  conceptualizing  what   sustainability  might  look  like  for  FFE  in  Guatemala.  The  pursuit  of  sustainability  should  occur  at  multiple   levels,  from  advocacy  at  the  policy  level,  government  budgets  to  support  improved  policy,  and   strengthened  collaborations  with  local  organizations  and  communities  to  improve  capacity  for   implementation.  USDA  representatives  indicated  that  CRS  has  been  active  at  each  of  these  levels,  and   there  is  evidence  to  suggest  that  the  Government  of  Guatemala  may  pass  a  bill  that  requires  school   feeding  for  all  children  in  the  country.  Importantly,  in  the  context  of  the  current  study,  all  CTAs   interviewed  noted  improvements  in  the  government  policy  and  regulatory  frameworks,  and  77%   reported  seeing  an  increase  in  the  capacities  of  government  institutions  during  the  course  of  FFE   implementation.  Project  staff  also  reinforced  the  need  to  pursue  sustainability  through  strengthened   collaborations  with  public  and  private  institutions,  and  community  members. 36 RECOMMENDATIONS 1) Conduct  a  barriers  analysis  to  explore  why  literacy  for  6th  graders  has   plateaued.   At  final  evaluation,  6th grade  boys  and  girls  alike  made  little  progress  on  reading  comprehension   compared  to  the  midterm  evaluation.  Yet,  at  midterm,  both  boys  and  girls  fared  considerably  better   compared  to  baseline,  such  that  the  plateaued  literacy  rates  at  final  evaluation  are  unexpected.  We  have   speculated  in  this  report  about  some  of  reasons  behind  a  lack  of  increase  in  literacy,  particularly  related   to  lower  enrollment  and  attendance  more  generally.  However,  more  information  is  needed  to  examine   the  root  causes  of  stalled  literacy  rates,  to  test  the  hypothesis  that,  at  6th grade,  children  are  balancing   the  competing  demands  of  schoolwork  and  economic  need  to  work. 2) Conduct  a  barriers  analysis  to  examine  reasons  behind  decreasing  school   enrollment  and  attendance.  Also  examine  why  teachers’  reports  of   attentiveness,  and  observed  attentiveness,  are  significantly  different.   Enrollment  dropped  substantially  for  both  boys  and  girls  since  the  midterm  evaluation,  and  especially   compared  with  the  baseline  evaluation.  Stakeholders  have  noted  that  schools  around  the  country  are   experiencing  related  declines  in  enrollment,  a  case  example  being  in  Huehuetenango.  We  speculated  in   this  report  that  economic  demands  and  migration  may  be  affecting  school  attendance  and  enrollment,   and  there  is  some  evidence  to  suggest  these  factors  are  increasing  in  the  region.  A  barriers  analysis  study   should  be  conducted  to  examine  the  drivers  of  lowered  enrollment  in  Totonicapán. At  final  evaluation,  girls  (both  3rd and  6th grade)  exceeded  their  targets  for  school  attendance.  Their   attendance  was  much  higher  than  at  midterm  where,  despite  increasing  since  baseline,  they  did  not   increase  as  much  as  boys.  However,  boys’  attendance  (both  3rd and  6th grade)  dropped  precipitously  since   midterm,  a  surprise  given  their  increased  midterm  attendance  compared  to  baseline.    A  barriers  analysis   study  should  therefore  be  conducted  to  examine  the  causes  of  lowered  attendance  especially  for  boys. This  study  also  revealed  a  significant  discrepancy  between  teachers’  reports  of  student  attentiveness  in   the  classroom,  compared  with  structured  observations  of  student  attentiveness. As  part  of  this   recommended  study,  CRS  should  examine  teachers’  expectations  for  student  attentiveness,  and  consider   training  teachers  on  expected  behavior  for  students  in  the  classroom.  The  structured  Observation  of   Classroom  Conditions presented  in  Table  5 of  this  report  may  be  helpful  in  reinforcing  the  importance  of   student  attentiveness  and  what  attentiveness  in the  classroom  might  look  like. 3) Conduct  a  barriers  analysis  to  examine  reasons  behind  low  teacher  attendance   rates.     At  final  evaluation,  teacher  attendance  increased  slightly  from  midterm,  but  was  still  lower  than  at   baseline,  and  below  the  target  for  the  results  indicator.  As  noted  in  the  midterm  evaluation  report,   relying  solely  on  directors’  observations  may  have  implicit  and  unknown  biases,  and  it  would  be   preferable  to  have  a  standardized  way  of  collecting  data  from  schools.  A  barriers  analysis  should  be conducted  to  examine  why  teacher  attendance  seems  to  have  stalled  below  the  final  target  indicator.  The   midterm  evaluation  also  recommended  a  number  of  incentives  to  improve  teacher  attendance:   rewarding teachers  each  semester  who  have  90%  or  more  attendance;  offering in-­‐school  trainings  to   assist  teachers  in  obtaining  specialized  certifications;  and  offering teachers  take-­‐home  rations,  the   37 amount  of  which  is  linked  to  their  attendance. A  barriers  analysis  could  also  examine  whether  the   introduction  of  such  incentives leads  to  increased  attendance. 4) Vary  the  food  provided  by  the  program,  and  ensure  that  food  is  culturally   appropriate.   As  in  the  midterm  evaluation,  many  parents  and  teachers  complained  about  the  pinto  beans  being   difficult  to  cook,  requiring extensive  wood  for  cooking,  and  that  the  children  did  not  like  pinto  beans.  A   conversation  with  a  USDA  representative  revealed  that,  based  on  earlier  feedback,  pinto  beans  will  be   replaced  with  the  more  culturally  appropriate  black  beans.  Respondents  also complained  about  the   monotony  of  children  eating  the  same  or  similar  foods  every  day.  The  research  team  learned  that   MINEDUC  also  provides  funding  for  school  snacks,  with  which  parents  can  purchase  fresh  fruits,   vegetables,  eggs,  and  other  provisions.  This  recommendation  is  therefore  twofold:  (a)  explore  with  USDA   whether  a  greater  variety  of  food  could  be  provided  through  FFE,  with  input  and  direction  from   community  stakeholders;  and  (b)  provide  trainings  to  parent  volunteers  charged  with  purchasing  food   from  MINEDUC  stipends,  related  to  dietary  diversity. 5) Reinforce  the  benefits  of  education  with  parents  and  improve  future   measurement  of  this  indicator.   At  16.81%,  the  number  of  parents  who  identified  three  reasons  why  education  is  important  is  extremely   low.  This  number  also  represents  a  substantial  decrease  from  the  midterm  (35.9%) after  being  assessed   at  96.4%  at  baseline.  Given  this  variability,  we  speculate  that  measurement  problems  account  for  this  low   rate.    As  stated  in  the  report,  a  better  question  to  be  asked  in  the  future  would  probably  be  as  follows:   “What   are   three   benefits   of   education  for   your   child?”.   Otherwise,   a   more   open-­‐ended   question   may   simply   be   measuring   a   parent’s   reticence   to   engage   with   the   enumerator,   rather   than   an   accurate   assessment  of  their  commitment  to  their  child’s  education.  Nevertheless,  parents  are  critical  resources  in   children’s   education.     Continued   efforts   need   to   be   made   to   reinforce   to   parents   the   importance   of   education  for  children,  not  just  in  terms  of  literacy  but  also  on  the  lifelong  benefits  education  can  provide. 6) Strengthen  and  coordinate  training  on  bilingual  education  for  teachers   A  number  of  issues  were  raised  around  bilingual  education  within  FFE,  which  are  described  in  the  table   below,  along  with  recommendations  for  addressing  each  issue: Table  27.  Issues  and  Recommendations  Related  to  Bilingual  Education Issue   Recommendation   Differences  in  K’iche’  language  not  accounted  for  in  Kemom   Ch’ab’al (Refer  to  page  10) Provide  the  Spanish  translation  for  words  and  passages   in  K’iche’  so  that  teachers  can  translate  from  Spanish   to  the  appropriate  K’iche’  dialect;  and/or,  include  a   K’iche’  dictionary  in  Spanish  to  provide  guidance Many  teachers  do  not  speak  K’iche’  are  not  comfortable   teaching  it (Refer  to  page 10) Conduct  teacher  trainings  to  enhance  their  capacity  to   apply  the  techniques  in  both  Spanish  and  K’iche’,  even   if  teachers  are  not  fluent  in  K’iche’ Schools  are  receiving  different  messages  about  teaching   bilingual  education,  e.g.,  USAID-­‐funded  projects  recommend   that  bilingual  education  begin  at  2nd grade  while  Kemom   Ch’ab’al  begins  in  1st grade (Refer  to  page  30) Continue  CRS  leadership  in  interorganizational  groups   to  address  this  challenge  and  develop  a  strategy  for   aligning  the  methodologies  for  consistent  messaging   across  MINEDUC,  USAID,  and  USDA  projects 38 7) Consider  expanding  the  Spaces  to  Grow  program,  and  highlight  potential  benefits   to  directors  and  teachers   While  the  goal  of  the  project  was  to  implement  the  Spaces  to  Grow  program  in  1/3rd of  the  schools,  and   this  target  was  almost  achieved  with  25.66%  of  schools  having  the  program  at  the  end  of  the  overall   project,  it  is  recommended  that  this  program  expand  and  that  all  school  consider  implementation.  The   vast  majority  of  principals  (91.51%)  indicated  they  would  be  interested  in  implementing  Spaces  to  Grow.   As  noted  in  the  midterm  evaluation,  efforts  could  be  made  to  expand  this  program  to  more  schools,  and   reinforce  the  program  in  existing  schools.  Spaces  to  Grow  might  also  be  a  centerpiece  around  which   further  extracurricular  activities  could  be  built.  In  addition,  Spaces  to  Grow  may  be  an  effective  way  to   identify  the  stalled  literacy  rates  for  6th graders  (see  Recommendation  #1)  and  further  study  could   identify  whether  the  program  helps  in  school  retention. 8)  Articulate  what  sustainability  looks  like  for  the  FFE  program,  communicate  this   with  beneficiaries,  and  enhance  beneficiaries’  capacities  to  operate   independently.   Most  beneficiaries  noted  that  their  belief  that  FFE  is  not  sustainable  once  CRS  and  USDA  support  ends.   Conversations  with  USDA  representatives  provided  a  helpful  framework  of  the  meaning  of  sustainability,   as  well  as  the  extent  to  which  CRS  is  involved  as  a  leader  in  Guatemala  in  terms  of  school  feeding  and   strengthening  academics  countrywide.  As  one  USDA  representative  noted,  sustainability  needs  to  occur   at  multiple  levels,  including  with  policy  advocacy,  interorganizational  work  (with  government  agencies,   and  local  and  international  NGOs),  and  capacity  building  of  local  communities,  schools,  and  parent   volunteers.   Given  the  responses  of  beneficiaries  in  relation  to  sustainability,  CRS  should  communicate  more  to   communities  and  schools  about  the  achievements  and  expectations  for  sustainability.  CRS  should   document  and  present  to  communities  the  variety  of  activities  in  which  CRS  is  engaged  to  build   sustainability  for  FFE  in  Totonicapán  but  also  in  Guatemala  as  a  whole.  Also,  as  noted  in  a   recommendation  from  the  midterm  evaluation,  CRS  should  also  train  school  staff  and  parent  volunteers   on  expectations  for  sustainability  after  CRS  and  USDA  support  is  removed.  This  training  would  likely   involve  capacity  building  in  the  purchasing  of  food,  the  selection  of  food  for  nutrition  and  dietary   diversity,  as  well  as  principles  and  skills  of  bilingual  education.  The  development  of  a  formal  phase-­‐out   plan  may  be  helpful  for  CRS  to  develop  in  anticipation  of  USDA’s  ending  support  for  FFE  – even  if  such  a   plan  is  not  shared  externally  until  it  is  needed. 39 REFERENCES Alderman,  H.,  &  Bundy,  D.  (2012).  School  Feeding  Programs  and  Development:  Are  We  Framing  the   Question  Correctly?  World  Bank  Research  Observer,  27(2),  pp.  204-­‐221.  doi:   10.1093/wbro/lkr005 Alderman,  H.,  Gilligan,  D.,  &  Lehrer,  K.  (2012).  The  impact  of  Food  for  Education  programs  on  school   participation  in  northern  Uganda.  Economic  Development  and  Cultural  Change,  61(1),  pp.  187-­‐ 218. Cheung,  M.  &  Perrotta,  M.  (2010)  The  Impact  of  Food  For  Education  Program  On  Schooling  in  Cambodia.   Asia  and  the  Pacific  Policy  Studies  (APPS),  2(1),  pp.  44-­‐57.   Martorell,  R.  (1999)  The  Nature  of  Child  Malnutrition  and  Its  Long-­‐Term  Implications.  Food  and  Nutrition   Bulletin,  20(3),  pp.  288-­‐292.   Moon,  B.,  &  Alarid,  L.  F.  (2015).  Negative  School  Environment  Index [Database  record].  Retrieved  from PsycTESTS.  doi: http://dx.doi.org/10.1037/t39206-­‐000 USDA  (2016).  McGovern-­‐Dole  Food  for  Education  program. Retrieved  from:   http://www.fas.usda.gov/programs/mcgovern-­‐dole-­‐food-­‐education-­‐program. WFP  (2007)  Full  Report  of  the  Thematic  Evaluation  of  the  WFP  School  Feeding  in   Emergency  Situations. (A  Report  from  the  Office  of  Evaluation  No.  OEDE/2007/06).  Retrieved   from   http://documents.wfp.org/stellent/groups/public/documents/reports/wfp127463.pdf?_ga=1.801 3938.1583969398.1468935841.     Zullig,  K.  J.,  Collins,  R.,  Ghani,  N.,  Hunter,  A.  A.,  Patton,  J.  M.,  Huebner,  E.  S.,  &  Zhang,  J.  (2015).  School   Climate Measure-­‐-­‐Revised [Database  record].  Retrieved  from  PsycTESTS.  doi: http://dx.doi.org/10.1037/t45748-­‐000 40 APPENDICES   APPENDIX  A:    RESULTS  INDICATORS  (see  attached) APPENDIX  B:    SCHOOL  ENVIRONMENT  MEASURE  (MEDICIÓN  DEL  AMBIENTE  ESCOLAR)  (see  attached) APPENDIX  C.  FOCUS  GROUPS  ANALYSIS  SUMMARY  (see  attached) APPENDIX  D.  THEMES  AND  CODES  FROM  QUALITATIVE  ANALYSIS  (see  attached) APPENDIX  E.  FINAL  EVALUATION  MEASUREMENT  INSTRUMENTS  (see  attached)