This publication was produced at the request of the United States Department of Agriculture. It was prepared independently by EQSA Consulting, S.A. Foreign Agricultural Service, United States Department of Agriculture Guatemala Nuestra Cosecha Local Regional Procurement Baseline Evaluation September 2019 DISCLAIMER: The author’s views expressed in this publication do not necessarily reflect the views of the United States Department of Agriculture or the United States Government. Nuestra Cosecha Baseline Evaluation Report Project Concern International (PCI), in association with Save the Children International (SCI) and Catholic Relief Services (CRS) will implement the “Nuestra Cosecha” project with funding from the United States Department of Agriculture (USDA). The project will focus on the local procurement of foods to support the school feeding activities promoted by these organizations under their own Food For Education (FFE) McGovern-Dole (MGD) programs. This baseline evaluation will determine the situation, context and indicator values at the start of the project, so that changes over the life of the project can be measured during a final evaluation Agreement Number: LRP-520-2018/014-00 Project Duration: 2018-2021 Implemented by: Project Concern International Evaluation Authored by: Julio de la Roca, EQSA, S.A. 3 Table of Contents Acronym List ...................................................................................................................................... 6 1. Executive Summary ............................................................................................................ 7 1.1 Introduction.............................................................................................................................. 7 1.2 Methodology ............................................................................................................................ 8 1.3 Results...................................................................................................................................... 8 1.3.2 Average number of days per year that are needed to complete the entire improved procurement process. ................................................................................................ 9 1.3.3 Percentage of schools serving school meals with an adequate dietary diversity, as a result of USDA assistance. ........................................................................................ 9 1.3.4 Percentage of students who state that they like the school meals, using a five-point facial hedonic scale ................................................................................................... 9 1.3.5 Percentage of schools using fresh fruit and/or vegetables in daily school meals, as a result of USDA assistance ......................................................................................... 9 1.3.6 Percent of school-aged children receiving a minimum acceptable diet .......................... 9 1.3.7 Percent of OPF members who demonstrate sufficient knowledge of the LAE and its regulation .................................................................................................................. 9 2. Introduction and Objectives .............................................................................................. 10 2.1 Background ............................................................................................................................ 10 2.2. Project Background and Objectives..................................................................................... 11 3. Context of the Target Area................................................................................................ 12 4. Baseline Methodology....................................................................................................... 16 4.1 Secondary Data Review......................................................................................................... 16 4.2 Inception and Coordination Sessions..................................................................................... 16 4.3 Sampling Strategy.................................................................................................................. 16 4.3.1 Treatment Schools.......................................................................................................... 17 4.3.2 Control Schools ............................................................................................................. 17 4.3.3 Interviews....................................................................................................................... 18 4.4 Instruments............................................................................................................................. 20 4.5 Process for Verifying Data Collection Quality ...................................................................... 20 4.6 Study Limitations................................................................................................................... 22 4. Results............................................................................................................................... 23 5.1 Improved Effectiveness of Food Assistance through Local and Regional Procurement (SO1) ...................................................................................................................................... 23 5.1.1 Improved Cost-Effectiveness of Food Assistance (LRP1.1)........................................... 23 4 5.1.2 Improved Timeliness of Food Assistance (LRP 1.2)...................................................... 27 5.1.3 Improved Utilization of Nutritious and Culturally Acceptable Food that Meet Quality Standards (LRP 1.3)................................................................................................ 29 5.1.4 Improved Capacity of Relevant Organizations (LRP 1.4.3).......................................... 36 5.2 Indicator Table ....................................................................................................................... 39 5.3 Qualitative analysis of key evaluation questions................................................................... 40 5.3.1 Relevance ................................................................................................................ 40 5.3.2 Performance ............................................................................................................ 42 5.3.3 Sustainability........................................................................................................... 43 5.3.4 Effectiveness............................................................................................................ 44 6. Conclusions................................................................................................................................... 44 7. Recommendations......................................................................................................................... 47 Annexes I. Minimum Acceptable Diet Survey II. Director Survey III. OPF Survey IV. Food Satisfaction Tool V. Significance tests Minimum Acceptable Diet VI. Significance Tests OPF VII. Significance Tests Food Satisfaction VIII. List of Suppliers IX. Evaluation Team X. Terms of Reference *Annexes are included as separate documents from this report. 5 Tables Table 1. Number of Schools to Evaluate in the Nuestra Cosecha Baseline ...................................... 17 Table 2. Classification of Intervention and Control Municipalities.................................................. 18 Table 3. People Reached During the Study....................................................................................... 20 Table 4. The Minimum Diet Instrument ........................................................................................... 21 Table 5. The LAE Knowledge Instrument ........................................................................................ 21 Table 6. Average price in kilograms for eggs, fresh fruit and fresh vegetables................................ 24 Table 7. Limitations for local producers to market their products in schools................................... 25 Table 8. Main food products produced by department ..................................................................... 25 Table 9. Average number of days per distribution for the procurement of perishables.................... 27 Table 10. Number of times per year each activity in the procurement process is conducted to purchase perishables.......................................................................................................................... 28 Table 11. Average number of days per year for the procurement of perishables.............................. 28 Table 12. Average number of days per distribution for the procurement of non-perishable products ........................................................................................................................................................... 28 Table 13. Number of times per year each activity in the procurement process is conducted to purchase non-perishable products..................................................................................................... 28 Table 14 . Average number of days per year for the procurement of non-perishables..................... 28 Table 15. Food groups consumed the previous day by Department ................................................. 33 Table 16. Summary of Results.......................................................................................................... 39 Graphs Graph 1. Number of food groups served with USDA assistance ...................................................... 29 Graph 2. How do you rate the food received?................................................................................... 30 Graph 3. Students who like school meals by type of school............................................................. 31 Graph 4. Students who liked school meals by sex and department................................................... 31 Graph 5. Fresh Fruits and/or Vegetables........................................................................................... 32 Graph 6. Food groups consumed the previous day ........................................................................... 33 Graph 7. Number of meals consumed the previous day.................................................................... 34 Graph 8. Children with a minimum diet by department and type of school ..................................... 34 Graph 9. Children with a minimum diet by department.................................................................... 35 Graph 10. Children with a minimum diet by type of school............................................................. 35 Graph 11. Students with minimum diet by sex ................................................................................. 35 Graph 12. Basic knowledge about the LAE...................................................................................... 36 Graph 13. Basic knowledge about the LAE by sex........................................................................... 36 Graph 14. Basic knowledge about the LAE by department and by group of schools (Nuestra Cosecha, Intervention and Control) .................................................................................................. 37 Graph 15.Basic knowledge on the LAE by school group ................................................................. 37 Graph 16. Training / Achievement.................................................................................................... 38 6 Acronym List CNB National Basic Curriculum CRS Catholic Relief Services DIGEFOCE General Directorate for Strengthening of the Educational Community EDUCAMOS Educación con Calidad y Alimentación Mejorada, Orientadas a la Sostenibilidad FAO United Nations Food and Agriculture Organization FFE Food for Education INE National Statistics Institute INTECAP Technical Training and Productivity Institute LAE School Feeding Law MAGA Ministry of Agriculture and Food MGD McGovern-Dole MINEDUC Ministry of Education OPF School Parent Organizations PCI Project Concern International SAT Tax Administration Superintendency SCI Save the Children International SEGEPLAN General Secretariat for Planning and Programming SPSS Statistical Package for the Social Sciences USDA United States Department of Agriculture 7 1. Executive Summary 1.1 Introduction Project Concern International (PCI), in association with Save the Children International (SCI) and Catholic Relief Services (CRS) will implement the “Nuestra Cosecha” project with funding from the United States Department of Agriculture (USDA). The project will focus on the local procurement of foods to support the school feeding activities promoted by these organizations under their own Food For Education (FFE) McGovern-Dole (MGD) programs. The project area includes 50 schools, 15 of which are located in the Department of Huehuetenango under PCI’s program (seven schools in the municipality of Santa Eulalia and eight schools in Barillas). SCI’s project coverage includes the department of El Quiché, where 15 schools were selected (12 in the municipality of Nebaj and three in Chajul). CRS covers the largest number of schools, with 20 in the municipality of Momostenango, department of Totonicapán. This project will be implemented from September 2018 through September 2021. The project’s application of the National School Feeding Law (LAE) seeks to improve student nutrition by incorporating locally-produced fruits and vegetables in school menus, which will also help invigorate the local economy. The goal of Nuestra Cosecha is to improve the effectiveness of food assistance through the procurement of local and regional food products that are highly nutritious and culturally acceptable, in a cost-effective and timely fashion, using quality standards. The project’s four objectives are to: a. Increase the capacity of municipalities, local producer groups, and schools to effectively and efficiently procure and deliver local, culturally acceptable food commodities to supplement school feeding programs in a timely and cost-effective way. This includes supporting local producer groups in registering with the Tax Administration Superintendency (SAT) and developing marketing actions. b. Foster capacity development of stakeholders to strengthen the transition to a school feeding program based on local markets, in line with the government of Guatemala’s national LAE. c. Support School Parents Organizations (OPF) to strengthen their capacity in developing procurement plans, selecting commodities, soliciting bids, and procuring locally grown fresh foods. d. Partner with key government stakeholders at national, departmental, and municipal levels in the operationalization of the LAE, using target schools as a model for the law’s national implementation. With achievement of the Nuestra Cosecha project objectives, the quality of student diets will improve significantly, resulting in an improved quality of education. Additionally, by promoting local productivity and economic impact, the project will help reduce the root causes of migration of parents and of entire families, who travel to other areas and internationally in search of livelihood opportunities. Thus, PCI, SCI, and CRS have the opportunity to generate a positive socio-economic impact for local families in the target areas, while improving the nutritional quality of school children’s diets, along with a model to promote family farming, based on the mandates of the LAE, that could be scaled at the national level. 8 1.2 Methodology a. Evaluators reviewed existing documentation and held meetings with PCI, CRS, and SCI staff to organize the field work. Three groups of schools were identified for this study, including: a) schools targeted by Nuestra Cosecha, b) schools benefiting only from the regular MGD, USDA program; and c) a control group of schools with resources provided only by the Ministry of Education (MINEDUC). b. Training sessions on the Mobenzi data collection tools and system were conducted for field staff. c. The study planned for an average of 25 students per school to be interviewed, including students from grades four, five, and six. Overall, 1,104 students were interviewed. d. Focus groups were planned with OPF members from 45 schools, using the METAPLAN1 technique, and a total of 44 school OPFs were interviewed. Members from one OPF in a control school (in Nimsituj, Momostenango) did not attend the focus group meeting because the principal did not invite them. e. All OPF members who attended the focus group sessions were also interviewed by the evaluators, completing 156 individual interviews. f. The field work was carried out by four teams, and each had a translator for the specific language of the area, including Q´anjob`al, Ixil and Quiché, to facilitate communication with the students and OPF members who did not speak Spanish. For the most part, OPF members did not require translation. Many of the students preferred to be interviewed in their native language, which facilitated the work. g. In all schools visited, whether they were part of Nuestra Cosecha, the regular MGD program, or the control group, school principals, students, and OPF members willingly contributed to this evaluation. 1.3 Results Below is a summary of the results obtained from measuring 11 target indicators (see Table 16, under Section 6.3 of this report). All contents of this report are based on the analysis of the data collected from 15 schools targeted by Nuestra Cosecha, 15 schools participating in the regular MGD program, (5 for each implementing organization), and 15 control schools not receiving USDA support. 1.3.1 Products prioritized in school menus Evaluators identified the prices paid by local schools for foods on the list of 12 basic products (eggs, chicken, cheese, broccoli, tomatoes, potatoes, carrots, onions, apples, plantain, avocados, and oranges) using MINEDUC funds. Fruits included: oranges (five units per kilo), plantain (five units per kilo), and tomatoes (which is a fruit according to the technical definition). The projected average price of fruits in December 2019 will be US $1.15. The projected average price of vegetables in December 2019 will be US $1.10. These include potatoes (technically a tuber, 2.2 units per kilo), onions (technically a bulb, 2.2 units per kilo), and carrots (a vegetable, four units per kilo). Finally, the average cost per kilo of fresh eggs (15 units per kilo) by December 2019 will be US $2.12. 1 METAPLAN is a qualitative data collection methodology focused on “brainstorming.” It involves community leaders developing ideas, solutions, recommendations and action plans. 9 1.3.2 Average number of days per year that are needed to complete the entire improved procurement process. Schools spend an average of 53.8 days per year on local food procurement activities (bids, tests, payment). In practice, evaluators identified that the averages should be calculated separately for perishables (74.78 days per year) and non-perishables (32.82 days per year), given that the schools do not have refrigeration equipment to store the perishable products with a short shelf life. In addition, the number of days of consumption covered for each purchase could not be calculated. 1.3.3 Percentage of schools serving school meals with an adequate dietary diversity, as a result of USDA assistance. All schools in the study, including both the intervention and control groups, have adequate dietary diversity in school meals. One contributing factor to this result is the application of the LAE, which promotes varied menus that include fruits, vegetables, and meat by allocating four quetzals per student per day. In intervention schools, OPFs promote greater diversity and quality in the menus. 1.3.4 Percentage of students who state that they like the school meals, using a five-point facial hedonic scale Of the students interviewed in Nuestra Cosecha schools, 91.9% liked the food they received, as did 94.1% in the regular MGD program and 84.4% in schools without a USDA intervention. 1.3.5 Percentage of schools using fresh fruit and/or vegetables in daily school meals, as a result of USDA assistance The value for this indicator in Nuestra Cosecha is 100% because all schools surveyed include fresh fruits and/or vegetables in daily school meals. This is because of MINEDUC contributions or donations from the community. Additionally, 100% of control schools included fruits and vegetables with vitamin A, and 92.9% included other fruits and vegetables in the school meals as a result of MINEDUC support. 1.3.6 Percent of school-aged children receiving a minimum acceptable diet The results showed that 67.7% of students in schools that will be targeted by Nuestra Cosecha received a minimum acceptable diet, while 74.7% of students in MGD schools received a minimum acceptable diet. Finally, 73.9% of students in schools that don’t receive USDA support (only MINEDUC) received a minimum acceptable diet. 1.3.7 Percent of OPF members who demonstrate sufficient knowledge of the LAE and its regulation This indicator was measured using a five-question test given to OPF members, where sufficient knowledge was indicated by answering at least four of the five questions correctly, for a score of 80% or higher. Of the OPF members interviewed, 43.6% obtained a score of 80% or higher. In Nuestra Cosecha schools, the percent was 34.3%, in regular MGD beneficiary schools, 52.2%, and in control schools, 48.8%. Women obtained the lowest average scores on these tests. 10 2. Introduction and Objectives 2.1 Background A basic rule for countries trying to find the road to development is to build an educational system that responds to the knowledge gaps of the population, so that when they grow and there is a generational shift, they have the capacities needed to achieve substantial improvements. In Guatemala, the educational system has serious weaknesses that should be overcome, from the application of the National Base Curriculum (CNB) to the differentiation of the school cycle depending on location, customs, cultures, languages, and internal migrations linked to sugar cane or coffee harvests. To achieve these structural changes, public policies are aimed at children, who should be fed with a minimum acceptable diet to ensure they will become healthy, active people to be able to learn. When statistics were analyzed for populations vulnerable to food insecurity, the difficulty of breaking the cycle of hunger and poverty was noted. In the 2018 Report on the Food and Nutrition Security Panorama in Latin America and the Caribbean, it was noted that the undernutrition index has not decreased and has been stagnant for more than eight years in Guatemala. For this reason, in addition to providing food, programs such as the MGD FFE program address a series of comprehensive activities that lead to community develop. These include health interventions through the promotion of a series of behavior patterns for personal hygiene in children, the production of fresh foods in school gardens, improvement of school infrastructure, improvement of educational quality through trainings for teachers and the provision of educational materials, the provision of literacy materials in Mayan languages, the formation of Women Empowered groups, which seek comprehensive empowerment of women, emphasizing economic, social and political aspects, and the diffusion of the LAE in different environments. Additionally, the Nuestra Cosecha project focuses on the local procurement of foods to support FFE school feeding projects to complement the interventions and directly impact students’ health and nutrition. The school feeding programs makes it possible for communities to unite and organize, which generated important, visible social capital. The OPF structure supports the school feeding commission, and the members understand that everyone should work together on a common objective: ensuring that children receive a daily school meal. Decree number 16-2017, the LAE, is an important solution for the students’ nutritional needs in Guatemala, with the allocation of four quetzals per student per day to schools and the expectation that this amount will increase to five quetzals by 2020. Additionally, the law is expected to reactivate and energize the local economy with a provision that promotes family farming (article 4 section 3 regulation), although given the challenges local families face in undertaking food production activities that generate a surplus, as well as a lack of water for irrigation, lack of capital, and the requirement to register with the SAT, this provision is not currently being implemented. Without an intervention to address these challenges, it will likely never be implemented. A possible solution is to develop a food production model at the community level based on collaboration and associativity, but it will take time to develop local capacities enough for schools to purchase from these associative entities, and thus the norm under this project will be to continue to buy from suppliers outside the targeted communities. 11 2.2. Project Background and Objectives The Nuestra Cosecha project that will be implemented by PCI, SCI, and CRS will focus on achieving the following objectives: a. Increase the capacity of municipalities, local producer groups, and schools for the efficient and effective procurement and delivery of local, culturally acceptable food products to complement the school feeding programs in a timely and cost-effective manner. This includes support for local producer groups in registering and marketing their business. b. Promote capacity development of local actors to transition to a school feeding program based on local markets, in compliance with the LAE of Guatemala. c. Support the OPF, strengthening their capacity to develop purchase plans, select food products, request bids, and procure fresh, locally produced foods. d. Partner with government stakeholders at the national, departmental, and municipal levels to operationalize the LAE, using the target schools as models for implementation of the law at the national level. The project will be implemented in 50 schools in three departments. PCI will work in the municipalities of Santa Eulalia and Barillas in the department of Huehuetenango, CRS will cover the municipality of Momostenango in the department of Totonicapán, and SCI will work in the municipalities of Nebaj and Chajul in the department of El Quiché. The project will strengthen local small and medium producers2 in the family farming element promoted in the LAE, strengthen the OPFs, and promote food safety and quality. School feeding activities will directly benefit 11,528 primary school students in the targeted schools in these communities. The identification of work areas was based on the criteria used to determine schools to work with in the MGD project: volume and variety of agriculture production, year-round accessibility, willingness on the part of the local government and educational community, high priority areas for the government of Guatemala, and adequate infrastructure for storing products. 2.3. Baseline Evaluation Objectives The following objectives were established for this baseline study: a. Determine the current situation of ownership, efficacy, and sustainability in Nuestra Cosecha schools and communities, based on the Results Framework and the main activities proposed in the project. b. Analyze and compare the data obtained from the schools that will participate in Nuestra Cosecha with those obtained from schools with only MGD support and those without any USDA support. c. Determine the baseline values for project indicators, including an analysis of the proposed targets. 2 Producer families were defined as: 1) A family group organized as a productive unit that is mobilized daily; 2) Labor mainly comes from family, but without limiting the rights of its members, they could hire temporary labor for complementary activities; 3) The production unit is managed by a member of the family; 4) 75% of the household’s income benefits the family unit; and 5) Small scale farming is carried out on rented land or on land that is owned but without legal security and with agricultural limitations. 12 In addition, the evaluation terms of reference specified the following learning questions that should be answered using results from the evaluation. Relevance o Determine potential problems or challenges across project beneficiaries to achieve progress towards results students, o Determine how well project interventions align with stakeholder policies and priorities Performance o Determine baseline values for required project indicators o Assess to what extent the indicator targets are realistic and appropriate o Assess beneficiaries’ knowledge and skills to refine capacity building and training activities o Identify opportunities and threats to project implementation Sustainability o To what extent should local actors (private sector, community entities/associations, local government agencies, etc.) be engaged and committed to ensure sustainability? o What specific institutional capacity needs (e.g., for key GoG, municipal, community and school level stakeholders) should the project address to foster greater engagement and accountability? Effectiveness o Determine to what extent the Results framework is realistic, appropriate and practical to implement, and identify if design, structure, logic or management need modifications to facilitate the achievement of desired outcomes and impact o Generate baseline data for comparative analysis (using quasi-experimental design) o Validate project strategies and assumptions 3. Context of the Target Area Department of Huehuetenango Huehuetenango is located in the northwestern area of the country at 15º19’14’’ latitude and 91º28’13’’ longitude. The department borders Mexico on the north and west, El Quiché to the east, and Totonicapán, Quetzaltenango, and San Marcos to the south. It covers an area of 7,429.77 square kilometers and is the fifth largest department in the country. The departmental capital is the city of Huehuetenango, located 269 kilometers from the capital city. It has 33 municipalities, the youngest of which, Petatán, was approved by Congressional decree 6-2015 and was divided off of Concepción Huista. The National Statistics Institute (INE), estimated that the population will reach 1,381,969 people in 2019. The census that was carried out in 2018 will not provide results until August 2019, and for that reason, INE’s projected data was used. The municipalities that were studied were Santa Cruz Barillas, which is estimated to have a population of 193,625 and where six schools were studied and Santa Eulalia, with a population of 58,571 and where nine schools were studied. Municipality of Santa Cruz Barillas 13 Santa Cruz Barillas covers 1,112 km2 . Its population are descendants of people who initially lived in the municipality of Santa Eulalia, and the primary language is Q´anjob´al. According to the General Secretariat of Planning and Programming (SEGEPLAN), the distance from the central park in the municipality of Huehuetenango to the urban center of Santa Cruz Barillas is 150 km. It is located in the northern part of the department, connected by a road in generally poor conditions. It is in the northwestern region of the country and the northern transversal strip and is part of the group of northern border municipalities. It is located 415 km. from Guatemala City at an altitude of 1,450 meters above sea level. According to INE, the political and administrative make-up of the municipality consists of 10 villages, 140 caseríos, 38 fincas, and the town, giving a total of 189 registered populations. The projected population for 2020 is 201,954 inhabitants, of which 51% will be women and 49% men. The agricultural vocation depending on climate conditions is annual crops, livestock, and forestry activities. Migration is common throughout the municipality. It is estimated that more than 85% of the population migrates temporarily to México to work in agricultural activities, especially from September to January. The families who do not go to México work in the coffee harvest in Huehuetenango, which takes place from November to May. Other employment activities when they migrate are masonry, metal work, bakeries, and cleaning services. Municipality of Santa Eulalia Santa Eulalia is located in the northern part of the department, in the Sierra of the Cuchumatanes. It covers 292 km2 , and its population generally speaks Q´anjob´al. According to SEGEPLAN, it is 88 km from the central part of the municipality of Huehuetenango to the urban center of Santa Eulalia, connected by a road in acceptable conditions. It is 355 km from the capital and is the coldest municipality in the department of Huehuetenango. The municipal seat is at 2,580 meters above sea level. According to INE, the political and administrative division of the municipality consists of 13 villages (five are not legally registered in the National Geographical Institute), 56 caseríos, 13 cantones, one finca, and the town, which leads to a total of 84 registered populations. The projected population for 2020 is 60,229 inhabitants. The land in the municipality is generally forest with high levels of rainfall. The land is not apt for agriculture, but it can be used for crops such as vegetables, flowers, annual agriculture, barley, wheat, and deciduous fruit trees. 14 Migration is common. It is estimated that 80% of the population, especially in the rural areas, migrates temporarily to México and the capital city of Guatemala to seek work in agriculture, informal labor, construction, masonry, cleaning, and security services (SEGEPLAN 2010b). Department of Totonicapán The department of Totonicapán is located in the southwestern region of Guatemala, at 14º 54’ 45’’N latitude and 91º 21’ 36’’O longitude. It borders Huehuetenango to the north, Sololá to the south, Quiché to the east, and Quetzaltenango to the west. It covers 1,061 km2 with 601,837 inhabitants. The study was carried out in the municipality of Momostenango, which has a total of 158,578 inhabitants. The study was carried out in 15 schools. Municipality of Momostenango Momostenango covers 305 km2 , and its inhabitants mainly speak Quiché. According to SEGEPLAN, it is 31 km from the central park of the municipality of Totonicapán to the urban center of Momostenango. It is located in the northern part of the department of Totonicapán, connected by a good road with several access points. It is located close to the CA-1 highway and the departmental seats of Quetzaltenango and Totonicapán and is 208 km from Guatemala City, at an altitude of 2,204 meters above sea level. Its political/administrative division includes the municipal seat which is categorized as a town with four neighborhoods, 13 villages, 10 caseríos, and two parajes. By 2020 the population is projected at 162,941 inhabitants of which, 52.4% are women and 47.6%, men. The land can be used for agriculture with soil and forest management techniques. It has great tourism potential due to its weavings, cliffs, hot springs and trekking. The people of Momostenango are travelers by nature. The main economic activity of the municipality is commerce, which they carry out throughout Guatemala and beyond its borders. In any market or sales arena, people from Momostenango are found selling products from their home area and from other sites. The small stores open in all city neighborhoods are one of their specialties. 15 Department of El Quiché The department of El Quiché is located in the northwestern region at 15º 02’ 14’’ N latitude and 91º 09’ O longitude. It borders México to the north, the departments of Sololá and Chimaltenango to the south, Alta and Baja Verapaz to the east, and Totonicapán and Huehuetenango to the west. It covers 8,378 square kilometers, and according to INE estimations, has a population of 1,237,939 inhabitants. The study was carried out in the municipalities of Nebaj, (which has 110,034 inhabitants) in 11 schools, Cotzal, (which has 33,384 inhabitants) in one school, and Chajul, (which has a population of 70,660) in three schools. The population of these municipalities mainly speak the Ixil language group, although the municipality of Chajul also has a segment of the population whose mother tongue is Quiché. Municipality of Nebaj Nebaj covers 608 km2 , and the predominant language group is Ixil. It is located 87 km from the department seat in the northern part of the department of El Quiché, with good road connections and various access points. The municipality is more strongly developed, given that it is the economic center of the Ixil area. It is 251 km from Guatemala City and 2,001 meters above sea level. The political-administrative composition includes the municipal seat that is categorized as a town, 68 villages, 19 caseríos, and 22 cantones. The population is projected to reach 113,902 inhabitants by 2020 (51% women and 49% men). Land is apt for agricultural purposes, particularly for basic grains, fruits, coffee, cardamom, pasture, and others. The Ixil area in general has important tourism potential given the beauty of its patrimony, and Nebaj benefits from this tourism as a large city with services such as hotels, restaurants, and tour guides. Migration occurs in Nebaj, especially to the cane and coffee harvests in the plantations of Cobán and the Pacific coast, especially Escuintla and Suchitepéquez, from October to April, and to the capital city. There is also migration to the United States, and those migrants send home remittances. Because it is a center for economic activities, it has its own internal migration, where people from the rural Ixil populations migrate to seek better living conditions in the city. Municipality of Chajul Chajul covers 608 km2 , and the population speaks Ixil. It is 107 km from the department capital. It is located in the northwestern part of the department of El Quiché and on the western side of the Sierra of the Cuchumatanes, connected by good roads to the other two municipalities of the Ixil area - Nebaj and Cotzal. It is 273 km from Guatemala City at an altitude of 1,991 meters above sea level. The political-administrative composition includes the municipal seat, which is divided int o three cantones, 48 villages, seven caseríos, seven fincas, and one colonia, for a total of 63 communities. The projected population for 2020 is 73,454 inhabitants, of which 50% are women and 50% are men. The most important economic activity in Chajul is agriculture, especially the production of corn, beans, and coffee, which is the only one of these three products that is marketed and generates 16 earning opportunities during the harvest season. Livestock is raised freely in households and offers additional income to the families. Chajul is one of the poorest municipalities in Guatemala, and because of that there is significant migration during the harvest season, especially for coffee crops. The United States is the destination of choice for those who can pay the cost of the trip. Family income from remittances helps subsistence level families survive. 4. Baseline Methodology The baseline evaluation uses a quasi-experimental design and a mixed-methods approach to conduct an impact evaluation. This baseline will serve as an ex-ante comparison point for an ex-post final evaluation at the end of the project. To conduct an impact evaluation, PCI selected two different control groups to provide a counterfactual – schools within the project area that are part of the USDA￾funded FFE programs but do not participate in the LRP program, and schools in the project area that do not participate in either the FFE or LRP programs. Quantitative data against 11 key performance indicators (KPI) was collected from intervention and control group school and will be analyzed at endline using a difference-in-difference methodology. Using this methodology, the evaluations will address the following research hypothesis: Nuestra Cosecha supported schools will demonstrate improved effectiveness of food assistance compared to schools not supported by USDA and schools supported only by Food for Education programming. 4.1 Secondary Data Review PCI provided relevant documents for the new Nuestra Cosecha project, the indicators with explanations, and maps of the locations of the intervention schools to provide a geographic overview to develop and assign routes. The evaluation team used these documents to structure the work to be conducted per the terms of reference, and the routes for the teams were planned to cover the target number of schools in the stipulated timeframe. 4.2 Inception and Coordination Sessions PCI and the consultant group held virtual meetings via Skype as well as in person meetings in Huehuetenango and Quetzaltenango with the project teams. The instruments for data collection were prepared to collect data relating to the 11 indicators established in the terms of reference. PCI requested that the consultants use the Mobenzi system and opened an account to conduct the necessary tests. The team communicated with the staff of PCI, CRS, and SCI for coordination, planning, and establishing the dates for staff training and the preparation of the Mobenzi forms. 4.3 Sampling Strategy The terms of reference for this baseline study required the inclusion of a treatment group of schools, students, parents, and producers that will participate in Nuestra Cosecha, as well as two control groups with schools that will not participate in the project, including schools participating in USDA MGD projects and schools without USDA support. In each department, five schools targeted by Nuestra Cosecha were included, as well as five from the regular MGD program implemented by each organization and five control schools not included in either of the two programs, for a total of 45 schools. 17 4.3.1 Treatment Schools To calculate the number of schools in the target municipalities, a percentage weight was determined based on the number of students in each school. The results are presented in the table below. Table 1. Number of Schools to Evaluate in the Nuestra Cosecha Baseline Organization Department Municipality No. Students % Students No. Schools PCI Huehuetenango Santa Eulalia 2131 19 3 PCI Huehuetenango Barillas 1741 16 2 CRS Totonicapán Momostenango 3317 30 5 SCI El Quiché Nebaj 3157 29 4 SCI El Quiché Chajul 596 5 1 Total 10,942 100 15 Prepared by consultant. Source: Annex IV of terms of reference Subsequently, the list of all treatment schools was randomly numbered for each of the departments to obtain a randomized sample of schools and determine the 15 intervention schools to be targeted for this study. 4.3.2 Control Schools The control schools were divided into two: a) Regular MGD program, and b) No USDA interventions. After identifying treatment schools, the evaluation team selected the two control groups, taking into consideration the schools with similar contexts as the treatment schools in terms of geographic location, language, access, number of students, Human Development Index, and degree of chronic malnutrition. These criteria were established in coordination with each organization during the general meeting organized by PCI in Huehuetenango. To reach five schools in Quiché, three were selected from Nebaj, one from Cotzal, and one from Chajul. For comparison purposes, the same instruments were applied to a total of 45 schools. The characteristics of the intervention and control municipalities, as well as the number of schools per municipality, are presented in the table below. 18 Table 2. Classification of Intervention and Control Municipalities No. of Schools Type Municipality No. Inhabitants HDI Chronic Malnutrition Main Language 3 Intervention Santa Eulalia, Huehuetenango 3 55,194 0.482 71.5% Q'anjob'al Regular program 3 Control 2 Intervention Santa Cruz Barillas, Huehuetenango 2 177,353 0.562 64.5% Q'anjob'al Regular program 2 Control 5 Intervention Momostenango, Totonicapán 158,578 0.538 Data not available 5 Quiché Regular program 5 Control 4 Intervention Santa María Nebaj, El Quiché 4 110,034 0.4390 61.9% Ixil Regular program 3 Control 1 Intervention Chajul, El Quiché 1 70,660 0.410 72% Ixil Regular program 1 Control 1 Control San Juan Cotzal 33,384 0.410 69.1% Ixil Notes. Prepared by consultant, collected from documents in Ixil Foundation and Municipal Development Plans of SEGEPLAN. The source for data on chronic malnutrition is the IV National Student Census for 2015 - 6 to 9 years of age. 4.3.3 Interviews a. Structure of the data collection team The team included the following roles3 and functions in the study. i. General Coordinator: Carried out meetings with contact persons from PCI, CRS, and SCI to coordinate field work, structure the school entry methodology, and define the field work process. Internally, the teams were composed of hired translators for each for the Q´anjob´al, Quiché, and Ixil speaking areas. The coordinator applied qualitative instruments and participated 100% of the time in the three weeks of field work. ii. Supervisor: Ensured that team activities flowed smoothly, obtained information on the quality of the data collected by means of supervision of questionnaires and periodic data checking. iii. Interviewers: Collected data by means of the research instruments using mobile devices with the Mobenzi digital platform. The interviewers’ role for maximizing data quality was adherence to the protocol and the supervisor’s instructions. The interviewers were fluent in the local languages: for Santa Cruz Barillas and Santa Eulalia, Q´anjob´al; for Nebaj, Cotzal, and Chajul, Ixil; and for Momostenango, Quiché. Women were chosen to facilitate communication, especially with girls. 3 The names of the evaluation team are presented in Annex IX. 19 b. Training of the field team Data collection began with a field team that was appropriately trained to clearly understand the study objectives and for skills in compiling data. The trainings were held as follows: i. Ixil Group – held at the Hotel Shalom of Nebaj on March 6 and 7 ii. Q´anjob´al Group – held at PCI offices on March 8 iii. Quiché Group – held at EQSA offices in Quetzaltenango on March 12 Staff training covered technical and ethical aspects, and the team practiced the field exercises so the interviewers could practice their skills (procedures for approaching those being interviewed, application of informed consent, manner of carrying out interviews, and resolution of doubts) under direction by EQSA and PCI. The training included three phases: i. Preparation: Guaranteed timely organization of all materials and logistics, including supplies such as electronic devices for data collection for monitoring field work ii. Implementation: The supervisors and person in charge of data quality control played the role of the participants so that all the interviewers practiced as if they were in a real interview situation. All the protocols, such as interview methods, were practiced. iii. Feedback: Discussions were held on the training and practice with the field team, including a discussion of the various components of the questionnaire, feedback by interviewers on the use of the questionnaire, and feedback by supervisors and quality control staff. Comments were also shared by the PCI team about the instruments. c. Field work To establish the number of students needed in the study, the total reported in Annex IV of the terms of reference was used as a base for applying the statistical formula with a 95% confidence level, giving the result of 372 students needed from the intervention schools. A similar number was set for the control schools. Out of a target of 1,116 students, 1,104 (98.9%) were interviewed. The remaining 1.2% were not interviewed due to previously scheduled activities outside the school for those students. The reason for selecting fourth to sixth grade students was because their age made them more likely to give objective answers about their satisfaction with the foods. For the indicator on minimum acceptable diet, the director of each school was interviewed, and all 45 provided information. Out of the targeted 45 OPFs, the team interviewed 44 (97.78%). The control school of Nimsituj in Momostenango did not receive information about the interview, so the director, who was also the only teacher for the six grades, did not invite OPF members to participate. 20 The team expected four people from each OPF to attend, for a total of 180 people to be interviewed individually. A total of 156 people in 44 schools participated (due to the one school’s OPF not participating), and the study achieved 86.67% of what was expected. The director was interviewed in each school, for 100% participation. Table 3. People Reached During the Study Sample Huehuetenango El Quiché Totonicapán Total # Total % Boys Girls Boys Girls Boys Girls Students from 4 th , 5 th and 6th grade 186 193 190 172 173 190 1,104 98.9% Directors 15 15 15 45 100.0% OPF as a structure 15 15 14 44 97.8% OPF individuals 56 49 51 156 86.7% 4.4 Instruments The Minimum Acceptable Diet instrument was used with boys and girls from fourth to sixth grades in each of the intervention and control schools. On average, 25 students were interviewed in each school (see Annex I Minimum Acceptable Diet). A tool was designed for the interviews with the directors of each school, and 100% of the 45 directors were interviewed and can be found in Annex II: Directors Survey. The METAPLAN tool was used for the focus groups and is a technical visualization in which the ideas of the OPF members were collected one at a time in short words on Post It notes and placed on a poster which, depending on the space provided by directors, was placed on the wall, a window, the door, or blackboards when possible. At the end of the workshop, interviews were held with each parent member of the OPF who was available to survey their knowledge of the LAE. This tool can be found in Annex IV: OPF Knowledge Survey. In accordance with the terms of reference, three weeks were provided for data collection in the field. Depending on the location, the team worked with two to four schools per day, surveying 15 schools per week in the different modalities. 4.5 Process for Verifying Data Collection Quality Measures to maintain a high quality of data to ensure the quantitative data wassignificant and accurate were taken during the entire study process: during the study design and planning, during data collection, and during the data analysis phase. The following activities were carried out: a. Data processing The objective of this activity was to verify that the data obtained had the required quality and were complete. The Statistical Package for the Social Sciences (SPSS) version 19 program was used to process the data and produce the corresponding products. 21 a. Data analysis The following statistical techniques were used for data analysis: Table 4. The Minimum Diet Instrument ID Variable Statistical Procedure A Classification of school and interviewee (department, type of school, grade, sex) Simple frequency B Incidence of meals and location Simple frequency A/B Classification of school and interviewee / Incidence of meals and location Cross tabulation C Groups of foods per meal Simple frequency D Number of meals Simple frequency A/D Classification of school and interviewee (department, type of school, grade, sex) / Number of meals Cross frequency tables E Identification of students who have at least 3 meals and 4 food groups Transformation, sum and recoding of variables A / E Classification of school and interviewee (department, type of school, grade, sex) / Identification of students who have at least 3 meals and 4 food groups Cross frequency tables F Hedonic scale for school feeding Simple frequency A/F Classification of school and interviewee (department, type of school, grade, sex) / Hedonic scale for school feeding Cross frequency tables Simple frequency: Number of times that a variable is repeated in a distribution of data. Cross Tabulation: Table that shows the repetition of a variable in relation to another study variable. Recoding of variables: Procedure from SPSS program where a variable is recoded in similar units for mathematic operation and subsequent grouping in categories. Table 5. The LAE Knowledge Instrument ID Variable Statistical procedure A Classification of school and interviewee (department, type of school, grade, sex) Simple frequency B Five questions on basic LAE aspects Simple frequency A/B Classification of school and interviewee/ 5 questions about basic aspects of the LAE Cross frequency tables C Training incidence Simple frequency D Institutions that provide training on LAE Simple frequency E Identification of members of OPF who responded correctly to at least 4 of 5 questions Transformation, sum and recoding of variables A / E Classification of school and interviewee (department, type of school, grade, sex) / Identification of members of OPF who responded correctly to 4 of 5 questions Cross frequency tables Additionally, analysis of two-tailed tests with a significance of 0.05 was carried out with both instruments to establish significant differences between variables. 22 b. Ethical aspects To guarantee an ethical approach in data collection and to ensure confidentiality in the management of participants’ data, study participants were informed of their following rights4 : i. Right to choose: Everyone who was approached has the right to decide if they want to participate or not in this study. Anyone can end their participation at any time (even after having given their consent) or refuse to respond to any specific question that makes them feel uncomfortable. If they decide not to participate, they will be told that that is fine and they will be assured that their decision will not bring adverse consequences. ii. Right to safety: The participants have the right to protection from physical or psychological harm. If there is any question that causes stress for any participant in the study, they should not be pressured in any way to respond. This right also is relevant for the presentation of data, and no participant should be stigmatized for any of their answers. iii. Right to be informed: Study participants have the right to be informed about all aspects of the study, its objectives, the time required for the survey, and what will be done with the data. This will enable each participant to make an informed choice about whether to participate in the study or not. iv. Right to privacy: Everyone has the right to privacy in terms of personal information as well as all the responses given. This information will not be transmitted or revealed by any means to any other private or public organization. Benefits: The people interviewed in the survey did not receive any direct material benefit nor were they given any incentives in cash or in kind for their participation in the activity. The participants were told that their participation in the study could lead to benefits over time via the implementation of programs in their community. Payment. Participants did not receive any payment for their participation in the study. Informed Consent. All participants gave their informed consent before beginning the interview. The consent summarizes several aspects: objectives of the study, procedures, voluntary participation, and confidentiality. The informed consent process was carried out individually and verbally so that the participants could approve their participation in the study. The interviewer was responsible for ensuring that the participants understood well before beginning the activity. 4.6 Study Limitations At the start of the study, PCI requested a kick-off meeting with EQSA and all of the partners. However, SCI was not able to attend that meeting which caused some delays in the planning of the baseline evaluation. There were challenges establishing the location of the control schools, mapping out the data collection route, and confirming days of data collection with control school directors. One of the control schools near Nebaj (Vicampanavitz) was 3.5 hours drive from the center of the town and has very few similarities with the rest of the intervention school. The inclusion of this school could skew comparisons between control and intervention schools. 4 Adapted from the rights of participants promoted by the Marketing Research Association, USA. 23 Enumerators were only able to collect data from 86% of the target OPF members due to challenges with school directors requesting the attendance of all OPF members. Therefore, results related to OPFs might be skewed due to the lack of respondents. 4. Results This section presents the comparison of data collected in the field from schools targeted by Nuestra Cosecha, from schools that had a regular MGD program, and from schools without school feeding support from a USDA project that only received MINEDUC support. Analysis is focused on the 11 indicators for which this study established baseline values. 5.1 Improved Effectiveness of Food Assistance through Local and Regional Procurement (SO1) 5.1.1 Improved Cost-Effectiveness of Food Assistance (LRP1.1) Indicator 1: Average USD cost per kilogram of fresh fruits, adjusted for inflation, procured and delivered to project schools Fruit found in schools included: oranges (five units per kilo), with a price of $0.755 per kilo, bananas (five units per kilo), with a price of $1.42 per kilo, and tomatoes, (2.2 units per kilo), with a price of $1.27 per kilo. These prices were projected to estimate December 2019 prices using an expected inflation of 4.5%, based on data presented in the Banco de Guatemala’s Economic Outlook report. The average price for indicator 1 was: $1.15 per kilo (which is the projected price for December 2019). Indicator 2: Average USD cost per kilogram of fresh vegetables, adjusted for inflation, procured and delivered to project schools Vegetables found in schools and their projected price for December 2019 included: potatoes (2.2 units per kilo), with a price of $0.79 per kilo, onions (2.2 units per kilo), with a price of $1.45, and carrots (four per kilo), with a price of $1.05 per kilo. Average price for indicator 2 was: $1.10 (which is the projected price for December 2019). Indicator 3: Average USD cost per kilogram of fresh eggs, adjusted for inflation, procured and delivered to project schools. Eggs (15 small eggs per kilo) will cost $2.12 per kilo (by December 2019) 5 The source used to calculate the exchange rate was the Banco de Guatemala, based on administrative resolution JM-126-2006 issued by the bank’s Monetary Board and it uses the reference exhcange rate for April 7, 2019, which is 7.67 quetzals per one dollar. 24 To calculate these indicators, the evaluation used the inflation estimated by the Banco de Guatemala for December 2018, which is expected to be similar in December 2019, based on their publication “Survey on Economic Outlook with a Panel of Private Analysts EEE," subsection 1: "inflation". The average price was obtained by reviewing the invoices in each school visited6 . For example, eggs were bought in some places by the box with 360 eggs per box, but in other schools they were bought in regular cartons. Evaluators calculated the average price per unit of egg. This procedure was repeated with all the remaining targeted products. Some products were not bought in schools, although there were substitutes that can be used and are culturally appropriate for the schools’ context; for example, broccoli can be replaced with cauliflower. In the control schools of Chocsunil, in Santa Eulalia in Huehuetenango, Chicorral and Xoljoc in Momostenango, department of Totonicapán, and in Xolacul, in Nebaj, department of El Quiché, students received apples as part of the contributions from parents when they were in season. Normally, these were from parents who grow apples for self-consumption and sent some to school. The table below lists the average prices for agricultural food products included as part of Nuestra Cosecha project. Table 6. Average price in kilograms for eggs, fresh fruit and fresh vegetables Product Average Price for April 2019 Expected inflation to Dec. 2019 1 Price for December 2019 Measure ment Units per kilo Expected price per Kilo Q Expected price per Kilo $ 1. Eggs Q 1.04 4.50% Q 1.08 Unit 15 Q 16.27 $ 2.12 2. Chicken Q 12.50 4.50% Q 13.06 Pound 2.2 Q 28.74 $ 3.75 3. Cheese Q 25.00 4.50% Q 26.13 Pound 2.2 Q 57.48 $ 7.49 4. Broccoli Product not found in recorded school procurement 5. Tomatoes Q 4.24 4.50% Q 4.43 Pound 2.2 Q 9.74 $ 1.27 6. Potatoes Q 2.63 4.50% Q 2.74 Pound 2.2 Q 6.03 $ 0.79 7. Carrots Q 1.93 4.50% Q 2.02 Unit 4 Q 8.08 $ 1.05 8. Onions Q 4.84 4.50% Q 5.06 Pound 2.2 Q 11.12 $ 1.45 9. Apple Product not found in recorded school procurement 10. Plantain Q 2.08 4.50% Q 2.18 Unit 5 Q 10.88 $ 1.42 11. Avocado Product not found in recorded school procurement 12. Oranges Q 1.10 4.50% Q 1.15 Unit 5 Q 5.75 $ 0.75 Evaluators. Source: Invoices found in visited schools in the three departments 1: Source: Banco de Guatemala, economic outlook report. 2: Expected price uses the exchange rate of Q7.67 for $1. OPF members who were asked about local producers and the barriers or limitations they face in marketing their products in schools indicated that there are local producers who grow products mainly for self-consumption. SAT registration was a limitation identified in all 44 focus groups. Other limitations included: 6 Annex XIII presents the list of current suppliers for each department. 25 Table 7. Limitations for local producers to market their products in schools Limitation Percentage Don’t have invoice 64.4% There is production but it is for self-consumption 22.2% No adequate land for production 13.3% Climate is not favorable 6.7% Do not have technical assistance 15.6% No water 10.8% Source: Focus groups N=44. It does not add up to 100% because this is a multiple select response. The percentage was established based on the number of answers divided by the number of focus groups. The main products identified in the focus groups are listed below: Table 8. Main food products produced by department Huehuetenango Quiche Totonicapán Chard Chard Avocado Avocado Avocado Annona Domestic animals Celery Celery Annona Sweet peas Peas Peas Chinese Peas Chinese Peas Pumpkin Pumpkin Pumpkin Banana Banana Banana Broccoli Broccoli Amaranth Coffee Coffee Broccoli Sugar Cane Sugar Cane Coffee Cardamom Onions Sweet potatoes Onions Gourd Sugar Cane Sweet pepper Chilly Chilly pepper Chilly pepper Chipilín Chinta Chistan Coriander Coriander Coriander Plum Plum Coconut Cauliflower Cauliflower Peach Peach Peach Beans Green Beans Beans Broilers Beans Fruit Guava Layers Layers Güisquil Chickpea Granadilla Nightshade Granadilla Güicoy 26 Huehuetenango Quiche Totonicapán Whitebrush Güisquil Güisquil Vegetables Broad beans Broad beans Eggs Mint Nightshade Lime Herbs Mint Lemon Vegetables Jocote Maize Lime Lettuce Malanga Lemon Limes Tangerine Maize Lemon Apples Malanga Maize Turnips Tangerine Tangerine Oranges Apples Apples Nispero Turnips Cantaloupe Pacaya Oranges Husk tomatoes Potatoes Pacaya Oranges Papayas Potatoes Nispero Pineapple Potatoes Potatoes Plantain Papayas Papayas Güisquil tips Plantain Parsley Radish Quilete Chicken Beet Radish Radish Cabbage Beet Beet Tomatoes Cabbage Cabbage Tamarillo Tomatoes Watermelon Wheat Carrots Tomatoes Yucca Wheat Carrots Vegetables Source: Focus groups N=44 List of local suppliers per community - see annex VIII. Yucca Of the twelve basic products, focus group participants identified those that were locally produced in the target communities: Avocado, Broccoli, Apples, Oranges, Potatoes, and Tomatoes. Products that were produced in two of the three target departments included: Onions (Huehuetenango and Quiché); Plantains (Huehuetenango and Quiché); Carrots (Huehuetenango and Quiché). Products that were available in only one of the three target departments included: Eggs (Huehuetenango); and, Chicken (Totonicapán). Cheese was not identified in any of the target departments. 27 5.1.2 Improved Timeliness of Food Assistance (LRP 1.2) Indicator 4: Average number of days per year to complete entire process of improved procurement (bidding, testing, payments), delivery and distribution for locally procured commodities All the schools targeted by this study had received supervision and technical support from MINEDUC’s General Directorate for Strengthening of the Education Community (DIGEFOCE) at the time of the survey, which established the following process to complete food procurement by OPF and school feeding commission members: 1. Request quotes from suppliers 2. Purchase products (visit the winning supplier) 3. Delivery 4. Liquidation For each school, DIGEFOCE performs four fund transfers or disbursements during the year, each covering 50 school days for a total of 180 effective school days, as established by MINEDUC. To receive a new disbursement, the school needs to first complete the procurement process for the prior disbursement with the respective liquidation at the corresponding DIGEFOCE office. Additionally, two types of purchases are made based on the expiration date of procured products: 1. Perishables: Fresh fruits and vegetables (perishable products) are purchased every week. Six purchases are made for each disbursement, to cover a total of 50 school days. 2. Non-perishables: Grain and other non-perishables are purchased every 25 days, or two purchases for every disbursement: once after receiving the funds and the other 25 days later, so the school will have products in stock Monday through Friday, until the new disbursement. To measure this indicator for the procurement process for perishable products, evaluators multiplied the number of times each activity is conducted in the procurement process by the average number of days that it takes that activity to be finalized, and the total was divided by the number of disbursements made throughout the year. This was calculated based on the indicator calculation manual provided by PCI, which defines the numerator and denominator. Table 9. Average number of days per distribution for the procurement of perishables Concept Process 1. Quotes 2. Purchase 3. Delivery 4. Liquidation Average days 2.96 1.22 1.40 2.22 Source: Field research n=45 28 Table 10. Number of times per year each activity in the procurement process is conducted to purchase perishables Concept Annually 1.Quotes 2.Purchase 3. Delivery 4. Liquidation Number of times 1 24 24 4 Source: Field research n=45 Table 11. Average number of days per year for the procurement of perishables Concept Process 1.Quotes 2.Purchase 3.Delivery 4. Liquidation Total Annual Average days 2.96 29.3 33.60 8.89 74.78 Source: Field research n=45 Therefore, it took schools an average of 74.78 days per year to complete all activities under the improved procurement process for the purchase, delivery, and distribution of locally available perishables for school feeding. Per distribution, schools spent 18.69 days (74.78 / 4) to complete the procurement activities to cover the 50 days. For non-perishable products, this indicator was measured as follows: Table 12. Average number of days per distribution for the procurement of non-perishable products Concept Process 1. Quotes 2. Purchase 3. Delivery 4. Liquidation Average days 2.96 1.22 1.40 2.22 Source: Field research n=45 Table 13. Number of times per year each activity in the procurement process is conducted to purchase non-perishable products Concept Annually 1.Quotes 2.Purchase 3. Delivery 4. Liquidation Number of times 1 8 8 4 Source: Field research n=45 Table 14 . Average number of days per year for the procurement of non-perishables Concept Process 1.Quotes 2.Purchase 3.Delivery 4. Liquidation Total annual Average days 2.96 9.78 11.20 8.89 32.82 Source: Field research n=45 Therefore, it took schools an average of 32.82 days per year to complete all activities under the improved procurement process for the purchase, delivery, and distribution of locally available non￾perishables for school feeding. Per distribution, schools spent 8.21 days (32.82 / 4) to complete procurement activities to cover the 50 days. When combining both perishables and non-perishables, the average number of days per year to complete the entire process of improved procurement (bidding, testing, payments, delivery, and 29 distribution) for locally procured commodities was 53.8. However, separate calculations should be made for purchases of perishable (74.78 days per year) and non-perishable products (32.82 days per year), given that schools do not have the necessary refrigeration equipment to store perishable products for long enough to match the non-perishables procurement time frame. 5.1.3 Improved Utilization of Nutritious and Culturally Acceptable Food that Meet Quality Standards (LRP 1.3) Indicator 5: Percentage of schools serving school meals with an adequate dietary diversity, as a result of USDA assistance Dietary diversity, according to the United Nations’ Food and Agriculture Organization (FAO) involves the consumption of seven food groups: 1. grains, roots, and tubers, 2. legumes and nuts, 3. dairy products, 4 meats, 5. eggs, 6. vitamin A foods (vegetable oil, fruits, and vegetables), and 7. other fruits and vegetables. All of the schools (100%) receiving USDA assistance offered their students five or more of the dietary diversity food groups, and 76.7% offered all seven recommended food groups every day. USDA￾donated commodities included corn-soy blend (CSB), beans, and fortified vegetable oil, covering food groups one, two, and six, respectively. All of the control schools not receiving USDA donated commodities also provided school meals with an adequate dietary diversity, as a result of the LAE’s promotion of the addition of fruits and vegetables to the school meals. The graph below illustrates the number of food groups served in school meals. Graph 1. Number of food groups served with USDA assistance 5.1.3.1 Improved Access to Culturally Acceptable Foods Indicator 6: Percent of students who report that they like school meals, using the 5-point facial hedonic scale, by sex. The five-point facial hedonic scale uses five facial expressions to classify mean acceptance or rejection, from one (hated it) to five (loved it) (see Annex V). Students are asked about what they eat 7.1% 0.0% 7.1% 85.7% 0.0% 3.3% 20.0% 76.7% 4 grupos 5 grupos 6 grupos 7 grupos No recibe asistencia de USDA recibe asistencia de USDA Elaboración propia. N=45 30 for a typical school meal then shown a series of five facial expressions that are associated with five levels of food acceptance. The student is asked to identify the face that most closely associates with their opinion about the typical school meal. Evaluators approached students after they received their meals in school. Results were as follows, per type of school: Graph 2. How do you rate the food received? Most students liked or loved the food served in school. Control schools showed a lower percentage of students who said they “loved it” when compared to schools in the regular program and those targeted by Nuestra Cosecha, but a higher percentage of students that “liked it.”. To measure this indicator, evaluators added the number of students who liked the food and scored it a four (I liked it) or five (I loved it) in relation to the total number of students interviewed. Statistically significant differences were observed in category five (I loved it) between the schools with the regular program’s intervention and Nuestra Cosecha schools when compared to schools without USDA assistance. See annex VII for statistical significance. Schools targeted by Nuestra Cosecha already received USDA-donated commodities under the MGD project (EDUCAMOS), and Nuestra Cosecha will complement those commodities with fresh, unprocessed products grown locally by farming families. 1.6% 1.3% 5.2% 42.6% 49.4% 0.2% 1.4% 4.3% 41.6% 52.5% 3.0% 3.0% 9.7% 49.3% 35.0% Detesté No me gustó Indiferente Me gustó Me encantó Nuestra Cosecha Programa regular Control Fuente: 1,104 Estudiantes entrevistados 31 Graph 3. Students who like school meals by type of school Schools targeted by the regular program had a higher percentage of students who liked the food received (94.1%) in comparison to schools without USDA support (84.4%). One of the reasons for this result was the monitoring and training activities conducted with OPF and the mothers in charge of preparing school meals. Graph 4. Students who liked school meals by sex and department When cross tabulating the results by sex and department, the evaluation found that Quiché had the lowest percentage of students that reported liking the food they eat at school, a result that was consistent between boys and girls. Schools in Huehuetenango had the highest percentage of students who liked the food they eat at school. 5.1.3.2 Increased Value Added to Post-Production Agricultural Products Indicator 7: Value of annual sales by USDA project beneficiaries (LRP3) The indicator definition states that the value is based on sales through the LRP program. As a result, the baseline value is zero. 5.1.3.3 Increased Access to Markets to Sell Agricultural Products Indicator 8: Volume (MT) of commodities sold by project beneficiaries (LRP 4) The indicator definition states that the value is based on sales through the LRP program. As a result, the baseline value is zero. 92.5% 95.3% 87.9% 84.9% 91.3% 88.9% Masculino Femenino Masculino Femenino Masculino Femenino Huehuetenango Quiché Totonicapan Fuente: M 186 F 193 M 190 F 172 M 173 F 190 91.9% 94.1% 84.4% Nuestra Cosecha Programa regular Control Fuente: 1,104 Estudiantes entrevistados 32 5.1.3.4 Improved Access to Nutritious Foods Indicator 9: Percentage of schools using fresh fruit and/or vegetables in daily school meals, as a result of USDA assistance Graph 5. Fresh Fruits and/or Vegetables All of the surveyed schools (100%) said they add fresh fruits and vegetables to the daily school meals, including the control schools. Fresh products are typically purchased with MINEDUC funds or are donated by the local community. . In Educamos schools, PCI has provide training on school menus, training on the preparation of fruits and vegetables and guidance on food purchasing plans that include fresh products. All visited schools had a registered OPF and received the daily monetary allowance of four quetzals per student for school feedings. OPFs used the menus established by the DIGEFOCE, which contained products from the food groups mentioned above, including fruits and vegetables. Indicator 10: Percent of school-age children receiving a minimum acceptable diet The criteria for this indicator measures both the minimum feeding frequency and minimum dietary diversity, based on FAO guidelines for measuring dietary diversity in households and individuals. Minimum dietary diversity for children is defined as consuming four or more food groups out of the following seven per day: 1. grains, roots and tubers, 2. legumes and nuts, 3. dairy products (milk, yogurt, cheese), 4. meats (beef, fish, chicken, and liver/organs), 5. eggs, 6. vitamin A enriched foods, including vegetable oil, fruits, and vegetables; and 7. other fruits and vegetables. The minimum meal frequency for children is defined as three or more feedings of solid, semi-solid, or soft food per day. When evaluating dietary diversity by food groups consumed the day before the survey, students indicated they consumed: 100.0% 100.0% 92.9% 100.0% Frutas / vegetales con vitamina A Otras frutas y hortalizas Fuente: 45 escuelas evaluadas Control Intervención 33 Graph 6. Food groups consumed the previous day Grains, roots, and tubers were available in all the schools, as they are considered staples in the target communities and as a result of USDA-supported school feeding activities. Fruits and vegetables with vitamin A were consumed the day before by 85.6% of the students, while legumes and nuts were consumed by 78.6% of the students. The rest of the food groups had lower consumption rates. There were very few differences by department in terms of the foods consumed by students: Table 15. Food groups consumed the previous day by Department Food Group Huehuetenango (n=379) Quiche (n=362) Totonicapán (n=363) Grains, roots and tubers 100% 100% 100% Legumes and nuts 78.6% 80.9% 76.1% Dairy products 33.8% 48.3% 35.7% Meats 52.5% 49.2% 59.9% Eggs 22.2% 23.8% 31.9% Fruits and vegetables with vitamin A 81.5% 91.2% 84.3% Other fruits and vegetables 54.9% 54.7% 27.7% Source: Field research Statistically significant differences in relation to the total number of food groups present in students' diet were only found in the department of Quiché vs Totonicapán: Quiché showed that 5.8% of students received all seven food groups, and Totonicapán showed that only 1.9% of students did. 100% 78.60% 39.20% 53.80% 25.90% 85.60% 45.90% Granos, raíces y tuberculos Legumbres y frutos secos Productos lácteos Alimentos de carne Huevos Frutas y verduras con vitamina A Otras frutas y hortalizas Fuente: Trabajo de campo con 1,104 estudiantes 34 Graph 7. Meals consumed the previous day With regards to the number of meals consumed during the day, this study found that afternoon snacks were the least consumed meal of the day by students, with only 50.7% of the students eating an afternoon snack. On the other hand, 91.1% of students said they have a morning snack at school, demonstrating the impact of the school feeding program. Statistically significant differences were found in the number of meals consumed by students by sex: while 36.4% of boys have five meals per day, 42.3% of girls have the same number of meals. This result indicates a possible paradigm shift, as boys, especially the older boys, have historically been given preference over girls in terms of nutrition. The percentage of school-aged children who received a minimum acceptable diet according to the criteria established for this indicator was similar across departments and type of school, though schools targeted by Nuestra Cosecha in Huehuetenango showed the lowest percentage. The significance for minimum diet can be found in Annex V. Graph 8. Children with a minimum diet by department and type of school 61.5% 76.3% 66.7% 74.6% 69.8% 81.3% 64.5% 83.0% 72.0% Huehuetenango Quiché Totonicapan Nuestra Cosecha Intervención Control Fuente: Trabajo de campo 379 estudiantes 362 estudiantes 363 estudiantes 94.7% 91.1% 92.6% 50.7% 94.6% Desayuno Refacción AM Almuerzo Refacción PM Cena Fuente; entrevistas 1,104 estudiantes 35 Graph 9. Children with a minimum diet by department Results by department indicated that Quiché had the highest percentage of students with a minimum acceptable diet with 76.5%; Totonicapán had 72.5%, and Huehuetenango had the lowest percentage with 68.6%, creating statistically significant difference between Quiché and Huehuetenango. One factor that played a role was food availability at home. Huehuetenango is characterized by higher levels of poverty and extreme poverty, which are analyzed separately given their different definitions. Graph 10. Children with a minimum diet by type of school The evaluation identified that even though the percentages are similar, schools targeted by Nuestra Cosecha had the lowest percentage of students with a minimum acceptable diet. Control schools had 73.9%, which demonstrates the positive results of the government’s school feeding program to promote varied school menus. Schools that received USDA support through the regular MGD program had the largest percentage of children with a minimum acceptable diet, which can be attributed to the fact that their activities were implemented for longer, including accompaniment and training for participants. Graph 11. Students with minimum diet by sex In relation to the percentage of boys and girls who received a minimum diet, the evaluation found that 75.3% of girls received a minimum diet, compared to 69.6% of boys in the 45 schools interviewed. This represents a statistically significant difference. 69.6% 75.3% Masculino Femenino Fuente: 45 escuelas investigadas 68.6% 76.5% 72.5% Huehuetenango Quiché Totonicapan Fuente: 1,104 estudiantes entrevistados 67.7% 74.7% 73.9% Nuestra Cosecha Programa regular Control Fuente: 1,104 Estudiantes entrevistados 36 5.1.4 Improved Capacity of Relevant Organizations (LRP 1.4.3) Indicator 11: Percent of OPF members who demonstrate sufficient knowledge of the LAE and its regulation. Graph 12. Basic knowledge about the LAE The criteria for determining sufficient knowledge about the LAE by OPF members was a score of 80% or above in a five-question questionnaire designed for this purpose. Almost half (43.6%) of OPF members obtained a score of 80% or higher. Graph 13. Basic knowledge about the LAE by sex Overall, 156 OPF members were interviewed, of which 72 were men and 84 women. When comparing scores by sex, the evaluation found that a greater percentage of men showed basic knowledge about the LAE than women. Results showed that 35.7% of women and 52.8% of men had basic knowledge about the LAE. See Annex VI for statistical significance testing. Results showed that Huehuetenango had the highest percentage of OPF members with basic knowledge about the LAE with 50.0%, while Quiché had 40.8% and Totonicapán 39.2%. Logro No logro Fuente: 156 OPF entrevistados 56.4% 43.6% 52.8% 35.7% Masculino Femenino Conocimientos básicos LAE Masculino Femenino Fuente: 156 OPF entrevistados 37 Graph 14. Basic knowledge about the LAE by department and by group of schools (Nuestra Cosecha, Intervention and Control) When comparing results by type of school, the evaluation found that schools that will participate in Nuestra Cosecha had the lowest percentage of OPF members with basic knowledge about the LAE, with 34.3%. Schools that participated in the regular USDA-funded project had 52.2%, and those without support from USDA had 48.84%. 50.0% 40.8% 39.20% Huehuetenango Quiché Totonicapán Fuente: OPF entrevistados, 56 de Huehuetenango, 49 de Quiché y 51 de Totonicapán 34.3% 52.2% 48.84% Nuestra Cosecha Intervención Control Fuente: OPF entrevistados, 56 de Huehuetenango, 49 de Quiché y 51 de Totonicapán Graph 15.Basic knowledge on the LAE by school group 38 Graph 16. Training / Achievement Additionally, the evaluation asked OPF members if they received any type of training on the LAE, and the results were compared with the percentage of members who had basic knowledge about the Law. OPF members in schools in Quiché demonstrated a corresponding match between the training received and the knowledge demonstrated, with the biggest differences between training and knowledge found in Momostenango in Totonicapan, where 66.7% of OPF members said they received training but only 39.2% scored above 80% in the interview. In Huehuetenango, there was a 10.7 percentage point difference between the percentage who said they received training and the percentage who scored 80% or higher in the interview. This result shows the need to strengthen OPF training activities. 60.7% 40.8% 66.7% 50.0% 40.8% 39.20% Huehuetenango Quiché Totonicapán Capacitación Logro Fuente: OPF entrevistados, 56 de Huehuetenango, 49 de Quiché y 51 de Totonicapán 39 5.2 Indicator Table Table 16. Summary of Results No. Result Indicator Nuestra Cosecha MGD EDUCAMOS Control Source 1 LRP 1.1 Average USD cost per kilogram of fresh fruits, adjusted for inflation, procured and delivered to project schools 1.15 1.11 1.16 School procurement invoices 2 Average USD cost per kilogram of fresh vegetables, adjusted for inflation, procured and delivered to project schools 1.10 1.14 1.12 School procurement invoices 3 Average USD cost per kilogram of fresh eggs, adjusted for inflation, procured and delivered to project schools 2.12 2.26 2.13 School procurement invoices 4 Average number of days per year to complete entire process of improved procurement (bidding, testing, payments), delivery and distribution for locally procured commodities 74.78 * 72.60 77.62 Principals / OPF 5 LRP 1.3 Percentage of schools which serve school meals with adequate dietary diversity, as a result of USDA assistance 100% 100% N/A Principals 6 Percent of students who report that they like school meals, using the 5-point facial hedonic scale, by sex. 91.9% 94.1% 84.4% Students 7 LRP 3 Value of annual sales of farms and firms receiving USDA assistance (USD) 0 0 0 Producers 8 LRP 4 Volume of commodities sold by farms and businesses receiving USDA assistance 0 0 0 Producers 9 Percentage of schools using fresh fruit and/or vegetables in daily school meals, as a result of USDA assistance 100% 100% N/A Principals 10 Percent of school-age children receiving a minimum acceptable diet 67.7% 74.7% 73.9% Students 40 No. Result Indicator Nuestra Cosecha MGD EDUCAMOS Control Source 11 LRP 1.4.3 Percent of School Parent Organization members who demonstrate sufficient knowledge of School Feeding law and its regulation 34.3% 52.2% 48.84% OPF Members Developed by evaluator. Source: PCI indicator table *Evaluator recommends include the indicator: average number of days per year dedicated to procurement of perishable products (74.78 days) because Nuestra Cosecha will work these products. 5.3 Qualitative analysis of key evaluation questions The terms of reference for the Nuestra Cosecha baseline required evaluators to answer key evaluation questions to improve project implementation. Based on the analysis of the interviews conducted with school principals in 45 schools and OPF members in 44 schools (evaluators were not able to interview OPF members in a school in Momostenango) in three departments, the evaluation team presented the analysis below to answer questions about the project’s relevance, performance, sustainability, and effectiveness, including recommendations to improve the project’s approach and strategies, whenever appropriate. 5.3.1 Relevance a. Determine the potential problems or challenges for project beneficiaries in making progress towards achieving project results. • Potential agricultural food producers near schools experience uncertainty and feel distrust in registering with the SAT. To address this challenge, awareness raising activities should be implemented to promote regularization and a culture of fiscal responsibility and information. • Local producers do not have extensive land available for production, so the project should promote crop production among farmer groups in the communities. Although associativity promotion requires additional processes, this initiative does have the potential to work in this area, given that the market for their products will be ensured and any surplus may be sold in local communities or destined for self-consumption. • Overall, local producers do not have the skills required to produce the crops selected by the project. To address this, the project should establish a training and agricultural technology transfer program in collaboration with government institutions, particularly the Ministry of Agriculture and Food (MAGA), the Technical Training and Productivity Institute (INTECAP), technical agriculture schools, and other existing actors, including cooperatives or agricultural associations. • OPF members lack knowledge about the LAE, which is the foundation for developing the transparent procurement processes at the school level that will benefit community members by allowing them to become suppliers. Therefore, an extensive training process should be carried out with OPF members that goes beyond the general information to learn about the operational aspects of the law where there are knowledge gaps. • School principals and school feeding commissions are currently the ones who, for the most part, analyze school menus, make decisions about the purchase of perishable and non- 41 perishable products, and lead the food procurement process. This represents a challenge because these activities should be the responsibility of OPFs, whose members generally avoid taking them on given their low schooling level, lack of time, or lack of interest in the procurement activities. OPF involvement and responsibility over these processes should be even greater if schools will be involved in weekly purchases of perishable products. Therefore, they should receive trainings on their role in the process to raise their awareness about the direct benefit these activities will generate for their children, as well as for their income by becoming school suppliers. • Because school principals already have close links with their current suppliers, changing the procurement strategy to purchase from many local farmers in the community could generate resistance. Schools will presumably prefer to deal with only one supplier to reduce administrative burden, and they likely have personal interests in purchasing from specific suppliers. The process needs to be made transparent by involving OPFs and establishing greater accountability to the community, while integrating local producers to ensure their quality products are consumed by schoolchildren. The vehicle to improve transparency in the process will be to empower communities to ensure the benefits from these activities directly impact the community itself, both in terms of improved school feedings and income generation for local farming families. • The LAE established a menu variation that includes agricultural products, which are already being distributed to some extent in local schools. However, it is also necessary for all actors to work together to ensure the menus contain not only the menus from Nuestra Cosecha, but other nutritious food products to improve the students’ diet quality. • Some products selected for Nuestra Cosecha are not currently purchased in local schools, so the project should analyze if those products need to be replaced by other products or if additional products that are currently available in target communities should be included. b. Determine how aligned project interventions are with the other stakeholder policies and priorities. • Given the recent passing of the LAE, government stakeholders, OPFs, and school directors do not have full knowledge or clarity about the specific roles everyone plays under the law. Therefore, Nuestra Cosecha activities align with LAE by promoting increased knowledge of the law, providing information about every stakeholder’s responsibility under the law, including at the governmental, school, and community levels, and disseminating knowledge about the opportunity to sell locally available products to schools. OPF training is a crucial element to help schools properly implement the mechanisms of the law. The private sector will become involved when they see the potential economic benefits or the opportunity to support school feedings. • The coordination and work dynamics between governmental and municipal institutions is erratic, mainly due to staff turnover related to budget constraints or political interests, so these institutions’ involvement will depend on the commitment and political will of these authorities to fully implement the law. This situation is expected to improve after the finalization of the electoral process in Guatemala, which will identify the next round of authorities at the municipal and national government levels. Meanwhile, extensive and clear negotiations are needed with all stakeholders. 42 5.3.2 Performance 1. Determine the baseline values for the required project indicators • This report presents the baseline values, which will allow the implementing team to analyze these results and determine the best processes, activities, and strategies to improve upon those values and generate impact during the life of the project. 2. Assess to what extent the indicator targets are realistic and appropriate • The Nuestra Cosecha implementing team should make this determination. These baseline results provide the data needed to analyze if the current staffing and resources available will be sufficient to achieve the targets proposed to USDA. In any case, the team should negotiate with the donor agency to lower the targets, based on the baseline values that were identified through direct field measurements. 1. Evaluate beneficiaries’ knowledge and skills to refine capacity building activities • The result of the indicator that measured OPF members’ knowledge about the LAE indicated that the project will improve their knowledge, which will enable them to use their purchasing power to encourage and improve the food production of local farmers in the communities, thus promoting the economic development of local families. • The project should undertake a strategy to train the mothers, whether volunteers or paid, involved in meal preparation at schools to adopt the MINEDUC school meal menus established by the LAE and make sure the meals include the agricultural products prioritized by Nuestra Cosecha. • OPF members, school principals, and school feeding committee members should improve their skills to more efficiently carry out procurement processes, including asking for quotes and presenting the necessary documents to receive disbursements and pay suppliers on time. 2. Identify opportunities and threats to project implementation Opportunities: • Nuestra Cosecha has the important opportunity to create a model for local purchases and the promotion of family farming that can be replicated at national level. The evaluation team did not find any similar project in Guatemala, even with the implementation of pilot projects in schools due to the LAE to generate knowledge and experiences related to local food production. This could be a successful project that could be adopted by the MINEDUC and other governmental organizations and scaled to other regions of the country. • PCI, CRS, and SCI will become an example at the national level in terms of the successful implementation of a local purchase model, which will boost the local economy in targeted communities, reduce migration, provide economic alternatives to local families traditionally dedicated to subsistence farming, and increase the availability of healthy food products for schoolchildren. Without a safe and secured food supply chain, schools currently do not know the source, quality, or safety of the food being distributed through school meals. • This opportunity to create the first large-scale production and marketing experiences at the local level will motivate schools in the surrounding communities but not targeted by Nuestra 43 Cosecha to undertake similar procurement processes, which would be an unexpected result of the project, generating an impact at the municipal, departmental, and national levels. Threats: • Schools could leave the project because of its effect on the personal interests of school principals, teachers, and/or OPF members. • The effects of climate change in the region could produce floods, droughts, frosts, or pests that could lead to a loss of crops for local farmers. • If the schools begin to buy only non-perishables from their current vendors, the vendors could pressure schools to continue their existing contracts by not selling any products to the schools or by increasing prices. • Given the expectation generated by the investment of USDA funds in the procurement process, the price of food products could increase at the community and municipal levels. 5.3.3 Sustainability a. To what extent should local actors (private sector, community entities/associations, local government agencies, etc.) be engaged and committed to ensure sustainability? To ensure the involvement and commitment of local stakeholders for project sustainability, the project should ensure the following components are included: 1. Proper implementation, by Nuestra Cosecha staff, of the strategies and activities that were included the project’s Results Framework. 2. An appropriate mechanism to disseminate project knowledge and learning with key government and municipal actors to encourage their participation in the project and help them understand their specific roles. 3. Awareness raising and the empowerment of key school actors, such as OPF members and administrators, so they can guide the procurement process with transparency and adopt mechanisms for accountability with the MINEDUC and the community. 4. The provision of clear tools and procedures to guide current suppliers, as well as local producers who can become suppliers, during and beyond the project. Sustainability will be measured not only in target schools but also in schools that decide to replicate the model. 5. The enabling of all other stakeholders, including public, private, and non￾governmental organizations, by the MINEDUC leadership to become proactively involved in implementing and complying with the LAE, which mandates the promotion of local economic development. b. What specific institutional capacity needs (e.g., for key government of Guatemala, municipal, community, and school level stakeholders) should the project address to foster greater engagement and accountability? 1. The training of all stakeholders at all levels, including departmental, municipal, and community, on the LAE and its regulations, so everyone involved understands their responsibilities and specific roles. Female OPF members should be specifically 44 targeted because even though they constitute the majority of OPF members, they were the least knowledgeable about the law. 2. Training on agricultural techniques and technification, including building linkages with other institutions, particularly the MAGA and INTECAP. 3. Engagement with and training for existing agricultural producer associations to improve their productive capacities and integrate new smallholder farmers, who will face difficulties in becoming formal food suppliers for the schools. 5.3.4 Effectiveness a. Determine to what extent the Results Framework is realistic, appropriate, and practical to implement and identify if design, structure, logic, or management needs modifications to facilitate the achievement of desired outcomes and impact. 1. Nuestra Cosecha includes a results framework containing the basic elements for a local and regional food procurement project that aims to improve cost￾effectiveness, reduce the number of days for procurement processes to supply schools, utilize culturally acceptable nutritious foods, and engage key stakeholders (government, private sector, OPF, etc.). 2. Success will depend on the development of strategies and activities to meet the results framework, including an active accompaniment by the MINEDUC, involvement by related government institutions, including municipal councils, OPF commitment, capacity strengthening, private sector engagement, procurement process transparency in school feedings, and effective coordination by PCI, CRS, and SCI to implement a unified and coordinated project in the departments of Huehuetenango, Totonicapán, and El Quiché. 6. Conclusions a. General Aspects 1. With the contributions guaranteed by the LAE (decree number 16-2017), which consists of four quetzals per student per day, schools under all three groups selected for this study already helped students achieve an acceptable minimum diet due to the allocated budget allowing schools to buy products based on the MINEDUC menus. 2. Additionally, local communities will likely experience an economic revitalization based on the LAE’s article 15, paragraph two, which states: "of the total financial resources assigned to each school, at least fifty percent (50%) must be used to purchase food products grown by family farmers, as long as the necessary supply exists in the local market." While this is an assumption that will be tested during the final evaluation, it should be noted that the market study conducted by PCI during project start-up collected data from schools, all of which procure fresh products from wholesalers or small retailers who source their products from outside of the municipality. So it is logical to conclude that increase local purchases from from near 0% to 50% will increase household incomes and improve local economies. To this end, efforts to increase local food production, ensure producers are in compliance with local laws, and enforce the school feeding laws are key to local food procurement functioning as an economic lever. 45 3. Menus facilitated by the MINEDUC and implementing organizations (PCI, CRS, and SCI), along with the commitment of school principals, the training of mothers who prepare school meals, and the increased awareness of OPF members on the importance of varying the menus, were all factors that contributed to a high level of satisfaction among students in the food they consume at school. 4. Although the LAE provides resources for school feedings, intensive work will be required in schools to strengthen the capacities of mothers who prepare the meals and of OPF members to select the products with the required quantity and quality and efficiently assume their role as managers of school procurement processes. Therefore, the activities currently implemented by USDA-funded organizations are crucial for achieving an integrated and quality diet for schoolchildren, in close collaboration with MINEDUC. 5. Most schools have between six and ten school meal menus and use one each day to ensure that no meals are repeated during the week. 6. According to the results of 44 focus groups conducted under this study, parents overwhelmingly believed that the new school feeding system motivated children to not miss school. 7. The three implementing organizations (PCI, CRS, and SCI) received special recognition in the intervention schools where they implemented their respective programs, both with school principals and OPF members, who expressed great appreciation for the contributions, the work, and the accompaniment provided. 8. The modality in which school meals are served supports family finances in local communities because children receive a formal meal at school, either breakfast or lunch, which helps families save money. Formal meals include the staple food groups, and non-formal meals generally include a cup of coffee and sometimes a tortilla, which is what students often consume at home. 9. Some products selected for Nuestra Cosecha are not currently purchased in local schools, so the project should analyze if those products should be replaced by other products or included with additional products that are available in target communities. 10. In the schools that were visited, the school principal was generally the person who decides where to buy the food. This was related to the fact that the school principal is seen as the most knowledgeable person in the community and the one with contacts and greater mobility, which resulted in the majority of schools having a single supplier. 11. The need to improve the quality of drinking water was mentioned in all 44 focus groups conducted with OPFs. In Huehuetenango, the main concern was the need for clean drinking water, and in some cases, the long distance to take the water home or to the production areas to grow fresh food. In Totonicapán, particularly in Momostenango, participants expressed interest in trainings on rainwater collection, given the scarcity in some of their communities, and the need to improve water containers in schools. In Quiche, OPFs expressed the need to improve the quality of drinking water. 46 12. In the schools visited during this baseline evaluation, the LAE did not fulfill its promise to boost local economies yet. Food was not purchased from community farmers because most local producers were subsistence farmers and did not generate surplus food to sell to the schools, or they did not have the right conditions in terms of soil characteristics, sloped terrains, and a lack of water for irrigation. 13. Based on data collected during the 44 focus groups conducted during the study, participants said there was no surplus production in most of these communities. The little they had that they could sell, they did not sell to the schools because they were not registered with the SAT, which was seen as an unsurmountable challenge for the time being. 14. Most schools had a single supplier, from which schools buy their products and take them to the school themselves, or the products are delivered by the supplier. Participants saw a single supplier as a benefit because they can buy everything they need in one place, and that place provides a formal invoice and facilitates compliance during the procurement process, which helps to avoid any future difficulties. 15. For the most part, students were satisfied and motivated with the food they received. They showed satisfaction for the varied menus, which ensured a different meal every day of the week, and they had a favorite day based on the meal served that day. 16. In control schools, there was a small difference in student satisfaction with the food received, although the students were still generally satisfied. 17. Regardless of the type of school they were in, students overwhelmingly demonstrated a high level of satisfaction with the food received in school, as a result of the new allocation of four quetzals per student per day by the MINEDUC to provide a varied menu for school feedings. This allowed schools to provide different meals. 18. The root grains and tubers food group was consumed by all students. Yucca was the most frequently consumed root, while maize and beans were the most frequently consumed grains, and potatoes and carrots were the most commonly consumed tubers. 19. The afternoon snack was the least frequently consumed meal of the day. 20. Schools in Huehuetenango had the lowest percentage of students consuming the minimum acceptable diet. Poverty and extreme poverty were the main reason, as families lacks the economic resources to buy food for lunch, afternoon snack, or dinner. 21. OPF members lacked knowledge about the LAE, which is the foundation for developing transparent procurement processes in schools, which will benefit community members by becoming suppliers. 22. Fewer female respondents and fewer respondents at schools participating in the LRP program demonstrated a basic understanding of the LAE. Among those who said they have received training on the LAE, respondents in Momostenango had the lowest percentage of respondents who demonstrated basic knowledge of the LAE. 47 b. Final conclusion Compliance with the LAE will elevate the quality of schoolchildren’s nutrition and serve as both an incentive for children to attend school and a deterrent from dropping out of school. The law should be supported and refined to not only provide a quality and varied diet, but to also emphasize and boost the local economy and become a vehicle for change for the new generation of children in the country. 7. Recommendations 1. It is important to identify the relevance and complementarity of the different products contributed by the project with the types of food students do not receive at home and which they like to maintain the motivation and desire to go to school, study, learn, and eat a nutritious and tasty meal. 2. OPF capacities should be strengthened to allow them to gradually assume greater decision￾making within the product selection and procurement processes for school feedings. 3. An awareness-raising and pedagogical intermediation program should be undertaken to reduce apprehension about registering with the SAT and allow local farmers to formally engage with schools as direct suppliers of fresh products for school feeding. 4. The experiences of existing successful production systems should be systematized and scaled to other schools. For example, lessons should be shared from the successes of producers who sold their surplus production to the schools, or investments by the program to develop production activities, such as the case of the poultry farms to supply eggs in Jolomtaj, Barillas, in the farm of local community member Robinson Ávila. 5. Overall, local producers do not have the skills and technification required to produce the crops selected by the project. To address this, the project should establish a training and agricultural technology transfer program in collaboration with government institutions, particularly the MAGA, the INTECAP, technical agriculture schools, and other existing actors, including cooperatives or agricultural associations. 6. The program should promote an associative production system that includes a mechanism to collect the products from many different smallholder farmers, negotiate with schools, and pay taxes, which would be a model that can be replicated to take advantage of the opportunities provided by LAE as the vehicle for boosting the local economy in the target communities and municipalities. 7. An extensive training process should be carried out with OPF members to provide additional knowledge about the LAE beyond the general information and to teach them about the operational aspects of the law where there are knowledge gaps. 8. The project should identify different ways to streamline administrative and control paperwork in schools to allow teachers to spend less time on these issues and have more time to teach students. 48 9. The project should identify ways to reduce any existing, unnecessary bureaucratic procedures and facilitate quick error and problem solving mechanisms to save OPF members the time and resources they put into liquidation meetings with technical administrative staff and to avoid the need for them to make multiple trips to their communities, which sometimes take a full day of travel due to road condition and lack of public transport. 10. Special attention should be paid to the recommendations in the chapter "Analysis of key evaluation questions," in section 6.3 of this document, which presents the evaluation team’s experience in the field based on the qualitative analysis conducted with members of the education community and the respective LAE beneficiaries. Final recommendation The existing school feeding program should be strengthened by improving the capacities of school parents to work with teachers to select quality, nutritious food for their children’s school meals. It is also crucial to initiate local procurement processes through the development of associative production systems, which can provide a significant and attractive volume of fresh food for the schools, allowing local farmers to become suppliers for groups of schools and serving as an economic boost for the local economy in the target communities and municipalities. Section 1. Clasificación 1.1 instrucciones 1.2 Sexo 1.3 Municipio 1.4 Tipo de escuela LB Dieta Mínima Last Modified by: Jose Mena on 25 Mar 2019 14:41:59 Revision number: 166 Field Count: 58 Instrucciones: (Lea al entrevistado/a) PCI en asocio con CRS y Save the Children está implementando un proyecto actualmente en los departamentos de Huehuetenango, Totonicapán y Quiché denominado Nuestra Cosecha, por lo que te queremos preguntar sobre los alimentos que tomaste el día de ayer Sexo del entrevistado Expects a single option response (required) Masculino [1] Femenino [2] Municipio Expects a single option response (required) Santa Eulalia [1] Barillas [2] Momostenango [3] Nebaj [4] Chajul [5] Tipo de escuela Expects a single option response (required) Nuestra Cosecha [1] FFE [2] Control [3] 1.5 Comunidad 1.6 Escuela Comunidad Expects a single option response (required) Paraje La Cumbre [12] Aldea Xequemeya [13] Caserio Canquixaja [14] Barrio Santa Ana [15] Paraje Paxaq [16] Paraje Sicalbe [17] Paraje San Rafael Racana [18] Caserio Rachoquel [19] Paraje Chacalte [20] Caserio Chonimatux [21] Paraje Chicorral [22] Xoljoc [23] Nimsituj [24] Centro Dos [25] Aldea Pitzal [26] Ixtenam [1] Bololac [2] Yulconop [3] Miramar [4] Temux Chiquito [5] Yulxac [6] Santa Rosa [7] Canton B las Brisas [8] Nueva Reforma [9] Chocsunil [10] Altamiranda [11] Xepiun [27] Saquil Grande [28] Acul [29] Cambalam [30] Pelax grande [31] Tucoral [32] Vicampanavitz [33] Xachkoral [34] Xolcuay [35] Cocup [36] Sajbuta [37] Xepucxuc [38] Xolacul [39] Xevitz [40] Vicotz [41] Nombre de la escuela Expects a single line text response (optional) 1.7 Consentimiento informado Entiendo que la información que estoy dando es confidencial y para usos del presente estudio, entiendo mis derechos y deseo participar el el presente estudio Expects a single option response (required), Default: Si Si [1] No [2] Branches If response Equals 'No [2]' then skip to finalizar (7.3) Section 2. Desayuno 2.1 Tomo desayuno 2.2 donde desayuno 2.3 Grupo 11 2.4 grupo 12 2.5 grupo 13 Tomó su desayuno el día de ayer? Expects a single option response (required) Si [1] No [2] Branches If response Equals 'No [2]' then skip to Tomo Refa am (3.1) Dónde tomó su desayuno? Expects a single option response (required) En la escuela [1] En Casa [2] Granos, raíces y tubérculos Expects multiple selected options (required) Cereal [1] Tortillas [2] Tamal, Chuchito, pache, etc [3] Arroz [4] Pan [5] Fideos, coditos u otras pastas [6] Avena / Mosh [7] Yuca [8] Papa [9] Atoles, CSB, incaparina [10] Platano, Ixintal [11] Ninguno [0] Legumbres y frutos secos Expects multiple selected options (required) Frijol [1] Ejote [2] Manía [3] Nueces [4] Ninguno [0] Branches If response Includes 'Ninguno [0]' then skip to grupo 13 (2.5) Productos lácteos Expects multiple selected options (required) Leche [1] Crema [2] Queso [3] Ninguno [0] 2.6 grupo 14 2.7 grupo 15 2.8 grupo 16 2.9 grupo 17 Alimentos de Carne Expects multiple selected options (required) Pollo [1] Res [2] Cerdo [3] Ninguno [0] Huevos Expects multiple selected options (required) Si [1] No [2] Alimentos con vitamina A Expects multiple selected options (required) Zanahoría [1] Brocolí [2] Berro [6] Acelga [7] Espinaca [3] Hierbas [10] Chile Pimiento [9] Tomate [8] Guicoy [11] Melón [4] Mango [5] Papaya [12] Naranja [13] Piña [14] Ninguno [0] Otras frutas y hortalizas Expects multiple selected options (required) Mandarina [2] Banano [Banano] Otra fruta: [4] Rábano [5] Lechuga [6] Pepino [7] Coliflor [8] Remolacha [9] Ninguno [0] Branches If response Includes 'Otra fruta: [4]' then skip to Otra fruta (2.10) If response Excludes 'Otra fruta: [4]' then skip to Tomo Refa am (3.1) 2.10 Otra fruta Qué otra fruta? Expects a single line text response (required) Section 3. Refaccion am 3.1 Tomo Refa am 3.2 donde refacciono 3.3 Grupo 11 Refa am 3.4 grupo 12 Refa am 3.5 grupo 13 Refa am Tomó refacción en la mañana el día de ayer? Expects a single option response (required) Si [1] No [2] Branches If response Equals 'No [2]' then skip to Tomo Almuerzo (4.1) Dónde refaccionó ayer en la mañana? Expects a single option response (required) En la escuela [1] En Casa [2] Granos, raíces y tubérculos Expects multiple selected options (required) Cereal [1] Tortillas [2] Tamal, Chuchito, pache, etc [3] Arroz [4] Pan [5] Fideos, coditos u otras pastas [6] Avena / Mosh [7] Yuca [8] Papa [9] Atoles, CSB, incaparina [10] Platano, Ixintal [11] Ninguno [0] Legumbres y frutos secos Expects multiple selected options (required) Frijol [1] Ejote [2] Manía [3] Nueces [4] Ninguno [0] Productos lácteos Expects multiple selected options (required) Leche [1] Crema [2] Queso [3] Ninguno [0] 3.6 grupo 14 Refa am 3.7 grupo 15 Refa am 3.8 grupo 16 Refa am 3.9 grupo 17 Refa am Alimentos de Carne Expects multiple selected options (required) Pollo [1] Res [2] Cerdo [3] Ninguno [0] Huevos Expects multiple selected options (required) Si [1] No [2] Alimentos con vitamina A Expects multiple selected options (required) Ninguno [0] Zanahoría [1] Brocolí [2] Berro [6] Acelga [7] Espinaca [3] Hierbas [10] Chile Pimiento [9] Tomate [8] Guicoy [11] Melón [4] Mango [5] Papaya [12] Naranja [13] Piña [14] Otras frutas y hortalizas Expects multiple selected options (required) Mandarina [2] Banano [Banano] Otra fruta: [4] Rábano [5] Lechuga [6] Pepino [7] Coliflor [8] Remolacha [9] Ninguno [0] Branches If response Includes 'Otra fruta: [4]' then skip to Otra fruta Refa am (3.10) If response Excludes 'Otra fruta: [4]' then skip to Tomo Almuerzo (4.1) 3.10 Otra fruta Refa am Qué otra fruta? Expects a single line text response (required) Section 4. Almuerzo 4.1 Tomo Almuerzo 4.2 Grupo 11 Almuerzo 4.3 grupo 12 Almuerzo 4.4 grupo 13 Almuerzo 4.5 grupo 14 Almuerzo Almorzo el día de ayer? Expects a single option response (required) Si [1] No [2] Branches If response Equals 'No [2]' then skip to Tomo Refa pm (5.1) Granos, raíces y tubérculos Expects multiple selected options (required) Cereal [1] Tortillas [2] Tamal, Chuchito, pache, etc [3] Arroz [4] Pan [5] Fideos, coditos u otras pastas [6] Avena / Mosh [7] Yuca [8] Papa [9] Atoles, CSB, incaparina [10] Platano, Ixintal [11] Ninguno [0] Legumbres y frutos secos Expects multiple selected options (required) Frijol [1] Ejote [2] Manía [3] Nueces [4] Ninguno [0] Productos lácteos Expects multiple selected options (required) Leche [1] Crema [2] Queso [3] Ninguno [0] Alimentos de Carne Expects multiple selected options (required) Pollo [1] Res [2] Cerdo [3] Ninguno [0] 4.6 grupo 15 Almuerzo 4.7 grupo 16 Almuerzo 4.8 grupo 17 Almuerzo 4.9 Otra fruta Almuerzo Huevos Expects multiple selected options (required) Si [1] No [2] Alimentos con vitamina A Expects multiple selected options (required) Zanahoría [1] Brocolí [2] Berro [6] Acelga [7] Espinaca [3] Hierbas [10] Chile Pimiento [9] Tomate [8] Guicoy [11] Melón [4] Mango [5] Papaya [12] Naranja [13] Piña [14] Ninguno [0] Otras frutas y hortalizas Expects multiple selected options (required) Mandarina [2] Banano [Banano] Otra fruta: [4] Rábano [5] Lechuga [6] Pepino [7] Coliflor [8] Remolacha [9] Ninguno [0] Branches If response Includes 'Otra fruta: [4]' then skip to Otra fruta Almuerzo (4.9) If response Excludes 'Otra fruta: [4]' then skip to Tomo Refa pm (5.1) Qué otra fruta? Expects a single line text response (required) Section 5. Refaccion PM 5.1 Tomo Refa pm 5.2 Grupo 11 Refa pm 5.3 grupo 12 Refa pm 5.4 grupo 13 Refa pm 5.5 grupo 14 Refa pm Tomó refacción en la tarde de ayer? Expects a single option response (required) Si [1] No [2] Branches If response Equals 'No [2]' then skip to Tomo Cena (6.1) Granos, raíces y tubérculos Expects multiple selected options (required) Cereal [1] Tortillas [2] Tamal, Chuchito, pache, etc [3] Arroz [4] Pan [5] Fideos, coditos u otras pastas [6] Avena / Mosh [7] Yuca [8] Papa [9] Atoles, CSB, incaparina [10] Platano, Ixintal [11] Ninguno [0] Legumbres y frutos secos Expects multiple selected options (required) Frijol [1] Ejote [2] Manía [3] Nueces [4] Ninguno [0] Productos lácteos Expects multiple selected options (required) Leche [1] Crema [2] Queso [3] Ninguno [0] Alimentos de Carne Expects multiple selected options (required) Pollo [1] Res [2] Cerdo [3] Ninguno [0] 5.6 grupo 15 Refa pm 5.7 grupo 16 Refa pm 5.8 grupo 17 Refa pm 5.9 Otra fruta Refa pm Huevos Expects multiple selected options (required) Si [1] No [2] Alimentos con vitamina A Expects multiple selected options (required) Zanahoría [1] Brocolí [2] Berro [6] Acelga [7] Espinaca [3] Hierbas [10] Chile Pimiento [9] Tomate [8] Guicoy [11] Melón [4] Mango [5] Papaya [12] Naranja [13] Piña [14] Ninguno [0] Otras frutas y hortalizas Expects multiple selected options (required) Mandarina [2] Banano [Banano] Otra fruta: [4] Rábano [5] Lechuga [6] Pepino [7] Coliflor [8] Remolacha [9] Ninguno [0] Branches If response Includes 'Otra fruta: [4]' then skip to Otra fruta Refa pm (5.9) If response Excludes 'Otra fruta: [4]' then skip to Tomo Cena (6.1) Qué otra fruta? Expects a single line text response (required) Section 6. Cena 6.1 Tomo Cena 6.2 donde desayuno Cena 6.3 Grupo 11 Cena 6.4 grupo 12 Cena 6.5 grupo 13 Cena Ceno el día de ayer? Expects a single option response (required) Si [1] No [2] Branches If response Equals 'No [2]' then skip to alimentacion escolar (7.1) Dónde tomó su cena? Expects a single option response (required) En la escuela [1] En Casa [2] Granos, raíces y tubérculos Expects multiple selected options (required) Cereal [1] Tortillas [2] Tamal, Chuchito, pache, etc [3] Arroz [4] Pan [5] Fideos, coditos u otras pastas [6] Avena / Mosh [7] Yuca [8] Papa [9] Atoles, CSB, incaparina [10] Platano, Ixintal [11] Ninguno [0] Legumbres y frutos secos Expects multiple selected options (required) Frijol [1] Ejote [2] Manía [3] Nueces [4] Ninguno [0] Productos lácteos Expects multiple selected options (required) Leche [1] Crema [2] Queso [3] Ninguno [0] 6.6 grupo 14 Cena 6.7 grupo 15 Cena 6.8 grupo 16 Cena 6.9 grupo 17 Cena Alimentos de Carne Expects multiple selected options (required) Pollo [1] Res [2] Cerdo [3] Ninguno [0] Huevos Expects multiple selected options (required) Si [1] No [2] Alimentos con vitamina A Expects multiple selected options (required) Ninguno [0] Zanahoría [1] Brocolí [2] Berro [6] Acelga [7] Espinaca [3] Hierbas [10] Chile Pimiento [9] Tomate [8] Guicoy [11] Melón [4] Mango [5] Papaya [12] Naranja [13] Piña [14] Otras frutas y hortalizas Expects multiple selected options (required) Mandarina [2] Banano [Banano] Otra fruta: [4] Rábano [5] Lechuga [6] Pepino [7] Coliflor [8] Remolacha [9] Ninguno [0] Branches If response Includes 'Otra fruta: [4]' then skip to Otra fruta Cena (6.10) If response Excludes 'Otra fruta: [4]' then skip to alimentacion escolar (7.1) 6.10 Otra fruta Cena Qué otra fruta? Expects a single line text response (required) Section 7. Alimentación escolar 7.1 alimentacion escolar 7.2 escala 7.3 finalizar Observa la siguiente gráfica y en base a ella dime como te sientes con respecto a la comida que te dieron en la escuela el día de hoy Señala la carita que más te parece que representa la comida de la escuela Expects a single option response (required) 1. Odie [1] 2. No me gusto [2] 3. Indiferente [3] 4. Me gusto [4] 5. Me encanto [5] Si desea terminar y enviar cuestionario presionar en siguiente Section 1. Clasificación 1.1 instrucciones 1.2 1Sexo 1.3 1Municipio 1.4 1Tipo de escuela LB directores Last Modified by: Jose Mena on 25 Mar 2019 14:47:28 Revision number: 28 Field Count: 19 Instrucciones: (Lea al entrevistado/a) PCI en asocio con CRS y Save the Children está implementando un proyecto actualmente en los departamentos de Huehuetenango, Totonicapán y Quiché denominado Nuestra Cosecha, por lo que nos interesa conocer su valiosa opinión sobre algunos temas, gracias Sexo del entrevistado Expects a single option response (required) Masculino [1] Femenino [2] Municipio Expects a single option response (required) Santa Eulalia [1] Barillas [2] Momostenango [3] Nebaj [4] Chajul [5] Tipo de escuela Expects a single option response (required) Nuestra Cosecha [1] FFE [2] Control [3] 1.5 Comunidad 1.6 Nombre Comunidad Expects a single option response (required) Ixtenam [1] Bololac [2] Yulconop [3] Miramar [4] Temux Chiquito [5] Yulxac [6] Santa Rosa [7] Canton B las Brisas [8] Nueva Reforma [9] Chocsunil [10] Altamiranda [11] Paraje La Cumbre [12] Aldea Xequemeya [13] Caserio Canquixaja [14] Barrio Santa Ana [15] Paraje Paxaq [16] Paraje Sicalbe [17] Paraje San Rafael Racana [18] Caserio Rachoquel [19] Paraje Chacalte [20] Caserio Chonimatux [21] araje Chicorral [22] Xoljoc. [23] Nimsituj [24] Centro Dos [25] Aldea Pitzal [26] Xepiun [27] Saquil grande [28] Acul [29] Cambalam [30] Pexlaj grande [31] Tucoral [32] Vicampanavitz [33] Xachkoral [34] Xolcuay [35] Cocop [36] Sajbuta [37] Xepucxuc [38] Xolacul [39] Xevitz [40] Vicotz [41] Nombre de la escuela Expects a single line text response (required) 1.7 total de alumnos 1.8 Consentimiento informado Total de alumnos de la escuela Expects a single line text response (required) Entiendo que la información que estoy dando es confidencial y para usos del presente estudio, entiendo mis derechos y deseo participar el el presente estudio Expects a single option response (required), Default: Si Si [1] No [2] Section 2. Diversidad dietetica 2.1 Sirven alimentacion 2.2 grupos 2.3 Grupo 11 2.4 grupo 12 2.5 grupo 13 Se sirve alimentación escolar en esta escuela? Expects a single option response (required) Si [1] No [2] Branches If response Equals 'Si [1]' then skip to grupos (2.2) If response Equals 'No [2]' then skip to finalizar (2.11) Mencione que alimentos se sirven en esta escuela como parte del programa de alimentación escolar Granos, raíces y tubérculos Expects multiple selected options (required) Cereal [1] Tortillas [2] Tamal, Chuchito, pache, etc [3] Arroz [4] Pan [5] Fideos, coditos u otras pastas [6] Avena / Mosh [7] Yuca [8] Papa [9] Atoles, CSB, incaparina [10] Platano, Ixintal [11] Ninguno [0] Legumbres y frutos secos Expects multiple selected options (required) Frijol [1] Ejote [2] Manía [3] Nueces [4] Ninguno [0] Productos lácteos Expects multiple selected options (required) Leche [1] Crema [2] Queso [3] Ninguno [0] 2.6 grupo 14 2.7 grupo 15 2.8 grupo 16 2.9 grupo 17 Alimentos de Carne Expects multiple selected options (required) Pollo [1] Res [2] Cerdo [3] Ninguno [0] Huevos Expects multiple selected options (required) Si [1] No [2] Alimentos con vitamina A Expects multiple selected options (required) Zanahoría [1] Brocolí [2] Berro [6] Acelga [7] Espinaca [3] Hierbas [10] Chile Pimiento [9] Tomate [8] Guicoy [11] Melón [4] Mango [5] Papaya [12] Naranja [13] Piña [14] Ninguno [0] Otras frutas y hortalizas Expects multiple selected options (required) Mandarina [2] Banano [Banano] Otra fruta: [4] Rábano [5] Lechuga [6] Pepino [7] Coliflor [8] Remolacha [9] Ninguno [0] Branches If response Includes 'Otra fruta: [4]' then skip to Otra fruta (2.10) If response Excludes 'Otra fruta: [4]' then skip to finalizar (2.11) 2.10 Otra fruta 2.11 finalizar Qué otra fruta? Expects a single line text response (required) Para salir y guardar dar clic en siguiente Section 1. Clasificación 1.1 instrucciones 1.2 1Sexo 1.3 1Municipio 1.4 1Tipo de escuela LB OPF Last Modified by: Jose Mena on 25 Mar 2019 14:52:25 Revision number: 56 Field Count: 20 Instrucciones: (Lea al entrevistado/a) PCI en asocio con CRS y Save the Children está implementando un proyecto actualmente en los departamentos de Huehuetenango, Totonicapán y Quiché denominado Nuestra Cosecha, por lo que nos interesa conocer su valiosa opinión sobre algunos temas, gracias Sexo del entrevistado Expects a single option response (required) Masculino [1] Femenino [2] Municipio Expects a single option response (required) Santa Eulalia [1] Barillas [2] Momostenango [3] Nebaj [4] Chajul [5] Tipo de escuela Expects a single option response (required) Nuestra Cosecha [1] FFE [2] Control [3] 1.5 Comunidad 1.6 Consentimiento informado Comunidad Expects a single option response (required) Ixtenam [1] Bololac [2] Yulconop [3] Miramar [4] Temux Chiquito [5] Yulxac [6] Santa Rosa [7] Canton B las Brisas [8] Nueva Reforma [9] Chocsunil [10] Altamiranda [11] Paraje La Cumbre [12] Aldea Xequemeya [13] Canquixaja [14] Barrio Santa Ana [15] Paraje Paxaq [16] Paraje Sicalbe [17] Paraje San Rafael Racana [18] Caserio Rachoquel [19] Paraje Chacalte [20] Caserio Chonimatux [21] Paraje Chicorral [22] Xoljoc [23] Nimsituj [24] Centro Dos [25] Aldea Pitzal [26] Xepiun [27] Saquil grande [28] Acul [29] Cambalam [30] Pelax Grande [31] Tucoral [32] Vicampanavitz [33] Xachkoral [34] Xolcuay [35] Cocop [36] Sajbuta [37] Xepucxuc [38] Xolacul [39] Xevitz [40] Vicotz [41] Entiendo que la información que estoy dando es confidencial y para usos del presente estudio, entiendo mis derechos y deseo participar el el presente estudio Expects a single option response (required), Default: Si Si [1] No [2] Section 2. Conocimientos ley alimentacion escolar 2.1 conoce lae 2.2 alimentos lae 2.3 cuales alimentos 2.4 proceso de compras 2.5 verificacion proceso 2.6 Registros 2.7 formas registro Conoce usted la ley de alimentación escolar? Expects a single option response (required) Si [1] No [2] Branches If response Equals 'No [2]' then skip to proceso de compras (2.4) If response Equals 'Si [1]' then skip to alimentos lae (2.2) Sabe que alimentos se debe adquirir para incluir en los menus escolares? Expects a single option response (required) Si [1] No [2] Branches If response Equals 'Si [1]' then skip to cuales alimentos (2.3) If response Equals 'No [2]' then skip to proceso de compras (2.4) Puede mencionar algunos alimentos que deben incluir los menus? Expects a single line text response (required) Sabe cuál es el proceso que debe seguir la OPF para realizar la compra de alimentos? Expects a single option response (required) Si [1] No [2] Branches If response Equals 'Si [1]' then skip to verificacion proceso (2.5) If response Equals 'No [2]' then skip to Registros (2.6) Puede mencionar algunos pasos del proceso a seguir? Expects a single line text response (required) Conoce la forma en la que deben llevarse los registro de alimentos en la escuela? Expects a single option response (required) Si [1] No [2] Branches If response Equals 'Si [1]' then skip to formas registro (2.7) If response Equals 'No [2]' then skip to SAT (2.8) Qué formas de registro conoce? Expects a single line text response (required) 2.8 SAT 2.9 MAGA 2.10 Capacitacion 2.11 Institucion 2.12 Otra institucion 2.13 aspectos 2.14 guardar El proveedor de alimentos a las escuelas debe estar inscrito en la SAT para poder vender a las escuelas? Expects a single option response (required) Si [1] No [2] No se [3] El proveedor de alimentos a las escuelas debe estar inscrito en el ministerio de Agricultura, ganaderia y alimentación (MAGA) para poder vender a las escuelas? Expects a single option response (required) Si [1] No [2] No se [3] Ha recibido alguna capacitación sobre la Ley de Alimentación escolar? Expects a single option response (required) Si [1] No [2] Branches If response Equals 'Si [1]' then skip to Institucion (2.11) If response Equals 'No [2]' then skip to guardar (2.14) Qué institución le brindo la capacitación? Expects multiple selected options (required) Mineduc [1] PCI [2] Catholic Relief Services [3] Save the Children [4] Otra [5] Branches If response Includes 'Otra [5]' then skip to Otra institucion (2.12) If response Excludes 'Otra [5]' then skip to aspectos (2.13) Qué otra institución? Expects a single line text response (required) Sobre que temas se trato la capacitación? Expects a single line text response (required) Para finalizar y guardar el cuestionario dar click en siguiente Sexo Masculino Femenino % del N de la columna Recuento % del N de la columna Recuento 3 tiempos 4 grupos No 30.4% 167 24.7% 137 Si 69.6% 382 75.3% 418 Total 100.0% 549 100.0% 555 Sexo Tipo de escuela Total Nuestra Cosecha % del N de la columna Recuento % del N de la columna Recuento 3 tiempos 4 grupos No 27.5% 304 32.3% 100 Si 72.5% 800 67.7% 210 Total 100.0% 1104 100.0% 310 Tipo de escuela Intervención Control % del N de la columna Recuento % del N de la columna Recuento 3 tiempos 4 grupos No 25.3% 107 26.1% 97 Si 74.7% 316 73.9% 274 Total 100.0% 423 100.0% 371 Tipo de escuela Departamento Total Huehuetenango % del N de la columna Recuento % del N de la columna Recuento 3 tiempos 4 grupos No 27.5% 304 31.4% 119 Si 72.5% 800 68.6% 260 Total 100.0% 1104 100.0% 379 Departamento Quiché Totonicapan % del N de la columna Recuento % del N de la columna Recuento 3 tiempos 4 grupos No 23.5% 85 27.5% 100 Si 76.5% 277 72.5% 264 Total 100.0% 362 100.0% 364 Departamento Total % del N de la columna Recuento 3 tiempos 4 grupos No 27.5% 304 Si 72.5% 801 Total 100.0% 1105 Comparaciones de proporciones de columnasa Sexo Tipo de escuela Masculino Femenino Nuestra Cosecha Intervención Control (A) (B) (A) (B) (C) 3 tiempos 4 grupos No B Si A Los resultados se basan en pruebas bilaterales con un nivel de significación 0.05. Para cada par significativo, la clave de la categoría con la proporción de columna menor aparece debajo de la categoría con mayor proporción de columna. a. Utilizando la corrección de Bonferroni, se han ajustado las pruebas para todas las comparaciones por pares dentro de una fila para cada subtabla situada más al interior. Comparaciones de proporciones de columnasa Departamento Huehuetenango Quiché Totonicapan (A) (B) (C) 3 tiempos 4 grupos No B Si A Los resultados se basan en pruebas bilaterales con un nivel de significación 0.05. Para cada par significativo, la clave de la categoría con la proporción de columna menor aparece debajo de la categoría con mayor proporción de columna. Comparaciones de proporciones de columnasa Departamento Huehuetenango Quiché Totonicapan (A) (B) (C) 3 tiempos 4 grupos No B Si A Los resultados se basan en pruebas bilaterales con un nivel de significación 0.05. Para cada par significativo, la clave de la categoría con la proporción de columna menor aparece debajo de la categoría con mayor proporción de columna. a. Utilizando la corrección de Bonferroni, se han ajustado las pruebas para todas las comparaciones por pares dentro de una fila para cada subtabla situada más al interior. Tabla personalizada 1 Sexo Masculino Femenino % del N de la columna Recuento % del N de la columna Recuento Alguna vez en el día grupo 11 No .0% 0 .0% 0 Si 100.0% 549 100.0% 555 Sexo Tipo de escuela Total Nuestra Cosecha % del N de la columna Recuento % del N de la columna Recuento Alguna vez en el día grupo 11 No .0% 0 .0% 0 Si 100.0% 1104 100.0% 310 Tipo de escuela Intervención Control % del N de la columna Recuento % del N de la columna Recuento Alguna vez en el día grupo 11 No .0% 0 .0% 0 Si 100.0% 423 100.0% 371 Tipo de escuela Departamento Total Huehuetenango % del N de la columna Recuento % del N de la columna Recuento Alguna vez en el día grupo 11 No .0% 0 .0% 0 Si 100.0% 1104 100.0% 379 Departamento Quiché Totonicapan % del N de la columna Recuento % del N de la columna Recuento Alguna vez en el día grupo 11 No .0% 0 .0% 0 Si 100.0% 362 100.0% 364 Departamento Total % del N de la columna Recuento Alguna vez en el día grupo 11 No .0% 0 Si 100.0% 1105 Comparaciones de proporciones de columnasb Sexo Tipo de escuela Masculino Femenino Nuestra Cosecha Intervención Control (A) (B) (A) (B) (C) Alguna vez en el día grupo 11 No . a . a . a . a . a Si . a . a . a . a . a Los resultados se basan en pruebas bilaterales con un nivel de significación 0.05. Para cada par significativo, la clave de la categoría con la proporción de columna menor aparece debajo de la categoría con mayor proporción de columna. a. Esta categoría no se utiliza en las comparaciones porque su proporción de columna es igual a cero o uno. Comparaciones de proporciones de columnasb Sexo Tipo de escuela Masculino Femenino Nuestra Cosecha Intervención Control (A) (B) (A) (B) (C) Alguna vez en el día grupo 11 No . a . a . a . a . a Si . a . a . a . a . a Los resultados se basan en pruebas bilaterales con un nivel de significación 0.05. Para cada par significativo, la clave de la categoría con la proporción de columna menor aparece debajo de la categoría con mayor proporción de columna. a. Esta categoría no se utiliza en las comparaciones porque su proporción de columna es igual a cero o uno. b. Utilizando la corrección de Bonferroni, se han ajustado las pruebas para todas las comparaciones por pares dentro de una fila para cada subtabla situada más al interior. Comparaciones de proporciones de columnasb Departamento Huehuetenango Quiché Totonicapan (A) (B) (C) Alguna vez en el día grupo 11 No . a . a . a Si . a . a . a Los resultados se basan en pruebas bilaterales con un nivel de significación 0.05. Para cada par significativo, la clave de la categoría con la proporción de columna menor aparece debajo de la categoría con mayor proporción de columna. a. Esta categoría no se utiliza en las comparaciones porque su proporción de columna es igual a cero o uno. b. Utilizando la corrección de Bonferroni, se han ajustado las pruebas para todas las comparaciones por pares dentro de una fila para cada subtabla situada más al interior. Tabla personalizada 1 Sexo Masculino Femenino % del N de la columna Recuento % del N de la columna Recuento Alguna vez en el día grupo 12 No 20.8% 114 22.2% 123 Si 79.2% 435 77.8% 432 Total 100.0% 549 100.0% 555 Sexo Tipo de escuela Total Nuestra Cosecha % del N de la columna Recuento % del N de la columna Recuento Alguna vez en el día grupo 12 No 21.5% 237 23.2% 72 Si 78.5% 867 76.8% 238 Total 100.0% 1104 100.0% 310 Tipo de escuela Intervención Control % del N de la columna Recuento % del N de la columna Recuento Alguna vez en el día grupo 12 No 16.5% 70 25.6% 95 Si 83.5% 353 74.4% 276 Total 100.0% 423 100.0% 371 Tipo de escuela Departamento Total Huehuetenango % del N de la columna Recuento % del N de la columna Recuento Alguna vez en el día grupo 12 No 21.5% 237 21.4% 81 Si 78.5% 867 78.6% 298 Total 100.0% 1104 100.0% 379 Departamento Quiché Totonicapan % del N de la columna Recuento % del N de la columna Recuento Alguna vez en el día grupo 12 No 19.1% 69 23.9% 87 Si 80.9% 293 76.1% 277 Total 100.0% 362 100.0% 364 Departamento Total % del N de la columna Recuento Alguna vez en el día grupo 12 No 21.4% 237 Si 78.6% 868 Total 100.0% 1105 Comparaciones de proporciones de columnasa Sexo Tipo de escuela Masculino Femenino Nuestra Cosecha Intervención Control (A) (B) (A) (B) (C) Alguna vez en el día grupo 12 No B Si C Los resultados se basan en pruebas bilaterales con un nivel de significación 0.05. Para cada par significativo, la clave de la categoría con la proporción de columna menor aparece debajo de la categoría con mayor proporción de columna. a. Utilizando la corrección de Bonferroni, se han ajustado las pruebas para todas las comparaciones por pares dentro de una fila para cada subtabla situada más al interior. Comparaciones de proporciones de columnasa Departamento Huehuetenango Quiché Totonicapan (A) (B) (C) Alguna vez en el día grupo 12 No Si Los resultados se basan en pruebas bilaterales con un nivel de significación 0.05. Para cada par significativo, la clave de la categoría con la proporción de columna menor aparece debajo de la categoría con mayor proporción de columna. a. Utilizando la corrección de Bonferroni, se han ajustado las pruebas para todas las comparaciones por pares dentro de una fila para cada subtabla situada más al interior. Tabla personalizada 1 Sexo Masculino Femenino % del N de la columna Recuento % del N de la columna Recuento Alguna vez en el día grupo 13 No 64.8% 356 56.9% 316 Si 35.2% 193 43.1% 239 Total 100.0% 549 100.0% 555 Sexo Tipo de escuela Total Nuestra Cosecha % del N de la columna Recuento % del N de la columna Recuento Alguna vez en el día grupo 13 No 60.9% 672 62.6% 194 Si 39.1% 432 37.4% 116 Total 100.0% 1104 100.0% 310 Tipo de escuela Intervención Control % del N de la columna Recuento % del N de la columna Recuento Alguna vez en el día grupo 13 No 67.6% 286 51.8% 192 Si 32.4% 137 48.2% 179 Total 100.0% 423 100.0% 371 Tipo de escuela Departamento Total Huehuetenango % del N de la columna Recuento % del N de la columna Recuento Alguna vez en el día grupo 13 No 60.9% 672 66.2% 251 Si 39.1% 432 33.8% 128 Total 100.0% 1104 100.0% 379 Departamento Quiché Totonicapan % del N de la columna Recuento % del N de la columna Recuento Alguna vez en el día grupo 13 No 51.7% 187 64.3% 234 Si 48.3% 175 35.7% 130 Total 100.0% 362 100.0% 364 Departamento Total % del N de la columna Recuento Alguna vez en el día grupo 13 No 60.8% 672 Si 39.2% 433 Total 100.0% 1105 Comparaciones de proporciones de columnasa Sexo Tipo de escuela Masculino Femenino Nuestra Cosecha Intervención Control (A) (B) (A) (B) (C) Alguna vez en el día grupo 13 No B C C Si A A B Los resultados se basan en pruebas bilaterales con un nivel de significación 0.05. Para cada par significativo, la clave de la categoría con la proporción de columna menor aparece debajo de la categoría con mayor proporción de columna. a. Utilizando la corrección de Bonferroni, se han ajustado las pruebas para todas las comparaciones por pares dentro de una fila para cada subtabla situada más al interior. Comparaciones de proporciones de columnasa Departamento Huehuetenango Quiché Totonicapan (A) (B) (C) Alguna vez en el día grupo 13 No B B Si A C Los resultados se basan en pruebas bilaterales con un nivel de significación 0.05. Para cada par significativo, la clave de la categoría con la proporción de columna menor aparece debajo de la categoría con mayor proporción de columna. Comparaciones de proporciones de columnasa Departamento Huehuetenango Quiché Totonicapan (A) (B) (C) Alguna vez en el día grupo 13 No B B Si A C Los resultados se basan en pruebas bilaterales con un nivel de significación 0.05. Para cada par significativo, la clave de la categoría con la proporción de columna menor aparece debajo de la categoría con mayor proporción de columna. a. Utilizando la corrección de Bonferroni, se han ajustado las pruebas para todas las comparaciones por pares dentro de una fila para cada subtabla situada más al interior. Tabla personalizada 1 Sexo Masculino Femenino % del N de la columna Recuento % del N de la columna Recuento Alguna vez en el día grupo 14 No 47.0% 258 45.4% 252 Si 53.0% 291 54.6% 303 Total 100.0% 549 100.0% 555 Sexo Tipo de escuela Total Nuestra Cosecha % del N de la columna Recuento % del N de la columna Recuento Alguna vez en el día grupo 14 No 46.2% 510 51.0% 158 Si 53.8% 594 49.0% 152 Total 100.0% 1104 100.0% 310 Tipo de escuela Intervención Control % del N de la columna Recuento % del N de la columna Recuento Alguna vez en el día grupo 14 No 44.7% 189 43.9% 163 Si 55.3% 234 56.1% 208 Total 100.0% 423 100.0% 371 Tipo de escuela Departamento Total Huehuetenango % del N de la columna Recuento % del N de la columna Recuento Alguna vez en el día grupo 14 No 46.2% 510 47.5% 180 Si 53.8% 594 52.5% 199 Total 100.0% 1104 100.0% 379 Departamento Quiché Totonicapan % del N de la columna Recuento % del N de la columna Recuento Alguna vez en el día grupo 14 No 50.8% 184 40.1% 146 Si 49.2% 178 59.9% 218 Total 100.0% 362 100.0% 364 Departamento Total % del N de la columna Recuento Alguna vez en el día grupo 14 No 46.2% 510 Si 53.8% 595 Total 100.0% 1105 Comparaciones de proporciones de columnasa Sexo Tipo de escuela Masculino Femenino Nuestra Cosecha Intervención Control (A) (B) (A) (B) (C) Alguna vez en el día grupo No 14 Si Los resultados se basan en pruebas bilaterales con un nivel de significación 0.05. Para cada par significativo, la clave de la categoría con la proporción de columna menor aparece debajo de la categoría con mayor proporción de columna. a. Utilizando la corrección de Bonferroni, se han ajustado las pruebas para todas las comparaciones por pares dentro de una fila para cada subtabla situada más al interior. Comparaciones de proporciones de columnasa Departamento Huehuetenango Quiché Totonicapan (A) (B) (C) Alguna vez en el día grupo 14 No C Si B Los resultados se basan en pruebas bilaterales con un nivel de significación 0.05. Para cada par significativo, la clave de la categoría con la proporción de columna menor aparece debajo de la categoría con mayor proporción de columna. a. Utilizando la corrección de Bonferroni, se han ajustado las pruebas para todas las comparaciones por pares dentro de una fila para cada subtabla situada más al interior. Tabla personalizada 1 Sexo Masculino Femenino % del N de la columna Recuento % del N de la columna Recuento Alguna vez en el día grupo 15 No 75.7% 415 72.6% 402 1 24.3% 133 27.4% 152 Total 100.0% 548 100.0% 554 Sexo Tipo de escuela Total Nuestra Cosecha % del N de la columna Recuento % del N de la columna Recuento Alguna vez en el día grupo No 74.1% 817 71.3% 221 15 1 25.9% 285 28.7% 89 Total 100.0% 1102 100.0% 310 Tipo de escuela Intervención Control % del N de la columna Recuento % del N de la columna Recuento Alguna vez en el día grupo 15 No 78.4% 331 71.6% 265 1 21.6% 91 28.4% 105 Total 100.0% 422 100.0% 370 Tipo de escuela Departamento Total Huehuetenango % del N de la columna Recuento % del N de la columna Recuento Alguna vez en el día grupo 15 No 74.1% 817 77.8% 294 1 25.9% 285 22.2% 84 Total 100.0% 1102 100.0% 378 Departamento Quiché Totonicapan % del N de la columna Recuento % del N de la columna Recuento Alguna vez en el día grupo 15 No 76.2% 276 68.0% 247 1 23.8% 86 32.0% 116 Total 100.0% 362 100.0% 363 Departamento Total % del N de la columna Recuento Alguna vez en el día grupo 15 No 74.1% 817 1 25.9% 286 Total 100.0% 1103 Comparaciones de proporciones de columnasa Sexo Tipo de escuela Masculino Femenino Nuestra Cosecha Intervención Control (A) (B) (A) (B) (C) Alguna vez en el día grupo 15 No 1 Los resultados se basan en pruebas bilaterales con un nivel de significación 0.05. Para cada par significativo, la clave de la categoría con la proporción de columna menor aparece debajo de la categoría con mayor proporción de columna. a. Utilizando la corrección de Bonferroni, se han ajustado las pruebas para todas las comparaciones por pares dentro de una fila para cada subtabla situada más al interior. Comparaciones de proporciones de columnasa Departamento Huehuetenango Quiché Totonicapan (A) (B) (C) Alguna vez en el día grupo 15 No C C 1 A B Los resultados se basan en pruebas bilaterales con un nivel de significación 0.05. Para cada par significativo, la clave de la categoría con la proporción de columna menor aparece debajo de la categoría con mayor proporción de columna. a. Utilizando la corrección de Bonferroni, se han ajustado las pruebas para todas las comparaciones por pares dentro de una fila para cada subtabla situada más al interior. Tabla personalizada 1 Sexo Masculino Femenino % del N de la columna Recuento % del N de la columna Recuento Alguna vez en el día grupo 16 0 14.8% 81 14.1% 78 1 85.2% 468 85.9% 477 Total 100.0% 549 100.0% 555 Sexo Tipo de escuela Total Nuestra Cosecha % del N de la columna Recuento % del N de la columna Recuento Alguna vez en el día grupo 16 0 14.4% 159 20.3% 63 1 85.6% 945 79.7% 247 Total 100.0% 1104 100.0% 310 Tipo de escuela Intervención Control % del N de la columna Recuento % del N de la columna Recuento Alguna vez en el día grupo 16 0 7.6% 32 17.3% 64 1 92.4% 391 82.7% 307 Total 100.0% 423 100.0% 371 Tipo de escuela Departamento Total Huehuetenango % del N de la columna Recuento % del N de la columna Recuento Alguna vez en el día grupo 16 0 14.4% 159 18.5% 70 1 85.6% 945 81.5% 309 Total 100.0% 1104 100.0% 379 Departamento Quiché Totonicapan % del N de la columna Recuento % del N de la columna Recuento Alguna vez en el día grupo 16 0 8.8% 32 15.7% 57 1 91.2% 330 84.3% 307 Total 100.0% 362 100.0% 364 Departamento Total % del N de la columna Recuento Alguna vez en el día grupo 0 14.4% 159 16 1 85.6% 946 Total 100.0% 1105 Comparaciones de proporciones de columnasa Sexo Tipo de escuela Masculino Femenino Nuestra Cosecha Intervención Control (A) (B) (A) (B) (C) Alguna vez en el día grupo 16 0 B B 1 A C Los resultados se basan en pruebas bilaterales con un nivel de significación 0.05. Para cada par significativo, la clave de la categoría con la proporción de columna menor aparece debajo de la categoría con mayor proporción de columna. a. Utilizando la corrección de Bonferroni, se han ajustado las pruebas para todas las comparaciones por pares dentro de una fila para cada subtabla situada más al interior. Comparaciones de proporciones de columnasa Departamento Huehuetenango Quiché Totonicapan (A) (B) (C) Alguna vez en el día grupo 16 0 B B 1 A C Los resultados se basan en pruebas bilaterales con un nivel de significación 0.05. Para cada par significativo, la clave de la categoría con la proporción de columna menor aparece debajo de la categoría con mayor proporción de columna. a. Utilizando la corrección de Bonferroni, se han ajustado las pruebas para todas las comparaciones por pares dentro de una fila para cada subtabla situada más al interior. Tabla personalizada 1 Sexo Masculino Femenino % del N de la columna Recuento % del N de la columna Recuento Alguna vez en el día grupo 17 0 56.3% 309 51.9% 288 1 43.7% 240 48.1% 267 Total 100.0% 549 100.0% 555 Sexo Tipo de escuela Total Nuestra Cosecha % del N de la columna Recuento % del N de la columna Recuento Alguna vez en el día grupo 17 0 54.1% 597 66.8% 207 1 45.9% 507 33.2% 103 Total 100.0% 1104 100.0% 310 Tipo de escuela Intervención Control % del N de la columna Recuento % del N de la columna Recuento Alguna vez en el día grupo 17 0 46.1% 195 52.6% 195 1 53.9% 228 47.4% 176 Total 100.0% 423 100.0% 371 Tipo de escuela Departamento Total Huehuetenango % del N de la columna Recuento % del N de la columna Recuento Alguna vez en el día grupo 17 0 54.1% 597 45.1% 171 1 45.9% 507 54.9% 208 Total 100.0% 1104 100.0% 379 Departamento Quiché Totonicapan % del N de la columna Recuento % del N de la columna Recuento Alguna vez en el día grupo 17 0 45.3% 164 72.3% 263 1 54.7% 198 27.7% 101 Total 100.0% 362 100.0% 364 Departamento Total % del N de la columna Recuento Alguna vez en el día grupo 17 0 54.1% 598 1 45.9% 507 Total 100.0% 1105 Comparaciones de proporciones de columnasa Sexo Tipo de escuela Masculino Femenino Nuestra Cosecha Intervención Control (A) (B) (A) (B) (C) Alguna vez en el día grupo 17 0 B C 1 A A Los resultados se basan en pruebas bilaterales con un nivel de significación 0.05. Para cada par significativo, la clave de la categoría con la proporción de columna menor aparece debajo de la categoría con mayor proporción de columna. a. Utilizando la corrección de Bonferroni, se han ajustado las pruebas para todas las comparaciones por pares dentro de una fila para cada subtabla situada más al interior. Comparaciones de proporciones de columnasa Departamento Huehuetenango Quiché Totonicapan (A) (B) (C) Alguna vez en el día grupo 17 0 A B 1 C C Los resultados se basan en pruebas bilaterales con un nivel de significación 0.05. Para cada par significativo, la clave de la categoría con la proporción de columna menor aparece debajo de la categoría con mayor proporción de columna. a. Utilizando la corrección de Bonferroni, se han ajustado las pruebas para todas las comparaciones por pares dentro de una fila para cada subtabla situada más al interior. Tabla personalizada 1 Sexo Masculino Femenino % del N de la columna Recuento % del N de la columna Recuento Total grupos diarios 1 .9% 5 .5% 3 2 7.1% 39 5.4% 30 3 22.0% 121 18.4% 102 4 29.7% 163 30.8% 171 5 24.0% 132 26.3% 146 6 13.1% 72 14.2% 79 7 3.1% 17 4.3% 24 Total 100.0% 549 100.0% 555 Sexo Tipo de escuela Total Nuestra Cosecha % del N de la columna Recuento % del N de la columna Recuento Total grupos diarios 1 .7% 8 1.6% 5 2 6.3% 69 9.0% 28 3 20.2% 223 20.6% 64 4 30.3% 334 34.5% 107 5 25.2% 278 21.6% 67 6 13.7% 151 11.0% 34 7 3.7% 41 1.6% 5 Total 100.0% 1104 100.0% 310 Tipo de escuela Intervención Control % del N de la columna Recuento % del N de la columna Recuento Total grupos diarios 1 .2% 1 .5% 2 2 4.7% 20 5.7% 21 3 20.1% 85 19.9% 74 4 30.7% 130 26.1% 97 5 23.6% 100 29.9% 111 6 16.1% 68 13.2% 49 7 4.5% 19 4.6% 17 Total 100.0% 423 100.0% 371 Tipo de escuela Departamento Total Huehuetenango % del N de la columna Recuento % del N de la columna Recuento Total grupos diarios 1 .7% 8 .5% 2 2 6.3% 69 7.9% 30 3 20.2% 223 22.4% 85 4 30.3% 334 26.1% 99 5 25.2% 278 26.1% 99 6 13.7% 151 13.5% 51 7 3.7% 41 3.4% 13 Total 100.0% 1104 100.0% 379 Departamento Quiché Totonicapan % del N de la columna Recuento % del N de la columna Recuento Total grupos diarios 1 .0% 0 1.6% 6 2 3.9% 14 6.9% 25 3 19.3% 70 18.7% 68 4 28.7% 104 36.0% 131 5 26.8% 97 22.5% 82 6 15.5% 56 12.4% 45 7 5.8% 21 1.9% 7 Total 100.0% 362 100.0% 364 Departamento Total % del N de la columna Recuento Total grupos diarios 1 .7% 8 2 6.2% 69 3 20.2% 223 4 30.2% 334 5 25.2% 278 6 13.8% 152 7 3.7% 41 Total 100.0% 1105 Comparaciones de proporciones de columnasb Sexo Tipo de escuela Masculino Femenino Nuestra Cosecha Intervención Control (A) (B) (A) (B) (C) Total grupos diarios 1 2 3 4 5 A 6 7 Los resultados se basan en pruebas bilaterales con un nivel de significación 0.05. Para cada par significativo, la clave de la categoría con la proporción de columna menor aparece debajo de la categoría con mayor proporción de columna. b. Utilizando la corrección de Bonferroni, se han ajustado las pruebas para todas las comparaciones por pares dentro de una fila para cada subtabla situada más al interior. Comparaciones de proporciones de columnasb Departamento Huehuetenango Quiché Totonicapan (A) (B) (C) Total grupos diarios 1 . a 2 3 4 A 5 6 7 C Los resultados se basan en pruebas bilaterales con un nivel de significación 0.05. Para cada par significativo, la clave de la categoría con la proporción de columna menor aparece debajo de la categoría con mayor proporción de columna. a. Esta categoría no se utiliza en las comparaciones porque su proporción de columna es igual a cero o uno. b. Utilizando la corrección de Bonferroni, se han ajustado las pruebas para todas las comparaciones por pares dentro de una fila para cada subtabla situada más al interior. Sexo Masculino Femenino % del N de la columna Recuento % del N de la columna Recuento Tiempos de comida diarios 1 .2% 1 .0% 0 2 1.1% 6 1.3% 7 3 13.7% 75 11.9% 66 4 48.6% 267 44.5% 247 5 36.4% 200 42.3% 235 Total 100.0% 549 100.0% 555 Sexo Tipo de escuela Total Nuestra Cosecha % del N de la columna Recuento % del N de la columna Recuento Tiempos de comida diarios 1 .1% 1 .3% 1 2 1.2% 13 1.3% 4 3 12.8% 141 11.3% 35 4 46.6% 514 41.6% 129 5 39.4% 435 45.5% 141 Total 100.0% 1104 100.0% 310 Tipo de escuela Intervención Control % del N de la columna Recuento % del N de la columna Recuento Tiempos de comida diarios 1 .0% 0 .0% 0 2 1.4% 6 .8% 3 3 11.1% 47 15.9% 59 4 53.7% 227 42.6% 158 5 33.8% 143 40.7% 151 Total 100.0% 423 100.0% 371 Tipo de escuela Departamento Total Huehuetenango % del N de la columna Recuento % del N de la columna Recuento Tiempos de comida diarios 1 .1% 1 .3% 1 2 1.2% 13 1.6% 6 3 12.8% 141 14.0% 53 4 46.6% 514 48.3% 183 5 39.4% 435 35.9% 136 Total 100.0% 1104 100.0% 379 Departamento Quiché Totonicapan % del N de la columna Recuento % del N de la columna Recuento Tiempos de comida diarios 1 .0% 0 .0% 0 2 1.1% 4 .8% 3 3 13.8% 50 10.4% 38 4 51.1% 185 40.1% 146 5 34.0% 123 48.6% 177 Total 100.0% 362 100.0% 364 Departamento Total % del N de la columna Recuento Tiempos de comida diarios 1 .1% 1 2 1.2% 13 3 12.8% 141 4 46.5% 514 5 39.5% 436 Total 100.0% 1105 Comparaciones de proporciones de columnasb Sexo Tipo de escuela Masculino Femenino Nuestra Cosecha Intervención Control (A) (B) (A) (B) (C) Tiempos de comida diarios 1 . a . a . a 2 3 4 A C 5 A B Los resultados se basan en pruebas bilaterales con un nivel de significación 0.05. Para cada par significativo, la clave de la categoría con la proporción de columna menor aparece debajo de la categoría con mayor proporción de columna. a. Esta categoría no se utiliza en las comparaciones porque su proporción de columna es igual a cero o uno. b. Utilizando la corrección de Bonferroni, se han ajustado las pruebas para todas las comparaciones por pares dentro de una fila para cada subtabla situada más al interior. Comparaciones de proporciones de columnasb Departamento Huehuetenango Quiché Totonicapan (A) (B) (C) Tiempos de comida diarios 1 . a . a 2 3 4 C 5 A B Los resultados se basan en pruebas bilaterales con un nivel de significación 0.05. Para cada par significativo, la clave de la categoría con la proporción de columna menor aparece debajo de la categoría con mayor proporción de columna. a. Esta categoría no se utiliza en las comparaciones porque su proporción de columna es igual a cero o uno. b. Utilizando la corrección de Bonferroni, se han ajustado las pruebas para todas las comparaciones por pares dentro de una fila para cada subtabla situada más al interior. Prueba de significancia 80% concoimiento opf sobre LAE 1Sexo Masculino Femenino % del N de la columna Recuento % del N de la columna Recuento 80% conocimiento lae No 47.2% 34 64.3% 54 Si 52.8% 38 35.7% 30 Total 100.0% 72 100.0% 84 1Sexo 1Tipo de escuela Total Nuestra cosecha % del N de la columna Recuento % del N de la columna Recuento 80% conocimiento lae No 56.4% 88 65.7% 44 Si 43.6% 68 34.3% 23 Total 100.0% 156 100.0% 67 1Tipo de escuela Intervención Control % del N de la columna Recuento % del N de la columna Recuento 80% conocimiento lae No 47.8% 22 51.2% 22 Si 52.2% 24 48.8% 21 Total 100.0% 46 100.0% 43 1Tipo de escuela Departamento Total Huehuetenango % del N de la columna Recuento % del N de la columna Recuento 80% conocimiento lae No 56.4% 88 50.0% 28 Si 43.6% 68 50.0% 28 Total 100.0% 156 100.0% 56 Departamento Quiché Totonicapan % del N de la columna Recuento % del N de la columna Recuento 80% conocimiento lae No 59.2% 29 60.8% 31 Si 40.8% 20 39.2% 20 Total 100.0% 49 100.0% 51 Departamento Total % del N de la columna Recuento 80% conocimiento lae No 56.4% 88 Si 43.6% 68 Total 100.0% 156 Comparaciones de proporciones de columnasa 1Sexo 1Tipo de escuela Masculino Femenino Nuestra cosecha Intervención Control (A) (B) (A) (B) (C) 80% conocimiento lae No A Si B Los resultados se basan en pruebas bilaterales con un nivel de significación 0.05. Para cada par significativo, la clave de la categoría con la proporción de columna menor aparece debajo de la categoría con mayor proporción de columna. a. Utilizando la corrección de Bonferroni, se han ajustado las pruebas para todas las comparaciones por pares dentro de una fila para cada subtabla situada más al interior. Comparaciones de proporciones de columnasa Departamento Huehuetenango Quiché Totonicapan (A) (B) (C) 80% conocimiento lae No Si Los resultados se basan en pruebas bilaterales con un nivel de significación 0.05. Para cada par significativo, la clave de la categoría con la proporción de columna menor aparece debajo de la categoría con mayor proporción de columna. a. Utilizando la corrección de Bonferroni, se han ajustado las pruebas para todas las comparaciones por pares dentro de una fila para cada subtabla situada más al interior. Prueba significancia escala hedonica estudiantes Sexo Masculino Femenino Total % del N de la columna Recuento % del N de la columna Recuento % del N de la columna Recuento escala 1 1.6% 9 1.4% 8 1.5% 17 2 1.3% 7 2.5% 14 1.9% 21 3 6.6% 36 6.1% 34 6.3% 70 4 47.2% 259 41.8% 232 44.5% 491 5 43.4% 238 48.1% 267 45.7% 505 Total 100.0% 549 100.0% 555 100.0% 1104 Tipo de escuela Nuestra Cosecha Intervención Control % del N de la columna Recuento % del N de la columna Recuento % del N de la columna Recuento escala 1 1.6% 5 .2% 1 3.0% 11 2 1.3% 4 1.4% 6 3.0% 11 3 5.2% 16 4.3% 18 9.7% 36 4 42.6% 132 41.6% 176 49.3% 183 5 49.4% 153 52.5% 222 35.0% 130 Total 100.0% 310 100.0% 423 100.0% 371 Tipo de escuela Departamento Total Huehuetenango Quiché % del N de la columna Recuento % del N de la columna Recuento % del N de la columna Recuento escala 1 1.5% 17 .8% 3 3.3% 12 2 1.9% 21 1.1% 4 3.6% 13 3 6.3% 70 4.2% 16 6.6% 24 4 44.5% 491 43.3% 164 40.1% 145 5 45.7% 505 50.7% 192 46.4% 168 Total 100.0% 1104 100.0% 379 100.0% 362 Departamento Totonicapan Total % del N de la columna Recuento % del N de la columna Recuento escala 1 .5% 2 1.5% 17 2 1.1% 4 1.9% 21 3 8.2% 30 6.3% 70 4 50.3% 183 44.5% 492 5 39.8% 145 45.7% 505 Total 100.0% 364 100.0% 1105 Comparaciones de proporciones de columnasa Sexo Tipo de escuela Masculino Femenino Nuestra Cosecha Intervención Control (A) (B) (A) (B) (C) escala 1 B 2 3 B 4 5 C C Los resultados se basan en pruebas bilaterales con un nivel de significación 0.05. Para cada par significativo, la clave de la categoría con la proporción de columna menor aparece debajo de la categoría con mayor proporción de columna. a. Utilizando la corrección de Bonferroni, se han ajustado las pruebas para todas las comparaciones por pares dentro de una fila para cada subtabla situada más al interior. Comparaciones de proporciones de columnasa Departamento Huehuetenango Quiché Totonicapan (A) (B) (C) escala 1 A C 2 3 4 B 5 C Los resultados se basan en pruebas bilaterales con un nivel de significación 0.05. Para cada par significativo, la clave de la categoría con la proporción de columna menor aparece debajo de la categoría con mayor proporción de columna. a. Utilizando la corrección de Bonferroni, se han ajustado las pruebas para todas las comparaciones por pares dentro de una fila para cada subtabla situada más al interior. TERMS OF REFERENCE (TOR) FOR REQUEST FOR PROPOSALS (RFP) For Nuestra Cosecha Local Regional Procurement (LRP) Project Baseline Assessment Consultant Services RFP Issuance Date: TBD RFP Closing Date: TBD Delivery Address: 20 Avenida “A” 3-25, Zona 15, Vista Hermosa I, Guatemala Email: pwagner@pciglobal.org 2 I. INTRODUCTION Project Concern International (PCI) in partnership with Save the Children (StC) and Catholic Relief Services (CRS) will implement a local procurement project to support its ongoing school feeding project in Guatemala focused on achieving the following objectives: • Increase the capacity of municipalities, local producer groups, and schools to effectively and efficiently procure and deliver local, culturally acceptable food commodities to supplement school feeding programs in a timely and cost-effective way. This includes the support the registration and commercialization of local producer groups. • Foster capacity development of stakeholders to strengthen the transition to a school feeding program based on local markets in line with the Government of Guatemala’s (GoG) National School Feeding Law. • Support School Parent Organizations (OPFs) to strengthen their capacity in developing procurement plans, selecting commodities, soliciting bids, and procuring locally grown fresh foods. • Partner with key government stakeholders at national, departmental, and municipal levels in the operationalization of the National School Feeding Law, using target schools as a model for the law’s national implementation. The project will be implemented in 50 schools from Santa Eulalia and Barillas municipalities in Huehuetenango department, Momostenango municipality in Totonicapan department, and Nebaj and Chajul municipalities in Quiche department. PCI is soliciting an external evaluation consultant to conduct a baseline evaluation of the Nuestra Cosecha project. Beneficiaries: PCI will implement program activities directly benefitting 468 small and medium sized local producers1 linked with 50 public primary schools in Santa Eulalia and Barillas municipalities in Huehuetenango department, Momostenango municipality in Totonicapan department, and Nebaj and Chajul municipalities in Quiche department. Local producers will benefit as part of the Family Agriculture component promoted in the Guatemalan Law on School Feed, Decree 16-2017. Further, school meal activities will directly benefit 11,456 primary students who participate in the 1 Family Farmers are defined as: 1) A family group organized as a productive unit that allows it to mobilize every day; 2) The labor force is predominantly family, without limiting the exercise of the rights of its members, being able to contract labor temporarily for complementary activities; 3) The productice unit is managed by a family members; 4) 75% of the household income comes from the family unit; and, 5) Small scale agriculture is practiced on leased lands, or owned without legal certainty and with agricultural limitations. 3 program at the school level within these communities. Additionally, through a focus on local capacity strengthening, 250 OPF members, and national and municipal governments agencies will benefit from project activities. Interventions and Strategies: PCI’s strategic approach is grounded in its Project-Level Results Framework. Based on a clear understanding of international and national donor and government involvement in targeted communities, Nuestra Cosecha will focus on achieving desired results under three key intermediate results (IRs): Improved Cost-Effectiveness of Food Assistance; Improved Timeliness of Food Assistance; Improved Utilization of Nutritious and Culturally Acceptable Food that Meet Quality Standards, and a series of cross-cutting Foundational Results. Activities and objectives to be achieved during the project are detailed in the Project-Level Results at the end of this document. PCI’s vision for the program centers on strengthening existing structures at national, departmental and municipal government levels, as well as school and local producer and value chain levels, to be able to effectively operationalize the local procurement component of the GoG’s school feeding law, while providing culturally acceptable and nutritious school meals and supporting the local economy and producers. PCI’s work will therefore be grounded in four foundational results: Increased Capacity of Government Institutions; Improved Policy and Regulatory Framework; Improved Capacity of Relevant Organizations; and Increased Leverage of Private-Sector Resources. II. EVALUATION APPROACH & METHODOLOGY The consultant will be responsible for finalizing the methodology for a quasi-experimental evaluation design, consistent with the USDA approved project Evaluation Plan that allows for rigorous impact and attribution analysis and accounts for any confounding factors. The design must also utilize a mixed-method, cross-sectional design that includes quantitative and qualitative (e.g., focus groups, in-depth interviews and observations) methods. The quantitative data collected through representative surveys will provide or confirm baseline values for all required project indicators (see section Required Indicators below). Qualitative methods will involve a broader selection of stakeholders and provide additional information to understand the program’s operating context, triangulate quantitative findings, strengthen credibility and validity and provide a greater level of detail and contextualization of findings. The baseline study must include a treatment group of schools, students, parents and producers participating in Nuestra Cosecha, and two control groups not participating in the project – schools and students that are part of the Food for Education projects, and schools and students not participating in any USDA supported projects. The consultant shall identify a well-defined group of comparison producers and schools matched on key characteristics. Counterfactual data from these schools and groups will measure attribution of program effects through the final evaluation. Key Evaluation Questions: Relevance o Determine potential problems or challenges across project beneficiaries to achieve progress towards results students, o Determine how well project interventions align with stakeholder policies and priorities 4 Performance o Determine baseline values for required project indicators o Assess to what extent the indicator targets are realistic and appropriate o Assess beneficiaries’ knowledge and skills to refine capacity building and training activities o Identify opportunities and threats to project implementation Sustainability o To what extent should local actors (private sector, community entities/associations, local government agencies, etc.) be engaged and committed to ensure sustainability? o What specific institutional capacity needs (e.g., for key GoG, municipal, community and school level stakeholders) should the project address to foster greater engagement and accountability? Effectiveness o Determine to what extent the Results framework is realistic, appropriate and practical to implement, and identify if design, structure, logic or management need modifications to facilitate the achievement of desired outcomes and impact o Generate baseline data for comparative analysis (using quasi-experimental design) o Validate project strategies and assumptions Sampling Strategy It is expected that the consultant will design and verify the sampling strategy in consultation PCI. Sample sizes of students, parents and producers for quantitative indicators should allow the detection of statistically significant changes (at the 95% confidence level, 5% confidence interval) over the life of the project. The consultant should consider using strategies such as cluster sampling to minimize the cost and logistical complexity of the sample. The final sample size will have to consider cost and complexity as well as statistical significance. A list of schools by location with the number of students is included in Annex IV of this document. Required Indicators The consultant will be responsible for collecting and reporting on the following required project indicators as part of the baseline study. The proposed evaluation design should provide data collection and analysis methodology that allows reporting on these indicators at project baseline and endline. Baseline values will be reported using data from the treatment group. However, to demonstrate impact results will be compared between control and treatment groups. A list of required baseline indicators and their data sources is listed in the table below. No. Indicator Source 1 Average USD cost per kilogram of fresh fruits, adjusted for inflation, procured and delivered to project schools Producers 2 Average USD cost per kilogram of fresh vegetables, adjusted for inflation, procured and delivered to project schools Producers 3 Average USD cost per kilogram of eggs, adjusted for inflation, procured and delivered to project schools Producers 4 Average number of days per year to complete entire process of improved procurement (bidding, testing, payments), delivery and distribution for locally procured commodities Producers 5 Percentage of schools which serve school meals with adequate dietary diversity, as a result of USDA assistance School Directors 5 6 Percent of students who report that they like school meals, using the 5- point facial hedonic scale, by sex. Students 7 Value of annual sales of farms and firms receiving USDA assistance (USD) Producers 8 Volume of commodities sold by farms and firms receiving USDA assistance Producers 9 Percentage of schools using fresh fruit and/or vegetables in daily school meals, as a result of USDA assistance School Directors 10 Percent of school-age children receiving a minimum acceptable diet Students 11 Percent of School Parent Organization members who demonstrate sufficient knowledge of School Feeding law and regulations Parents Qualitative Data Collection Requirements: Additional qualitative data will be collected from key respondent groups and stakeholders to triangulate and contextualize the quantitative findings and to help answer the key baseline study questions outlined above. The beneficiary groups who should be interviewed and/or engaged during the baseline study, and proposed data collection methodologies, are listed in Table 4 below. III. KEY TASKS OF CONSULTANCY • Review project documentation and meet with project staff o Understand the project design and contextual framework o Understand full context of the project activities, types of beneficiaries, expected outcomes, and monitoring and evaluation strategies • Design and develop an Inception Report that addresses the baseline questions outlined above and fits a quasi-experimental evaluation design that can be applied to the final evaluation. The Inception Report should include, at minimum: o A mixed-methods study design (framework, methodology, etc.) that follows the guidelines listed below o Sampling strategy o Data collection plan to report on all required indicators and disaggregations o Identify data collection tools available or needed to collect all required data o A Quality Assurance Plan o An analysis plan detailing methodologies for analyzing data, presenting and visualizing results and conducting tests for statistical significance o An updated timeline for consultancy completion • Meet with USDA as needed to review design • Develop baseline data collection tools and protocols in collaboration with PCI staff o Review and refine tools and protocols as appropriate o Use mobile devices to collect data o Hire and train enumerators, team leaders/supervisors o Field test all instruments • Conduct data collection o Manage and secure all data sets o Clean data o Ensure data quality through an established data quality assurance protocol 6 o Deliver all complete, cleaned, translated datasets to PCI (including transcripts from interviews and focus groups) • Conduct baseline data analysis o The consultant will be responsible for consolidating quantitative and qualitative findings from the analyses into a comprehensive final report • Produce the comprehensive baseline study report o Follow PCI report guidelines o Provide PCI and stakeholders appropriate time for review and feedback of draft reports o Incorporate the feedback of PCI and other stakeholders and submit the final report • Provide a summary presentation o Conduct a presentation of key findings for project staff and other stakeholders in Guatemala IV. DELIVERABLES The consultant shall deliver the following products: 1. An evaluation design that includes the sampling strategy and approach to qualitative data collection, data collection methods and tools and a plan for data analysis 2. A work plan and timetable outlining key evaluation activities and products 3. A final report outline approved by PCI 4. A draft final baseline evaluation report 5. Cleaned complete dataset(s) of all quantitative data 6. A data dictionary/code book for the quantitative dataset, using a template provided by PCI 7. Output of statistical analysis of quantitative results in MS Word 8. Complete transcripts of all qualitative data collected 9. Content analysis or other analytical output of the qualitative data, including key themes identified and frequencies of those themes 10. Documentation of the qualitative topics analyzed and frequency analysis. V. RESPONSIBILITIES Consultant Responsibilities • Submit technical and financial proposal with clear statement of the proposed methodologies • Lead consultations with all informants of the evaluation • Hire, train and manage enumerators for quantitative data collection • Administer data collection, data entry and analysis of quantitative surveys • Conduct qualitative data collection with experienced qualitative facilitators • Review of relevant documents and literature • Analyze qualitative and quantitative data • Provide deliverables as per the pre-agreed timeframe of the evaluation • Develop survey tools in Mobenzi and train consultant on usage PCI Responsibilities • Ensure involvement of government and other relevant stakeholders • Facilitate consultations with the relevant stakeholders • Ensure contacts in areas of the study 7 • Review the draft report and provide feedback to the consultant • Approve all final deliverables • Disburse payment of the service as per the contract agreement. • Participate on the training of quantitative data collection to enumerators VI. ESTIMATED TIMELINE The consultancy is expected to begin on the first week of January 2019 and conclude by March 31, 2019, with an estimate of 40 days level of effort (LOE) to execute the responsibilities of the consultancy, distributed in the following manner: Activity Expected Timeframe Expected Days LOE Desk review of project documentation January 2019 3 Create implementation workplan, finalize technical design (sampling strategy, analysis plan, data collection tools), submit inception report January 2019 4 Train data collection team and finalize logistics January 2019 4 Data collection February 2019 15 Data analysis, review and preparation of first report draft February 2019 10 Incorporate PCI feedback and draft final report March 2019 3 Submit all final deliverables March 2019 2 VII. REVIEW COORDINATION Technical management of the evaluation report review process will be coordinated by the PCI Guatemala Nuestra Cosesha team, comprised of the Chief of Party, Project Director and M&E technical officer, in close collaboration with PCI’s headquarters MERL Technical Advisor and implementation partners (Save the Children, CRS). The primary point of contact will be the Nuestra Cosecha M&E technical officer. VIII. QUALIFICATIONS The applicant or team should meet the following minimum requirements: • Academic background in social / health sciences or related field; minimum of Master’s Degree • Extensive experience in leading project evaluations using a range of quantitative and qualitative data collection, analysis and sampling methods for quasi-experimental designs • Experience in leading evaluations in the areas of international development/public health; experience in evaluating school feeding programs strongly preferred • Experience in the evaluation of projects funded by U.S. government/USDA strongly preferred • Knowledge of Guatemalan national legislation relating to school feeding • Demonstrable capacity to deliver high quality outputs within the proposed timeframe • Verbal and written proficiency in English preferred • Professional experience leading evaluations in the project areas 8 IX. PREPARATION OF PROPOSAL Applicants are requested to submit a proposal for these services by submitting separate Technical and Financial Proposals as detailed below. The standard forms in this proposal document may be retyped for completion but the applicant is responsible for their accurate reproduction. Required formats are hereby attached: • Appendix I (Technical Proposal) • Appendix II (Financial Proposal) 1. Preparation of Technical Proposal The Technical Proposal shall be submitted in the format provided in Appendix I and describe the approach and plans for accomplishing the work outlined in this TdR. The Technical Proposal should be limited to ten (10) pages and must contain the following: A. Technical Proposal Cover Sheet (does not count towards the 10-page limit) B. A minimum of two references from organizations with which they have previously conducted evaluation consultancy work or a list of three references with contact information (does not count towards the 5-page limit) C. A description of the applicant’s direct prior relevant experience including summary of evaluations previously conducted D. Description of Methodology E. Explain how the evaluation design addresses the research questions • Sampling strategy • Approach to primary data collection • Plan for data analysis F. Timeline for Activity G. Include a timeline of activities and level of effort required for each activity H. Team Composition and Roles, including: • Key Team Member List and Core Qualifications • CVs of Key Team Members (does not count toward the 10-page limit) I. One example of evaluation reports that have been produced by the applicant (does not count towards the 5-page limit) 2. Preparation of Financial Proposal The Financial Proposal shall be submitted in the format provided in Appendix II and establish a breakdown of consultancy price according to the table provided. Applicants are solely responsible for their own costs in preparing the applications. X. ADDITIONAL INFORMATION If the applicant is an agency, include certified copies of: • Organizational profile • Relevant registration certificates Applicants may provide any other information which may be relevant to this proposal. 9 XI. MODIFICATION OF TERMS PCI reserves the right to add, modify or omit certain portions of the RFP at any time at its sole discretion. This includes the right to cancel at any time prior to entering into a contract with the successful applicant. XII. APPLICATION PROCESS 1. Interested candidates should Request a copy of the RFP by writing an email to the PCI Director of Operations, Hugo Diaz (HDiaz@pciglobal.org), indicating in the title “Soliting RFP for Nuestra Cosecha Baseline Evaluation Consultant). Due date - XXX 2. For any question regarding this Request for Letters of Interest and Proposals, contact: Jason Rubin (jrubin@pciglobal.org) and Yeison Cobón (ycobon@pciglobal.org). Due date – XXX 3. Technical and financial proposals should be sent to Pascale Wagner (pwagner@pciglobal.org) and Hugo Diaz (hdiaz@pciglobal.org) with the subject line “Proposal Submission for Nuestra Cosecha Baseline” by XXX XIII. PROPOSAL CONDITIONS 1. Applicants must expressly respond to specifications as stipulated in the Request for Proposals and failure to do so may result in disqualification of proposals. 2. Applicants must clearly indicate: A. Consultancy Period (include details of availability) B. Validity Period of Proposal 3. Quoted prices shall remain valid for a period of not less than one month after the deadline for submission. 4. The rates quoted by the Applicant shall be fixed for the duration of the contract and shall not be subject to adjustment on any account except through mutual agreement and amended in writing. 5. Prices should be quoted in US Dollars. 6. PCI will not return any documents submitted. 7. PCI may contact candidates who are short-listed for interviews. 8. The successful applicant will be notified for the award of contract by PCI prior to expiration of the quotation validity period. 9. Notwithstanding the above, PCI reserves the right to accept any proposals and reject any or all proposals at any time prior to the award of contract. XIV. AWARD CRITERIA & EVALUATION OF BIDS 1. Eligibility Criteria: Applicants are required to meet the following criteria to be eligible to participate in this consultancy: 10 A. Possess the ability to perform successfully under the terms and conditions of the proposed procurement; B. Be legally authorized to carry out such services in Guatemala; C. Not be insolvent, in receivership, bankrupt or being wound up, not have had business activities suspended and not be the subject of legal proceedings for any of the foregoing; D. Have fulfilled all obligations to pay taxes according to the tax laws of Guatemala; E. Are not suspended or excluded from participation in any procurement exercise be in public or private sector in Guatemala; F. Are not suspended, debarred, or excluded from receiving US government funds under US Government regulations; G. Have not been convicted, or any directors or officers been convicted, of any criminal offence relating to obtaining or attempting to obtain a contract or sub-contract; H. Not have any current or past corporate, personal, or other interests that may, in PCI’s opinion, give rise to a conflict of interest. 2. Qualification Criteria: Eligible proposals will be initially evaluated by reference to the following qualification criteria: A. Completeness of documentation as specified; B. Stated ability of the applicant to meet all the minimum requirements specified in this document. 3. Award Criteria: The contract will be awarded from eligible and qualifying bids on the basis of the highest quality and best value to PCI, based on the following criteria: A. Technical knowledge in the area of the required focus; B. Proof of relevant experience and record of past performance; C. Financial and technical resources or accessibility to other necessary resources to carry out the Required Services; D. Qualifications and skills composition of the Team who will undertake the exercise; (including qualifications under section VIII) E. Cost of undertaking exercise. NOTE: It is understood that PCI is not bound to necessarily accept the lowest price, or any bid offered. XV. CONFLICT OF INTEREST PCI requires that Services Providers provide professional, objective, and impartial advice and at all times hold PCI’s interests paramount, strictly avoid conflicts with other assignments or their own corporate interests and act without any consideration for future work. Without limitation on the generality of the foregoing, the consultant and any of their affiliates, shall be considered to have a conflict of interest and shall not be recruited, under any of the circumstances set forth below: 1. A Service Provider (including its Personnel) that has a business or family relationship with a member of staff of PCI who is directly or indirectly involved in any part of (i) the preparation of the Terms of Reference of the assignment, (ii) the selection process for such assignment, or (iii) supervision of the Contract, may not be awarded a Contract, unless the conflict stemming from this relationship has 11 been resolved in a manner acceptable to PCI throughout the procurement process and the execution of the Contract. 2. Service Provider has an obligation to disclose any situation of actual or potential conflict that impacts their capacity to serve the best interest of PCI, or that may reasonably be perceived as having this effect. Failure to disclose said situations may lead to the disqualification of the Service Provider or the termination of its Contract. 3. If a short-listed applicant could derive a competitive advantage from having provided services related to the assignment in question, PCI shall make available to all short-listed applicants together with this application all information that would in that respect give such applicant any competitive advantage over competing applicants. XVI. INTELLECTUAL PROPERTY: PCI shall own all Work Products. All Work Products shall be considered work made for hire by Consultant and owned by PCI. 12 Appendix I: Technical Proposal Note to Applicant: Complete this form with all the requested details and submit it as the first page of your technical proposal, with required documents attached. Ensure that your proposal is authorised in the signature block below. A signature and authorisation on this form will confirm that the terms and conditions of this Request for Proposal prevail over any attachments. If your proposal is not authorised, it may be rejected. A. Technical Proposal Cover Sheet: Subject of RFP: Request for Proposals for LRP Baseline Study Consultant Services Name of Applicant: Postal & Physical Address: Telephone Number: Fax Number: Website (if applicable): Name, Title, Mobile Number and Email Address for Contact Person #1: Name, Title, Mobile Number and Email Address for Contact Person #2 (if applicable): Date of Technical Proposal: The Service Delivery Period of our proposal: Begins on/around this date: ___________________ And ends on/around this date: _________________ Other Period(s) of Availability for our Services: The Validity Period of our Application is: ________ days/weeks/months (circle one) from the time and date of the submission deadline. 13 B. Applicant’s Certification: We offer to provide the services described as per the RFP in accordance with the terms and conditions stated in your RFP Terms of Reference above. We confirm that we are eligible to participate in public as well as private procurement and meet the eligibility criteria specified in “Eligibility Criteria” in your Request for Proposal referenced above. In addition to the Technical Proposal Cover Sheet included as Section A of this Technical Proposal, we enclose the following in response to instructions in “Preparation of Proposals” in your RFP Terms of Reference above: ____ References ____ Description of Applicant’s Prior Relevant Experience ____ Description of Methodology ____ Timeline for Activity ____ Team Composition and Roles ____ Examples of Previous Evaluation Reports Produced by Applicant We also enclose a Financial Proposal in accordance with instructions outlined in your RFP TOR above. C. Applicant’s Authorization of Technical Proposal: Signature: Name: Position: Date: Authorised for and on behalf of (if applicable): (DD/MM/YY) Company (if applicable): Official Stamp (if applicable): 14 Appendix II: Financial Proposal Note to Applicant: Complete this form with details of all your costs and submit it as part of your Financial Proposal. Prices should be indicated in US Dollars and be exclusive of VAT. Where your costs are in another currency, explain reasons for use of another currency and submit a separate form for each currency. Authorise the rates quoted in the signature block below. Where this is a unit price contract, the breakdown will be used as the cost estimates and payment will be made for the services actually performed and cost actually incurred. A. Financial Proposal Cover Sheet Subject of RFP: Request for Proposals for LRP Baseline Study Consultant Services Name of Applicant: Postal & Physical Address: Telephone Number: Fax Number: Website (if applicable): Name, Title, Mobile Number and Email Address for Contact Person #1: Name, Title, Mobile Number and Email Address for Contact Person #2 (if applicable): Date of Financial Proposal: Applicant’s Payment Terms: The Validity Period of our Application is: ________ days/weeks/months (circle one) from the time and date of the submission deadline. B. Breakdown of Contract Price Instructions: The cost of the applicant’s services should include the applicant’s daily rate x number of days, per diem for field work (hotel, meals and incidental expenses) x number of days, hired enumerators’ daily rate x number of days, local and international transportation (if applicable) and other proposed costs which shall be incurred to carry out the required services. All costs shall be itemized including unit costs and number of units required. Currency of fees shall be clearly indicated. Format of this section shall be determined by applicant and shall be clear and logical. VAT, if applicable, shall be indicated on a separate line. 15 (ATTACH BREAKDOWN OF CONTRACT PRICE TO THIS SECTION) Total Price of Contract: _________________ Currency of Fees: ___________________ C. Bidder’s Certification: We offer to provide the required services in accordance with the terms and conditions stated in your RFP TOR above and the rates stated herein. We confirm that we are eligible to participate in public as well as private procurement and meet the eligibility criteria specified in “Eligibility Criteria” in your RFP TOR referenced above. We also enclose a Technical Proposal in accordance with instructions outlined in your RFP TOR above. We confirm that the rates quoted in this proposal are fixed and firm for the duration of the validity period and will not be subject to revision or variation. Other terms and conditions of the proposal and/or supporting documentation may be attached to the bid. D. Bidder’s Authorization of Financial Proposal: Signature: Name: Position: Date: Authorised for and on behalf of (if applicable): (DD/MM/YY) Company (if applicable): Official Stamp (if applicable): 16 Appendix III: Results Framework 17 Annex IV: List of Schools 1 PCI Huehuetenango Santa Eulalia Aldea temux Grande E.O.R.M. Aldea Temux Grande 179 2 PCI Huehuetenango Santa Eulalia Alde Molna E.O.R.M. Alde Molna 169 3 PCI Huehuetenango Santa Eulalia Cantón Belén E.O.R.M. Cantón Belén 275 4 PCI Huehuetenango Santa Eulalia Aldea Ixtenam E.O.R.M. Aldea Ixtenam 352 5 PCI Huehuetenango Santa Eulalia Santa Eulalia E.O.U.M. JV Santa Eulalia 477 6 PCI Huehuetenango Santa Eulalia Aldea Temux chiquito yulais, EORM Aldea Temux chiquito yulais, 184 7 PCI Huehuetenango Santa Eulalia Aldea Pett E.O.R.M Aldea Pett 495 8 PCI Huehuetenango Barillas Aldea Palestina La Unión E.O.R.M. Aldea Palestina La Unión 270 9 PCI Huehuetenango Barillas Aldea Nuca E.O.R.M. Aldea Nuca 473 10 PCI Huehuetenango Barillas Aldea El Quetzal E.O.R.M. Aldea El Quetzal 309 11 PCI Huehuetenango Barillas Yulconop E.O.R.M. Yulconop 183 12 PCI Huehuetenango Barillas Caserío Nueva Providencia E.O.R.M. Caserío Nueva Providencia 170 13 PCI Huehuetenango Barillas La Providencia E.O.R.M. La Providencia 120 14 PCI Huehuetenango Barillas La Florida E.O.R.M. La Florida 120 15 PCI Huehuetenango Barillas Buena Vista Jolomtaj E.O.R.M. Buena Vista Jolomtaj 159 16 CRS Totonicapan Momostenango Paraje Junajpu E.O.R.M. Paraje Junajpu 346 17 CRS Totonicapan Momostenango Paraje Chirijquemeya E.O.R.M. Paraje Chirijquemeya 150 18 CRS Totonicapan Momostenango Paraje La Cumbre E.O.R.M. Paraje La Cumbreoc 246 19 CRS Totonicapan Momostenango Caserio Chobaquit E.O.R.M. Caserio Chobaquit 194 20 CRS Totonicapan Momostenango Paraje Los Itzep E.O.R.M. Paraje Los Itzep 180 21 CRS Totonicapan MomostenangoAldea Xequemeya E.O.R.M. Aldea Xequemeya 107 22 CRS Totonicapan MomostenangoMomostenango E.O.R.M. Barrio Santa Isabel 221 23 CRS Totonicapan Momostenango Caserio San Jose Siguila E.O.R.M. Caserio San Jose Siguila 128 24 CRS Totonicapan Momostenango Paraje Chisiguan E.O.R.M. Paraje Chisiguan 248 25 CRS Totonicapan Momostenango Paraje Kanil E.O.R.M. Paraje Kanil 155 26 CRS Totonicapan Momostenango Paraje Pajumet E.O.R.M. Paraje Pajumet 121 27 CRS Totonicapan Momostenango Paraje El Mirador E.O.R.M. Paraje El Mirador 116 28 CRS Totonicapan MomostenangoAldea Xolajap E.O.R.M. Aldea Xolajap 164 29 CRS Totonicapan Momostenango Caserio de Pueblo Viejo E.O.R.M. Caserio de Pueblo Viejo 147 30 CRS Totonicapan Momostenango Caserio Canquixaja E.O.R.M. Caserio Canquixaja 155 31 CRS Totonicapan Momostenango Paraje Bella Vista E.O.R.M. Paraje Bella Vista 86 32 CRS Totonicapan Momostenango Paraje Chuiquinom E.O.R.M. Barrio Santa Ana 100 33 CRS Totonicapan Momostenango Paraje Chipuerta E.O.R.M. Paraje Chipuerta 100 34 CRS Totonicapan Momostenango Paraje Paxaq E.O.R.M. Paraje Paxaq 130 35 CRS Totonicapan Momostenango Caseria Jutacaj E.O.R.M. Caseria Jutacaj 223 36 SCI Quiche Nebaj Xepiun E.O.R.M. Aldea Xepium 161 37 SCI Quiche Nebaj Xemamatze E.O.R.M. Xemamatze 357 38 SCI Quiche Nebaj Acul EORM Aldea Acul JM 290 39 SCI Quiche Nebaj Xexuxcap EORM Aldea Xexuxcap 181 40 SCI Quiche Nebaj Salquil Grande EORM Aldea Salquil Grande JM 334 41 SCI Quiche Nebaj Río Azul EORM Aldea Río Azul 323 42 SCI Quiche Nebaj Pulay Nebaj EORM Aldea Pulay Nebaj 289 43 SCI Quiche Nebaj Caserío Xecax EORM Xecax 156 44 SCI Quiche Nebaj Barrio Simocol EORM Simocol 277 45 SCI Quiche Nebaj Vipecbalam EORM Vipecbalam 162 46 SCI Quiche Nebaj Actxumbal EORM Actxumbal JM y JV (*) 281/224 47 SCI Quiche Nebaj Cambalam EORM Cambalam 122 48 SCI Quiche Chajul Xolcuay JV EORM Aldea Xolcuay JV 211 49 SCI Quiche Chajul Choncholá EORM Aldea Choncholá 181 50 SCI Quiche Chajul Choncholá EORM Santa Clara 204 # de Estudiantes Organization (PCI, Save, CRS) Departmento Municipio Comunidad Nombre de Escuela