Mid-Term Evaluation Project CREAN – World Vision 1 Report Mid-Term Evaluation Project CREAN – World Vision December 2016 Authors Claudia Pompa Denis Pavón Travis Burke Agreement Number: FFE-524-2014/045-00 Project Duration: 2014 - 2019 Mid-Term Evaluation Project CREAN – World Vision 2 Index INDEX 2 ABBREVIATIONS 4 ACKNOWLEDGEMENTS 5 EXECUTIVE SUMMARY 7 INTRODUCTION 12 COUNTRY CONTEXT 13 EDUCATION IN NICARAGUA 13 CHARACTERISTICS OF THE EDUCATION SYSTEM 14 CHILDREN’S NUTRITION IN NICARAGUA 14 THE COMPREHENSIVE SCHOOL NUTRITION PROGRAM 15 CREAN NICARAGUA 17 BACKGROUND 17 PROJECT OBJECTIVES 17 INTERVENTION AREA 17 BENEFICIARIES 18 PROJECT STRUCTURE AND ACTIVITIES 18 DAILY SCHOOL MEAL 18 EDUCATION ACTIVITIES 20 NUTRITION AND HEALTH ACTIVITIES 21 PROJECT MANAGEMENT 22 PROJECT PARTNERS 23 METHODOLOGY 23 EVALUATION OBJECTIVES 23 EVALUATION APPROACH 24 SAMPLING PLAN AND SITE SELECTION 24 EVALUATION METHODOLOGY 25 DATA ANALYSIS 29 FINDINGS 29 1.PROJECT RELEVANCE 29 STRATEGIC FIT AND RATIONALE 30 PROGRESS OF PROJECT IMPLEMENTATION 30 EXTERNAL AND INTERNAL FACTORS AFFECTING THE IMPLEMENTATION OF THE PROJECT 30 RELATIONSHIP WITH PROJECT PARTNERS 32 2.PROJECT EFFECTIVENESS AND EFFICIENCY 32 PROJECT DESIGN 32 Mid-Term Evaluation Project CREAN – World Vision 3 PROJECT MANAGEMENT 33 MONITORING AND EVALUATION 36 IMPLEMENTATION OF SEVERAL COMPONENTS OF THE PROJECT 37 SCHOOL MEAL 37 ACTIVITIES OF THE EDUCATION COMPONENT 41 ACTIVITIES OF THE HEALTH AND NUTRITION COMPONENT 44 3.SUSTAINABILITY AND EVIDENCE OF CHANGE 45 SUSTAINABILITY 45 INTERVENTION AREA 46 EVIDENCE OF CHANGE 47 SCHOOL MEAL 47 EDUCATION 47 HEALTH AND NUTRITION 48 RECOMMENDATIONS 49 REFERENCES 55 ANNEX 58 TABLE 3: KEY INFORMANT INTERVIEWS 27 TABLE 4: FOCUS GROUP DISCUSSIONS - STUDENTS 27 TABLE 5: FOCUS GROUP DISCUSSIONS - PARENTS 29 TABLE 6: FOCUS GROUP DISCUSSIONS – TEACHERS 29 Mid-Term Evaluation Project CREAN – World Vision 4 Abbreviations CAE Comité de Alimentación Escolar (School Feeding Committee) CREAN Children Reading and Nourished FGD Focus Group Discussion KII Key Informant Interview MTE Mid-Term Evaluation MINED Ministry of Education MINSA Ministry of Health M&E Monitoring and Evaluation PINE Programa Integral de Nutrición Escolar (School Nutrition Program) SAPR Semi-Annual Performance Report UFE Utilization-Focused Evaluation UNICEF United Nations Children’s Fund USDA United States Department of Agriculture WASH Water, sanitation and hygiene WV World Vision Mid-Term Evaluation Project CREAN – World Vision 5 Acknowledgements The consultant team acknowledges and appreciates the valuable contribution of each of the participants in this evaluation – students, parents, teachers, school directors, and municipal and departmental delegates from the Ministry of Education and the Ministry of Health – who dedicated part of their time to share their opinions and experiences regarding this project. We also wish to express our appreciation to the technical, administrative, and operational team of World Vision Nicaragua and World Vision USA who shared their perspectives and experiences of the project. Without their contributions and their time this evaluation would not have been possible. Composition of the evaluation team Claudia Pompa, Team Leader Travis Burke, Consultant Denis Pavon, Consultant Melvin Chavarria, Consultant Mid-Term Evaluation Project CREAN – World Vision 6 Notification The opinions expressed in this report belong strictly to the Evaluation Team and do not necessarily reflect the opinion of World Vision. Responsibility for the opinions expressed in this report rests solely with the authors. The publication of this document does not imply that World Vision supports the expressed opinions. This publication was produced at the request of the United States Department of Agriculture. It was prepared by an independent third-party evaluation firm. The author’s views expressed in this publication do not necessarily reflect the views of the United States Department of Agriculture or the United States Government. Accessibility Note: An accessible version of this document can be made available by contacting fas.monitoring.evaluation@usda.gov Mid-Term Evaluation Project CREAN – World Vision 7 Executive summary Background The Children Reading and Nourished (CREAN) project, funded by the United States Department of Agriculture (USDA), aims to improve educational, health and nutrition practices in primary school children in the most vulnerable areas of Estelí and León in Nicaragua. CREAN is set to run from 2015 to 2017 and seeks to strengthen the Ministry of Education of Nicaragua (MINED) Comprehensive School Nutrition Program (Programa Integral de Nutrición Escolar – PINE), which has been in place since 2007, by integrating elements of education, nutrition, hygiene, and health within it. The project has two main components: an Education component, for which the main objective is to improve the literacy of school-age children through increased student retention and improvement of literacy instruction; and a Health and Nutrition component, for which the main objective is to spread better health and dietary practices. The school meal, although falling under the education component, will be treated in this report as an independent component given the complexity of its implementation and scope. Mid-Term Evaluation The purpose of this Mid-Term Evaluation (MTE) is to understand the effectiveness of the project and the extent to which it has contributed to advancing its stated objectives. It also aims to identify the lessons learned to date, areas for improvement in the second phase of the project, and challenges constraining progress. We make recommendations to help the project to achieve expected outcomes and objectives within the project timeframe. In October 2016, our team of four researchers visited the site of the CREAN project to conduct the MTE. The evaluation used a context-sensitive and qualitative method approach. We drew on an appropriate mix of qualitative methods to understand what works best and to identify factors that appear to enhance or undermine the quality, acceptability and usefulness of implementation and outputs. 17 focus group discussions were held with students and parents, representing the main beneficiaries. 35 key stakeholder interviews were conducted with relevant stakeholders at the local, municipal and state levels, as well as project staff. A total of eight schools were visited in León and Estelí during a one-week period. Approximately 160 people participated. Results Our evaluation finds that the project CREAN is progressing well according to the current plan despite the delays and changes caused by the extended period of review of the projecy by MINED. The findings indicate that there are certain aspects of the project implementation that are well defined (for example, alignment with national plans regarding both literacy and nutrition) and some aspects that are progressing satisfactorily (for example the school meal, travel backpacks and health and hygiene clubs). There are few immediate concerns associated with the project, and some of the issues that do arise are probably beyond the scope of the Mid-Term Evaluation Project CREAN – World Vision 8 project and clearly in the domain of MINED. However, there were a number of points raised by stakeholders and staff that could strengthen the project. Some of these issues have been addressed recently or are in the process of being addressed. Project relevance and implementation progress In general, all stakeholders and project beneficiaries consider CREAN a successful project, integral to the needs of the beneficiaries, and highly aligned with government policies. During this evaluation, it was observed that the project has made significant progress in implementation, despite suffering considerable delays. These delays were mainly due to external factors related to the project partner (a request for change of partner and delay in formalizing the agreement) which affected the schedule of activities. The modification of the project calendar meant that instead of implementing a three-year project, the project team has attempted to do two years’ worth of work in one year. This change has decreased the team’s ability to evaluate progress, adapt as needed, and institutionalize their work. To improve the implementation of the project, we recommend: • Evaluating certain functions and redesigning job descriptions if needed. This practice will help to redistribute tasks and responsibilities, reduce stress, improve productivity and job satisfaction. • Recruiting technical staff with specific knowledge and technical skills for the construction or rehabilitation of water, sanitation and hygiene (WASH) infrastructure and for establishing school gardens so that progress can be made in the remainder of the implementation period. • Creating an executive council as a mechanism to improve the internal communication of the project team. The aim is to discuss priorities, share agendas, plan intervention strategies and ensure that each area has the information it needs. • Establishing effective communication channels within the project team to ensure that information is properly distributed, and that teams are informed of changes and decisions affecting their work areas. • Developing permanent organizational stress-management strategies to ensure good team performance and synergy. These initiatives also contribute to the successful achievement of objectives, as well as helping to motivate and retain staff. Effectiveness and efficiency of the project Both the effectiveness and efficiency of the project have been remarkable. CREAN is an extremely ambitious project relative to its three-year life cycle. Likewise, the project has a large number of interrelated activities and sub-activities that reflect its holistic character but also present a series of difficulties during implementation, as will be explained in the Findings sections. These activities make major demands on both time and personnel as well as requiring a meticulous process of support and monitoring. Mid-Term Evaluation Project CREAN – World Vision 9 The challenges in implementing the project are seen not only in terms of the number of activities, but also in the excessive amount of monitoring indicators to report. This has generated a disproportionate level of work and effort on the part of the project team to achieve the objectives. Projects of this nature inevitably lead to questions over the number of activities with respect to their impact. Although it is still too early to assess the impact of each of these activities and their interrelation with each other, it is important that the project team learn from the process to identify which activities have generated the greatest impact and which are the activities that, although important, could be implemented either in another phase of the project or by other coordinated projects. To improve the effectiveness and efficiency of the project, we recommend: • Reducing the number of indicators and prioritizing those that are appropriate, useful and relevant to the project objectives. This change will help improve the efficiency of monitoring and evaluation (M&E) data collection, and the performance of the project team. • Optimizing data collection tools (e.g. M&E sheets) to simplify the process for both field monitors and teachers. This change would generate significant savings of time and effort. • Digitizing the data collection process to speed up the process, reduce the margin of error and improve efficiency (e.g. use of tablets or smartphones). Sustainability Although the project is on the path to accomplishing its goals, it is unlikely to achieve sustainability, mainly due to its short life cycle. It is important for the project team to consider which of the activities implemented by CREAN have the capacity to be systematized and institutionalized in the time remaining for implementation and to concentrate their efforts on achieving them. Furthermore, the project must evaluate how to structure its resources, both human and financial, to work more closely with key actors to deliver stronger and more established capacity after completion. Through these steps, the project can leave a legacy of increased capacity and institutional methodologies that may be continued or replicated once the project has come to an end. Evidence of change Despite the short implementation time, evidence of substantial changes has been recorded. School meal Overall, school meals have been identified as a key factor in improving school retention. Teachers and municipal and departmental authorities have stated that they have seen improvements in school retention and there are several cases of children with 100% school attendance, which they say is linked to the provision of school meals. However, the food options have caused problems in the implementation of this component. The type of corn meal used stands out as a product that neither children nor parents like to prepare Mid-Term Evaluation Project CREAN – World Vision 10 or eat. This aversion has caused wastage and frustration, and there is a need to review the appropriateness of the products to the local food culture. Most schools lack the infrastructure to prepare meals on site, especially in rural areas, and this has led to problems with the preparation of the meal. Parents are responsible for meal preparation, and while School Feeding Committees (Comité de Alimentación Escolar – CAEs) have been established, not all parents have the time or resources to invest in preparation every day. Some parents have collaborated to design a schedule for preparation or to hire someone who is responsible for cooking and distribution of the meal. To improve the implementation of the school meal, we recommend: • Considering alternative products to corn flour due to the high rate of rejection of this item. Products that are more adjusted to Nicaraguan food culture should be identified. • Creating strategies that promote and encourage the on-going participation of parents in the process of preparation of the school meal. • Providing certified and periodic training on food management and preparation, which would strengthen the parents’ links to the project activities and their relationship with the school. • Strengthening the role of CAEs and generating ideas and strategies to increase the engagement of parents in the preparation of school snacks. Education Most aspects of the educational component have been successful. Teachers and principals report that establishing reading corners has helped to motivate children to read, while the educational materials and textbooks are aligned with the MINED curriculum. One drawback has been a lack of space for books, which might be overcome with the provision of bookshelves. Traveling backpacks are a major success: nearly every school visited requested more backpacks, and all students expressed their interest in them. This intervention was simple, easy to implement, and created benefits outside of the school when children read with parents, siblings, and relatives. The only related concern is the low level of book purchasing and distribution and how this might effect other activities of the component. Teacher training has been successful but needs to be better incorporated into a broader context of professional development rather than being a series of one-off activities. The strategies implemented by the Education component have also been received positively by all the schools visited and by the municipal and departmental authorities responsible for supervising the educational centers. Teachers and principals participating in this evaluation have noted the benefits and progress made through the activities implemented. To improve the implementation of the Education component, we recommend: • Advising schools not to restrict the use of reading materials outside schools, and ensuring they are decodable, leveled and age-appropriate for students and that all materials are easily accessible for both students and teachers. Mid-Term Evaluation Project CREAN – World Vision 11 • Laminating reading materials and providing bookshelves. These practices will help keep books in good condition and facilitate access. • Increasing the number of traveling backpacks as that will increase the number of students benefiting. • Laying the foundations for more structured teacher training processes, which would allow for a higher level of specialization while being more attractive to teachers. Health and nutrition The results of the Nutrition, Hygiene, and Health component were mixed: some aspects and activities were found to be successful (health and hygiene clubs) while others might not reach the stated objectives (construction and rehabilitation of water and sanitation infrastructure). Health and hygiene clubs have been successful both in promoting healthy habits and in encouraging students to spread those habits across grades and at home. Students are more knowledgeable about hand washing, brushing teeth, cleaning dishes and utensils, and other healthy habits. Difficulties arise when those schools do not have access to water to help the students practice these habits, although addressing this issue is well beyond the scope of the project. The construction and rehabilitation of water and sanitation infrastructure may not meet its objective. A lack of technical expertise made it a lower priority, and its late start may mean that this aspect of the project has encountered difficulties in its implementation. School gardens are also unlikely to succeed with the current level of investment in the activity. Nearly all schools visited lacked a successful school garden due to a number of factors – land and water issues, lack of technical knowledge, and lack of time for work from parents – and the project has not prioritized this aspect. School gardens are meant to both supplement the school meal and help students learn agricultural techniques, but without the expertise and capacity to implement this activity, the objective is unlikely to be achieved. To improve the implementation of the Health and Nutrition component, we recommend: • Systematizing and institutionalizing the establishment and maintenance of health clubs. • Analyzing the feasibility of implementing school gardens and identifying schools in which the experience of establishing a garden has been successful in order to identify best practices and lessons learned. • Prioritizing the construction/rehabilitation of WASH infrastructure so that the project can achieve its objectives. Mid-Term Evaluation Project CREAN – World Vision 12 Introduction In recent years, Nicaragua has moved from the instability of the past into an era of sustainable development, which has enabled it to maintain an above-average economic growth rate for a Central American country. Growth and increased macroeconomic stability have afforded Nicaragua to focus on long-term poverty reduction strategies and programs, especially in remote and rural districts. While Nicaragua’s poverty rate has declined, the country remains one of the least developed in Latin America. Rural areas often demonstrate an acute lack of educational facilities, adequate medical care, functioning infrastructure and robust local markets. Joining forces to combat illiteracy and child malnutrition, World Vision (WV) partnered with the Ministry of Education of Nicaragua (MINED) for the implementation of the project Children Reading and Nourished (CREAN). This initiative of the United States Government is funded through the US Department of Agriculture (USDA). While Nicaragua reports literacy rates of 78%1 and net primary school enrollment rates of 91.2%, 2 these numbers decline dramatically in rural areas, where there is a net enrollment rate of 78.9%, 3 and where 63% of the population live at or below national poverty lines.4 CREAN works within these communities to ensure that children receive the benefits of better education and nutrition, which are fundamental to decreasing poverty levels at the national level. Over three years, the project aims to strengthen the educational resources and capacity of rural schools while supporting healthy hygiene habits and the nutrition of rural children. The project seeks to directly benefit 38,000 children from 637 schools in the municipalities of Achuapa, Sauce, Santa Rosa del Peñón, El Jícaral, San Nicólas, Limay, La Trinidad and Estelí in the departments of León and Estelí in Nicaragua. 5 WV, currently midway through project implementation, seeks to better understand the effectiveness of CREAN, evaluate its progress in achieving stated objectives, and identify areas for improvement. 1 Education Policy and Data Center (2014), Nicaragua. National Education Profile. Washington DC: EPDC. 2 Castillo, M., Elvir, A, and Vijil, J. (2016), Prioridades de la Educación Nicaragüense para el Siglo XXI. Managua: Centro de Investigación y Acción Educativa Social. 3 UNESCO (2007), Urban and rural disparities in Latin America: their implications for education access. Buenos Aires: UNESCO 4 UNICEF (2013), Infancia, Seguridad Alimentaria y Nutrición: Modelo Integrado TUKTAN YAMNI – MUIH BIN MUIHNI YAMNI Accessed November 2016 http://mptf.undp.org/document/download/12165 5 Equipo DME Proyecto CREAN (2015), Estrategia de Seguimiento y Monitoreo, Vision Mundial. Mid-Term Evaluation Project CREAN – World Vision 13 Country context Nicaragua is the second poorest country in the Americas after Haiti. While poverty in the country has declined considerably over the past two decades – from 50.3% in 1993 to 42.7% in 2012 – it remains principally a rural problem.6 More than 43% of the population in Nicaragua lives in rural areas, and two out of three of these people (68% of the rural population) survive on less than US$1 a day. The country’s rural population is characterized by limited access to basic services – such as healthcare and education – land rights, and markets. Unpredictable environmental conditions and poor economic growth at the rural level further complicate the situation of these impoverished and struggling families.7 The exclusion and poverty in which many rural families live is exacerbated for those residing in the Pacific and north central regions, which together form part of the corredor seco or dry corridor. The region is characterized by a more rural population, higher rates of dependency on agricultural and livestock production, limited development of road infrastructure, and a general lack of public services.8 The agricultural sector has been most negatively affected by prolonged droughts, and small-scale producers, who engage in subsistence farming, have borne the brunt of these adverse effects. The lack of access to safe drinking water and increase in malnutrition has had direct consequences for child welfare in the region.9 As well as momentous political and socio-economic changes, Nicaragua has also seen significant demographic shifts in recent decades. Approximately, 62% of the population is economically active, and 34.6% of Nicaraguans are under 14 years of age. 10 The high percentage of the population that is school-age – around two million children – highlights the benefits the country could reap by giving citizens access to an improved education system, the main pillar of this WV project. Education in Nicaragua The education system in Nicaragua has achieved significant results in the past decades. The net primary school enrollment rate is 91.2%,11 and in 2009, MINED declared Nicaragua a “territory free of illiteracy” with only a 4.1% illiteracy rate. 12 However, challenges still exist in terms of early dropout rates, absenteeism, and grade repetition. The United Nations Children’s Fund (UNICEF) estimates that approximately 500,000 children between the ages of three and 17 are not in 6 Pan American Health Organization (2015), Nicaragua. Perfil de país. Managua: PAHO. 7 IFAD (2014), Rural Poverty Portal. Accessed November 2016 www.ruralpovertyportal.org/ar/country/home/tags/nicaragua. 8 Pan American Health Organization (2015), Nicaragua. Perfil de país. Managua: PAHO. 9 Molina (2013), Las Condiciones de sequia y estrategias de gestion en Nicaragua. Accessed November 2016 http://www.droughtmanagement.info/literature/UNW-DPC_NDMP_Country_Report_Nicaragua_2013.pdf 10 INIDE, (2011). Anuario Estadístico. Accessed November 2016 http://www.inide.gob.ni/bibliovirtual/Anuarios/ANUARIO11/ANUARIO11/assets/downloads/ANUARIO%20ESTADISTICO%2020 11.pdf 11 Castillo, M., Elvir, A, and Vijil, J. (2016), Prioridades de la Educación Nicaragüense para el Siglo XXI. Managua: Centro de Investigación y Acción Educativa Social. 12 PNUD Nicaragua (2011), Informa Nacional sobre Desarrollo Humano 2011. Las juventudes construyendo Nicaragua. Managua: PNUD. Mid-Term Evaluation Project CREAN – World Vision 14 school.13 Recent studies show that only six out of ten students entering first grade complete primary school, and only half of those who enter the first year of high school will finish it. 14 The data show dramatic setbacks when statistics are disaggregated across rural areas: 28% of children in such regions are not in primary school, and this number jumps to 60% when focusing on secondary education. 15 Characteristics of the education system The education system is structured in three discrete cycles: preschool (three years), primary (six years), secondary (five years). Regular primary education: this modality lasts six years and has normal teaching hours. It is aimed at children between the ages of six and 11, and functions in both urban and rural areas.16 Multi-grade primary education: this modality completes the objectives of general basic education. In multi-grade primary, one teacher simultaneously teaches two or more grades, supported by direct and indirect teaching techniques, participatory active methodologies, and cooperative work. It is aimed at children between the ages of six and 11, but in contrast to the first category it is taught only in rural areas. Accelerated primary education: this modality is aimed at children between the ages of nine to 15, and is taught in urban and rural areas. The objective of this category is to include children and adolescents over eight years old and less than 16, who due to different circumstances have been unable to enter school or have dropped out. 17 Children’s nutrition in Nicaragua The country ranks 52nd worldwide among nations whose children under five are suffering from malnutrition. 18 According to the World Health Organization, chronic malnutrition in children under five years of age declined from 21.7% in 2007 to 17.3% in 2011-12. However, very marked inequalities persist between rural and urban populations. 19 Chronic malnutrition is twice as high in rural areas than urban ones, and the risk of malnutrition is six times higher for the poorest quintile of the population. 13 UNICEF (2013), Nicaragua. Country Programme Document 2013-2017. Accessed November 2016 https://www.unicef.org/about/execboard/files/2012-PL31_Nicaragua_CPD-final_approved-English.pdf 14 Castillo, M., Elvir, A, and Vijil, J. (2016), Prioridades de la Educación Nicaragüense para el Siglo XXI. Managua: Centro de Investigación y Acción Educativa Social. 15 UNESCO (2011), The hidden crisis: Armed conflict and education. Regional Overview: Latin America and the Caribbean. Paris:UNESCO. Education Policy and Data Center (2014), Nicaragua. National Education Profile. Washington DC:EPDC. 16 Gobierno de Nicaragua (2014), Ley de Reformas y Adicionales a la Ley N582, “Ley General de Educacion”. Accessed November 2016 http://legislacion.asamblea.gob.ni/SILEG/Gacetas.nsf/15a7e7ceb5efa9c6062576eb0060b321/8524dc0b4dd5b68c06257cb7004 ac2a7/$FILE/2014-03-26-%20G- %20Ley%20No.%20861,%20Ley%20de%20reforma%20y%20adiciones%20a%20la%20Ley%20No.%20582,%20Ley%20General% 20de%20Educaci%C3%B3n.pdf 17 MINED (n.d.), Educación Primaria. Accessed November 2016 http://www.nicaraguaeduca.edu.ni/principal/educacionprimaria 18 UNICEF (2013), Mejorar la Nutrición Infantil. El imperativo para el progreso mundial que es posible lograr. New York: UNICEF 19 Pan American Health Organization (2015), Nicaragua. Perfil de país. Managua: PAHO. Mid-Term Evaluation Project CREAN – World Vision 15 The underlying causes of malnutrition in the country vary and are influenced by social, economic, environmental, cultural, and political factors. The main aspects that generate malnutrition in the family – chiefly in children – are the lack of economic resources for food purchases, poor maternal health, inadequate feeding and childcare practices, and a lack of access to safe drinking water and sanitation. 20 According to UNICEF data, 27.2% of school children aged six to nine years old suffer from chronic malnutrition. 21 However, in 2013, the government-supported National Census of Nutrition calculated an 11.6% decrease in the indices of chronic child malnutrition, representing a drop of almost half of the number calculated in 2006. 22 The MINED, in collaboration with the Ministry of Health (MINSA), is currently conducting a nutritional census aimed at primary schools to gauge the nutritional levels of children and support follow-up measures. The Comprehensive School Nutrition Program The Comprehensive School Nutrition Program (Programa Integral de Nutrición Escolar – PINE) is a strategic plan of the MINED to contribute to the improvement of education, nutrition, and food￾related cultural conditions of those children enrolled in public pre-schools and primary schools. This program started as a campaign by the MINED to combat non-nutritious food usage during the year 2007, within the context of implementation of the new curriculum.23 The PINE-MINED program consists of four components: School meal: the main objective of the school meal is to improve educational, health, and nutritional indicators of children, guaranteeing their attendance in a sustained manner throughout the school year. 24 At present, school meals benefit 1.2 million students in 12,000 pre￾school and primary schools. 25 The meals consist of 145 grams of food including the following items: rice, beans, vegetable oil, fortified cereal, cornmeal, and milk powder. The meal, which is provided once a day, contributes 30% of the required nutritional intake for pre-school and primary children.26 Due to drought and high levels of malnutrition in the dry corridor, the MINED decided to reinforce the meal in 51 municipalities, giving both male and female students two hot meals per day. 27 20 Ibarra, A., Ramírez, D. and Salgado, B. (2014), Factores socioeconómicos que inciden en los altos niveles de desnutrición infantil en los niños y niñas de 0 a 5 años. Leon: Universidad Nacional Autonoma de Nicaragua. 21 UNICEF (2013), Nicaragua. Country Programme Document 2013-2017. Accessed November 2016 https://www.unicef.org/about/execboard/files/2012-PL31_Nicaragua_CPD-final_approved-English.pdf. 22 MINSA (2016), Nicaragua ha reducido a la mitad el índice de desnutrición crónica Accessed November 2016 http://www.minsa.gob.ni/index.php/noticias/329-nicaragua-ha-reducido-a-la-mitad-el-indice-de-desnutricion-cronica 23 MINED (2016), Programa Integral de Nutrición Escolar. Accessed November 2016 http://www.mined.gob.ni/index.php/pine/ 24 MINED (2016), Programa Integral de Nutrición Escolar. Accessed November 2016 http://www.mined.gob.ni/index.php/pine/ 25 WFP (2016), Nicaragua refuerza la merienda escolar. Accessed November 2016 http://es.wfp.org/historias/nicaragua￾refuerza-la-merienda-escolar 26 Zúñiga, S. (2015), Estado nutricional, habitos alimentarios y merienda escolar de estudiantes que cursan el 5°-6° grado del Colegio Publico del Poder Ciudadano Experimental México. Periodo Noviembre 2014-Febrero 2015. Managua: Universidad Nacional Autónoma de Nicaragua 27 WFP (2016), Nicaragua refuerza la merienda escolar. Accessed November 2016 http://es.wfp.org/historias/nicaragua￾refuerza-la-merienda-escolar Mid-Term Evaluation Project CREAN – World Vision 16 Table 1: Size of ration for pre-school and primary education Food type Ration (grams) Energy (Kcal) Rice 35 126 Cornmeal 40 104 Mixed cereal 35 195 Beans 25 84 Oil 10 88 Recommended daily nutrition 2000 Nutritional support from school meal 29.85% Source: Zúñiga, 201528 It is worth mentioning that 76% of school-meal funding comes from the Nicaraguan government, which has a large operational and logistical chain that guarantees the purchase, storage, and distribution of food in every public school in the country.29 PINE-MINED promotes the creation of a School Feeding Committee (Comité de Alimentación Escolar – CAE) in each beneficiary school. The CAEs are made up of principals, teachers, and parents whose role is to ensure proper collection, preparation, distribution, and storage of the school meal. School gardens: the objective of school gardens is to teach students in the final grades of primary and secondary school about land use, while improving the school meal with supplements produced in gardens.30 School kiosks: these kiosks are established in schools and aim to promote healthy and local use of food resources from each region to strengthen healthy eating practices. 31 Food Security and Nutritional Education (Educación en Seguridad Alimentaria y Nutricional; SAN): this activity aims to support comprehensive training of students on the appropriation and application of knowledge about production and consumption of nutritious foods, the advantages of agro-forestry systems, and the protection of natural resources. 32 28 Zúñiga, S. (2015), Estado nutricional, habitos alimentarios y merienda escolar de estudiantes que cursan el 5°-6° grado del Colegio Publico del Poder Ciudadano Experimental México. Periodo Noviembre 2014-Febrero 2015. Managua: Universidad Nacional Autónoma de Nicaragua. 29 WFP (2016), Nicaragua refuerza la merienda escolar. Accessed November 2016 http://es.wfp.org/historias/nicaragua￾refuerza-la-merienda-escolar 30 MINED (2016), Programa Integral de Nutrición Escolar. Accessed November 2016 http://www.mined.gob.ni/index.php/pine/ 31 MINED (2016), Programa Integral de Nutrición Escolar. Accessed November 2016 http://www.mined.gob.ni/index.php/pine/ 32 MINED (2011), Guía didáctica para docentes del nivel de educación inicial. Seguridad Alimentaria y Nutricional. MINED: Managua. Mid-Term Evaluation Project CREAN – World Vision 17 CREAN Nicaragua Background WV has 25 years of experience in Nicaragua and 15 years of experience implementing USDA￾funded nutrition and education projects in Afghanistan, Mongolia and Mozambique. 33 In June 2013, the organization identified the need to improve literacy and school nutrition in the departments of León and Estelí. Thanks to its work history and presence in the area of intervention, WV is not only familiar with local needs but also has the credibility and support of local communities to implement a project of this scale. WV, in partnership with the USDA’s McGovern-Dole program, has joined efforts to support the PINE-MINED comprehensive school nutrition initiative through the CREAN project, which aims to improve learning, health, and nutritional practices among primary school children. This project lasts three years – starting in 2015 and ending by 2017 – and aims to strengthen the PINE-MINED program, which has been in place since 2007, while integrating elements of education, nutrition, hygiene, and health. Project objectives The CREAN project established the following objectives: 34 • Improve the reading and writing skills of children through training of teachers and technical staff and the delivery of textbooks, school supplies, and supplemental materials for reading. • Improve school attendance through reducing hunger and increasing access to food through school meals and the establishment of school gardens. • Improve school attendance through the implementation of activities that promote reading and community awareness of the benefits of education. • Increase the involvement of community organizations and groups through parent-teacher partnership training and the development of action plans. • Increase the use of best food and health practices by increasing access to clean water services through water filter systems and improving access to sanitation through the construction or rehabilitation of latrines and hand-washing stations. Intervention area USDA, MINED, and WV jointly selected the areas of implementation. The selection criteria included the identification of geographic areas with high priority nutritional needs and the prior presence of WV in the intervention area. 35 In the department of León, the project is implemented across all of the schools in the municipalities of Achuapa, El Sauce, Santa Rosa del Peñón, and Jicaral. Similarly, in the department of Estelí, the project works with all the schools located in the municipalities of San Juan de Limay, San Nicolás, Estelí, and La Trinidad. 33 World Vision (2016), Evaluation Plan Project CREAN Children Reading and Nourished, World Vision-USDA. 34 World Vision (2014), Attachment A Plan of Operation, World Vision. 35 World Vision (2014), Attachment A Plan of Operation, World Vision. Mid-Term Evaluation Project CREAN – World Vision 18 Working within all schools in each municipality levels the playing field, facilitates cooperation with existing government structures, avoids initiating conflict between communities, and discourages students from switching schools based on the provision of meals. Beneficiaries Approximately 38,000 students and 1,705 school personnel have been participating and benefiting directly from the project. 36 Project structure and activities The CREAN project has two main components: the Education component, which aims to increase the literacy of school-age children through school retention and improvement of education; and the Health and nutrition component, which seeks to improve health and nutrition practices. The school meal, although it belongs to the Education component, will be treated in this report as an independent component, which given its complexity and scope will allow for clearer description. Under these three components, the project implements a series of activities, as described below. Daily school meal CREAN provides daily school meals for children and teachers in pre-school and elementary schools located in the departments of Estelí and León in order to increase attendance and literacy. Meals promote good health and beneficial eating habits, as food provides the energy and nutrients that schoolchildren need to perform their daily activities. In line with MINED’s school meal program, every student in each beneficiary school receives 150 grams of food that includes rice, beans, vegetable oil, fortified cereal, and cornmeal. CREAN provides yellow cornmeal instead of the whole grain white flour provided by the MINED. CREAN also provides fortified cereal and rice, while MINED distributes locally produced cereal and regular rice. The delivery of the meal to each child is the result of a long and complex logistical process, which includes a series of stages and sub-activities, described below. Food handling logistics CREAN is responsible for the USDA food application process and customs, controls, and transportation once the food arrives in country. Once the country’s customs authority approves the shipment, food is transferred to a warehouse in Matagalpa where the verification of quantity and quality of the different products occurs. WV manages the flow of information regarding food shipments through the Commodity Tracking System. This is a global data base system maintained by World Vision Food Program Management Group. All World Vision Food Programs have to use this auditable accounting system for monthly reporting. After these control mechanisms are complete, CREAN coordinates the transport of food to different distribution points in the intervention areas with PINE-MINED. Distribution centers are 36 World Vision (2016), Evaluation Plan Project CREAN Children Reading and Nourished, World Vision-USDA. Mid-Term Evaluation Project CREAN – World Vision 19 selected based both on their location and their ability to receive and store large quantities of food. Delivery dates to the distribution centers, once established, are communicated to the schools, which are responsible for collecting their food rations from the distribution centers. Each school organizes collection as it sees fit, and in most cases, the process is carried out either by the CAE or jointly by principals, teachers, and parents. There are three food distribution schedules per year (January, May, August), following the school calendar; each distribution seeks to cover 60 days of food. At the time of distribution, those responsible for collection at each school receive a list noting the number of students and teachers and the corresponding amount of food. This list is the basis for the school meal monitoring process. Food reception and storage Each school must assign an appropriate place for proper food storage. While some schools have a storeroom, most schools designate dedicated spaces in a classroom, library, or at the principal’s office for storage. As much as possible, the food storage location must be clean, tidy, and sheltered from the external environment. CREAN has distributed four 55-gallon plastic containersto each school participating in the project to facilitate the adequate and hygienic storage of food rations; plastic containers help preserve food and prevent rodent infestation. Cutlery for each student (dish, spoon, and cup) has also been provided. Training in food preparation and handling The CREAN team uses MINED’s official school nutrition guides to train the CAEs to manage and store food properly. The project provides training to those responsible for the preparation of the meal – typically parents – on the best hygiene and nutrition practices for meal preparation. The project also trains the CAEs in each beneficiary school on how to create an action plan that includes meal preparation schedules and menus that incorporate complementary vegetable and fruit from the school gardens. They are also given a list of hygienic practices for the preparation and storage of food. Preparation and distribution of school meals Parents typically prepare school meals on a rotational basis. There have also been cases where parents collectively raise a small amount of money to pay someone to prepare the meal every day. Since most schools do not have a kitchen, parents responsible for food preparation need to move food to their homes. Each parent prepares the school meal and, as far as possible, supplements Mid-Term Evaluation Project CREAN – World Vision 20 it with locally available products. Once the meal has been prepared, the parent transfers it to the school for distribution and consumption. Children usually receive the school meal every morning between 8 and 10 a.m. The parent responsible for preparing the meal is also responsible for assisting the classroom teacher in the distribution of the meal. Monitoring and control of school meals In order to ensure the proper and effective use of food for the provision of school meals, CREAN has established a monitoring system employing field monitors. These monitors are responsible for visiting each school at least once per distribution schedule for a total of three times per year. During the visits, monitors compare and validate food delivery lists against student attendance data. CREAN uses student daily attendance collected by the teachers as a verification mechanism to control consumption of the meal. Field monitors also provide assistance and advice to CAEs and teachers regarding food storage, management, and preparation, as well as answering questions or concerns regarding the various activities carried out in each school. Education activities CREAN seeks to improve the educational needs of children, strengthening the learning/teaching capacity of children and teachers. This component seeks to help children and teachers achieve the required level of competence according to the MINED curriculum. This larger activity breaks down into a series of undertakings described below. Establishment of reading corners The project delivers reading books, storybooks, activity books, dictionaries, and other types of educational materials to assist teachers in promoting a culture of reading both at school and in the home. Through this delivery, every classroom or school can establish a mini-library (or reading corner) where students can access appropriate reading material that motivates them to read. Traveling backpacks This initiative uses a ‘backpack’ or pouch with books that children can take to their homes. These backpacks are to be used by students to take home books they find interesting to read, strengthening their literacy skills and also promoting reading habits among other family members. Once the student returns the backpack, they record the name of the borrowed book, the dates, and number of pages read in their ‘traveler passport’. Students also report to rest of the class about the main points in the books they read at home. This enables the teacher to record the usage of the books. Generally, students may borrow up to three books depending on content; the approximate lending time for the backpack is three days. Supplies and school materials Mid-Term Evaluation Project CREAN – World Vision 21 As part of the project activities, CREAN has evaluated the needs of each participating school, and – based on this need – has delivered to selected schools a kit of materials including pencils, markers, cards, colored ribbons, etc. Teachers at the beneficiary schools use these materials to decorate the classroom and/or develop classroom activities. Teacher training CREAN provides teacher training on five basic reading skills: phonological awareness, phonetics, vocabulary, fluency and comprehension of MINED’s methods and approach. The content of the training is aligned with MINED’s study plan, which includes pre-literacy skills, letter recognition, phonological awareness, vocabulary, reading fluency and comprehension. The project also emphasizes the important of Pedagogic Circles, which are informal meetings for teachers to share good teaching practices with their peers. Nutrition and health activities CREAN seeks to contribute to an integrated and healthy environment for learning. CREAN provides garden tool kits, seeds, and guidance manuals to help schools start their own school gardens. CREAN also plans to improve water and sanitation by educating beneficiaries on good hygiene practices (hand washing, tooth brushing); the project further supports small infrastructure works, such as improving latrines and lavatories, and providing and promoting filters to improve access to clean drinking water. Health and hygiene clubs CREAN has helped establish health and hygiene clubs in participating schools. These clubs – composed of fourth, fifth, and sixth grade students and a teacher – have been trained in topics related to healthy and hygienic daily practices. These include hand washing with soap, personal grooming, brushing teeth, and using clean water for drinking and cooking. Health and hygiene clubs use a child-to-child education methodology whereby students transfer lessons learned to other children through events, health fairs, information, education, communication materials, and ‘learning by doing’. Hygiene kit The CREAN project has also distributed to students hygiene kits containing soap, a towel, a toothbrush, toothpaste, shampoo a fine comb (to prevent lice). Within each school, hand washing is encouraged before and after the school meal, and brushing teeth after the school meal. Hygiene kits are stored in classrooms so that they are accessible to students when necessary; however, not all schools have the infrastructure, conditions, and/or appropriate security to store the kits. School gardens CREAN has evaluated the availability of plots, conditions, and types of land of selected schools that offer the best conditions for school garden development. The plan focuses on working with community agricultural promoters, teachers, parents, students, and other community members to initiate and maintain school gardens. Mid-Term Evaluation Project CREAN – World Vision 22 CREAN’s school gardens are established in line with the Methodological Guide for the Establishment of School Gardens and Guide for the Operation of School Gardens from MINED. Seeds, plants, and tools have been delivered. The fruit and vegetables harvested in the gardens are for use as complements to the school meal. Construction and rehabilitation of water and sanitation (WASH) infrastructure CREAN has worked with MINED to evaluate sanitary conditions in each participating school in order to identify the need for rehabilitating existing latrines and constructing or rehabilitating hand-washing stations. A hand-wash station consists of a faucet, a sink and a foundation. Water filters The project provides participating schools with a water filtration and purification system. Depending on their size, location, and conditions, schools receive filters with a capacity of eight gallons. These filters are filled with water from a well, spring, or other source, which is then filtered for consumption. If access to water is a problem in the school, each child is asked to bring a small bottle of water. This water is deposited in the filter for consumption. Project management Project administration and management is the responsibility of a team composed of 50 staff members located in the Sébaco office and two staff members located in the central office in Managua. The functions of the team are divided into Management, Merchandise, Operations (overseeing Education and Health and Nutrition), Monitoring and Evaluation, Administration, Purchases and Finance and Human Resources. The management team is made up of the Director and Deputy Director of the project. They are responsible for managing and supervising all project activities. They also deal directly with the donor and with key project partners, notably to coordinate activities with MINED. The merchandise team, made up of 26 members, is responsible for all food logistics, including reception in-country, storage and controls in the warehouse, and transfers to the final destination point. Three members of the team are in charge of tracking the commodities through the Commodity Tracking System. Three other members work in the warehouse. The team includes 12 field monitors responsible for the control and periodic monitoring of the school meal. The field monitors are also responsible for the M&E sheets that track the progress of activities across components. The team also has six field monitors funded by CREAN but working directly with PINE-MINED. They carry out the same work as the CREAN field monitors, and at the same time seek to institutionalize the oversight methodology within the ministry. Mid-Term Evaluation Project CREAN – World Vision 23 The operations team comprises 10 people responsible for implementing the different activities related to Education, Health and Nutrition. The monitoring and evaluation team has three members, and is responsible for the design and implementation of monitoring plans, as well as the collection and analysis of information relating to progress of the project. The administration, accounting and procurement team has eight members and is in charge of all the project’s administrative and accounting processes, including the acquisition of goods and services for the implementation of the different project activities. In addition, the project has one person responsible for human resources and a cleaner. Project partners The CREAN project is implemented in close collaboration with local partners. The main partners are MINED and the Ministry of Health (MINSA). The project also works closely with other local non-governmental organizations (NGOs) to coordinate initiatives and with the teachers and parents in each school to ensure the effective implementation of the project. Methodology In line with the monitoring and evaluation strategy of CREAN, WV engaged a team of consultants to conduct the project’s mid-term evaluation which seeks to understand project effectiveness, analyze the progress made towards the objectives and goals, and identify areas for improvement in the implementation of the project. Evaluation objectives The main objectives of this evaluation are as follows: • Evaluate the strengths and weaknesses of program implementation and the quality of intermediate results, the degree in which the interventions of the project have adhered to the approved plans, and the perceived value for beneficiary communities. • Present evidence of changes (intended and unintended) associated with program interventions and activities of the project; assess how well the observed changes reflect the project framework, and identify factors that impede or promote the advancement of these changes. • Make recommendations in terms of structure, dedicated resources, administration, monitoring and evaluation, implementation, and other areas where the project can be improved to increase the chances of achieving the objectives. The recommendations are to be limited to actions that can be implemented within the remaining time and budget of the project. Mid-Term Evaluation Project CREAN – World Vision 24 Feedback gleaned from the results are expected to be useful to the CREAN project team and project partners in identifying best practices that can be replicated and scaled, both internally at the project level and at the wider organizational level, as well as in highlighting critical areas that need to be improved. Evaluation approach This research was built upon an evaluation approach known as Utilization-Focused Evaluation (UFE). 37 The core tenet of UFE is that the evaluation process and outcome should be developed so as to enhance the utility of the findings to the final intended users. It is not a specific methodology with narrowly prescribed methods, but a guiding framework oriented towards facilitated learning and shared ownership. The UFE process is meant to yield concrete decisions and facilitate improvements in performance. Therefore, the evaluation users must be engaged early in the process and throughout to ensure that their key questions are identified and answered. The evaluation should generate clear, actionable findings and suggestions that can help the users learn and improve. The evaluation approach was founded upon key principles of effective and ethical evaluation practice. The team: • Utilized systematic inquiry techniques and adhered to the highest technical standards to ensure the accuracy of the findings. • Employed honesty and integrity in their own behavior and in the evaluation process; any conflicts of interest were clearly disclosed and all possible efforts were made to interpret the data with fidelity. • Displayed respect to all stakeholders and ensured that all participants in the process provided their informed consent. • Ensured that every effort was made to acknowledge and accommodate differences in language, culture, gender, age, etc. among evaluation participants in planning, conducting, analyzing, and reporting on research findings. Sampling plan and site selection We used a stratified, purposeful sampling method in order to ensure that the data reflects the experiences of specific sub-groups and to ensure that there was an appropriate balance of sampling between León and Estelí departments in the country. Site selection was conducted through analysis of data provided by WV. A representative sample was constructed based on a series of factors: geographical location, type of school (urban vs. rural), type of intervention at school, number of students, number of teachers, and modality of education (pre-school, primary, multi-grade, special education, etc.). A first sample of schools was identified based on the data analysis conducted. This sample was shared with CREAN to discuss availability and accessibility. A refined list was finalized and shared with MINED for approval. 37 For more detail on UFE, please refer to the following for a primer on the approach: http://www.managingforimpact.org/sites/default/files/resource/ufeenglishprimer.pdf Mid-Term Evaluation Project CREAN – World Vision 25 A mix of urban and rural schools of different sizes, levels of access, and providing different modalities of education were selected. The schools selected were deemed to represent the universe of schools that CREAN serves. Table 2: Selected schools Department Municipality School Urban/Rural Estelí La Trinidad Teodoro Rodriguez Rural Estelí La Trinidad Especial Uri Molina Urban Estelí Estelí Rodolfo Ruiz Cruz Urban Estelí Estelí Monsenor Oscar Arnulfo Romero Urban León Achuapa Wiquili Rural León Sauce Sabana Grande Rural León Jicaral Alejandro Vegas Matos Urban León Santa Rosa Jose de la Cruz Mena Urban Evaluation methodology We employed a context-sensitive and qualitative method approach. We drew on an appropriate mix of qualitative methods to understand what works best and to identify factors that appear to enhance or undermine the quality, acceptability and usefulness of implementation and outputs. As part of this evaluation we used the following methods: • Desk-based review • Key informant interviews (KIIs) • Focus group discussions (FGDs) • Observation guide Field research was conducted in Nicaragua over a 12-day period from 17-28 October 2016. KIIs and FGDs were organized with relevant stakeholders. The research team was responsible for the design of all research instruments, including: interview questionnaires, discussion guides, consent forms, and others (see Annex). The instruments were reviewed and approved by WV and MINED. In collaboration with WV, we identified the target sub-groups (e.g. gender and geographic region) and generated lists of potential respondents within each category. Desk-based review The first phase of the evaluation was to conduct a desk-based review. This task constituted the review of information produced by WV, including monitoring, evaluation and learning tools, the baseline study, and research and evaluation studies. The review helped the team understand the structure of the project and its accomplishments to date and to identify potential areas that needed to be improved. The information from the project also helped shape the type of questions that formed the basis of the instruments used during research. Mid-Term Evaluation Project CREAN – World Vision 26 Key informant interviews 35 KIIs were conducted with members of the staff of CREAN, state and municipal representatives from MINED and MINSA, school principals, teachers and parents. School principals and teachers involved in the implementation of project activities were interviewed during school visits depending on availability. When the school principal was not present during the visit, a teacher with knowledge of the project was interviewed instead. Interviews with members of MINED and MINSA were conducted at their local offices. Representatives were selected based on their location (municipal representatives were selected based on the schools visited) and direct involvement and knowledge about the project. KIIs with members of staff were conducted at the Sébaco office. Members of staff representing different teams and levels of responsibility were interviewed in order to get a more comprehensive understanding of the operations of the project. All the interviews were semi-structured and conducted in Spanish. They helped interpret detail and nuances of the CREAN project. Mid-Term Evaluation Project CREAN – World Vision 27 Table 1: Key informant interviews Interview group Number of interviews MINED – state representatives 4 MINED – municipal representatives 6 MINSA – state representatives 1 MINSA – municipal representatives 1 School principals 6 School teachers 5 CREAN staff members 12 Total 35 Focus group discussions We conducted 17 FGDs with students, parents and teachers to strengthen the findings and provide tangible examples of their experiences. All FGDs were conducted in Spanish and took place on the school premises. A sample of students from each school was selected to participate in FGDs based on their age, gender, participation in project activities, and availability. In total, 73 students took part in the FGDs. School authorities helped convene parents. Due to work responsibilities, most parents present for the FGDs were mothers who had varying levels of responsibility for the preparation of the school meal. 51 parents participated in FGDs. One FGD took place with teachers during one of the school visits. Informed consent was obtained prior to the initiation of the focus group, giving each participant the opportunity to ask questions regarding the purpose of the activity, its benefits and the potential risks associated with such an exercise. A discussion guide was employed, including open-ended questions to encourage group participation and discussion on specific topics. The FGDs were recorded, with the consent of participants, in order to facilitate their analysis at a later stage. In the case of the special education school Uri Molina, students’ disabilities meant the FGD was conducted with only two students. These students were nominated based on their ability to communicate with the evaluation team. Table 2: Focus Group Discussions - Students Schools Wiquili Sábana Grande Alejandro Vega Matos Juan de la Cruz Mena Especial Uri Molina Teodoro Rodríguez Rodolfo Ruiz Cruz Monseñor Arnulfo Oscar Romero State León León León León Estelí Estelí Estelí Estelí Number of students 6 13 10 9 2 12 10 11 Grade 4,5,6 6 3,4,5,6 4,5,6 6 4,5,6 4,5,6 4,5,6 Female 3 6 5 7 1 7 5 3 Male 3 7 5 2 1 5 5 8 Mid-Term Evaluation Project CREAN – World Vision 28 Mid-Term Evaluation Project CREAN – World Vision 29 Table 3: Focus Group Discussions - Parents Schools Wiquili Sábana Grande Alejandro Vega Matos Juan de la Cruz Mena Especial Uri Molina Teodoro Rodríguez Rodolfo Ruiz Cruz Monseñor Arnulfo Oscar Romero State León León León León Estelí Estelí Estelí Estelí Number of parents 4 9 3 10 7 5 8 5 Female 4 8 3 10 7 5 8 4 Male 0 1 0 0 0 0 0 1 Table 4: Focus Group discussions – Teachers Group Number of Teachers Teachers 8 Observation guide An observation guide was developed to assist the evaluation process. The observation indicators included: storage of products, education materials (reading corners, traveling backpacks, classroom decoration), health and hygiene materials (posters and signs, hygiene kits), water filters, school gardens, and sanitary installation. Data analysis Members of our evaluation team conducted qualitative analysis of KIIs and FGDs. The FGDs and the KIIs were transcribed, analyzed and coded based on a priori and emergent themes, all in the language in which they were conducted. Specific quotes were then translated into English where appropriate. The analysis looked at strengths, weaknesses, and other factors that appear to enhance or diminish the quality, acceptability and usefulness of implementation and outputs. Findings This section presents the results of the mid-term evaluation according to the key questions set out in the terms of reference. During the process of this evaluation, a number of stakeholders were consulted about the implementation process and activities of the project. A series of findings were made, which are detailed below. 1. Project relevance Both the importance and relevance of the USDA-funded CREAN project – as well as project alignment with the MINED strategies and priorities – has been verified at the state and municipal levels through multiple KII with representatives from MINED and MINSA. The results were also validated through interviews and FGDs with principals, teachers and parents from the schools visited. Both the interviews carried out and the data collected during the evaluation process confirm that the activities conducted by CREAN are relevant to the beneficiaries, particularly in the case of the school meal, which contributes approximately one-third of the dietary requirements of pre- Mid-Term Evaluation Project CREAN – World Vision 30 school and primary school children. School principals, teachers and parents agree that the provision of school meals directly influences the daily attendance of students. In some schools visited, stakeholders reported that the provision of school meals has resulted in major improvements in school attendance rates by students. Strategic fit and rationale CREAN is highly aligned with the priorities of the government of Nicaragua in general, and those of MINED in particular, which has facilitated the acceptance of the project by the various stakeholders, especially the schools where it is implemented. The main strategies promoted by MINED are based on improving (1) school retention and (2) academic performance. The project promotes both strategies through a series of interrelated activities. The school meal helps with student attendance and retention; however, the project recognizes that increased school attendance does not necessarily translate into better academic performance. Thus, CREAN works closely with school principals and teachers to provide them with the skills and materials necessary to improve academic achievements. Moreover, the different activities implemented by the project are based on the MINED curriculum and are aligned with the methodology implemented by the PINE-MINED. This has facilitated implementation and avoided unnecessary adaptation efforts by the participating schools. CREAN’s support of meal distribution in 637 schools in the departments of León and Estelí has allowed PINE-MINED to extend its coverage to other regions and areas of higher priority. Likewise, activities in the Education, and Health and Nutrition components have sought to initiate processes to generate and improve current capacity in the educational centers in which they are implemented. Furthermore, MINSA has been involved in the Health and Nutrition component through anti-epidemic campaigns and the nutritional census of children. Progress of project implementation At the time of the evaluation, most of the project’s planned activities were being implemented and it is highly likely that CREAN will meet the objectives set out in its plan. Although the project has made significant progress in implementation, it has suffered considerable delays that have affected the schedule of activities and the quality of their implementation. This had also placed the project team under significant levels of stress, which could potentially jeopardize the quality of activities carried out. The most delayed activities are the distribution of books and the construction and rehabilitation of WASH infrastructure, which has not yet been initiated. Unless resources and efforts are dedicated to these activities during the remainder of the implementation process, the project risks being unable to fulfill the related objectives. External and internal factors affecting the implementation of the project External factors Mid-Term Evaluation Project CREAN – World Vision 31 Our research revealed two main external factors that have delayed project implementation, which are outlined below: Changes in project partners: Initially, the project was designed to be jointly implemented by WV (which would be responsible for the school meal, and Health and Nutrition component) and Fe y Alegría Nicaragua (originally responsible for the Education component). Subsequently, MINED requested that WV be the sole executor of the project, which led WV to take total responsibility for the implementation of project activities. The change in the number of project implementers represented not only a significant delay in the implementation schedule but also a need for internal re-structuring in terms of activities, personnel and resources in order to fulfill the needs of the project. The sudden change in project partners also had an impact on the area of intervention. The municipalities of Estelí, located mainly in urban and peri-urban sectors, were initially proposed by Fe y Alegría Nicaragua based on their areas of intervention and are not located in the WV area of operations. Delay in the signing of the agreement: The implementation of the project has also been considerably delayed due MINED’s prolonged review time prior to signing the agreement. The lack of definition and fulfillment of established dates, the long decision-making process, and the concentration of work by key decision-makers have resulted in approximately a year’s delay in terms of implementation. As a result of the delays, 2015 saw only two of the three food distributions planned for the year. Since the signing of the agreement, the project team has been working diligently in the implementation of project activities in order to recover some of the lost time. However, the delay has meant a significant alteration in the schedule of activities, affecting the quality of implementation in some cases. This has generated high levels of stress among the project team, who are under a tremendous amount of pressure to achieve goals and objectives. It is important to note that the main reasons for the delayed implementation of the project have been outside of WV’s control; nevertheless, the project team has tried to minimize the adverse effect of this delay on the implementation of activities. Internal factors Aside from the two external causes, two internal organizational factors have contributed to the delay of the project, but in a less significant way: Delays in the establishment of the project team: This was manifested in delays in the recruitment and hiring process at the beginning of the project, as well as in the distribution of tasks between the different areas, which generated further delays during implementation. However, it is worth noting that the project could not begin the recruitment processes without a signed agreement from MINED. Delay in baseline results: Several of the activities envisaged as part of the project could not be implemented without baseline results. A delay in obtaining the baseline information has thus generated delays in terms of activity implementation. Mid-Term Evaluation Project CREAN – World Vision 32 Relationship with project partners Although the initial project negotiations saw several setbacks (the change of partners and delays in the signing of the agreement), the CREAN management team has since then dedicated much of its effort to building a positive and productive relationship with its main partner, MINED. These efforts have generated tangible results leading to the establishment of effective communication channels that have allowed significant advances in the implementation of activities. Today, WV has a close relationship with MINED, which contributes to the joint planning of activities, the discussion of topics of common interest, and the sharing of relevant project information. This positive and productive relationship at the national level has also been replicated at the state and municipal levels with the corresponding local representatives. This has allowed not only greater efficiency in the implementation of activities, but has also helped to accelerate decision-making processes. While progress in the relationship with project partners has been significant, MINED must continue to strengthen its internal planning and decision-making processes because of the significant impact each decision has on the project. MINED faces issues that are much larger than this project can address: partnering with it has required effort and ingenuity in the implementation of activities in order to respond to its institutional needs. Several cases were reported to us where MINED unexpectedly cancelled or changed dates that had been agreed months in advance. Sudden changes in the activities schedule entail significant financial losses for the project and are demotivating for a project team that is working under pressure and against the clock to achieve the objectives. Although there are contextual and political situations that are outside the control of the parties in this project, it is important that MINED appraises the influence of its actions on the implementation of the project. 2. Project effectiveness and efficiency Overall, the effectiveness and efficiency of the project has been commendable. Nonetheless, it could be further improved if the project was able to overcome some of the challenges highlighted below. This section details the strengths and weaknesses of project design, management, and administration, as well as the process of implementation of each of the components and activities of the project. Project design CREAN is an extremely ambitious project given to its three-year life cycle. The project has a large number of inter-related activities and sub-activities that, while reflecting its holistic character, also present a series of challenges. In order for project activities to be successful, significant time and personnel are required to ensure proper execution and constant monitoring. While the project design provides for the financial and human resources needed to implement these activities, there are significant shortfalls with what is actually needed in practice. For Mid-Term Evaluation Project CREAN – World Vision 33 example, while the project plan includes the purchase and distribution of reading materials for schools, it only takes into consideration the resources needed for purchase and distribution. In fact the process is much more complex, necessitating a bidding round, a purchase process, receipt of books, control of those books received (individual book accounting), re-packaging of books as required for each school, the creation of monitoring mechanisms for the distribution and reception of the materials, coordination with the school for reception, and reporting on the reception process. The lack of planning for such a demanding level of effort needed for each activity has increased the workload for the project team and raised stress levels significantly. The challenges in implementation were not only noticed in terms of the number of activities, but also in the excessive number of monitoring and evaluation indicators to report, which have generated a disproportionate level of work for the project team (this topic is further explored in the monitoring and evaluation section). Projects of this nature inevitably lead to questioning the number of activities versus their impact. Although it is still too early to assess the impact of each of these activities and their relation with each other, it is important that the project team learns from the current process. Project management The management and administration of the project is characterized by rigorous and adequate procedures. The project’s relative autonomy, compared to other WV activities, has helped minimize bureaucratic hurdles that would have further slowed down implementation. The project is also characterized by a high level of decentralization in planning and a high level of participation by the partner, PINE-MINED. Offices The CREAN project occupies a physical space in the WV headquarters in Managua, which allows greater communication with government actors such as MINED and MINSA. In the same way, CREAN has offices in Sébaco, which allows easier control and mobilization both to the beneficiary communities and to the warehouse. Project team The team in charge of the implementation of this project, mainly at the managerial and coordination level, is professional and competent with a high level of academic training, extensive experience in the sector, and the capacity to carry out the necessary tasks. In particular, the Director’s previous project management experience in similar initiatives has been a great help in enabling the team to respond to the different competing demands that have arisen. Although CREAN has 52 people who support the different activities and phases of project implementation, a number of situations have been identified that need to be addressed in order to improve the effectiveness of the project: Important differences regarding size versus responsibilities among teams: The Health and Nutrition team has only one coordinator and two field officers to implement all the activities of Mid-Term Evaluation Project CREAN – World Vision 34 this component in the 637 schoolsthe project serves. The geographical location of schools, which are quite distant from each other, as well as the number of activities for which the team is responsible, have made it difficult to implement activities and has forced the team to prioritize interventions at the expense of other important project aspects. While the Health and Nutrition team has relied on other teams for the implementation of activities (such as the Education team and the Merchandise team through field officers and field monitors), this has caused confusion in terms of responsibilities, tasks, and chains of command. The same problem has been identified in the Monitoring and Evaluation team, which has only two field officers. The excessive number of indicators the team needs to track has generated high levels of information and has overwhelmed the team in the process. The team has been left with no capacity to respond to project needs and/or analyze the information in a timely manner. Lack of technical capacity in key areas: The most affected team is Health and Nutrition, which lacks several technical capacities necessary for the implementation of activities. The team requires an agricultural specialist who can advise schools on issues related to setting up and managing school gardens. The team also lacks an engineer who can advise and assist with WASH infrastructure activities. This combination of factors – the limitation of team personnel and the lack of specific technical profiles – has meant that the team has had to prioritize the implementation of certain activities over others, causing delays in the overall schedule, as well as frustration from partners based on expected results. Confusion in responsibilities, tasks, and chains of command: Field monitors play a key role within the project. Their main responsibility is the proper and effective use of food for the provision of meals, controlled through regular visits to schools. During these visits, the field monitors report pertinent information regarding the receipt, storage, and use of food. At the same time, the monitors support the work of other teams: collecting relevant information for the M&E team using technical sheets for recording data on a series of activities; ensuring the correct implementation of school gardens; guaranteeing the proper use of reading corners and traveling backpacks; and more. Due to their regular presence in schools and the links they have built within the communities, the monitors have become the ‘eyes and ears’ of the project and are the main point of contact for the beneficiaries of CREAN. This has made monitors a kind of ‘one-stop shop’ – a single instance for all the paperwork, questions, queries, and complaints associated with the project. This ‘one-stop-shop’ task has generated much confusion regarding responsibilities, tasks, and chains of command. Although they report to the Merchandise team regarding the school meal, the monitors also receive requests to perform tasks in different areas, and therefore become recipients of vital information about various activities for different teams. However, due to the structure of the project, this information is often not properly shared, losing important learning opportunities about what is working and what is not within the project and on several occasions causing conflicts regarding responsibilities and chains of command. This trend of overloading monitors with activities has also meant that they have had to sacrifice some of the time they dedicate to each school in order to fulfill the many responsibilities they have. Taking into account Mid-Term Evaluation Project CREAN – World Vision 35 these circumstances, the project needs to re-evaluate the profile of the monitors’ position and decide what are the most appropriate chains of command for facilitating improved performance. Administrative bottlenecks: During the process of conducting this evaluation, different areas have noted a number of administrative processes that generate ‘bottlenecks’ or significant delays. One area identified in particular was purchasing, whereby the procurement process and insufficient personnel combined to cause delays on several occasions. At the time of this evaluation, the purchasing area had incorporated an extra person to assist in these tasks and, together with the administrative sector, a series of adjustments had been made in the process to make it more efficient. Although it is still to early to determine if these changes will be sufficient to address the problem, it does suggest progress being made. However, in the future, it is important to closely monitor the administrative processes in order to ensure that they facilitate rather than hinder the implementation of the project. High levels of stress and exhaustion among the project team: Delays in the schedule of activities, unexpected changes introduced by the MINED, limitations on staffing, and lack of clarity regarding tasks, responsibilities, and chains of command have generated high levels of stress. As a result, exhaustion has been detected not only at the managerial and coordination level, but also at field staff level in all areas. The project team has tried to implement in one year a volume of activities corresponding to two years of work. This has resulted in substantial workloads, long working days, and excessive pressure to achieve the established objectives. Situations of high stress such as this generate higher margins of error and greater probabilities of mistakes that can affect the quality of the activities. The accelerated and compressed implementation process also limits chances for creating learning opportunities and feedback loops that can benefit the project. While the project team has so far been able to sustain the volume and intensity of activities, this pace of work is hardly sustainable in the long run. The level of stress and exhaustion of the team not only undermines the wellbeing of the project staff, but also generates a series of questions regarding the retention of personnel. The project management needs to evaluate strategies that could help alleviate the situation and relieve the levels of stress. Communication and coordination challenges: During the evaluation,several people indicated the need to improve communication and coordination between several areas. At the moment, far from being institutionalized, coordination between areas is ad hoc and relies on personal relationships. Different staff members have expressed the need for a space to discuss common issues, share information and evaluate progress. An institutionalized communication process could help improve the effectiveness of the project by keeping all parties informed on the most relevant aspects. Budget The budget allocation process and structure has also generated a number of constraints on the project. There are budget lines that have sufficiently allocated capital for the implementation of Mid-Term Evaluation Project CREAN – World Vision 36 activities, sometimes above the necessary amount. For example, the budget line for traveling backpacks was higher than needed, allowing for more backpacks to be distributed. However, the budget line for books, for which there is a very ambitious goal of purchasing a million books, is insufficient, and there are other budget lines that are also insufficient. Furthermore, the current budget also does not reflect the particular needs of the different activities, such as the construction and rehabilitation of WASH infrastructure, which has a small budget but many pressing and expensive demands. In certain cases, such as infrastructure rehabilitation and construction, the budget has severely underestimated the financial needs of the component. Budgetary restrictions among different budgetary lines have made it difficult to re-allocate resources. Likewise, the different mechanisms and requirements for budget execution (both at the WV and donor level) have created a lengthy and complicated reporting process. Monitoring and evaluation M&E is a key area, responsible for monitoring progress while providing timely, pertinent, and relevant information that helps to inform decisions regarding the direction of the project. Unfortunately, the strategic role of the M&E team has been diluted by the high number of indicators (111) that the project needs to report on regularly. The situation is exacerbated by the short life cycle of the project. On top of the 111 indicators, the M&E team is also responsible for baseline, mid-term and final evaluations. These 111 indicators, established by the donor, have consumed much of the time and resources of the M&E team and have exhausted their capacity to respond to other internal needs. The large number of indicators has also had adverse repercussions, as it has raised doubts about the relevance and credibility of the process, leading to questions over why and how the information collected is used. In the specific case of CREAN, the number of indicators required has overwhelmed the capacity of the team and has been counterproductive in terms of project effectiveness. CREAN’s M&E system is extremely complex and highly labor-intensive. This has created a number of problems. For one thing, the data collection tools are extensive and take a long time to be completed. This leads to a decrease in quality in relation to the number of completed technical sheets. For another, the tools are collected and recorded manually, which not only leads to a higher margin of error and/or probability of confusion when digitizing the information, but also increases the time needed for processing and analysis. Finally, delays in processing and analysis of information have meant that the information provided is not always timely, pertinent, and relevant, generating further delays and unnecessary bottlenecks. Mid-Term Evaluation Project CREAN – World Vision 37 While some of these problems can be solved by revising and streamlining the M&E tools as well as digitizing the process, the underlying problem will remain unless there is a significant revision of quantity and relevance of the current 111 indicators. Implementation of several components of the project School meal During the course of this evaluation, direct observations of the following processes were made: food storage at the WV warehouse, food distribution documentation, food storage at schools, and the distribution and consumption of school meals. In addition, key informant interviews were conducted at both the state and municipal level, and relevant information was gathered from teachers, principals, parents, and students. Based on the information collected, the following observations were made: Warehouse: WV has a 1,000m2 warehouse for the storage and distribution of food for the school meal. The warehouse is located on the Sébaco-Matagalpa road, on Km125.1. The facilities are rented and therefore maintenance is performed by a third party. During the visit, the warehouse was found to be in good condition with respect to its cleanliness, access, lighting, ventilation, organization of food, protection against pests, and security. That said, the warehouse team is small, with only three people on site, so in periods of food delivery, CREAN hires additional staff to reinforce the logistical work of food distribution. During the weekends, there are no staff working in the warehouse, but there is a security guard who is responsible for oversight. In case of an emergency, designated personnel who live near the warehouse are available to visit the warehouse to resolve the problem. The warehouse team has the proper safety equipment for food handling, as well as clear and adequate signs for the different areas of the warehouse, evacuation routes, and fire-fighting equipment. No evidence of irregularities in the handling process and control of the warehouse was found. The inventory control, however, is carried out manually, which increases the risk of errors in accounting and potential manipulation of information. Food storage: Food storage was examined during all visits to schools. All schools had the plastic containers, which were provided by the project, properly identified. The location for food storage depends on the structure and design of each school: some have a storeroom specifically dedicated to food storage; others store food in a corner of the library, the principal’s office, or in a classroom. Particularly in those schools without a storeroom, food was stored in ventilated classrooms exposed to the outside environment, therefore increasing the chances of contamination. Additionally, in several cases, the food storage location was also used for storage of other types Mid-Term Evaluation Project CREAN – World Vision 38 of products such as garden tools, external materials, etc., which also increases the likelihood of food contamination. Several centers have reported problems with food management. In a specific school, pests and rodents ruined a portion of the food received. In several other cases, weevils in cornmeal and rice were reported (in two schools), as well as rancid rice and beans (in three schools). In most cases, schools reported losses or difficulty in maintaining food in the last weeks of consumption prior to new distribution. Products: While the products for the school meal distributed by CREAN are similar to those provided by the government, there are some differences that have made general acceptance difficult, particularly related to cornmeal. In Nicaragua, white corn tortillas are a daily diet staple. White cornmeal does not need a lard product for its preparation since it can be made easily with water. CREAN distributes cornmeal as part of the school meal, but it is a yellow type of the flour that usually requires lard in the preparation of tortillas. This difference in product specification has presented a series of challenges for the project. CAEs and parents responsible for preparing the yellow cornmeal stated that it cannot be used easily for tortillas. While CREAN provides oil as part of the meal, this is used to fry the beans. The need for oil to prepare the cornmeal is seen as an extra cost, especially in rural areas where resources are limited, with parents saying they could not afford it. Several parents also mentioned that yellow cornmeal deters some parents from participating in the preparation of the school meal. Students have also expressed their aversion to the yellow cornmeal and have stressed the difference in flavor. Difficulties in the use of cornmeal have resulted in a low consumption, and there are indications that the unconsumed cornmeal has been discarded or wasted. The use of yellow cornmeal has had repercussions at all levels of the project: from students who mention the difference in flavor, to the members of CAEs, parents, and teachers who note the difficulties of using the cornmeal, to the municipal and departmental authorities who have echoed the complaints from the schools and have stated that the use of yellow cornmeal “violates the country’s food culture”. CREAN has recognized this challenge and developed strategies to help alleviate the problem. Training sessions on the preparation of different products with yellow cornmeal have been conducted by the Health and Nutrition team, supported by field monitors. Likewise, recipes have been distributed that show how cornmeal can be used and community members have been invited to test different ways to prepare the cornmeal and demonstrate the results in food fairs. While these strategies have helped promote concrete actions to address this challenge, the underlying problem remains one of cultural acceptance that will be difficult to overcome. Mid-Term Evaluation Project CREAN – World Vision 39 Infrastructure for food preparation: Only one of the schools visited had the necessary infrastructure for the preparation of the meal on the school premises. The other schools visited lacked this kind of infrastructure and it can be safely assumed that they reflect the situation of the majority of schools in CREAN. The lack of a school kitchen forces CAEs or designated parents to transport the amount of food corresponding to one day of consumption to their homes for preparation. This has a number of disadvantages: the designated parent must go to pick up a significant volume of food from the school the day before, get it home, cook it, and bring it back to school. Especially in rural areas, where distances are significant and there is limited access to public transport, this creates additional difficulties and costs relating to the preparation of the meal. We understand that the provision of school kitchens is completely outside of the scope of the project; however, it is important to highlight the limitations of infrastructure in order to understand how this affects other aspects in the implementation of activities – and the popular perception of the project’s success. Parent involvement in preparation of the school meal: The evaluation also identified problems concerning the assignment of responsibility for the preparation of the school meal. The PINE￾MINED, and therefore CREAN, assumes that parents will take responsibility for the meal preparation on a rotating basis. The principle idea is that each grade organizes a roster of parents responsible for the meal with assigned dates per month. It is important to note that when mentioning ‘parents’ we are actually referring to ‘mothers’, who bear the responsibility of preparing the school meal. During our school visits, we found out that not all grades receive meals every day, mainly due to the lack of mothers available to prepare the meal. In one case, students of the third grade at a particular school stated that they do not receive the meal because none of the mothers in the grade can prepare it. In other cases, we found the meal is distributed on an intermittent basis – i.e. on some days but not on others, depending on the availability of mothers to prepare it. We also came across cases were the responsibility falls heavily on certain mothers who are forced to prepare the meals several times per week, or else the children are deprived of school meals. During the field research, the consultants interviewed a mother who told them that she prepares the meal approximately twice a week (eight times per month) because there are no other mothers who can do it in her child’s grade. As a result, the preparation of the school meal has in some cases become a contentious issue in the relationship between the parents and the school. There have been cases in which parents have instructed their children not to have the school meal in order to dissociate themselves from the responsibility of having to prepare it. The underlying shortcoming with the MINED’s assumption is the traditional conceptualization of the role of women: the housewife should have the time to carry out a series of activities, regardless of prior workload and responsibilities. In addition, this assumption does not take into account mothers who work outside of their residences and therefore do not have the flexibility Mid-Term Evaluation Project CREAN – World Vision 40 or availability to meet the responsibility of preparing school meals. This assumption also overlooks mothers who are sick, live far from the school, or those who have several children under their care. Much research on the role of women in rural contexts indicates that mothers already have multiple responsibilities and long working hours, excluding reproductive work. There is also an expectation that the mothers responsible for preparing the school meal complement the food with other products such as fresh juice, fresh cheese, plantains, etc., according to their financial capacity. In some cases, especially in rural areas, mothers are reluctant to accept responsibility for food preparation because of the extra costs this activity adds to the family budget, which is already extremely limited. This problem has been identified not only by teachers and parents, but also by the children themselves, who have expressed the desire that “everyone receive school meals”, and that “parents should change their attitude about preparation of the school meal”. The allocation of responsibility for the school meal should not be an obligation, and should not create tension and conflict in the relationship between parents and the school. This evaluation acknowledges the preparation of the meal is beyond the scope of the project; however, developing strategies to help address the issue should be considered. In some schools, parents have opted to collect extra money to pay a person assigned for daily food preparation. Still, this model needs to be refined and alternative solutions considered in cases of sickness or leave. Control of food consumption: CREAN controlsfood consumption on the basis school attendance: i.e. the food consumed should reflect the level of student attendance. However, a number of challenges have been identified associated with the use of this control mechanism. Teachers told us that at the beginning of the project the control process was burdensome. They needed to manually prepare a separate attendance list for school meal control, which not only took time but also represented an additional burden on teachers with an already high workload. However, this was solved thanks to the provision of computerized assistance lists by CREAN. This has helped speed up the process and save time and effort. As the project continues to make progress in its implementation, periodic checks of attendance lists must be made in order to avoid mismatches of enrollment. As mentioned before, not all grades prepare the school meal every day, which has in some cases created a food surplus. This evaluation has made inquiries about how different schools manage this situation. In some cases, the food is redistributed and served either as larger portions or double portions. The team also identified cases in which the surplus food was distributed to some of the poorest families in the community. The evaluation also identified some worrisome practices, particularly in one school, where it has been reported that food is discarded at a nearby landfill for fear that the surplus would prevent the school from receiving the subsequent distribution, or would receive less. Mid-Term Evaluation Project CREAN – World Vision 41 While in some schools the problem is a food surplus, others have reported having insufficient food for the distribution period. One of the centers visited reported being left without food for two weeks before the next distribution schedule. To address the problem, parents decided to make a collection to raise the funds to provide the school meal for those two weeks. This same center has stated that the food they received during the second distribution of 2016 will not be enough to reach the end of the year. The school has started conversations with a nearby pre￾school that has surplus food to donate some of the surplus. Despite what the field monitors have said on the matter, the problem has not been addressed. The main reason for this problem has been the difference in school enrollment. Although CREAN collected information regarding school enrollment at the beginning of the project, the numbers have varied during the implementation period. These issues emphasize the need to revise school enrollment periodically to ensure their accuracy. Taking into account the number of schools receiving daily meals, the same situation could be occurring in other schools. Besides, at least two of the schools visited mentioned fluctuations in the amount of food received between the first and second distribution. It remains unclear why these fluctuations occurred. School meal distribution: The distribution and consumption of school meals was observed during the school visits. In all schools visited, children had their own cutlery (dish, spoon, and cup) for the school meal. In several cases, children were observed washing their hands before the meal and brushing their teeth afterwards. Due to a lack of adequate infrastructure at the school level, the meals are usually consumed in the classroom. This practice is not necessarily conducive to maintaining standards of hygiene and cleanliness. It has also been observed that in certain schools dirty dishes are stored in a bucket, which is not washed everyday, undermining efforts to promote good hygiene and health practices. These problems continue to be observed due to several reasons. Teachers cannot be held responsible for cleaning the dishes. Notably, teachers have undertaken extra responsibilities associated with the school meal: they need to help get children ready beforehand (washing hands, collecting school materials, and arranging children to receive the meal) and assisting after the meal (brushing teeth, classroom cleaning, etc.). In addition, there are schools that do not have the facilities needed to clean the dishes (washing station, sponge and soap or detergent). In schools where access to water is limited, the situation is further complicated. Activities of the Education component We made direct observations regarding the presence of books and reading materials, reading corners, traveling backpacks, and other educational material available for teachers. Interviews were also conducted with key informants at various levels to gather relevant information. Reading corners: The project target was to establish 637 reading corners (1 per school) during the three-year project (200 first year, 300 second year and 113 last year). However, in 2016 the Mid-Term Evaluation Project CREAN – World Vision 42 Semi-Annual Performance Report FY/16 (SAPR) reported there were 858 reading corners established, which represents 35% more from the original target. The majority of schools visited had received reading materials – both textbooks and storybooks. In some cases, reading materials were located in the classroom and were easily accessible to children. The team observed some very colorful and artistic reading corners. In other schools, the books were in the principal’s office or in the library. Some schools were very cautious about the use of books, as they were concerned that they could get damaged or lost. This is a normal, as books are considered an important school asset, but it prevents students and teachers from benefiting from their use. Several schools have also mentioned difficulties in storing the books, as they do not have the shelves to do so (one of the reasons why books are not available in the classroom). In the majority of cases, students demonstrated that they knew the titles of books, and commented extensively on classroom reading activities and their favorite books. While this evaluation did not examine the benefits of each type of book, we did find commonalities across children’s favorite books. In most schools they mentioned their preference for colorful storybooks and students where able to comment and discuss the content of the stories. Positive comments were also made about the types of books selected, which include both local and foreign authors. MINED authorities said the books selected met the guidelines established by the ministry on themes and authors, which has facilitated their use in the schools. Traveling backpack: Thisis one of the most successful project activities. The backpacks have been positively perceived by the students, with teachers and parents mentioning the wider benefits of the activity. The traveling backpack allows children to take books home, and promotes reading among other members of the family. The traveling backpack has reinforced the bond of trust between the teacher and the student through the loan of a book that is taken home and then returned to school – students are trusted to preserve the original conditions of these books. Several parents have reported reading along with their children, and similarly several students have mentioned that the activity has been an opportunity to spend some quality time with their parents and siblings. The project had a goal of 540 traveling backpacks; however by 2016 SAPR reported over 1,010 backpacks had been distributed. Despite the significant increase in numbers, the main limitation of the traveling backpack continues to be their quantity – the limited number of backpacks and books per grade has made it difficult for more children to benefit from them. Considering that a child borrows each book for between three and seven days, it is easy to understand why the most popular books are not easily accessible. Mid-Term Evaluation Project CREAN – World Vision 43 Distribution of books and supplementary reading materials: In 2016, CREAN distributed 28,715 books. The target for the three-year project is 1,000,000. To date, this indicator is only 2.8% of the way to completion. The limited distribution of books and reading materials – when compared to overall objectives – can have a detrimental effect on reading corners and traveling backpacks. Students are interested in reading but the materials might not be available. It is highly unlikely this activity will be completed by the end of the project. The purchase and distribution of one million books will require significant effort, financial and human, and a grand￾scale logistical process that is simply unattainable considering the project activities and structure. Supplies and school materials: These are distributed to teachers in the form of a kit and according to teachers have been very helpful. We observed that most classrooms were decorated and provided a festive, colorful environment. Several teachers mentioned the materials in the kit have helped in decorating the classroom, increasing the students’ engagement. The kit has also been seen as a financial relief for teachers; several of them mentioned having to spend their own money to decorate their classrooms. Although the MINED has a budget line dedicated to these kinds of expenses, the government does not have the financial capacity to dedicate those scarce resources to all schools and classrooms. Teacher training: The teacher training sessions provided by the project have been well received both at the institutional level by the state and municipal authorities, and by schoolteachers and principals. Municipal and state authorities have reported these training sessions have provided teachers with the skills necessary to develop methodological strategies to improve teaching, which have helped improve the learning environment. Both teachers and MINED authorities also provided suggestions on how to improve or strengthen this component. Both groups mentioned the importance of conducting a more detailed teacher training needs assessment; for example: refresh courses to reinforce mathematics and language skills, and providing specific training for multi-grade teachers. It was also suggested that CREAN field officers participate in MINED’s workshops on evaluation, programming and training so that they could share the progress of the project and learn about the activities conducted in each district. Authorities also recommended expanding the scope of the project beyond early childhood education to include higher elementary school grades, as the project risks losing the important progress achieved at an earlier stage once students transition into the higher grades. Mid-Term Evaluation Project CREAN – World Vision 44 Activities of the Health and Nutrition component Health and hygiene clubs: According the SAPR, by September 2016 the number of established health and hygiene clubs accounted for 85% of the target goal. We had the opportunity to talk with members of the hygiene club and to observe some of the practices implemented in schools, mainly hand washing and oral hygiene. These clubs have been widely accepted by children and are considered to be another instance of the project’s successful activities. The clubs are seen as innovative and their child-centered methodology, which is based on peer-to-peer training, has helped achieve noticeable benefits. This methodology has been widely accepted by students, teachers and MINED itself. Students from the different schools visited were aware of the health and hygiene clubs and showed knowledge regarding different hand-washing practices. They also showed knowledge about tooth-brushing practices. The students mentioned sharing this knowledge with parents and siblings, extending the benefits of the activity beyond the school premises. Despite these successes, several of the schools visited did not have hand-washing stations, which causes children to get wet, creates mud around the tap, and wastes significant quantities of water. Hygiene kits: Individual hygiene kits have been distributed and are used to support the implementation of the hygiene habits promoted by the health and hygiene clubs. In most cases, students have reported keeping their respective kits in the classroom; however, in schools that lack the space or facilities necessary to do this, students keep the kits in their backpacks and bring them to school every day. Although schools and students recognize the benefits of hygiene practices as a path to a healthier life, concerns have been voiced about excessive water consumption, particularly in schools where access to safe drinking water remains an issue. Likewise, in certain schools teachers asked what happens after the kit supplies run out. For many rural families living in poverty, the purchase and provision of these supplies are out of their reach. In several schools visited, teachers and students mentioned the kits were distributed one day before the team of consultants had conducted the evaluation. This raises questions about the seriousness of the implementation of this particular activity in all schools. School gardens: Based on data from the SAPR and the IPTT, 22 school gardens and 24 plots were established in 2016. Several of schools visited had already established gardens or had received the materials and items necessary for establishing them. However, the condition of the gardens is far from satisfactory. Mid-Term Evaluation Project CREAN – World Vision 45 There are several factors that influence the establishment of a successful school garden. One important factor is access to and quality of land. Several schools mentioned the land on which the school is located is poor for sowing seeds, while other schools noted that they do not have adequate space. Another important aspect is the limited available time that parents and teachers have to get involved in tending to the school garden. Several schools have set up a garden committee (different from the CAE), which is made up of teachers, students, and parents, but the latter have mentioned considerable time constraints to fully engage in proper gardening. Teachers, who are busy with other school activities, do not have time to dedicate to the school garden. Another major challenge has been establishing school gardens in dry areas with limited access to water. Water restrictions have forced schools to ration its use. In these cases gardens have understandably been a low priority. Construction/rehabilitation of WASH infrastructure: Major delays have been associated with the implementation of this activity. Although an initial process of infrastructure assessment and data collection was carried out, this process has had to be reviewed since each school has a different layout, and therefore requires different interventions and levels of investment. Schools’ needs for WASH infrastructure are much greater than the budget available for addressing these needs, and so the project team will need to prioritize its interventions. As the project starts implementing this activity, it is imperative to differentiate between cases when a rehabilitation of the current facilities is warranted, and others that require new construction. For example, although it may be possible to repair the door or roof of a latrine, it is important to consider the overall life cycle of the latrine; a latrine whose use is to be discontinued soon may not merit rehabilitation. Water filters: We observed the presence of water filters in selected schools. Students demonstrated knowledge regarding the use of a water filter and were aware of the filtering process. While in some schools filters are filled with water accessed through the wells or from other sources, in schools without access to water, children are asked to bring a small water bottle from their homes. This water is filtered at the school and distributed among students. Although schools with access to potable water do not receive filters, at least one school with both potable water and water purifying filters was identified. It is important that the project carry out a detailed needs assessment in order to optimize the use of resources. 3. Sustainability and evidence of change Sustainability This mid-term evaluation has assessed the sustainability of the project and the extent to which PINE-MINED is committed to the project and to its eventual success. Mid-Term Evaluation Project CREAN – World Vision 46 While the project is on the path to achieving its goals, it is unlikely that it will accomplish the planned sustainability, mainly due to the short life cycle of the project. All the interviewees agree that MINED is highly committed to the project activities, especially with regard to the school meal. This can be evidenced by MINED’s continual support to the project implementation. However, limited financial resources at the country level could make it difficult for the government to cover the project intervention area. It should also be noted that uptake of other components of the project (mainly Education, Health and Nutrition) by MINED is unfeasible due to the lack of available resources and personnel. It should be emphasized that sustainability, especially in education, is subject to a process and not a project. Processes of this magnitude – which require the uptake and implementation of certain strategies and behavioral changes – require a ripening period longer than the three-year life cycle of this project in order to become successfully institutionalized. Intervention area This evaluation has identified that the greatest benefits of the activities carried out by the project are found in schools in rural areas characterized by a larger part of the population living in poverty with low levels of access to public services. These schools have limited economic and material resources and parents often cannot cope with educational needs beyond those covered by the school. Schools located in the department of León (in the municipalities of Achuapa, El Sauce, Santa Rosa del Peñón, and Jicaral) have benefited greatly from their participation in the project. However, the same level of benefits has not been identified in schools in the municipalities of Estelí and La Trinidad. These locations are characterized by schools located in urban and peri￾urban areas, with populations that have higher incomes and higher levels of access to public services and markets. The schools in these areas have the option to select which aspects and activities of the project interest them the most, and to focus on those of particular interest. Children in these schools have greater access to different educational materials at home; they therefore value the resources provided by the project less than rural counterparts. In total, 231 schools and approximately 19,450 students (roughly 50% of the total beneficiaries) belong to these two municipalities. If the project is extended to a second phase, it should reassess its support to urban schools, where the same levels of support and investment produce limited benefits. Instead, the project should focus its efforts on rural schools where the needs are higher and the benefits of the project activities are more noticeable, and where families do not have access to the same financial or material resources needed to support education as those in urban areas. It should be mentioned that CREAN operates in the department of Estelí due to a proposal put forward by the NGO Fe y Alegría prior to its withdrawal, and not because of its original implementation strategy. Mid-Term Evaluation Project CREAN – World Vision 47 Evidence of change Since the project has only effectively been underway for one year, the short timeframe limits the ability to gauge mid-and-long-term effects on project activities. The short life cycle of the project has raised concerns from different stakeholders about the project’s ability to generate significant behavioral changes – both at the individual and institutional levels. However, in the course of this evaluation, it has been possible to identify a series of changes associated with the implementation of project activities. Although it is still too early to assess the impact of each of these activities and their relation with each other, this evaluation represents an opportunity for the project team to identify which activities have generated the greatest impact and which ones, although important, could be implemented either in a second phase of the project or by other projects. School meal School meals have been identified as a key factor in helping improve student retention and higher attendance rates. Teachers and PINE-MINED authorities have stated that they have seen improvements in student retention. Both report cases of children with full school attendance, which they attribute to the provision of the school meal. There is widespread recognition that school meals have helped mainly those children who come from families living in poverty. These children have found in the school meals a solution (albeit partial, though no less important) to their nutritional needs. For these families, the provision of school meals represents a financial relief and becomes a clear incentive to send children to school. In some schools, the logistics necessary around the provision of school meals has served as an excuse to more strongly link parents to school activities. Education The strategies implemented by the Education component have been well received by all the schools that participated in this mid-term evaluation and by MINED authorities at the state and municipal levels. The school principals and teachers who contributed to the evaluation also noted the benefits of the different activities implemented by the project and the progress made as a result. The combination of reading corners and traveling backpacks has allowed teachers to develop literacy promotion strategies – such as asking children to comment about the books they read as part of in-classroom activities, transcribing sections from the books to notebooks, and learning new vocabulary from books – which are beginning to show promising results. In some schools visited, children demonstrated their reading ability. Students have been able to easily identified the advantages of these strategies, with many specifically mentioning that “we read better”, “we Mid-Term Evaluation Project CREAN – World Vision 48 understand words”, “I can read fluently”, “I read faster”, “I have improved my writing”, and, “we do not stutter so much when reading”. All of these improvements in children’s reading ability are indicative of the concrete benefits of the project. Likewise, teachers have acknowledged progress in the levels of reading comprehension, interpretation, and analysis, as well as improvements in the vocabulary, reading skills and fluency. Some schools cited the implementation of a 15-minute daily reading practice. According to the teachers implementing this activity, such approach has not only helped improve reading habits, but also fostered interdisciplinary learning and the promotion of certain values associated with the readings. Additionally, MINED authorities at the state and municipal level have reported that training has allowed teachers to develop methodological strategies aimed at improving teaching, which in turn has helped to improve the learning environment. Teachers have appropriated these teaching and learning strategies and have shown interest in implementing them. Municipal authorities have also mentioned significant improvements in test scoring in the Early Grade Reading Assessments, which they indicate is related to better pedagogical practices. This has greatly motivated teachers as they begin to directly observe the benefits of these practices. Health and nutrition The hygiene club is a good example of how a relatively simple activity, which does not require significant investment, can generate substantial behavioral changes and create positive externalities that extend beyond the school. Students from the different schools visited were aware of the health and hygiene clubs, and showed knowledge regarding different hand-washing practices. They also showed knowledge about tooth-brushing practices. The students mentioned sharing this knowledge with parents and siblings, extending the benefits of the activity beyond the school environment. They were able to clearly articulate the importance of nutrition, hygiene and health practices. In addition, all the schools visited had posters placed in strategic sections of the school showcasing the right way to wash hands or brush teeth, among other hygiene practices. The makeup of the health and hygiene clubs, which includes children of different grades, has also been conducive to promoting the integration of children of different grades and encouraging sustainability in the activity. Several parents interviewed also stated that the activities of the hygiene club have had a positive influence at home, where children have begun to ask their parents to implement certain practices such as washing their hands before eating. The students have also shared their knowledge with parents and siblings, extending the benefits of the activity beyond the school. The project team should consider the institutionalization of this activity at the municipal, state, and national levels. Mid-Term Evaluation Project CREAN – World Vision 49 The same positive effect that the hygiene club has had on the teaching environment, especially among students, has not been seen in other implemented activities such as the school gardens and the construction/rehabilitation of WASH infrastructure. The use of water filters is an activity that has yielded mixed results. In one of the schools visited, teachers acknowledged the direct benefits observed since obtaining the filters. The school suffered from low school attendance because of water-borne bacteria that affected the health of several children. Since the filter arrived in the school, diseases caused by contaminated water have reduced considerably. However, we also identified schools where the use of the filter is not systematic and has not been institutionalized. While the school garden aims to both fulfill an educational function and provide supplements for the school meal, at the time of this evaluation the observed benefits were limited. The CREAN team needs to identify schools in which the experience of establishing a garden has been successful in order to identify prerequisites and best practices that can be replicated by other schools. Regarding the construction/rehabilitation of WASH infrastructure, the project needs to clarify and redefine its intervention strategy. The project has a very limited life cycle and unless rehabilitation and construction activities are started as soon as possible, the objectives will not likely be met. Recommendations This section provides a set of recommendations, based on the findings of the evaluation, to be considered for adoption by the project management. The project team needs to consider their implementation as a way of improving the efficiency and effectiveness of the project. Project management and administration Redesign of certain functions: Job redesign of certain positions should be considered, particularly in the case of field monitors. Job redesign includes the review of tasks and responsibilities, and how these might affect the current organogram. Undertaking such a task could lead to improvements in both productivity and job satisfaction. Appropriate training should support this process so that staff feel ready to take on changes in function and responsibility. Recruitment of technical profiles: In order to fulfill all the objectives of the project, CREAN needs to increase the footprint of its technical staff, specifically in the area of Health and Nutrition. The recruitment of personnel with specific experience in the construction or rehabilitation of WASH infrastructure is fundamental if progress is to be achieved in the remainder of the implementation period. Similarly, the project needs to consider hiring technical personnel in the area of nutrition and school gardens. The hiring of technical staff will help achieve a better balance of the workload in this area and make progress towards objectives. Mid-Term Evaluation Project CREAN – World Vision 50 Creation of an executive council: Weaknesses in communication among the project team were clearly identified during the evaluation. The establishment of an executive council will improve internal communication, which in turn would help strengthen the project team internally. The executive council – composed of senior management team members – will be responsible for discussing priorities, sharing agendas, planning joint intervention strategies, and ensuring that information about project progress and results is communicated to all team members. In order to ensure its success, it is important that the executive council become institutionalized through regular meetings where specific agenda topics are discussed. Establishment of effective communication channels: This is another area that needs further development. It is important that communication is timely and multi-directional. Newsletters and/or email distribution lists can be effective methods for disseminating relevant information to the entire project team. General meetings of the entire project team on a bi-weekly or monthly basis, wherein team members have the opportunity to ask questions and/or express concerns, is another effective method of keeping employees informed and engaged and improving the team’s cohesiveness. Organizational stress management: CREAN is characterized as a complex project with several activities and sub-activities that have experienced significant delays. Those challenges have therefore generated high levels of stress in the implementation team. It is important that the project management acknowledges the problem and develops specific responses to help alleviate the situation. There are several strategies that CREAN could implement to better manage organizational stress: 38 • Provide the necessary materials and equipment needed for work: This is a very simple way to reduce high levels of stress and tension, particularly among the field monitors who only have a single computer for report writing for a team of 12 people. This has not only caused significant delays in the production of monthly reports, but also generated high levels of stress. The project staff need to have the necessary materials and equipment to be able to efficiently do their job. • Staff training should be a priority, especially for managers and coordinators who play a critical role in establishing the organizational climate for the team. Due to the nature of the project, many team members have been asked to accept additional roles and/or substantially modify their initial job descriptions. These employees need proper, comprehensive training to ensure their chances of success. Training programs based on the specific needs of the staff also help improve motivation, and can become an important differentiating component on the part of the employer. • Functional cross-training (different staff capable of performing multiple functions) is also useful in streamlining processes, and enabling role rotation and stress reduction. However, it 38 World Health Organization (2004), Work Organization and Stress. Geneva: WHO. Manning, D. and Preston, A. (2003), Organizational Stress: Focusing on Ways to Minimize Distress. CUPA-HR Journal, Summer 2003, Vol. 54 No. 2. Knoxville: College and University Professional Association for Human Resources Mid-Term Evaluation Project CREAN – World Vision 51 is important that cross-training is not perceived as a threat to jobs, but rather as a mechanism to assist staff. • Establish and promote career development opportunities. According to Manning and Preston: “having a career goal to work toward will help alleviate role ambiguity, reduce turnover, enhance job performance, and encourage other high-performing employees to apply for positions within your institution.” 39 • Flexible work: Although there are certain functions that necessarily require the physical presence of the staff, there are opportunities to implement flexible work schemes that can help better balance the workload while increasing motivation levels. • Other potential interventionsto manage stress include developing reward programs, wellness programs, employee assistance programs, restrictions on working outside office hours or during weekends and others. CREAN and WV must evaluate which of these (or other) options are best adapted to their organizational and team culture and characteristics and, based on this, develop a strategy that helps them better manage stress at the organizational level. Monitoring and evaluation Reduce the number of indicators: We strongly recommend a review of the monitoring plan, and a reduction in the number of indicators that are reported by the project staff. As described in previous sections, data collection on 111 indicators not only has significant implications in terms of time and resources, but it has also exhausted the M&E team’s ability to respond to other internal needs. Indicators should be selected based on their relevance to results, not simply because they are relatively easy to collect. This ensures that the indicators provide data that is appropriate, useful, and relevant to assess progress. Optimize the data collection tools: Specific tools such as the M&E sheets (fichas de monitoreo) need to be reviewed and streamlined. The data sheet, which is very extensive, collects information that is not necessarily useful or relevant. Completing the sheet, which takes approximately two hours, slows the task of the field monitor, affecting the quantity and quality of the data collected. Completing the sheet also creates a high volume of work for teachers, who have expressed resistance in participating in the data collection process. There are clear opportunities to simplify the M&E sheet to generate significant savings in time and effort. The M&E team should assess the type and frequency of data that need to be collected. There are data points that can be collected monthly, and others that can be collected on a quarterly or semi-annual basis. Digitize the data collection process: Modernizing the M&E tools and data analysis will accelerate processes, reduce margins of error, and improve efficiency. One option the project should 39 Manning, D. and Preston, A. (2003), Organizational Stress: Focusing on Ways to Minimize Distress. CUPA-HR Journal, Summer 2003, Vol. 54 No. 2. Knoxville: College and University Professional Association for Human Resources Mid-Term Evaluation Project CREAN – World Vision 52 explore is the use of smartphones for data collection. Smartphones are a better alternative to tablets because they are less visible and flashy, reducing the risk of theft or robbery for those who carry out the data collection process. There are a number of data collection and analysis software programs, which have the ability to be customized and encrypted at different levels, allowing storing and processing of information in a single format in the cloud. This not only reduces margins of error but also prevents problems of information loss or corruption. Some of this software does not require the user to be connected to the internet at the time of data collection, further easing the process of collecting data in rural areas. Intervention area Improve mapping of intervention areas: The evaluation noted that the more rural and remote the school is, the greater the perceived benefits. In the future, projects of this type should concentrate their efforts on rural areas, where the benefits are higher. The ‘dry corridor’, characterized by high poverty rates and population needs, should be prioritized. School meal Digitize inventory control systems in the warehouse: The implementation of barcode technology would facilitate real-time entry and exit of food and the physical distribution of inventory. This technology also minimizes the margin of error, which would reduce losses due to potential data corruption or loss. There are simple methods of inventory control using barcodes that do not require high investments and can be configured manually through Microsoft Excel and a barcode scanner device. The Instituto Nicaragüense de Codificación, which offers coding and advisory services for implementation of the technology, could be an initial point of contact for the process. Consider alternative products to corn flour: It is important that CREAN reviews USDA’s list of products, and assesses options available to identify potential alternative products that are more appropriate to Nicaragua’s food culture. Problems with product acceptance are likely to continue due to entrenched cultural customs and practices. Alternatively, the provision of oil or some kind of extra lard/fat to assist during tortilla preparation could help alleviate the problem. Encourage continuing participation of parents in the preparation process of the school meal: The project should develop strategies that motivate parents to participate in the preparation of the school meal. Recognizing and appreciating parents who have collaborated in the preparation of the school meal, or offering small contributions in the form of food, could serve as incentives to encourage more parents to participate in the process. Provide periodic training on food handling and preparation: While CREAN has developed training to teach people about food handling and how to prepare the cornmeal, this must be continuously repeated to reach a greater number of parents. One mechanism to encourage parental involvement in the activities is through the provision of additional and certified training on specific topics of interest (such as short bakery courses, dressmaking, etc.). This would ensure that parents receive training in food handling and preparation while strengthening their links with both the school and the project. Mid-Term Evaluation Project CREAN – World Vision 53 Provide a cleaning kit for the schools: CREAN should consider the provision of a cleaning kit (detergent, sponge, pail, and rags) to each grade or school to promote the cleaning of school meal cutlery. Improve school infrastructure in kitchens and canteens: This activity is completely beyond the scope of the project, but in the future WV should consider working together with other donors to help improve school facilities in areas where the project is implemented. In many infrastructure projects, donors’ greatest fear is that new facilities will not be properly used. In this case, CREAN could act as assurance that they are used correctly and appropriately through the life of the project. Strengthening the role of CAEs: In response to the problems in food control based on school attendance levels, we recommended that the project team continue the task of strengthening the CAEs as a support mechanism in food control. Strengthening these committees will allow for a gradual process of appropriation of the school meal and thus a greater involvement in the provision process. Strengthening the CAEs will also promote the generation of ideas and specific strategies aimed at the greater involvement of parents in the preparation of school meals. Education Promote the use of reading materials: Many schools restrict the use of reading materials for fear of damage or breakage. It is important to promote their use on an ongoing basis and to ensure that these materials are easily accessible to students. Laminate reading materials and provide bookshelves: The lamination of reading materials, which is not very costly, would help maintain book quality and address teachers’ fears that books will get damaged or worn out. Bookshelves will allow reading materials to be properly stores in classrooms, and make them more accessible to students. Increase the number of traveling backpacks: This is one of the most successful activities; increasing the number of backpacks will increase the number of students benefiting. The benefit of this activity is in the quality of books students are borrowing. The project should ensure that the books are of different types to build letter knowledge, phonemic awareness, vocabulary, fluency and comprehension. This means that the books should be short decodable stories to build knowledge of letters and phonemes accompanied with pictures, pitched at different reading levels according to children’s grades. Build the foundations for more structured teacher training processes: Currently, teacher training is presented as a series of activities and not necessarily a learning process. Shifting to a more structured training process would allow a higher level of specialization, which makes it more attractive to teachers. The teacher training process should also be a space for reflection on the basic competencies required for teachers and how to ensure their fulfillment. Mid-Term Evaluation Project CREAN – World Vision 54 Health and Nutrition Systematize and institutionalize health and hygiene clubs: Systematization, alongside the compilation of practical improvements and lessons learned in the implementation process, will help the institutionalization of the activity. While it may not be possible to institutionalize the activity at the national level, there are opportunities for uptake at the state and municipal levels. Analyze the viability of the implementation of school gardens: In the sample of schools visited, there were currently no clear benefits from the implementation of school gardens. With this in mind, the project should assess the viability of implementing this activity. One way to assess viability would be to develop a set of criteria or prerequisites for garden implementation, such as access to and availability of adequate land for planting, access to water, commitment of CAEs or parents, etc. The project should also identify schools in which the experience of establishing a garden has been successful in order to identify best practices and lessons learned that could be replicated by other schools. Prioritize the construction/rehabilitation of WASH infrastructure: In order for the project to achieve its objectives, it needs to prioritize the implementation of this activity. CREAN can benefit from the school vacation period for the implementation of these activities and thus accelerate the process. Mid-Term Evaluation Project CREAN – World Vision 55 REFERENCES Castillo, M., Elvir, A, and Vijil, J. (2016), Prioridades de la Educación Nicaragüense para el Siglo XXI. Managua: Centro de Investigación y Acción Educativa Social. Education Policy and Data Center (2014), Nicaragua. National Education Profile. Washington DC: EPDC Equipo DME Proyecto CREAN (2015), Estrategia de Seguimiento y Monitoreo, Visión Mundial. FAO (2013), Alimentación Escolar y las posibilidades de compra directa de la agricultura familiar. Estudio Nacional de Nicaragua. Managua: FAO Gobierno de Nicaragua (2014), Ley de Reformas y Adicionales a la Ley N582, “Ley General de Educacion”. Accessed November 2016 http://legislacion.asamblea.gob.ni/SILEG/Gacetas.nsf/15a7e7ceb5efa9c6062576eb0060b321/8524dc0b 4dd5b68c06257cb7004ac2a7/$FILE/2014-03-26-%20G- %20Ley%20No.%20861,%20Ley%20de%20reforma%20y%20adiciones%20a%20la%20Ley%20No.%20582 ,%20Ley%20General%20de%20Educaci%C3%B3n.pdf Ibarra, A., Ramírez, D. and Salgado, B. (2014). Factores socioeconómicos que inciden en los altos niveles de desnutrición infantil en los niños y niñas de 0 a 5 años. León: Universidad Nacional Autónoma de Nicaragua. IFAD (2014), Rural Poverty Portal. Accessed November 2016 www.ruralpovertyportal.org/ar/country/home/tags/nicaragua INIDE, (2011), Anuario Estadístico. Accessed November 2016 http://www.inide.gob.ni/bibliovirtual/Anuarios/ANUARIO11/ANUARIO11/assets/downloads/ANUARIO% 20ESTADISTICO%202011.pdf Manning, D. and Preston, A. (2003), Organizational Stress: Focusing on Ways to Minimize Distress. CUPA￾HR Journal, Summer 2003, Vol. 54 No. 2. Knoxville: College and University Professional Association for Human Resources. MINED (2016), Programa Integral de Nutrición Escolar. Accessed November 2016 http://www.mined.gob.ni/index.php/pine MINED (2011), Guía didáctica para docentes del nivel de educación inicial. Seguridad Alimentaria y Nutricional. MINED: Managua. MINED (n.d.), Educación Primaria. Accessed November 2016 http://www.nicaraguaeduca.edu.ni/principal/educacionprimaria MINSA (2016), Nicaragua ha reducido a la mitad el índice de desnutrición crónica. Accessed November 2016 http://www.minsa.gob.ni/index.php/noticias/329-nicaragua-ha-reducido-a-la-mitad-el-indice-de￾desnutricion-cronica Mid-Term Evaluation Project CREAN – World Vision 56 Molina (2013), Las Condiciones de sequia y estrategias de gestion en Nicaragua. Accessed November 2016 http://www.droughtmanagement.info/literature/UNW￾DPC_NDMP_Country_Report_Nicaragua_2013.pdf Pan American Health Organization (2015), Nicaragua. Perfil de país. Managua: PAHO PINE-MINED. (2010), Merienda Escolar. Managua: MINED PNUD Nicaragua (2011), Informa Nacional sobre Desarrollo Humano 2011. Las juventudes construyendo Nicaragua. Managua: PNUD MAGFOR (2009), Política de seguridad y soberanía alimentaria y nutricional desde el sector publico agropecuario y rural. Accessed November 2016 http://www.magfor.gob.ni/descargas/SeguridadAlimentaria/Politica%20SSAN%20UV%20140509.pdf Rojas et al (2007), Guía de alimentación para comedores escolares. Madrid: Dirección General de Salud Publica. UNESCO (2011), The hidden crisis: Armed conflict and education. Regional Overview: Latin America and the Caribbean. Paris: UNESCO. UNESCO (2007), Urban and rural disparities in Latin America: their implications for education access. Buenos Aires: UNESCO UNICEF, (2016), Niñez y Adolescencia Nicaragüense. Accessed November 2016. http://www.unicef.org.ni/ninez-adolescencia UNICEF (2013), Infancia, Seguridad Alimentaria y Nutrición: Modelo Integrado TUKTAN YAMNI – MUIH BIN MUIHNI YAMNI. Accessed November 2016 http://mptf.undp.org/document/download/12165 UNICEF (2013), Nicaragua. Country Programme Document 2013-2017. Accessed November 2016 https://www.unicef.org/about/execboard/files/2012-PL31_Nicaragua_CPD-final_approved-English.pdf UNICEF (2013), Mejorar la Nutricion Infantil. El imperativo para el progreso mundial que es posible lograr. New York: UNICEF World Health Organization (2004), Work Organization and Stress. Geneva: WHO World Food Program (2016), Nicaragua refuerza la merienda escolar. Accessed November 2016 http://es.wfp.org/historias/nicaragua-refuerza-la-merienda-escolar World Vision (2014) Attachment A Plan of Operation, World Vision. World Vision (2015) CREAN Project Baseline Study. World Vision. Managua World Vision (2016) Evaluation Plan Project CREAN Children Reading and Nourished, World Vision-USDA. Mid-Term Evaluation Project CREAN – World Vision 57 Zúñiga, S. (2015), Estado nutricional, habitos alimentarios y merienda escolar de estudiantes que cursan el 5°-6° grado del Colegio Publico del Poder Ciudadano Experimental México. Periodo Noviembre 2014- Febrero 2015. Managua: Universidad Nacional Autónoma de Nicaragua. Mid-Term Evaluation Project CREAN – World Vision 58 Annex Informed Consent – Focus Group Discussions Getting informant consent consist of two parts: Part 1) Information data sheet; and Part 2) Informed consent form. Part 1: Information data sheet: Child Reading and Nourished (CREAN) Project 1. General information about the research We are working on a mid-term evaluation of the CREAN project implemented by World Vision. The purpose of this study is to better understand what are the project’s strengths and weaknesses and how to improve it. We are asking you to take part in a focus group discussion with approximately 8-12 other people from the community. We will collect some information on your age and background. During the discussion, a facilitator will ask about your opinions about school meals, reading materials and school infrastructure. The focus group discussion will last approximately 1 hour. 2. Voluntary participation & benefits/risks of the study We do not anticipate that participation in this study will pose physical or psychological risks beyond what you encounter in everyday life. However, if you are uncomfortable answering a particular question, you are free to not answer the question. This study will inform our report and will help better understand how to improve the project implementation. 3. Confidentiality We will ask all participants not to share what is discussed with people outside of the group. We will audio-tape the discussion and make a written copy for later analysis. The information obtained from your and other people’s interviews will be used for reports on improving the project. Your name won’t appear in any report/publication and can’t be traced back to you. 4. Withdrawal from the study Your participation is voluntary and you can withdraw at any time without giving reasons and you will not be penalized for withdrawing or questioned on why you have withdrawn. Whom to Contact: If you have any questions on this research, please contact: Marisol Gomez, World Vision Nicaragua Marisol_Gomez@wvi.org or Claudia Pompa, +44 7717 536989, claudia.pompa@gmail.com, in England. Mid-Term Evaluation Project CREAN – World Vision 59 Mid-Term Evaluation Project CREAN – World Vision 60 Part 2: Informed Consent Form: Child Reading and Nourished (CREAN) Project I have read the participation information sheet, or it has been read to me. I have had the opportunity to ask the researcher any further questions I may have had. I understand that my participation in this research is voluntary and I can withdraw any time. I understand that information from me will be used for research reports and I consent for it to be used in this manner. If I have any concerns or complaints regarding the way the research is or has been conducted I can contact Marisol Gomez, World Vision Nicaragua Marisol_Gomez@wvi.org, World Vision Nicaragua or Claudia Pompa, +44 7717 536989, claudia.pompa@gmail.com, in England. By signing below I am consenting to (please tick): □ participating in the focus group □ letting the focus group be recorded □ having my statements referenced anonymously such as ‘youth, x region’ Print Name ________________________ Signature __________________________ Date _______________________________ Day/month/year If under 18 years of age: Name of guardian Signature/Thumbprint of guardian Date Day/month/year Mid-Term Evaluation Project CREAN – World Vision 61 Focus Group Discussion Guide Background Hello and welcome [Introduce self and co-moderator] Thank you for taking the time to join our discussion about the CREAN project, implemented by World Vision. Everyone here today was invited because you share something in common related to the topic. Our goal is to better understand what are the project’s strengths and weaknesses and how to improve it. There are no right or wrong answers, because everyone experiences things differently. We are interested in the full range of experiences, so please feel free to share your point of view even if it differs from what others have said. Discussion group rules Before we begin, let me suggest some guidelines or rules that will make our discussion more productive. • Please speak up—but only one person should talk at a time. We’re recording the session because we don’t want to miss any of your comments. If you have trouble hearing any of the comments, please let the group know. • In the discussion, we’ll be on a first-name basis. In our reports of the results no names will be attached to any comments. Your name will be kept confidential. We are using name badges just to help us remember each other’s names during our conversation. • My role here is to ask questions and to listen. I won’t be actively participating in the conversation, only guiding it. I want you to feel free to talk to the group and not just to me. I’ll move the discussion from one question to the next to try to keep us on track so that we can finish by [insert time]. • Sometimes, people in focus groups think of things they want to say after the discussion has moved on to other questions. If you would like to add to your comments after the group, we will be around to talk with you privately. • Please do not share what has been discussed today with people outside of the group. We want everyone to feel free to speak openly during our discussion together. • Any questions before we begin? Mid-Term Evaluation Project CREAN – World Vision 62 Questions – Focus Group Parents Questions 1. Do you know the activities that the project CREAN has implemented in the school? 2. What do you think of them? 3. Have any of you participated/collaborated directly in any of these activities? How? How was the experience? 4. What has been the reaction of the community in general in relation to these activities? Who has supported these activities? Was there opposition from any group? 5. Do you think that teachers/directors/students/parents/community members have changed their behavior as a result of the implementation of this project? How? o Teaching methodology o Hygiene habits o Reading habits o Learning o Reading and Writing o Nutrition 6. What do you think are the benefits of participation in CREAN regarding: o The nutrition and hygiene of children at the school o Learning o Reading and Writing 7. What changes could be made to improve the implementation of the project? / What aspects could be improved? How? o How could the project improve sustainability? 8. Are there any other comments you would like to add? Mid-Term Evaluation Project CREAN – World Vision 63 Questions – Focus Group Students Questions 1. Do you know the activities that the project CREAN has implemented in the school? 2. Do you think these activities are important? Why? 3. Have any of you participated directly in any of these activities? How? How was the experience? 4. What have you learned up till now thanks to your participation in these activities? 5. What do you think are the advantages/benefits of (choose as applicable) a. Lunch at school b. Reading Corners c. Participation in Health and Hygiene Clubs d. Traveling Backpacks e. Etc. 6. When you learn new things at school in these activities, do you tell your parents, siblings, relatives, and friends what you have learned? 7. How have you put into practice what you are learning? 8. What do you think could be done to improve the activities? 9. Are there any other comments you would like to add? Mid-Term Evaluation Project CREAN – World Vision 64 Informed Consent – Key Informant Interviews Getting informant consent consist of two parts: Part 1) Information data sheet; and Part 2) Informed consent form. Part 1: Information data sheet: Child Reading and Nourished (CREAN) Project 1. General information about the research We are working on a mid-term evaluation of the CREAN project implemented by World Vision. The purpose of this study is to better understand what are the project’s strengths and weaknesses and how to improve it. We are asking you to take part in a key informant interview, which will last approximately one hour. During the discussion, the interviewer will ask about your opinions about the project and the activities it implements. 2. Voluntary participation & benefits/risks of the study We do not anticipate that participation in this study will pose physical or psychological risks beyond what you encounter in everyday life. However, if you are uncomfortable answering a particular question, you are free to not answer it. This study will inform our report and will help better understand how to improve the project implementation. 3. Confidentiality We will ask all participants not to share what is discussed with people outside of the group. We will audio-tape the discussion and make a written copy for later analysis. The information obtained from your and other people’s interviews will be used for reports on improving the project. Your name won’t appear in any report/publication and can’t be traced back to you. 4. Withdrawal from the study Your participation is voluntary and you can withdraw at any time without giving reasons and you will not be penalized for withdrawing or questioned on why you have withdrawn. Whom to Contact: If you have any questions on this research, please contact: Marisol Gomez, World Vision Nicaragua Marisol_Gomez@wvi.org or Claudia Pompa, +44 7717 536989, claudia.pompa@gmail.com, in England. Mid-Term Evaluation Project CREAN – World Vision 65 Part 2: Informed Consent Form: Child Reading and Nourished (CREAN) Project I have read the participation information sheet, or it has been read to me. I have had the opportunity to ask the interviewer any further questions I may have had. I understand that my participation in this evaluation is voluntary and I can withdraw any time. I understand that information from me will be used for research reports and I consent for it to be used in this manner. If I have any concerns or complaints regarding the way the research is or has been conducted I can contact Marisol Gomez, World Vision Nicaragua Marisol_Gomez@wvi.org, World Vision Nicaragua or Claudia Pompa, +44 7717 536989, claudia.pompa@gmail.com, in England. By signing below I am consenting to (please tick): □ participating in the focus group □ letting the focus group be recorded □ having my statements referenced anonymously such as ‘youth, x region’ Print Name ________________________ Signature __________________________ Date _______________________________ Day/month/year Mid-Term Evaluation Project CREAN – World Vision 66 Key Informant Interview Guide Stakeholders Name Organization Position Date of Interview Role in CREAN 1. What is your involvement in the project, and what does it entail? 2. Who are the principal counterparts with whom you have worked as part of this project? Relevance 3. Are the objectives of the project aligned with the priorities of the government and/or other key partners of the project? How so? 4. Do you think the established objectives are achievable and realistic? 5. What do you see as the main factors (internal and external) that help/hinder the implementation of the project activities? How does this affect the achievement of the set objectives? 6. Have strategic priorities/activities been adjusted to respond to changes in context? Effectiveness 7. Speaking generally, how effective are the implemented activities to achieve the project objectives? 8. Are there activities of the project that have seemed particularly successful or challenging? Which ones? Why? 9. Have the format and quality of the activities been appropriate to achieve the objectives of the project? Efficiency and Implementation 10. Do you think that the counterparts have received the necessary support for the implementation and coordination of the project? 11. What are the strengths and weakness of the project team? The structure? The lines of communication? 12. Do you think the implementing team has the skills, knowledge, and attitudes appropriate to develop and complete the expected activities? 13. Up till now, what aspects of the project are functioning well? Which are not functioning as well? 14. Are there aspects that could be improved? Impact 15. What do you think are the main benefits of the project? What are or will be the impacts of the project in the short and long term? 16. Are there examples of good practices that have been institutionalized as a result of the project? 17. What do you think are the lesson learned that could be applied to other projects? 18. Are there unexpected positive or negative impacts that the project has already generated? Design Improvements Mid-Term Evaluation Project CREAN – World Vision 67 19. What are the positive aspects of the project? 20. What are the weakest aspects of the project? 21. What changes could be made to improve the implementation of the project / What aspects could be improved? How? 22. Do you have any other comments you would like to add? Mid-Term Evaluation Project CREAN – World Vision 68 Key Informant Interview Guide Staff Name Position Interview Date Role in CREAN 1. Please describe your role in the project 2. What are your main tasks and responsibilities? 3. Who are the main counterparts with whom you work, and how are they related to the project? Relevance 4. Were the activities of the project established on the basis of a needs analysis or evidence of another type? How were the various activities of the project designed and decided upon? 5. Are the objectives of the project aligned with the priorities of the government and/or other key partners of the project? How so? 6. Do you think the established objectives are achievable and realistic? 7. What do you see as the main factors (internal and external) that help/hinder the implementation of the project activities? How does this affect the achievement of the set objectives? 8. Have strategic priorities/activities been adjusted to respond to changes in context? Effectiveness 1. Speaking generally, how effective are the implemented activities to achieve the project objectives? 2. Are there activities of the project that have seemed particularly successful or challenging? Which ones? Why? 3. Have the format and quality of the activities been appropriate to achieve the objectives of the project? Efficiency and Implementation 9. Do you think the project works with the appropriate counterparts? a. Have the counterparts received the necessary support to implement the project? 10. What are the principle mechanisms for implementation and control of the project? a. Do you think the structures and procedures have been adequate? b. What more could be done or changed? 11. Are there internal procedures that have been particularly effective/challenging? Why? 12. Up till now, are there aspects of the project that function well? What are the aspects that are not functioning well? In the following terms: a. Design b. Implementation c. Management d. Internal Controls e. Communication Mid-Term Evaluation Project CREAN – World Vision 69 f. Cooperation g. Coordination/Organization with different project partners 13. What are the strengths and weaknesses of the project team? The structure of the project? The lines of communication? 14. Are there aspects that could be improved? Impact 15. Do you believe that teachers/directors/students/parents/community members have changed their behaviors as a result of the implementation of the project? How? 16. Who do you think are the principal beneficiaries of the project? What are or will be the short and long term impacts of the project? a. Students b. Teachers c. School d. Community e. Participation in CREAN 17. Are there examples of good practices that have been institutionalized as a result of the project? 18. What do you think are the lesson learned that could be applied to other projects? Design Improvements 23. What are the positive aspects of the project? 24. What are the weakest aspects of the project? 25. What changes could be made to improve the implementation of the project / What aspects could be improved? How? 19. Do you have any other comments you would like to add? Mid-Term Evaluation Project CREAN – World Vision 70 Key Informant Interview Guide School Principals / Teachers Name Education Center – Location Position Interview Date Total number of students Type of school Questions 1. What activities have been implemented in the school as part of the CREAN project? 2. What was the process wherein the school became involved in the project? 3. How were the implementation activities selected? 4. Are there activities of the project that seem particularly successful or challenging? Which ones? Why? 5. What have been the main challenges so far in the implementation of the activities? 6. What has been the reaction of the educational community and the community in general in relation to these activities? Who supported the activities? Was there opposition from any group? 7. Up till now, what aspects of the projects have functioned well? What aspects are not functioning well? 8. Are there aspects that could be improved? 9. Do you think student attendance has improved thanks to the activities implemented by CREAN? 10. Do you think that teachers/directors/students/parents/community members have changed their behaviors as a result of the implementation of the project? How? 11. What are the benefits for (select a group) for participating in CREAN: a. Students b. Teachers c. The school d. The Community 12. What changes could be made to improve the implementation of the project? / What aspects could improve? How? 13. Do you have any other comments you would like to add?