PAISANO 2015 Midterm Evaluation Report Table of Contents Table of Contents ...................................................................................................................... i Acknowledgements ................................................................................................................. iii Acronyms .................................................................................................................................. iv Glossary .................................................................................................................................... vi Executive Summary .................................................................................................................vii I Introduction .......................................................................................................................... 10 II Evaluation Objectives and Methodology ............................................................................ 12 Purpose and Objectives......................................................................................................................................12 Evaluation Team.............................................................................................................................................13 Methodology and Evaluation Process.................................................................................................................14 Process Overview....................................................................................................................................14 Methodology ............................................................................................................................................14 Sample .......................................................................................................................................................15 Analysis .....................................................................................................................................................15 Limitations and Challenges ..............................................................................................................................15 III Findings ............................................................................................................................. 16 Strategic Objective 1: Household Access to Food ...............................................................................................16 Introduction .............................................................................................................................................16 Table 1: Strategic Objective (SO) 1 with Intermediate Results (IR)................................................16 Description of Activities.........................................................................................................................17 Strengths and Weaknesses in Strategic Objective 1 Activities..........................................................18 Strategic Objective 1 Targeting .............................................................................................................23 Figure 1: PAISANO SO 1 Targeting....................................................................................................24 Strategic Objective 1 Conclusions.........................................................................................................25 Strategic Objective 2: Reduction of Malnutrition................................................................................................29 Introduction .............................................................................................................................................29 Table 2: Strategic Objective 2 with Intermediate Results (IR) and Lower Results (LR)...............29 Description of Activities.........................................................................................................................29 Technical Components...........................................................................................................................32 Preventing Malnutrition in Children under 2 Approach Food Distribution..................................39 Strategic Objective 2 Targeting .............................................................................................................40 Strategic Objective 3: Improved Resilience to Food Insecurity..............................................................................40 Introduction .............................................................................................................................................40 Table 3: Strategic Objective (SO) 3 with Intermediate Results (IR)................................................40 Description of Activities.........................................................................................................................42 Table 4: Strategic Objective 3 Targets..................................................................................................42 Technical Components...........................................................................................................................43 Table 5: Summary of Baseline WASH Conditions in PAISANO Project Area.............................45 Table 6: Tracking of PAISANO WASH activities.............................................................................48 Approaches for Strengthening Community Organizations...............................................................50 IV Gender ................................................................................................................................ 50 V Monitoring and Evaluation .................................................................................................. 51 M&E Staffing and Work Structure................................................................................................................51 Figure 2: PAISANO M&E Organigram .............................................................................................51 Data Collection Processes..................................................................................................................................51 Figure 3: SAMI M&E System................................................................................................................52 ii Strengths of M&E ..........................................................................................................................................52 Challenges for M&E.......................................................................................................................................53 VI Commodity Management................................................................................................... 53 VII Sustainability ..................................................................................................................... 54 VIII Conclusions ..................................................................................................................... 56 IX Recommendations .............................................................................................................. 58 Strategic Objective 1 Recommendations..............................................................................................................58 Strategic Objective 2 Recommendations..............................................................................................................61 Strategic Objective 3 Recommendations..............................................................................................................64 M&E Recommendations ................................................................................................................................. 65 Sustainability Recommendations .......................................................................................................................66 General Recommendations................................................................................................................................67 Annex A: Evaluation Protocol ................................................................................................. 68 Annex B: Illustrative Informed Consent Forms ...................................................................... 78 Annex C: Timeline ................................................................................................................... 82 Annex B: List of People Interviewed ....................................................................................... 83 Annex C: List of Communities Visited ................................................................................... 94 Annex D: Annual Survey Indicators – PAISANO ................................................................... 95 iii Acknowledgements The members of the evaluation team extend our gratitude to Save the Children, Catholic Relief Services, Caritas, Asociación de Desarrollo Integral para el Occidente , and Project Concern International program staff, USAID/Guatemala mission staff and management, for all the support received during this evaluation, including interview time, support to field visit planning and logistics, and sharing of project documentation. We also thank Government of Guatemala officials for taking the time to share their perspectives on the project. While it is not possible to thank all individuals who have assisted, the team is particularly grateful to Rodrigo Arias (Chief of Party, PAISANO), Carlos Cárdenas (Country Director, Save the Children), Monica Rodriguez (Chief of Party, Seguridad Alimentaria Enfocada en los Primeros 1,000 Días), Karen Latham (Head of Program, Catholic Relief Services), and Pascale Wagner (Head of Program, Project Concern International). We would also like to recognize our interpreters for their excellent services: Angela Alvarado Chew, Fernando Galindo, Carolyn Minera, Jonathan Porta, Coralia Rivera, Enrique Tabush, and Maria Sol Torres. We also thank the drivers, who carried us safely and on time throughout the fieldwork. Thanks are also due to the Technical and Operational Performance Support (TOPS) Program of USAID/ Food for Peace for funding the support of Monica Mueller (TANGO International) for assembling and editing team contributions and finalizing the evaluation reports. We are indebted to the individuals and families who gave freely of their time and company to be interviewed by our teams. Without their generosity and openness in welcoming us into their homes and sharing invaluable information about their lives, this important review would not have been possible. Evaluation Team Kyla Gregoire (USAID/FFP) Adam Reinhart (USAID/FFP) Suzanne Poland (USAID/BFS) Melanie Thurber (USAID/FFP) Angel Lopez (USAID/Guatemala) Gabriel Santos (Save the Children) Rupert Best (Catholic Relief Services) Claudia Flores (Catholic Relief Services) Elena McEwan (Catholic Relief Services) Flor Idalia Muñoz (Catholic Relief Services) Lucy-Ann Manfredo (Catholic Relief Services) Tom Spangler (Project Concern International) Salvador Baldizon (Project Concern International) Arif Rashid (USAID/FFP), Evaluation Team Leader iv Acronyms ACIC Asociación de Cooperación Integral Cunenense Translation: Cunen Integrated Cooperation Association ADECHIQ Asociación de Agricultores para el Desarrollo de Concepción Chiquirichapa, Quetzaltenango Translation: Farmers’ Association for the Development of Concepción Chiquirichapa, Quetzaltenango ADIPO Asociación de Desarrollo Integral para el Occidente Translation: Development Association for the West ADS Automated Directives System AGEXPORT Asociación Guatemalteca de Exportadores Translation: Association of Guatemalan Exporters ANACAFÉ Asociación Nacional del Café Guatemala BFS Bureau for Food Security CEPROCAL Centro de Producción Caprino del Altiplano Translation: Highlands Goat Production Center CLTS Community Led Total Sanitation COCODE Consejo Comunitario de Desarrollo Translation: Community Development Council COCOSAN/ COSAN Comisión Comunitaria de Seguridad Alimentaria y Nutricional Translation: Community Commission for Food and Nutrition Security COLRED Coordinadora Local para la Reducción de Desastres Translation: Local Coordinator for Disaster Reduction COMUDE Consejo Municipal de Desarrollo Translation: Municipal Development Council COMRED Coordinadora Municipal para la Reducción de Desastres Translation: Municipal Coordinator for Disaster Reduction COMUSAN Comisión Municipal de Seguridad Alimentaria y Nutricional Translation: Municipal Commission for Food and Nutrition Security CRS Catholic Relief Services CSB+ Corn-soya blend plus DIP Detailed Implementation Plan DFAP Development Food Assistance Program ET Evaluation team FFP FFW Food for Peace Food for Work FSN Food security and nutrition FTF Feed the Future GHI Global Health Initiative HAZ Height for Age Z Score GMP Growth monitoring and promotion v GoG GTQ Government of Guatemala Guatemalan Quetzal (Currency) IPTT Indicator Performance Tracking Table IR Intermediate result LR Lower result MCHN Maternal and child health and nutrition MIS Management Information System M&E MoH Monitoring & Evaluation Ministry of Health MSPAS Ministerio de Salud Pública y Asistencia Social Translation: Ministry of Public Health and Social Assistance MT Metric ton MTE Midterm evaluation MUAC Mid-upper-arm circumference MYAP Multi Year Assistance Proposal NP Nutrition Promoters OMAS Oficina Municipal de Agua y Saneamiento Translation: Municipal Water and Sanitation Office PAISANO Programa de Acciones Integradas de Seguridad Alimentaria y Nutricional del Occidente Translation: Integrated Food Security and Nutrition Program in the West PCA Program Constraints Assessment PCI Project Concern International PEC Programa de Extensión de Cobertura Translation: Expansion of Coverage Program PHAST Participatory Hygiene and Sanitation Transformation PLW Pregnant and lactating women PM2A Preventing Malnutrition in Children under 2 Approach POU Point-of-use PROMASA Maya Food Security Program RFA Request for Application SAM Severe Acute Malnutrition SAMI Sistema Automatizado para el Manejo de Información Translation: Automated Information Management System SBCC Social and Behavior Change Communication SC Save the Children SEGAMIL Seguridad Alimentaria Enfocada en los Primeros 1,000 Días Translation: Food Security in the First 1,000 Days SESAN Secretaría de Seguridad Alimentaria y Nutricional Translation: Local Surveillance and Warning System SILVYA Sistema Local de Vigilancia y Alerta Translation: Local Surveillance and Warning Systems vi SO Strategic Objective USAID U. S. Agency for International Development USD U.S. Dollar (Currency) WASH Water, Sanitation and Hygiene WHIP Western Highland Integrated Program WHZ Weight for Height Z-score Glossary Spanish English Cuerda Unit of measurement equivalent to 21m2 or 0.0441 ha Líder agropecuario Agricultural leader Masfrijol A Feed the Future project Municipio Municipality Pila Basin; large sink Técnico Technician (usually refers to an agricultural technician) vii Executive Summary In FY 2012, the U. S. Agency for International Development Office of Food for Peace (FFP) issued an award to Save the Children (SC) Guatemala to implement the project, “Integrated Food Security and Nutrition Program in the West,” known by its Spanish acronym, PAISANO.1 SC implements PAISANO with Project Concern International (PCI) in three departments in Guatemala’s western highlands. The project’s overall goal is to reduce food and nutrition insecurity for the most vulnerable rural families in targeted regions. To achieve the goal, SC and PCI aim to increase household access to food, reduce malnutrition among girls and boys under age five, and improve community resilience. PAISANO’s main strategies to improve food security are to improve agricultural productivity and production, enhance farm and non-farm income to improve food availability and access, and improve health and nutritional outcomes of poor and vulnerable households. SC directly implements activities in eight municipalities of Quetzaltenango and Quiché, and PCI implements in four municipalities in Huehuetenango. PAISANO serves 26,500 vulnerable rural families in the region. In July and August 2015, FFP, SC and Catholic Relief Services (CRS) jointly and concurrently implemented midterm evaluations (MTEs) of PAISANO and of CRS’ FFP-funded project, “Food Security in the First 1,000 Days,” (SEGAMIL,2 by its Spanish acronym). The overall purposes of the evaluation were to review the extent to which project delivery matches the original, approved plan; assess the effectiveness of the project approach; and identify implementation or contextual factors that are contributing to intended objectives, results, and impacts. The evaluation sought to extract lessons learned and best practices, inform modifications of strategies that are not serving project goals, and identify activities for scale-up. The design of the joint undertaking of PAISANO and SEGAMIL evaluations allowed cross-fertilization of lessons and best practices across projects. A 14-member, multi-disciplinary evaluation team (ET) reviewed both projects and was comprised of USAID technical specialists and specialists from SC, PCI and CRS, with an equal mix of men and women. The team’s combined expertise encompassed program evaluation, qualitative analysis, community mobilization, agriculture, livelihoods, food security, maternal and child health, nutrition, rural value chains, monitoring and evaluation, social and behavioral change communications, and resilience. The Senior Monitoring & Evaluation (M&E) Advisor of FFP served as team leader for both evaluations. The team spent 36 days in country, with a short break midway through fieldwork. The evaluation employed qualitative methodologies. The mission included introductory meetings with USAID and awardees, qualitative data collection and analysis, two ground-truthing workshops, and debriefing presentations on evaluation findings. PAISANO has demonstrated progress on its strategic objectives and at midterm, appears on track to meet output indicator targets. The project implements several approaches or activities notable for attaining objectives at output level and receiving praise from participants. These include: 1 Programa de Acciones Integradas de Seguridad Alimentaria y Nutricional del Occidente 2 Seguridad Alimentaria Enfocada en los Primeros 1,000 Días viii • Homestead gardens, which the ET assessed to be contributing to nutrition and dietary diversity, and at times to allow households to generate income by selling surplus produce; • Cooking demonstrations, which are well linked to gardening activities and whose lessons appear to be taken up readily by project participants; • Savings and lending groups, which are instilling a culture of savings among members; and • Engaging community members in training and empowerment-oriented activities. These are important successes. At the same time, they should be measured by the project’s higher aspirations for outcomes, impact and sustainability at project, population and system levels. The MTE presents a moment for strategic reflection on what implications the observed presence or absence of progress to date – and more importantly, project strategies and processes – have for the short, medium, and long term. It is an opportunity to adjust mid-cycle, to optimize progress on different scales and timespans. However, PAISANO is only one in a decades-long series of food security projects in Guatemala. Yet despite government and donor investments in the western highlands, the rate of chronic malnutrition remains at an alarming level and continues to rise. The 2008-09 Reproductive Health Survey found that 49.8 percent of all Guatemalan children under age 5 are stunted.3 The rate of stunting among indigenous children was even higher (58 percent).4 The 2014 -15 National Maternal and Child Health Survey reported a slight decline in overall rate of stunting among Guatemalan children age below five years (46.5), but there is no change in the stunting rate for indigenous children.5 The baseline survey shows that more than three-quarters of children (75 percent) under five years of age suffer from moderate and severe stunting. Ethnicity is a key factor associated with food and income insecurity in Guatemala, as are suboptimal infant and young child feeding practices, low dietary diversity, food insecurity, lack of clean water and sanitation, poor hygiene and poor access to health services.6, 7 With regard to infant feeding, both the amount of food young children are fed and the variety of food they are given are inadequate to meet their nutritional needs. The SEGAMIL baseline survey also reports poor dietary practices, sub optimal practices related to infant and young child feeding, and poor water, sanitation and hygiene behaviors. The context has changed and knowledge has increased since PAISANO was designed so the project should revisit its theory of change to achieve food and nutrition security outcomes. 3 MSPAS. Encuesta Nacional de Salud Materno Infantil 2008 (ENSMI 2008-2009). Ministerio de Salud Publica y Asistencia Social (MSPAS), Institución Nacional de Estadística (INE)/ Centro de Control y Prevención de Enfermedades (CDC), Guatemala 2010. WHO Child Growth Standards. 4 2008-2009 ENSMI, cited in USAID/Guatemala BEST Action Plan for Family Planning, MNCH, and Nutrition, Best Practices at Scale in the Home, Community, and Facilities, submitted to USAID’s Global Health Bureau: April 27, 2011. 5 Ministerio de Salud Pública y Asistencia Social (MSPAS), Instituto Nacional de Estadística (INE), ICF Internacional, 2015. Encuesta Nacional de Salud Materno Infantil 2014-2015. Ciudad de Guatemala, Guatemala. 6 Chaparro, C. 2012. Household Food Insecurity and Nutritional Status of Women of Reproductive Age and Children under 5 Years of Age in Five Departments of the Western Highlands of Guatemala: An Analysis of Data from the National Maternal-Infant Health Survey 2008–09 of Guatemala. Washington, DC: FHI 360/FANTA. 7 De Pee, S. and Bloem, M.W. 2009. “Current and Potential Role of Specially Formulated Foods and Food Supplements for Preventing Malnutrition among 6 to 23 Month-Old Children and for Treating Moderate Malnutrition among 6 to 59 Month-Old Children.” Food and Nutrition Bulletin. Vol. 30 (3 Suppl), pp. S434–63. ix Taking this longer view, the ET finds several strategic and operational areas that need modification and strengthening, and acknowledges – as FFP has acknowledged – that PAISANO’s original project design sought to respond a Request for Application (RFA) that was prescriptive, and warrants reassessment. As far as project design, beneficiary targeting is foremost among areas to re￾assess. As noted in Section IV, PAISANO is viewed by communities as a “women’s project.” This stems from the 1,000 Days approach and corresponding emphasis on women. The ET agrees that women need to be a cornerstone of food security approaches. However, the quality of women’s engagement in learning and empowerment processes in the current project needs to be improved. Moreover, the project has not clearly articulated men’s role in these processes, nor is it understood – and thus valued – by communities. In practice, project assumptions on synergies between Strategic Objectives (SO) 1 and 2 targeting do not bear out, as the de facto audience for health and nutrition interventions does not overlap significantly with the one most directly engaged in household agricultural production – whether crops or livestock – and value chain work. Regardless of target group, adult education techniques and topics need to be more demand-driven, more flexible, and more engaging. Learning goals of all project activities need to be clearer to all participants: rations and material handouts are the draw for participation, which suggests that absent an improved understanding of how learning activities add value to current and future daily life, interest in applying project learning will fade when the project finishes. In addition, the program needs a strategy transferring knowledge beyond project participants to the whole community, and for embedding this knowledge in community governance structures. Given its proven role in improving chronic health and nutrition outcomes, the evaluation identified water, sanitation and hygiene (WASH) – particularly community-level WASH – as a strategic gap of the project. Moreover, WASH needs to be part of a larger learning and capacity development strategy that involves government and community structures. The data collection and organization aspects of PAISANO’s M&E system are well planned and well implemented. However, staff and partners do not consistently receive the data and analysis generated from M&E processes, including the baseline study. As a result, project strategies and interventions do not use the information that they should to design, target, and adjust based on needs and achievements in project coverage areas on a regular basis. Continuing PAISANO activities with a clearer eye on sustainability requires stronger consideration of contextual factors such as the role and capacity of Guatemalan institutions such as Community Development Council8 (COCODEs) Municipal Water and Sanitation Office9 (OMAS), health centers and agricultural extension services, as well as the private sector, to deliver the services essential to building and maintaining food security and nutrition. PAISANO needs to include, plan, coordinate and consult more with these entities and with the private sector to ensure that the services currently provided or facilitated primarily through PAISANO remain sufficiently resourced, particularly in the most vulnerable communities, and managed with adequate technical competency. 8 Consejo Comunitario de Desarrollo 9 Oficina Municipal de Agua y Saneamiento x PAISANO and partners also may need to build or strengthened demand for certain services or goods. This will require enhancing community awareness of returns on household investments, such as investment in agricultural inputs (e.g., improved seed varieties) and services (e.g., fee-for-service agricultural extension), or in health care (e.g., investing time and resources to attend pre-natal visits). The MTE finds “graduation” strategies at participant level, and exit strategies at project level, to be either poorly defined or poorly understood. PAISANO needs to work more, both project-internally and with local Guatemalan institutions and the private sector, to strategize how to optimize resource investments in the medium and long term in a context of chronic vulnerability. As implemented, PAISANO suffers to some extent from inclusion error in its targeting practices at the beneficiary level; this is one example of an area where effective use of M&E could assist in decision making. Finally, the project has made important and promising connections with fellow USAID-funded initiatives, such as other members of the Western Highlands Integrated Program (WHIP), as well as with the Guatemalan private sector, especially in agriculture. There are also incipient relationships with entities that can play a role in capacity development in some technical areas after project exit. These collaborations, especially ones where the partner has a vested interest and commitment beyond PAISANO project life, can be an important part of the PAISANO phase-out and exit strategy; here again, meaningful engagement with native institutions in this planning is critical. The potential within PAISANO and in the stakeholder environment exists for consolidating project achievements and ensuring continued progress toward PAISANO objectives, commonly held objectives for food security in Guatemala’s western highlands. The remaining time in PAISANO would be well spent reworking implementation approaches – based on recommendations not only from this MTE but dating back to the baseline – and on solidifying the relationships and competencies in Guatemala that are critical to continued improvements in food security goals in the region. There is also an opportunity to enhance sharing and learning between SEGAMIL and PAISANO given the similarity of the projects and the challenges they face. This evaluation has yielded recommendations – some specific to individual strategic objectives and some crosscutting – in strategic and operational areas. The recommendations cover targeting, service quality, adult learning strategies, synergies across strategic objectives, institutional capacity development, M&E, gender-sensitive programming, phase-out/ exit strategies and sustainability, among other categories. The evaluation team presented and reviewed all recommendations in November 2015 during a three-day workshop in Guatemala City with key staff of PAISANO and SEGAMIL, their corresponding partner organizations, and the USAID Mission in Guatemala. The workshop intended to validate the relevance and feasibility of the recommendations, refine them based on participant feedback, and enable implementers to begin planning actions in response to the evaluation. I. Introduction In FY 2012, the U.S. Agency for International Development (USAID) Office of Food for Peace (FFP) issued an award to Save the Children (SC) Guatemala to implement the project, “Integrated 11 Food Security and Nutrition Program in the West,” known by its Spanish acronym, PAISANO.10 SC implemented PAISANO with Project Concern International (PCI) in three departments in Guatemala’s western highlands. It is a follow-on to its predecessor projects, Maya Food Security Program (PROMASA) I and II. To tackle the increasing rate of chronic malnutrition, the RFA called for a preventive approach focusing on the first 1,000 days between a woman’s pregnancy and her child’s second birthday that should include at a minimum: • Improving infant and young child feeding practices (including early initiation and exclusive breastfeeding, adequate and timely complementary feeding, promotion of improved complementary feeding using locally available and affordable foods, feeding during and after illness, etc.); • Improving maternal nutrition and health, with a focus on pregnant and lactating women as well as adolescent girls; • Improving nutritional status and nutrition awareness among single and newly married adolescent girls and their families; • Improving access to safe drinking water and appropriate sanitation and improving hygiene practices; • Increasing diet diversity and dietary quality at the household level, with particular focus on women and children; • Improving adoption of key essential nutrition, hygiene and health practices through effective use of social and behavior change communication (SBCC) interventions; • Improving prevention and treatment of childhood illnesses in children under five years of age; • Improving access and quality of health services, including family planning services, as well as nutrition services in coordination with partners; and • Improving detection and referral of children under 5 years of age with Severe Acute Malnutrition (SAM). PAISANO’s overall goal is to reduce food and nutrition insecurity for the most vulnerable rural families in targeted regions. To achieve the goal, SC and PCI aim to increase household access to food, reduce malnutrition among girls and boys under age five, and improve community resilience. PAISANO’s main strategies to improve food security are to improve agricultural productivity and production, enhance farm and non-farm income to improve food availability and access, and improve health and nutritional outcomes of poor and vulnerable households. Activities include training farmers and agricultural leaders in improved crop production techniques and natural resource management, creating micro-watershed communities, facilitating savings and lending groups, improving preventative health services and referrals, providing conditional food rations, strengthening community groups for disaster preparedness and emergency response, and 10 Programa de Acciones Integradas de Seguridad Alimentaria y Nutricional del Occidente 12 establishing municipal-level early warning systems. PAISANO aims to ensure gender equality, mitigate environmental consequences, and influence behavior change. PAISANO addresses critical aspects of the USAID Forward initiative. SC and PCI work closely with Guatemalan institutions; with micro-credit institutions to improve access to credit for men and women; and with local partners to whom PAISANO will gradually transfer project implementation responsibilities. After transfer to local partners, SC and PCI will retain a mentoring role and provide continuous technical support to the end of Year 6, ensuring program quality, accomplishment of PAISANO’s objectives, and increased local capacity for sustainability. PAISANO targets rural communities suffering or at risk of suffering from food insecurity, and communities with a high prevalence of chronic malnutrition. SC directly implements activities in eight municipalities of Quetzaltenango and Quiché, and PCI implements in five municipalities in Huehuetenango.11 PAISANO serves 26,500 vulnerable rural families in the region. In July and August 2015, FFP, SC and Catholic Relief Services (CRS) jointly and concurrently implemented midterm evaluations (MTEs) of PAISANO and of CRS’ FFP-funded project, “Food Security in the First 1,000 Days,” (SEGAMIL,12 by its Spanish acronym). This is a report of the PAISANO MTE. II. Evaluation Objectives and Methodology Purpose and Objectives The overall purposes of the MTE are to review the extent to which project delivery matches the original, approved plan; assess the effectiveness of the project approach; and identify implementation or contextual factors that are contributing to intended objectives, results, and impacts. The MTE sought to extract lessons learned and best practices, inform modifications of strategies that are not serving project goals, and identify activities for scale-up. The joint undertaking of PAISANO and SEGAMIL MTEs allowed cross-fertilization of lessons and best practices across projects. The MTE also reviews the implementation processes outlined in the Detailed Implementation Plan (DIP). It assesses whether the program logic reflected in the results framework is likely to result in achieving targets defined in the Indicator Performance Tracking Table (IPTT). The MTE offers prioritized recommendations to improve project effectiveness; the project will use the recommendations to aid in adjusting strategies to maximize project impact on participants. The MTE’s specific objectives are to: 11 In Quiché: Chajul, Chichicastenango, Cunén, Nebaj, San Juan Cotzal, and Uspantán; in Quetzaltenango: Concepción Chiquirichapa, and San Juan Ostuncalco; in Huehuetenango: Chiantla, Concepción Huista, San Antonio Huista, and Todos Santos 12 Seguridad Alimentaria Enfocada en los Primeros 1,000 Días 13 1. Assess the strengths and challenges of project implementation and outputs in terms of meeting terms agreed with FFP and their acceptability and usefulness to the targeted communities, and identify factors that appear to enhance or detract from the efficiency, quality, acceptability, and usefulness of the activities’ implementation and outputs. 2. Present, through qualitative methods and available routine monitoring data, evidence of changes (intended and unintended) associated with project activities and outputs, assess how well the observed changes support the logic of the results frameworks, and identify factors in the implementation or context that impede or promote the achievement of target results. 3. Review SEGAMIL and PAISANO’s plans to adjust programming per the recommendations from the USAID team based on the Assessment of the Western Highlands Integration Program (WHIP) in Guatemala.13 Review the recommendations in line with the findings from the midterm evaluation team and suggest adjustments if necessary. Assess the progress in coordination and collaboration with other actors within WHIP, Department Development Committees, Local Surveillance and Warning System14 (SESAN), Zero Hunger Pact, NutriSalud and Rural Value Chain Program. 4. Review logic of both projects’ results frameworks to assess the hypotheses, risks, assumptions and progress made by using the project service and delivery mechanisms and recommend changes to help achieve the goal. 5. Review the processes for capturing and documenting lessons learned, including feedback from the perspective of stakeholders and beneficiaries. Assess the systems in place to adjust project strategies to incorporate learning. Determine the sustainability of outcomes, systems and services, and the progress towards implementing sustainability strategies. 6. Determine how well the projects considered gender equality with regard to access to, participation in, and benefitting from the activities, and how the choices and focus of activities are gender-sensitive. Based on these objectives, the evaluation team (ET) developed key evaluation questions; the evaluation protocol is in Annex A. Evaluation Team FFP, SC and CRS jointly implemented the MTEs of PAISANO and SEGAMIL. A 14-member multi-disciplinary ET reviewed both projects and comprised of USAID technical specialists and specialists from SC, PCI and CRS, with an equal mix of men and women. The team’s combined expertise encompassed program evaluation, qualitative analysis, community mobilization, agriculture, 13 “Key Takeaways and Recommendations from the Assessment of the Western Highlands Integration Program (WHIP) in Guatemala. January 11-17, 2015” 14 Secretaría de Seguridad Alimentaria y Nutricional 14 livelihoods, food security, maternal and child health, nutrition, rural value chains, monitoring and evaluation, social and behavioral change communications, and resilience. The Senior M&E Advisor of the Office of Food for Peace served as team leader for both ETs. Six translators accompanied the ET to provide simultaneous Spanish-English translation. SC and CRS identified local translators to translate from Spanish to Mam, Quiche and other indigenous languages. The ET had a one-day meeting with the local translators to jointly review and familiarize them with the key questions and tools and to explain the objectives of the questions. Methodology and Evaluation Process Process Overview The evaluation consisted of three stages: a preparation stage, fieldwork stage, and reporting stage. Preparations began in May 2015 and included reviewing the draft statement of work, document review, protocol and tools development, site selection and logistics planning. The evaluation mission took place in July and August 2015. The ET spent 36 days in country, with a short break midway through fieldwork (mission dates July 5-18 and July 26 – August 8). The mission included introductory meetings with USAID and awardees, two ground-truthing workshops, and debriefing presentations on evaluation findings. The ET returned to Guatemala in November 2015 for a three￾day workshop involving staff from PAISANO, SEGAMIL and implementing partners; USAID mission staff, and some staff from SC and CRS headquarters. The workshop aimed to review recommendations, make adjustments as appropriate, and initiate action plans to implement the recommendations. Methodology A detailed evaluation protocol (Annex A) guided the evaluation methodology for both PAISANO and SEGAMIL.15 The MTE used a primarily qualitative approach. The evaluation began with desk review of project documentation such as project proposal, baseline survey report, annual reports, M&E plan, monitoring reports and data, IPTT, Performance Monitoring Plan, food distribution reports, and Feed the Future formative research and barrier analyses. The team also reviewed Government of Guatemala (GoG) poverty reduction and food security initiatives strategies, and the national nutrition plan. During the mission, the ET applied primary data collection methods including semi-structured in￾depth-interviews and focus group discussions with program managers, technical staff, service providers (frontline staff and volunteers), and project participants, as well as some non-participants. Inquiry focused on perceived program effectiveness, constraints inhibiting effectiveness, and suggested means of overcoming these constraints. Key informants included staff of Feed the Future; other USG offices in Guatemala; the awardee and implementing partners at country office and field 15 The Evaluation Protocol contains all interview guides and further details about the evaluation process. 15 levels; WHIP; project participants, indirect beneficiaries, non-beneficiary community members, and community leaders; and GoG representatives at national and local levels. The ET conducted 149 key informant interviews (inclusive of both projects); see Annex D for a list of people interviewed. Fieldwork also included direct observation of project activities such as nutrition school sessions (SC), ECADI sessions (CRS), growth monitoring and promotion, knowledge and technology transfer sessions, demonstration plots, pass-on model demonstrations and other household and community-based activities. Sample The MTE selected a purposive sample of interview sites, representing variation in agro-ecological zones, livelihood strategies, proximity to major infrastructure (markets, roads, towns) and access to resources and services, quality of service delivery, and coverage/intensity of project services. Other considerations for sample selection included physical accessibility and the ET’s ability to visit all sites in the time available for the evaluation. The ET visited 16 communities in nine of the 13 municipalities that implement PAISANO. It visited 10 communities in six of the 12 municipalities that implement SEGAMIL. Annex E lists communities visited. Analysis The ET recorded field notes daily and convened daily to discuss and process emerging findings. The ET thus began preliminary analysis in country during and after fieldwork, vetted initial findings through the ground-truthing workshops, and continued its analysis post-mission using qualitative analytical methods. Limitations and Challenges16 Lack of consistent participation of all ET members was one of the limitations of the evaluation. Since all ET members have full-time jobs and other responsibilities, a number of ET members participated in different segments of the evaluation. This posed challenges to the interpretation of the observations because of varied experience. To mitigate this challenge, whenever possible the ET members overlapped in the field or had meetings to share their observations and interpretations. In addition, the team leader accompanied the new members in the field to ensure consistency in observation and interpretation. Considering the time, the ET could not interview an adequate number of non-project participants. The travel policy of USAID, SC, CRS and PCI defined the time that the ET could spend in the communities. Some of the communities selected were remote and it took significant travel time to 16 See also the notes and comments prepared by CRS, PCI and SC for the discussion of the Guatemala MTE experience with interested FFP officers and staff, held during FFP’s global conference on December 10, 2015. Document title: “Rethinking MTEs – The Guatemala Experience – Panel Discussion FFP Global Conference”. Filename: 121015 CRS PCI Save Consolidated Feedback on Guatemala Midterm Process. 16 reach them. This limited interview time in the communities. Since the ET tried to interview a large number of direct participants, limited time was available to interview non-participants. Getting technical specialists from CRS, SC, PCI and FFP to participate in the evaluation who could commit of his/her time for two months was a challenge. This was not the original evaluation plan; hence, it was not easy to obtain all required technical specialists. Therefore, the ET lacked a number of technical specialists, including gender. Given the timing and the allotted time for fieldwork, and given the availability of ET members with requisite expertise, the ET did not review all interventions promoted by the project(s). The ET prioritized active interventions where the projects made the greatest resource investments, and interventions considered as making a relatively high contribution toward achieving strategic objectives. The ET recognizes that the resulting trade-off was inadequate attention to a number of interventions, including natural resource management. This report includes some observations on these interventions but they are not a major focus of the analysis. Although individual members drafted their own sections, assembling the draft reports has been a challenge; this took longer than anticipated. To address this issue, FFP requested TOPS to help put together the reports. III. Findings Strategic Objective 1: Household Access to Food Introduction PAISANO seeks to complement the provision of food rations and training in improved household feeding practices with activities aimed at improving household agricultural productivity and dietary diversity. In addition to promoting improved agricultural and natural resource management practices, SO 1 activities include promoting women’s engagement in economic activities and support for improved access to agricultural markets (Table 1). Table 1: Strategic Objective (SO) 1 with Intermediate Results (IR) As described in the project proposal, and witnessed by the ET, SO 1 activities in PAISANO target areas face a range of contextual challenges. First and perhaps foremost is that most participating households have limited land holdings. With a reported average land size of 0.5 to two hectares,17 17 Bellmon Analysis (2011). SO 1: Household Access to Food Increased • IR 1.1 Household agricultural production diversified and increased • IR 1.2 Women’s business and leadership opportunities increased • IR 1.3 Access to markets increased 17 most households participating in PAISANO have great difficulty achieving sufficient agriculture production to meet household food needs throughout the entire year. Compounding challenges such as climate change (reduced precipitation and/or late onset of rains); erosion and poor land management practices also limit the agricultural productivity of many smallholder farmers. Finally, traditional cultivation practices (particularly of maize and beans) and frequent labor migration – often for extended periods – tend to limit the adoption of improved agricultural practices among men in participating households. In response to these challenges, SC has worked with its partners to design and implement PAISANO in an effort to boost household agricultural and economic productivity as a means of achieving greater food and nutrition security. In general, activities carried out under SO 1 of PAISANO fall into one of three categories, each of which described briefly below.18 Description of Activities Strengthened Crop and Livestock Production Practices for Household Consumption PAISANO aims to improve crop and livestock production techniques by providing inputs (e.g., seeds, animals and enclosure materials) and technical training to participating farmers; farmers can also purchase irrigation equipment at subsidized cost. Training to individual household participants is typically provided by líderes agropecuarios (agricultural leaders) who are trained by agricultural técnicos (technicians) employed by the projects. PAISANO provides training to program participants in accordance with specific technical instruction modules covering topics such as soil and water management, home gardens, livestock management and improved post-harvest storage. Establishment of Community-based Savings and Lending Groups In a limited number of communities, PAISANO has promoted the establishment of informal savings and lending groups – called Women Empowered groups – among participants as a means of sustainably financing household productive activities and contributing to women’s economic and social empowerment. In addition to providing a mechanism for participating households to accumulate savings on a monthly basis, PAISANO expects that these groups will enable the profitable engagement of members in productive activities, help link successful entrepreneurs with formal microfinance institutions, and promote more equitable decision making among men and women at the household level. 18 The PAISANO technical narrative (p.20) also describes activities under “LR1.2.3 Natural resource management (NRM) capacity increased,” mostly related to reforestation through planting of seedlings. The ET did not conduct extensive review of these activities because SC discontinued tree nurseries during the second half of project. The ET did visit one newly established nursery in a PAISANO community: it seemed to primarily benefit a private school. This raises the question whether targeting was based on need. Given the cessation of the nursery activities under the project, the NRM activities thus far have not had a tremendous impact and do not seem sustainable. 18 Enhanced Income Generation Through Marketing, Agro-Enterprise Skills and Inclusion in Value Chains Existing farmer groups or associations, some of whose members are included in SO 2, are selected for agro-enterprise training based on the members’ access to productive assets (land, water and markets), and previous marketing experience. PAISANO provides training to members on good agricultural practices, brokerage of links with markets and the construction of warehouses where the members’ products can be stored after harvest and bulked for sale. A training curriculum for strengthening the marketing capacities of existing and new associations is in preparation. Strengths and Weaknesses in Strategic Objective 1 Activities Crop and Livestock Production Homestead Gardens PAISANO has made notable progress toward improving the diversification of agricultural production among participating households. One of the key outputs under IR 1.1, cultivation and consumption of native plants and non-native vegetables in home gardens, appears widespread among PAISANO participants. Interviews with several participants and ET observations of home gardens revealed that throughout the PAISANO target area, vegetable gardens are well cared for and households maintaining them are consuming their own produce. In the vast majority of households visited, women took pride in showing ET members gardens where they are cultivating a range of nutritious plants including beets, carrots, chard, radishes, spinach, amaranth, etc. Women consistently report that the household gardens have directly contributed to household dietary diversity and observable changes in child nutrition. Many also claim that the added household production from gardens has enabled them to avoid spending money to purchase vegetables and enabled them to reallocate the savings to other household expenses including clothes, education and health care. The ET found that home gardens integrate with SO 2 in that they provide a tangible means for participants to practice improved nutrition behaviors. Specifically, the relatively small vegetable gardens within the homestead do not substantially increase the work burden of participating women, and they enable them to utilize their own produce in adopting improved household feeding practices such as those promoted through SO 2 cooking demonstrations. While the majority of participants interviewed reported that households used production from home gardens exclusively for household consumption, some stated that they also sell vegetables in local markets if they generate a surplus. A number of respondents claimed to have received training in fruit tree cultivation from PAISANO SO 1 field staff. Those who had received training were appreciative, indicating their belief that fruit tree cultivation will contribute to household nutrition and, if there is surplus harvest, to supplemental income generation. Added benefits to fruit tree cultivation include the low labor requirement for planting and maintenance, and limited land and water requirements. Noteworthy limitations of this activity are that it will take a minimum of three years for trees to bear fruit 19 consistently, and that even mature fruit trees will only meet a small portion of household nutrition requirements. In a limited number of cases, participant households interested in establishing household gardens have been unable to do so. In several cases, the primary reasons given were the limited precipitation and lack of access to water for irrigation. In other cases, implementers told households that they are not eligible to receive seeds because the land at their homestead is too steep and/or the land they would use to establish a garden plot is too far away from the homestead. PAISANO did not take into consideration the area of land that the poorest households can access to practice home garden. To meet the minimum area defined by PAISANO, many households have to use an area that is too far away from their homestead. As with the other SOs, the sustainability of SO 1 activities is uncertain. For example, while home gardens have been a notable success, some stakeholders and participants question their sustainability because the project provides the improved vegetable seeds used in the gardens and may be less accessible in local markets. PAISANO does intend to introduce strategies to increase sustainability. One is to promote extension services by project-trained individuals on a fee-for-service basis; this would expand income opportunities for individuals who, through the training they received under PAISANO, have demonstrated experience and competence in crop and livestock production techniques. Given that the demand for such services will continue beyond the life of the project, this strategy would support the dual objectives of sustaining needed services and expanding livelihood opportunities. Another sustainability approach is to support youth engagement in agricultural livelihoods by establishing the Integral Center of Production and Training on Agro-Ecology in conjunction with the Barbara Ford Center in Santa Cruz Quiche. This initiative had not yet started not yet started at the time of the MTE fieldwork. The ET understands that it will be a tuition-based program; it is unclear what criteria would be used for selecting participating youth, and how much (or for how long) PAISANO would subsidize tuition, leaving an open question regarding its sustainability. A third strategy is the promotion of community vegetable seed and seedling centers since 2014. Community members who own or have access to larger plots of land come together to receive training and some minimal inputs from PAISANO’s technical staff on how to properly grow and multiply the improved, quality vegetable seed and seedlings. These community centers serve as a means for farmers to grow and sell, meeting the increasing demand for seeds and seedlings from other community members. The project did not implement the seed centers in the communities sampled for this evaluation and were not mentioned by staff or beneficiaries; the ET assumes they are implemented on a limited scale. Livestock Production PAISANO is promoting improved livestock production practices. Many participants interviewed demonstrate a clear understanding of the advantages of keeping livestock, chickens and pigs in pens for the purposes of livestock production. The primary reasons given are that enclosing their livestock helps owners to better manage what animals consume, and keeps them from wandering off or being attacked by other animals. Some of the beneficiaries who adopted the practice also reported 20 a basic understanding of the health benefits of animal enclosure, primarily by keeping animal feces separate from young children and food preparation areas. Several participants with animals also noted the benefits of applying goat and chicken manure to agricultural crops, though it was not clear to the ET whether this understanding is attributable to PAISANO activities. PAISANO provides wire fencing, nails and corrugated roofing for animal enclosures to those deemed eligible for livestock support under SO 1. As with other interventions (e.g., water catchment and filters under SO 2), the ET is concerned that this type of support is not targeted to those in participating communities who are most vulnerable to food insecurity. For instance, those currently without animals or the sufficient assets to maintain them (land, fodder, etc.) are not eligible for support for livestock production or other means of livelihood diversification. Based on the interviews and observations in a number of communities, households keep turkey in the same areas where they rear free-range poultry: in home yards and inside of the houses. Some households never mentioned the hygiene objective when asked about the purpose of rearing poultry in enclosures, suggesting that hygiene messaging is not fully effective at reaching all of the beneficiaries. Several PAISANO participants who rear goats have realized substantial benefits beyond the anticipated nutritive value of the milk produced. They specifically mention the improved agricultural production in home gardens and bean plantations resulting from applying goat manure and goat urine, a practice promoted by PAISANO. While the project originally envisioned goat manure as a key element of organic composting activities, the project proposal made no mention of collection and application of goat urine. Farmers who are engaging in this practice attest to its effectiveness for both pest prevention and plant fertilization. In fact, one farmer interviewed claimed that he requested a goat from PAISANO specifically due to the benefits of goat urine he observed on his neighbor’s bean field. Some project participants are profitably selling goat urine to neighbors. PAISANO is actively engaging agricultural promoters in the demonstration of collection and application of goat urine. For instance, a growing number of farmers who have received improved bean seeds through the partnership with the Feed-the-Future (FTF)-funded Masfrijol are effectively using goat urine. CEPROCAL (Highlands’ Goat Production Center), founded during the prior Multi Year Assistance Proposal (MYAP), PROMASA II, plays an important role in the promotion of goats by the project. CEPROCAL has four functions: a) to be a training center for technicians, farmers and students, b) to promote goat genetic improvement, c) to undertake scientific research on goat production and d) to be a processing center for transforming surplus goat milk into industrial food products. The Center is playing a critical role in promoting goat rearing in Guatemala. Youth Engagement in Productive Activities According to reports, SC established 25 youth groups with the intent to support their engagement in productive activities. Youth trained on beekeeping and strawberry cultivation, and they will soon provide training on tilapia production. Project staff estimate that 500 youth are currently participating in the groups. 21 Household-level Linkages with Feed the Future19 PAISANO is effectively coordinating with Masfrijol, a USAID’s Feed the Future (FTF) initiative, in the dissemination of improved varieties of bean seeds to participating households. In several communities visited, the ET observed the distribution and cultivation of improved varieties provided through Masfrijol. Participants note that the improved seeds have higher germination rates, mature more quickly and produce a higher yield than traditional bean varieties. As a result, recipient households report that they dedicate larger areas of land to, and realize greater yield from, the improved bean varieties. The combination of Masfrijol seeds and the application of goat manure and fermented urine are reportedly providing higher bean yields than the combination of Masfrijol seeds and commercial fertilizer. This linkage with FTF has reportedly resulted in both economic and nutritional benefits, as families have greater access to a high-protein, high-value staple food that many PAISANO beneficiaries typically purchase in local markets at relatively high prices. Savings and Lending In Huehuetenango, PCI has established Women Empowered groups in PAISANO target communities, though on a limited scale; these groups reflect PCI’s approach to savings, lending and building entrepreneurial skills. Given the narrow scope of savings and lending activities and the MTE’s focus on other aspects of SO 1, the ET did not select a qualitative sample of communities specifically to assess the effectiveness of savings groups. Accordingly, the number of savings groups interviewed by the ET was limited. According to the latest report containing relevant data,20 “the Women Empowered Initiative continued its steady growth in number of participants (2,302 women and 84 men, organized in 93 groups).” The Women Empowered groups should to serve as vehicles and catalysts for women’s social and economic empowerment. Interviews with current members suggest that thus far, impact in both of these areas is ongoing but limited. Women are still predominantly dependent on their husbands to give them small amounts of money to contribute to monthly savings. Discussions with participating women did not suggest that participation in savings and lending groups has had a demonstrable influence on their social empowerment in terms of increased roles in household decision making or participation in community-level activities. The final evaluation report conducted in April 2015 contains a more focused examination of the effectiveness of the Women Empowered groups.21 Among the groups interviewed, the standard monthly contribution ranged from 5 to 10 Guatemalan Quetzal (GTQ). Few group members interviewed reported taking a loan from the group for productive investments; most reported utilizing their savings for food, school expenses, and/or clothing. While this is certainly a valid use of savings and an effective coping strategy, the poorest 19 See also discussion below under Market, agro-enterprise and value chain development. 20 PAISANO FY 2015 Q3 Quarterly Report July 20, 2015 21 USAID, Save the Children and PCI. “Initial Findings from Women Empowered Initiative in Guatemala (PAISANO)”. July, 2015. 22 households may need more time to make a positive contribution to the economic empowerment of members in the form of productive investments. At the same time, participants expressed great appreciation for the creation of savings groups; they claimed that they previously had no access to savings and that loans were informal and only accessible at relatively high interest rates. Participants stated their expectation that the groups would prove sustainable beyond the life of the project. It appears that the Women Empowered groups are having positive impacts, but the increasing gender equity in household decision-making and on promoting the use of loans for productive investments could increase impact and help transform livelihood opportunities for vulnerable women and households. Market, Agro-Enterprise and Value Chain Development Prerequisite conditions for a successful value chain include road infrastructure, access to markets, access to inputs, capacity for collective marketing, timely access to market price information, and consistent regulation of production and sales negotiation. Without these conditions, prospective entrants to rural agricultural value chains tend to suffer from low (or poor quality) production, poor organization for marketing, and a lack of clear basis for negotiation. Key informants from Asociación Nacional del Café Guatemala (ANACAFÉ) and Asociación Guatemalteca de Exportadores (AGEXPORT) added that in the Western Highlands, farmer-level organization with policy support from community and municipal-level authorities is critical for successfully entering and profiting from agricultural value chains. To overcome the obstacles to value chain participation identified by PAISANO, the project needs to dedicate product-specific technical assistance in group business organization, production and post-production, and marketing, and make farmers aware of the risks associated with commercial agriculture and the ways of mitigating these risks. The project will need to weigh the cost benefit of these additional investments, which might be at the expense of other SO 1 activities. Due to the co-location of Food for Peace-funded Development Food Assistance Programs (DFAPs) and Bureau for Food Security (BFS)-funded market development initiatives, there is considerable stakeholder interest in improving coordination and complementarity among USAID￾funded activities in Guatemala’s western highlands. A primary focus of these discussions has been collaboration between SEGAMIL and PAISANO, funded by FFP, and the BFS-financed Rural Value Chains projects, managed by ANACAFÉ and AGEXPORT, to complement and strengthen the work of SEGAMIL and PAISANO in promoting agricultural value chains.Agreements have been reached on collaboration between PAISANO and AGEXPORT in support of the associations Asociación de Cooperación Integral Cunenense (ACIC) and Asociación de Agricultores para el Desarrollo de Concepción Chiquirichapa, Quetzaltenango (ADECHIQ). PAISANO supports training in supply-chain￾management functions, the construction of the warehouse, computing equipment and brokering relationships with buyers. AGEXPORT provides training in good agricultural practices, accounting and marketing. Apple and bean producer groups seek additional support from the Rural Value Chains Program. 23 The PAISANO proposal describes plans to support participant engagement in value chains. However, representatives from AGEXPORT and ANACAFÉ reported that PAISANO beneficiaries represent a particularly vulnerable and asset-poor segment of the population that is incapable of profitable participation in commercial agriculture. They also added that the level of organization needed to develop a group of farmers to participate in a value chain could be a major challenge given the agro-ecological and socio-economic conditions of the participants. The ET directly observed the agro-ecological and socio-economic conditions in target communities and generally agrees with the conclusion that most beneficiaries are not yet viable participants in agricultural export markets. However, some segments of PAISANO’s target population do have access to assets that allow them to engage in value chains and are doing so. In addition, effective efforts can, and should, promote sustainable, profitable, small-scale involvement of project participants in agricultural and off-farm enterprises that supply local or regional markets. Furthermore, given the enthusiasm demonstrated by some members of Women Empowered groups, this mechanism should be an entry point to the promotion of group or individual income￾generating enterprises. Strategic Objective 1 Targeting SC and PCI conducted geographic targeting at the department and municipal levels based on available data on the severity of poverty and prevalence of chronic malnutrition (stunting) of children; the number of children under 2 years of age and pregnant and lactating women (PLW); and previous programming experience in specific areas. Both implementing agencies then worked with other key actors – SESAN, MSPAS22 and local civil society organizations – to select the neediest and most underserved communities within selected municipios.As part of their overall commitment to address chronic malnutrition among women and children, awardees then developed feasible estimates of the number of targeted beneficiaries (mother-child pairs) as a percentage of the total number of PLW and under-two children in the selected communities (for further detail see household targeting see SO 2 discussion). 22 Secretaría de Seguridad Alimentaria y Nutricional (SESAN) and Ministerio de Salud Pública y Asistencia Social (MSPAS) 24 It is critical to note that targeting of SO 1 activities was based on the assumption that participant households targeted under SO 2 warrant inclusion in sustainable agriculture and natural resource management training and activities under SO 1. The SO 1 targeting strategy reflects an understanding that households vulnerable to malnutrition among PLW and children under 2 years of age are at risk partly due to inadequate food production. It also reflects a deliberate effort on the part of PAISANO to integrate agriculture and nutrition activities. While this shared strategy for targeting SO 1 activities is defensible in theory, the ET found that in practice it creates challenges for promoting improved agricultural practices in target communities. First, since SO 1 targeting centered on those receiving rations under SO 2, the primary participants of SO 1 agricultural activities are, by default, PLW. However, women in rural Guatemala often do not have the primary responsibility for production decisions around maize and beans: they often do not have the time to adopt promoted agricultural practices; moreover, they are not the ones who make such decisions at the household level – the men do. (Women’s influence in agricultural activities tends to be more associated with livestock, horticulture and processing commodities, such as wool). Yet while men in participating households are heavily engaged in agricultural production, they rarely participate in the project’s agricultural trainings because they do not receive a ration or other incentive for doing so, and rarely spend time away from more productive activities. In addition, PAISANO is widely perceived as a project “for women,” which hinders perceptions of its relevance to household and community members by both sexes. Second, the ET met with many food insecure households in target communities – including those with children – that would be interested in participating in PAISANO’s agricultural training (though Figure 1: PAISANO SO 1 Targeting Source: Adapted from PAISANO Proposal Technical Narrative (February 3, 2012). The figures align with those in the project proposal provided to the evaluation team. SO2: 29,226 families receive food rations and comprehensive MCHN support SO1: 17,500 families receive agricultural production support SO1: 4,375 families receive agricultural marketing support Approximately 10% of agricultural marketing beneficiaries will not be SO2 beneficiaries as they will be lead adopters of higher risk agricultural marketing practices 25 it is possible that the primary interest is to receive the ration, given that existing members typically stop participating when they become ineligible to receive the ration). Since those households did not have a pregnant woman or a child under two, they were not eligible to receive assistance under the core, non-value chain, SO 1 activities. Interviews with non-participants in target communities revealed that there is minimal transfer of knowledge regarding improved agricultural practices by project participants; the project does not have any strategies to promote knowledge transfer to non￾project participants. Given the number of topics covered in SO 1 training, the infrequency of visits of agriculture promoters (discussed below), and seasonality, the limited window of time to introduce and review training topics is likely insufficient to ensure sustainable adoption of improved agricultural practices. The ET discovered another problem: demonstrated need was not a factor in the selection of families to receive goats. Rather, the selection was made based on their ability to care for them, e.g., as demonstrated by having sufficient land for constructing a goat pen, sufficient time for collecting fodder, and sufficient experience in animal husbandry to care for goats. These were the criteria reported by both beneficiaries and field staff. This practice warrants further investigation and monitoring: it is contrary to project design and can result in inclusion error whereby less needy households receive more project assistance than households that are more vulnerable. Strategic Objective 1 Conclusions Selection Criteria for Participation in Strategic Objective 1 in Many Cases Exclude Those Most Vulnerable to Food Insecurity The ET observed the distribution of materials aimed to contribute to behavior change under SO 1. While it is a reasonable strategy to give materials to those who can make the most immediate and productive use of them for training purposes, those receiving them were relatively well-off and therefore not the people most in need of material assistance. In addition, poor, food insecure households in target communities were reportedly excluded from SO 1 support because they did not have children under 2 years of age, or, in the case of vegetable gardens and goat rearing, did not have sufficient land to participate. Although having access to land was not a criterion to be eligible to receive a goat from PAISANO, staff, promoters and project participants reported that poor people often did not request a goat because they do not have the means to feed it: the household needs to have the ability to collect fodder. Moreover, PAISANO requires that the goat recipients build a pen, yet the poorest households could not afford to build one. In sum, the current selection criteria result in excluding some of the most vulnerable households, which are in fact the intended beneficiaries of the project. Insufficient Attention to Knowledge Transfer to Indirect Beneficiaries In communities visited, the ET sought to interview non-participant households in addition to participant households. In the majority of cases, the primary reason given for not participating in PAISANO is that the household did not have any pregnant women or children under 2. In others, female heads of household claimed that while they were interested in learning about improved 26 nutrition and agricultural production, they did not have time to attend training sessions. Among non-participant households, respondents consistently claimed that they had little knowledge of the improved practices promoted by PAISANO and had not had any direct interaction with project participants, promoters or staff in order to learn from them. These findings indicate that the project needs to strengthen strategies for ensuring that the people targeted for training do not face constraints to their participation, and/or mitigate these constraints. They also suggest that the project needs to find means to facilitate the transfer of training knowledge beyond training participants to others in the household and community for whom the training would be useful and necessary to make a population-level food security impact. Overly Rigid, Top-Down Approach to Capacity Building The ET’s interviews and observations highlight that consistent, quality engagement of field staff is a key determinant of the impact of SO 1 activities. Presently it appears that PAISANO provides a topically structured training curriculum to all field staff in all targeted communities; by virtue of the curriculum structure and/or the guidance on how to deploy the training topics, there is little flexibility to adapt the training topics or sequence to community need and demand. The lack of a demand-driven approach hampers participants’ interest and take-up, as the topics given are not necessarily the ones they find most relevant. Furthermore, field staff are reportedly notified of periodic training topics only shortly before they are required to deliver training to project participants. This top-down approach and the short notice of topics do not encourage field staff to respond directly to the needs of the beneficiaries or to use their own creativity to tailor trainings to the specific agro-ecological contexts in which the beneficiaries live. It should be noted that some of the interviewed PAISANO agricultural leaders and promoters appear to be highly skilled and motivated. Field observations and interviews suggest that some of the higher-performing agricultural leaders and promoters tailor their trainings to the needs of the community rather than strictly follow the training schedules as presented by the overall project sector coordinators. This flexibility could allow the trainings to respond to participant-identified needs and wishes in a way that a top-down￾managed process could not. An opportunity exists to utilize some of the highly capable staff to train other staff who may not be as strong such as through stand-alone capacity building sessions or by encouraging those in need of skills strengthening to observe the higher-quality trainings. A related constraint to the impact of SO 1 capacity building activities is that field staff are apparently overburdened both in terms of time that can be allocated to project work and to attend adequately to all the households they are responsible to train and visit. Among other causes, this has sometimes led to spotty uptake of project-disseminated interventions, and beneficiaries’ poor recall of what they have learned. Top-down training style that does not take advantage of optimal adult learning methodologies compounds the problem. To that point, in some cases the promoter or agricultural leader, sometimes in concert, appear to simply tell the attendees what to do and then do most of the work themselves, missing the opportunity to help the trainees learn by doing. 27 Integration of SO 1 Activities with SO 2 Activities The PAISANO project proposal describes its intent to integrate activities by “design[ing] and target[ing] so households with children under 2 and PLWs participate in and benefit from as many different health, nutrition, livelihoods, and risk mitigation activities as possible.” It describes strategies to accomplish this such as cross-training of technical staff and community workers to share messaging across sectors with target groups, devising a cross-sectoral SBCC strategy, and building the capacity of COCODEs and COMUDES to develop integrated community plans that ensure stronger linkages between commissions, local NGOs and government ministries across all sectors. The ET observed cross-SO impacts when the project actively promoted integration, such as in the use of home gardens for diet diversification and income generation. In the vast majority of cases, however, community-level respondents seemed to view PAISANO primarily as a project aimed at helping women and children through the provision of food rations, rather than as an integrated development program. The current approach to targeting SO 1 and SO 2 activities presents multiple challenges to effective integration and sustainability at the project level. SO 1 Intervention Strategies Are Insufficiently Tailored to Diverse Community Contexts The activities of PAISANO take place in 187 communities in 13 municipalities across three departments. The agro-ecological, socio-economic, market access and institutional contexts in these areas vary considerably. In SO 1, the base intervention strategies of PAISANO partially take into account these variations to tailor interventions to the specific needs of the communities. This tailoring needs to be extended and deepened when it comes to identifying agricultural and non￾agricultural income-generating opportunities for the target communities. These opportunities will differ according to the natural, social, human, productive and financial asset base of a particular community, its trade relations with surrounding communities, and access to markets within and beyond the community. The current design and implementation of PAISANO value chain activities are effective for enhancing the economic productivity of small- to medium-scale agricultural producers who are relatively food secure; they are less effective for the highly vulnerable because this population is excluded from value chain activities The work on agro-enterprise and value chains in PAISANO is directly impacting a relatively small proportion of the total targeted population (17 percent or 4,375 of the 26,51723 household end-of￾program target; this represents 24 percent of the 17,930 households targeted for agricultural production support). Those farmers involved in the value chains are among the better-endowed households within their respective communities, and enjoy access to national and export agricultural markets that most PAISANO participants lack. The project needs to consider how it can adjust its targeting strategy to effectively support smallholders who face greater barriers to entry to agricultural 23 SC-approved narrative dated June 27, 2012. 28 value chains to optimize their income streams through alternative livelihood activities, e.g., unskilled or semi-skilled labor, providing services locally to agricultural or non-agricultural activities, etc. Effective Use of Market, Enterprise and Value Chain Development for Improved Food and Livelihood Security The PAISANO proposal infers that support for farmer engagement in income-generating activities will enable improvements in food and livelihood security among participating households. While the hypothesized link between increased income through market engagement and household food security is plausible, the ET was unable to confirm, through its interaction with households visited, if this is borne out in practice. It was evident that the majority of PAISANO households are not viable candidates for linking to commercial national or export markets. They do not have the necessary prerequisite conditions (discussed under Market, Agro-Enterprise and Value Chain Development) to participate in a value chain and do not meet the current main selection criterion for participation in value chain activities. However, the market studies undertaken do identify opportunities in local and regional markets where purchase requirements in terms of volume, quality and presentation are not as stringent. These, and other non-agricultural income-generating opportunities, were not explored systematically, could provide options for those households that do not have assets to engage in national and export value chains. At the community level there needs to be a more consistent and participatory process of identifying, assessing and developing market-oriented income-generating enterprises that will be most appropriate, inclusive and sustainable for the communities in question. The project should endeavor to measure whether increased income generated through these enterprises does indeed improve food and livelihood security. Sustainability and Exit/Graduation Strategy Several respondents from non-participating households claimed to have little or no knowledge about the agricultural practices promoted by PAISANO. This strongly suggests that field staff have not effectively encouraged participation in agricultural training activities beyond registered program participants (e.g., men; women who do not have children under 2. Likewise, participants who have received agriculture and/or livestock training under SO 1 have apparently not been encouraged to share what they have learned with their non-participant neighbors. Without a deliberate strategy for involving food insecure households who fall outside of the 1,000 Days targeting criteria, especially men in both participant and non-participant households, it is unlikely that a sizeable number of households in the community will adopt SO 1 activities. This would result in limited demand for critical services and inputs and may not encourage private-sector investments to meet the demand. This is a major constraint to achieving sustainable impact beyond the targeted population. 29 Strategic Objective 2: Reduction of Malnutrition Introduction Strategic Objective (SO) 2 aims to reduce malnutrition among girls and boys under age five. The SO 2 strategy assumes that targeted households and communities will use quality maternal and child health nutrition (MCHN) preventive services promoted by the project, and demonstrate MCHN practices at household level. PAISANO also assumes that when quality of community-level MCHN services improves and the referral system between the community and Ministry of Health (MoH) services strengthens, the use of MCHN preventative services will increase. PAISANO also assumes that when knowledge of and attitudes toward key MCHN practices and household sanitation and hygiene improve, use of MCHN practices at the household level will increase. Table 2: Strategic Objective 2 with Intermediate Results (IR) and Lower Results (LR) SO 2 includes the following specific approaches: • Implement Guiding and Learning by Doing24 and AMIGA (ASPIRE) methodologies to promote the adoption of health and nutrition practices. • As part of Social and Behavior Change Communication (SBCC), implement nutrition schools for fathers and mothers, food preparation demonstrations, and meetings of pregnant women groups to promote health and nutrition practice adoption. • Partner with the University of San Carlos of Guatemala to obtain technical support from three students from the nutrition school to strengthen the quality of field activities while building the capacity of the students. Description of Activities PAISANO works in 13 municipalities of Quiché, Huehuetenango, and Quetzaltenango departments, reaching 187 communities and 10,698 households targeted with MCHN interventions.25 Project staff 24This methodology includes the following steps: learn the family experience regarding a practice and present a new practice; reflect on new practice implementation; support the new practice implementation; and plan for daily implementation of the new practice. SO 2: Malnutrition among girls and boys under five years reduced • IR 2.1: Use of quality maternal child health and nutrition preventative services increased • LR 2.1.1 Quality of community-level MCHN services improved • LR 2.1.2 Referral system between the community and MoH services strengthened • IR 2.2: Use of MCHN practices at household level increased • LR 2.2.1 Knowledge of and attitudes towards key MCHN practices improved • LR 2.2.2 Knowledge of and attitudes towards household sanitation and hygiene practices improved 30 interact directly with local government and local MoH staff to get their support on food security nutrition (FSN) activities in target communities. PAISANO also works with local leaders and volunteers to train and provide them with tools to promote MCHN best practices in target communities and individuals/ households. Under IR 2.1, PAISANO carried out an analysis of the MoH in the target communities, and created a training plan for MoH community staff based on this analysis. The plan includes the Guiding and Learning by Doing methodology to promote health and nutrition practice adoption, which the MoH has incorporated into its new behavioral change strategy, Sown Practices are Reaped. PAISANO trained mother leaders and members of the Community Commissions for Food and Nutrition Security26 (COSANes) in using this mechanism, aimed at increasing the utilization of health services and following up on referral cases through home visits. During FY 2014, 407 cases (209 girls and 198 boys) with danger signs and acute malnutrition were referred to health facilities. Since the end of FY 2014, 842 home visits of referral cases (pregnant or breastfeeding women, as well as boys and girls) were carried out by the PAISANO technical team with the support of mother leaders and COSAN members, to verify nutritional recovery treatment compliance and to promote proper breastfeeding, complementary feeding, hygiene and safe water consumption practices. Under IR 2.2, PAISANO implements several activities: nutrition schools, cooking demonstrations with education sessions, growth monitoring and home visits, pregnant women circles, and household nutrition plans and recipe books. In San Antonio Huista municipality Huehuetenango, PAISANO is using a modified care group approach in addition to activities listed above. PAISANO promotes a “seek services when danger signs are present in neonate” approach, with support from fathers, grandmothers, mothers-in-law or other family members. PAISANO conducted an annual survey in 2014 that showed 89 percent of participants recognize at least two danger signs during pregnancy and 98 percent sought health services when danger signs are present. The recognition of neonate danger signs achieved 71 percent out of planned 34 percent, and the seeking of services when danger signs are present achieved 72 percent out of planned 78 percent. A cultural practice where mothers rest for 40 days without leaving the home during the immediate postpartum period, likely hindered the completion of the latter target. During FY 2014, PAISANO distributed 107,549 rations (1,781 metric tons or MT) to MCHN participants, which represents 72.26 percent of the projected target. Some of the main obstacles for participants not collecting rations for this activity are temporary migration, disease, lack of awareness of schedules and emergencies. PAISANO also recruited and trained 341 Nutrition Promoters (NP), 25 FY 2014 PAISANO Annual Results Report 26 Comisión Comunitaria de Seguridad Alimentaria y Nutricional 31 71 Mother leaders, and two Father Leaders to support PAISANO activities. Nurse Promoters and Mother leaders support the growth monitoring and promotion (GMP) of boys and girls under 30 months of age, carried out every three months (four sessions per year) in all participating communities. The Mother leaders are responsible for counseling mothers/fathers and caregivers, while the Nutrition Promoters organize and coordinate the families around the GMP sessions and provide support during the sessions. Quarterly GMP allows the project to obtain regular anthropometric data and determine health actions (i.e., MoH referral) for optimal MCHN outcomes. GMP currently covers 94 percent of the overall project area (187 of 198 planned communities): 94 percent of Quetzaltenango communities, 92 percent in Quiché, and 95 percent in Huehuetenango. Common reasons for non-attendance included temporary family migration in search of jobs, and changes in the project’s activity calendar. The COSAN promotes attendance by emphasizing the importance of this activity; it is inviting participant families to GMP sessions and in collaboration with the NP, performing joint home visits to families who are not attending them. Nutrition Promoters (NP) conduct home visits to strengthen the learning process and support the adoption of practices through interpersonal communication. To support PAISANO technicians, mother leaders, and promoters, PAISANO designed a series of materials, including nutrition action plans with key behaviors and recipe flipcharts, in order to promote group dialogue and interpersonal communication. In addition, father leaders, mother leaders, and promoters have attended 190 workshops on topics such as skill and ability strengthening to facilitate learning through dialogue, Guiding and Learning by Doing27 methodology, breastfeeding, complementary feeding, and care during pregnancy. PAISANO focused on the following SBCC approaches: nutrition schools for father and mothers (60 established), food preparation demonstrations (5,800 conducted), and meetings of pregnant women groups (77 groups met 426 times). In FY 2014, the nutrition schools reached 73 percent of its target, with a high participation of women. Despite the significant efforts made to engage men, including timing sessions around their availability and home visits to encourage attendance, men’s participation rates remain low. PAISANO is in the process of identifying new strategies so men can engage in the health and nutrition activities. Recently, nutrition schools modified its strategy, disaggregated topics, and increased the duration of the schools to 12 months from the originally proposed six months. During food preparation demonstrations, mothers learned to prepare quality nutritional and high-energy recipes that are micronutrient-rich, incorporating animal source protein, and combined in-kind food (Title II and MoH) as well as produce from their own home gardens. Pregnant women groups met to promote pregnancy care practices, prenatal care, and nutrition for pregnant women; these meeting also provided a space for the women to share their experiences, solve concerns, and be mutually supportive. 27 Learning by Doing methodology promotes action-based learning in a participative and active way; it seeks to aid households to acquire knowledge and skills to implement health and nutrition practices. 32 PAISANO also conducted a study on youth in the intervention area in order to identify needs, interests and potential activities for capacity building opportunities to improve FSN at the community level, and to support ongoing PAISANO activities. The findings28 of this analysis will be the basis of the working strategy design for teenagers, which will be developed and implemented in FY 2015. PAISANO supported safe water consumption through the distribution of 1,484 water filters and conducted 108 trainings to ensure the proper use of filters and promote hygiene and sanitation practices at home. The beneficiaries signed a commitment for responsible management of the filter and payment. The filter unit must be replaced every 24 months. To further engage local partners and increase promotion of MCHN best practices, PAISANO partnered with the University of San Carlos of Guatemala to obtain the support of three students from the nutrition school to strengthen the quality of field activities and provide students the opportunity to gain experience working on an FSN project. Each student assists three communities, where they provide support to the technical team in the development of activities such as pregnant group sessions, nutrition schools, GMP, and home visits to referrals, among others. Technical Components IR 2.1: Use of Quality Maternal Child Health and Nutrition (MCHN) Preventative Services Increased LR 2.1.1 Quality of Community-level MCHN Services Improved TRAINING FOR COMMUNITY HEALTH WORKERS TO BECOME SELF-SUFFICIENT EXTENSION AGENTS PAISANO employs a regional supervisor with health and nutrition expertise who supervises four to six nutrition technicians, who each supervise three to five mother leaders and community Nutrition Promoters. Mother leaders receive a modest stipend and in some cases oversee activities in multiple communities. The technicians provide monthly training sessions for mother leaders and promoters and conduct supportive supervision site visits. The monthly training sessions cover technical topics and provide communication tools for the mother leaders and promoters to use for community promotion activities. The training session observed by the ET was in Spanish because participants spoke multiple local languages. While the technician asked questions throughout the session, the use of participatory methods and adult 28 Some findings: there is an upbringing pattern that creates gender stereotypes; teens are interested in participating in teenage groups; and teens identified actions for their self-development and that of their communities, involving local authorities. An update will be provided in the FY 2015 Q3 report. “Since becoming mother leader, some people refer to me as the light of the community. It’s enough motivation to continue with this work even if I wasn’t receiving a stipend.” 33 education methods were limited. During the training for mother leaders and promoters, the technician used a video played by a laptop. The video content was related to chronic malnutrition and provided some statistics, as well as key messages on chronic malnutrition in Guatemala and why it is important to address. The same videos were used during the nutrition schools discussed below. The use of this medium is an interesting approach, and while the videos are probably more appropriate for advocacy purposes, PAISANO could expand on the idea of using this type of media and perhaps create short videos using skits, role-play, or soap operas that show problem solving, negotiation, and dialogue about malnutrition. The promoter often is a beneficiary and receives a ration as payment for her services. Her main roles are to promote attendance and do outreach for the various PAISANO activities and conduct home visits with the mother leader. Once a promoter’s child turns 30 months, she no longer needs to be a promoter but is able to continue if desired. While the Mother leaders and promoters were highly motivated and clearly made a strong effort to carry out activities, their level of expertise to help problem-solve or negotiate behavior change practices with mothers – particularly women who have health issues during pregnancy or have a child who is ill – is very limited. While home visits and education sessions focused on prevention need, facilitators train in observation, negotiation, and dialogue. Mother leaders know when they see a serious situation they should seek out the necessary assistance for the woman or child. The ET observed clear differences in the abilities and capacities across the varying levels of SO 2 staff and community agents. Providing mother leaders with a small stipend clearly had an impact on self-efficacy of them and on the overall motivation of both mother leaders and promoters. While sustainability beyond PAISANO is questionable, PAISANO should continue to provide financial or commodity-based incentives to community agents. Staff at each level, including promoters, found value in the monthly visits from their supervisors. There seemed to be a strong relationship between the regional specialist, the technician, and the mother leader. One promoter interviewed was in contact with 45 women and conducts five home visits a month. She does not read or write but is able to help the technician translate words from Spanish to the local language when needed. She enjoys her role as promoter and would like to be a mother leader if she had better skills. 34 LR 2.1.2 Referral System between the Community MoH Services Strengthened The cancellation of the GoG rural health extension project Programa de Extensión de Cobertura (PEC) has had negative effects on the access to health services for rural Guatemalans. While the level of function and supplies of the health posts and health centers varied before the PEC, the disappearance of the convergence center where many beneficiaries accessed rural health services such as vaccinations and growth monitoring has been detrimental. Because the MoH is in a transition period and the new government is setting priorities for health and nutrition, the ET is not suggesting any threshold recommendations beyond continuing advocacy for health services for PAISANO beneficiaries and coordinating program inputs in an effort to have sustainable health activities. The health centers visited had a general sense of what PAISANO aimed to achieve, and one health center had a variety of PAISANO communication materials stocked with other materials that are provided to households with children under 2 that come to the health center. This particular health center, in the Nebaj region, had an impressive knowledge of PAISANO activities, and while they did not know the exact communities where PAISANO is implemented, they had working relationships with the PAISANO technician and some mother leaders, and understood that community workers could refer children with acute malnutrition to the health center. IDENTIFICATION AND REFERRALS OF ACUTE MALNUTRITION The technicians and Mother leaders interviewed were well aware of the protocol for community management of acute malnutrition Staff at one of the health centers visited confirmed that they knew the PAISANO technicians screened for acute malnutrition, though they could not cite when the last time a referred case was brought to the health center. In one example of a referral case, the ET interviewed a mother who was referred to the health center because she was told that her child was malnourished, but the health center did not provide the mother with any specialized food. The mother did not understand why her child was referred, since she received no additional services at While several community health workers mentioned they receive training once a month, one Mother Leader interviewed said she had been employed from the beginning of the project, and that she had been trained only four times. She had learned different technical topics (pregnancy danger signs, key messages related to newborn health, complementary feeding, home hygiene, food preparation, food security, and others) and had been trained on how to facilitate dialogue and on different nutritious recipes. This same mother leader had participated in the previous Save the Children program PROMASA II as a COSAN coordinator. When PAISANO started, she expressed interest in becoming a mother leader and was given a test. She receives 500 GTQ/month and works about 98 hours a month. She spends a little over 400 GTQ/month on travel between the two communities she works in, but mentioned that she is motivated by the work even if she does not have much of a profit. 35 the health center. The ET looked at the growth card, which showed the child was stunted – but did not have acute malnutrition, for which the child was referred to the health center. This suggests that there was some confusion on the part of either the Mother Leader/ nutrition promoter who made the referral, or on the part of the beneficiary, regarding what type of malnutrition warrants referral. Even though PAISANO reported that beneficiaries with acute malnutrition were being referred to health centers, the health centers visited in the same region reported they had not received any referred children. These findings indicate a break in communication either between PAISANO and the health centers (e.g., misunderstandings of what type of referral should occur), or between the beneficiary and the mother leader/nutrition promoter. Coverage of MoH nutritionists based at the health center was not consistent, and therefore it was difficult for the ET to have a full understanding of the operational level of MoH services and effectiveness in determining nutritional status of referred children. While PAISANO technicians and mother leaders use Mid-upper-arm circumference (MUAC) to measure for acute malnutrition, once referred, the health center reportedly uses weight for height/length. International guidelines suggest that it is best to use the same admission criteria between community and health center screenings, and that MUAC has been shown to be a stronger tool to show risk of mortality. If PAISANO could advocate for the MoH health services to use MUAC for any PAISANO referral, it may alleviate confusion if a child is referred and turned away because the Weight for Height Z-score (WHZ) did not measure -2 to -3. IR 2.2: Use of MCHN Practices at Household Level Increased LR 2.2.1 Knowledge of and Attitudes Towards Key MCHN Practices Improved NUTRITION SCHOOLS Nutrition schools are facilitated by PAISANO technicians with support from the mother leaders. They are participatory education sessions for up to 20 women with children who have acute malnutrition or severe stunting. Not all communities have nutrition schools; PAISANO technicians are only able to facilitate a maximum of four schools at a time due to time requirements and availability of technicians. At the time of the MTE, there had been 60 nutrition schools implemented. Communities with the highest levels of stunting are prioritized for the schools. If a beneficiary is identified to participate in nutrition school, it becomes a condition to receiving the food ration. Nutrition schools promote the 19 USAID-approved behaviors related to health, nutrition, and hygiene. The PAISANO technician typically chooses the topic of each session from a pre-determined schedule to ensure key PAISANO behaviors are being addressed. “We can think about nutrition in our children like healthy corn. We want the corn ears to be long and full of kernels that make it look wide. We do not want to see red kernels like we don’t want to see red in the MUAC tape. And we make sure the corn is clean for our tamales.” 36 The nutrition schools observed were interactive: the technicians encouraged mothers, and used some adult learning techniques and different media such as video and large images used for games. The nutrition and hygiene education session used analogies to aid understanding, such as comparing human health to growing healthy crops. Technicians taught in the local language, showed respect to the women, and pointed out positive behaviors women were practicing during the session (e.g., healthy snacks, washing hands.). Both the mother leader and promoter are present in the sessions, and the mother leader reinforces topics discussed at nutrition school during cooking demonstrations and home visits. The nutrition school is one of the most promising activities for PAISANO SO 2. Yet despite the quality of the interactions in the schools, beneficiaries did not have a clear understanding of the purpose of nutrition school, did not seem to be empowered to take action related to their children’s health and nutrition, and did not see saw the nutrition school as a defined activity to improve the health and nutrition of their child. Suggested areas of improvement for nutrition schools to have maximum impact and community ownership are: • The overall reach of nutrition schools generally in the PAISANO implementation zone and in the community is limited. The model and quality is impressive, but more women could benefit from this type of interaction, either with preventive or recuperative nutrition communications. • No men were observed attending the sessions. While it is understood that migration is an issue, because nutrition schools target households with already-malnourished children, male participation is extremely important. • If mothers and/or other participants had an opportunity to choose the topic presented during nutrition schools and other activities, it may lead to stronger comprehension and ownership of the topic. There may be other methods, such as those used in facilitated peer￾support group settings, which allow the technician and mother leader to engage in more in￾depth discussions of participant-generated topics with beneficiaries. 37 CARE GROUP APPROACH PCI pilots a modified care-group approach in San Antonio in Huehuetenango (12 communities) to try to address issues related to coverage of SO 2 SBCC activities. In addition to the other PAISANO SBCC activities, PCI uses a Volunteer Counselor to give health and nutrition lessons to groups of 10-12 neighbor women. In reality, these neighbor gatherings tend to be more like community meetings with many people, where the counselor stands and teaches lessons instead of facilitating dialogue in small groups of women. While this approach has potential to improve coverage of preventive and recuperative health and nutrition messages, it is implemented in parallel to other SBCC activities, rather than in coordination with them. Ideally, the PAISANO care group approach would be implemented closer to the care group model guidelines, and link with other PAISANO community actors such as the mother leader and promoter alongside training opportunities for the counselor. This would lead to increased synergy among activities and may lead to increased shifts in community and household behaviors. COOKING DEMONSTRATIONS Cooking demonstrations were one of the most valued activities that the beneficiaries reported participating in. The demonstration, led by the mother leader, uses crops that are promoted in the project. During the demonstration, the mother leader and promoter conduct short education sessions on various PAISANO-promoted behaviors. However, the ET observed that in many instances, the physical space for the cooking demonstration was not conducive for maximum In one community in Nebaj municipality, the ET interviewed two beneficiaries who were participants in nutrition schools. One beneficiary could not recall the previous month’s education session, and was confused at the difference between nutrition school and cooking demonstrations. In another household visited, the ET interviewed two sisters, “Maria” and “Rosie.”1 Maria has three children, one who is under 2.5 years old, and benefits from all PAISANO SO 2 activities except nutrition school. Rosie has one child who is 18 months, and benefits from all PAISANO SO 2 activities and attends nutrition school. Maria was very familiar with PAISANO overall, and had been a beneficiary under PROMASA 2; she could clearly state project objectives, but was unable to articulate why she was not told to attend nutrition school. Rosie was told that she had to attend nutrition school because her son was malnourished. After further dialogue, Rosie exclaimed that she did not understand why her son is malnourished. She said that she follows the PAISANO recipe book, takes her son to the health center for Incaparina (a corn-soy flour mix) and other 1,000-day services provided by the health center, and participates in all of the PAISANO activities, but is told every quarter that her son is malnourished. She attends nutrition school because she has to in order to receive rations, but she said she does not believe her son is malnourished; she said, “He is just small.” 38 impact; there was space for only a few mothers to see the demonstration and help prepare the foods. In addition, most demonstrations observed were using unimproved cooking stoves, which were a distraction and a potential health hazard for the mother leader, promoter, the beneficiaries and the small children watching the demonstration due to the physical and visual discomfort experienced in a room filled with smoke. PAISANO designed a flipchart-style recipe book for all beneficiary households. The cooking demonstrations and recipe books promote recipes that use the food provided as part of the Title II in-kind rations. Participants reported that the project taught different ways to prepare food, including mixing vegetables and animal-source foods with the rations. Beneficiaries reported having recipes for using the Corn-soya blend plus (CSB+) beyond its usual use: some beneficiaries make tortillas and puree, and many responded that they prepare the CSB+ like the local drink atolé, which participants made with salt, sugar, and cinnamon. GROWTH MONITORING AND PROMOTION AND HOME VISITS FOR MALNOURISHED CHILDREN PAISANO conducts quarterly growth monitoring sessions for children beneficiaries who are 6-30 months beneficiaries. The PAISANO technician takes the anthropometric measurements, the mother leader provides counseling, and the promoter ensures attendance. Measurements are taken using project-provided floor digital scales and height boards while the promoter records weight and height measurements onto PAISANO-created growth cards. The cards are large and easy to follow, enabling mother leaders and promoters to easily see growth trends and presumably to allow mothers to understand them easily as well. By measuring both height and weight for age, PAISANO provides an opportunity for mothers to see that growth is not just about weight. It can be very empowering for a mother to understand how to track the weight and height of their child and view how it is recorded. However, mothers are unable to keep their growth cards from the project, even though they hold onto their vaccination/growth cards from the MoH. While the PAISANO growth cards are easier to understand, the mothers are unable to refer to them because the promoter holds onto them. Home visits take place quarterly. The mother leader and promoter conduct the home visits together, targeting 5 to10 families a month. According to project documents, each household should have a household nutrition action plan (based on a selected behavior from a “wheel” of different behaviors); during the home visits, the mother leader and promoter review the wheel and encourage the mother to choose another behavior the household should focus on. Home visits give an opportunity to the promoters and mothers to discuss and negotiate key practices and understand the unique behavior change opportunities specific to that household. Typically, but not observed in PAISANO, home visits include negotiation and dialogue; trust and learning facilitate the adoption of optimal practices. Some households visited did not have the nutrition behavior wheels posted, and in some households that did have them, the mothers did not have the same understanding of the purpose of the wheel and reported never having talked about it with the mother leader or promoter. 39 INFORMATION EDUCATION COMMUNICATION MATERIAL DEVELOPMENT It is evident that PAISANO has devoted appropriate time and resources to develop creative and well-thought-out communication materials. As mentioned, different media such as videos, flipcharts, images, and storybooks provide an interesting opportunity to reach people who have different learning needs, and most households visited did have recipe books and the nutrition wheel showing their nutrition action plans. PAISANO staff reported that technicians and mother leaders received toolkits of all of the materials to use in education sessions and demonstrations; however, many mother leaders and promoters reported not having received the tool kits. Nutrition schools utilized the communication tools, but the ET did not observe them in the interactions between mother leaders and promoters. With the amount of resources and time put into the creation of the materials, PAISANO should ensure their maximum utilization. LR 2.2.2 Knowledge Of and Attitudes towards Household Sanitation and Hygiene Practices Improved Household sanitation was notably absent from the project strategy, despite highly variable access rates from community to community. Please refer to the WASH discussion under SO 3 for specific findings on sanitation and hygiene. Preventing Malnutrition in Children under 2 Approach Food Distribution Across all PAISANO communities visited, the food rations were by far the most valued and important benefit the beneficiaries cited receiving. Rations are distributed monthly and include 11 pounds each of rice, beans, and CSB+; one gallon of oil is distributed bimonthly to avoid giving participants a full gallon at once. The food basket and ration sizes were designed using Preventing Malnutrition in Children under 2 Approach (PM2A) technical reference materials29 and aim to fill 20 percent of the household dietary requirement. While for some households the rations did represent dietary requirements, this varied widely: in some households, the rations lasted for only a short time, while in others the rations were purely supplementary, as they had enough food – even without the ration – to meet their dietary needs. The majority of beneficiaries interviewed reported that the rations last most of the month, and that the entire ration, including CSB+, is shared among family members. The MCHN ration (individual plus household ration) is for supplementary purposes, to provide for a dietary gap. If the rations last through the end of the month, this would indicate the ration is less of a necessity to some households, creating dependence and exceeding the purpose of the PM2A ration that can eventually be provided by the family to ensure appropriate nutrition in the first 1,000 days. While beneficiaries reported paying approximately 13 GTQ for their rations, after further investigation, the ET understood that this money is used to transport the commodities from a central location to their homes. The small amount of money beneficiaries provide was also reported 29 Preventing malnutrition in children under 2 approach (PM2A): A Food-assisted approach; http://www.fantaproject.org/sites/default/files/resources/TRM_PM2A_RevisedNov2010_ENGLISH.pdf 40 going towards the 500 GTQ stipend mother leaders and agriculture leaders receive monthly. While some beneficiaries did report that their contribution went toward transport of the commodities, the ET only learned from project staff that the contribution goes toward the mother and agriculture leader stipend. If beneficiaries understood well that their contribution was to assist their peers, it will help with the sustainability of the activity. Most beneficiaries understood that the rations would end after two years. When asked what beneficiaries would do when the rations ended, replies varied; two common responses were, “I will look for work,” or “I’ll return to work once I’m no longer receiving food,” and “I will have another child,” or “I will become pregnant.”30 Strategic Objective 2 Targeting PAISANO uses the PM2A approach, which targets all women and children within the 1,000-day period for the prevention of chronic malnutrition. While PM2A theoretically targets children up to 24 months, PAISANO provides project inputs to households with children up to 30 months. Beneficiaries did not have a strong understanding of the importance of 1,000 days to prevent chronic malnutrition, which is a key message for mothers and their future children. Other than periodic radio spots with health and nutrition messaging, activities related to improving nutrition sessions are only available to beneficiaries of 1,000-day rations. Beyond being pregnant or having a child under 30 months, and being interested, there are no other eligibility criteria for receiving food rations or attending education sessions. Strategic Objective 3: Improved Resilience to Food Insecurity Introduction Table 3: Strategic Objective (SO) 3 with Intermediate Results (IR) The final version of the results framework for PAISANO is a bit different from the one described in the proposal. In the results framework in use (corresponding to Table 3), the first IR focuses on “increasing community preparedness to manage and respond to adverse shocks.” 31 It emphasizes enhancing good governance and advocacy knowledge, skills and practices of the municipal and community groups expected to play critical roles in reducing food and nutrition insecurity. However, in the project proposal, IR3.1 is stated as “strengthening municipal governments to establish linkages and coordination between municipalities, communities and market actors to build economic stability via market/value chain-based production with the GoG, NGOs and a range of private 30 One woman who responded that she will become pregnant again also noted she has no control over the decision whether or not to have children and cited lack of help for family planning as an issue in the community. 31 PAISANO Results Framework, p. 5. SO 3: Community Resilience Improved • IR 3.1 Community preparedness to manage and respond to shocks increased. • IR 3.2 : Community and municipality governance improved 41 sector business as partners to support sustained economic growth in isolated rural communities.”32 Although the original focus on economic growth was removed from IR 3.1, several activities related to community economic development continue to appear in SO 3 annual detailed implementation plans.33 IR 3.2 is described in the results framework34 as “improving community and municipality governance.”35 In the project proposal, this IR is stated as improving the “community disaster risk management capacity” by mobilizing “communities and institutions to take measurable steps to prevent, mitigate and respond to shocks, both large and small.”36 PAISANO has been working with the three main community organizations that, according to Guatemalan law, are expected to exist and function in each community: COCODEs, responsible for leading community development; COLREDes, responsible for emergency mitigation and response; and COCOSANes, responsible for the reduction of food insecurity and malnutrition. The three groups are expected to function under the leadership of the COCODEs and work in close coordination with each other. In communities where these groups already exist, PAISANO is expected to strengthen their capacity to function effectively by supporting them to adopt good governance practices and advocate with municipal governments to attend community needs and priorities. In communities where the groups do not exist, PAISANO aims to organize and strengthen them. Strengthening these community organizations is expected to promote changes in social rules whereby these groups adopt participatory, equitable and competency-based practices that improve the local decision making processes. These actions are also expected to enhance community resilience by adequately prioritizing needs and risks to be addressed, and by developing feasible plans for mitigation of, and response to, shocks. To complement and respond to community initiatives, PAISANO also works with the 13 municipal governments in its implementation area37 to raise awareness in three main themes: the high levels of food insecurity and chronic malnutrition among their constituencies, the main causes of these problems, and the priority interventions needed to address them in municipal development plans. Within the municipal government, PAISANO works with the Municipal Development Council (COMUDE), the Municipal Food Security and Nutrition Council (COMUSAN) and the Municipal Disaster Reduction Council (COMRED). 32 PAISANO Proposal, p. 40. 33 PAISANO FY 2015 Detailed Implementation Plan. 34 In January 2013, USAID and FANTA hosted a M&E workshop to provide guidance for revisions to the proposal Results Framework, which was used to develop the approved PREP for the second year of program implementation. 35 PAISANO Results Framework, p. 5. 36 PAISANO Proposal, p. 42. 37 SC works with eight, and PCI with five. 42 Description of Activities The mandate of PAISANO SO 3 at the community level is to train COCODEs members on good governance and advocacy principles and skills, food security and nutrition country policies, government-related programs (e.g., Zero Hunger Pact), civic participation and organization, equal participation for men and women, and the COCODEs’ role in the community. In addition to these topics, specific training is provided to the COCOSAN on identifying and addressing causes of food insecurity, logistics management of food distribution, and financial management of the cash contributions made by program participants in exchange for food rations. Additional training is also provided to the COLREDes in mapping community risks and the resources to mitigate those risks, and in the periodic collection, analysis and reporting on key emergency early warning indicators. SO 3 also has a WASH component. By applying the training, communities are expected to increase their awareness and capacity to produce community vulnerability and resource maps, and to prepare development plans to reduce their food insecurity and malnutrition levels using existing resources or by securing municipal and other external assistance. At the municipal level, PAISANO trains COMUDE members on advocacy and good governance practices such as transparency, efficacy, efficiency, conflict management, negotiation, decision making, time management and effective communication. The program also trains COMUSAN members on food security and nutrition, advocacy and the implementation of municipal early warning systems. COMREDes receive training on all topics mentioned, as well as on emergency preparedness and response, increasing community resilience and organizing municipal networks of COLREDes. The program targets related to the above activities are detailed in Error! Reference source not found.. Table 4: Strategic Objective 3 Targets Capacity Building/ Training of Community Groups COCODEs: 198 COCOSANes: 198 COLREDes: 198 COMUDEs: 13 COMUSANes: 13 Train 198 community organization promoters Production of 198 community risk and resource maps, with equal number of emergency response plans, DRR plans, food insecurity reduction plans and economic development plans Establishing sentinel sites for collection and reporting of early warning information Source: SC and PCI staff. The figures are based on the fact that PAISANO was expected to cover 198 communities (assuming that each would have one COCODE, one COSAN/COCOSAN, and one COLRED) in 13 municipalities (each with one COMUDE and one COMUSAN). 43 Technical Components Community Organizations, Participation and Management of Food Insecurity, Shocks and Risks COCOSANes are the most visible and approachable groups at the community level. Their composition varies from mainly women in some places, to mainly men in others, but program staff report that on average the groups are 30 percent women and 70 percent men. COCODEs also exist in most communities but were more difficult to contact given that, except for Food for Work (FFW) activities, they are not as closely involved with program implementation as COCOSANes. COCODEs’ composition seems to be almost exclusively men. COLREDes are also expected to exist in each community, but the 50 that exist have been recently organized and are still being trained in their roles and responsibilities, and in basic disaster risk reduction concepts. Regarding their current participation in managing activities to cope with and reduce food insecurity and increase community resilience, COCODEs have been involved only in the implementation of the 15 FFW activities (rehabilitation of roads, water systems or irrigation systems) accomplished through FY 2014 and are expected to participate in the 21 planned for FY 2015. Most COLREDes are still in the process of receiving basic training on their roles and responsibilities, organizing commissions to perform different functions during emergency response, and learning to use the basic equipment provided by PAISANO during the few emergency response simulations that have been implemented. COCOSANes are the groups most actively involved in program implementation and are responsible for all food logistics and financial management activities at the community level. This includes securing a proper place to store food, preparing food rations before distribution, distributing food rations, collecting cash contributions from each participant receiving food rations, and paying monthly incentives to farmer leaders and mother leaders. COCOSANes also collect quarterly payments from families that have received water filters; they deposit the funds in a designated account to ensure that the filtering unit will be replaced by the filter factory every 24 months. Such activities consume most of the hours per month that COCOSAN members are expected to work for PAISANO in exchange for a food ration. At the time of the MTE, PAISANO had not yet implemented any sentinel sites for collecting and reporting early warning indicators. The FY 2015 target is the implementation of four Local Surveillance and Warning Systems (SILVYA by its acronym in Spanish) in four locations representing different regions. However, there has been hesitation to move forward with this plan due to the realization that the government organization responsible for receiving and processing this information has no capacity or resources to respond to any community where any emergency is identified via the early warning system. Consensus among program staff is building to refocus this activity as one for exclusive use by community members, whereby the collected information is used to increase awareness of climate change impacts and to identify community-level mitigation measures to reduce the negative impact of climate change. 44 Horizontal and Vertical Linkages of the Community Organizations PAISANO’s approach to working with community-based organizations is based on Guatemalan legal frameworks such as the Decentralization Policy and Municipal Code, which establish that these organizations must function in an integrated and complementary manner under the leadership of the COCODEs. However, field observations indicated that the links between these organizations are very weak and in some cases non-existent. Even though some members of one group are often also members of the other two, there is no formal mechanism for sharing the vision, plans or activities across the groups. The COCODEs’ interaction with PAISANO seems to be limited to FFW activities, while COCOSANes focus more on food logistics and the financial management of families’ cash contributions and COLREDes are relatively new or non-existent. In most of the interviews, COCODEs members indicated that they were not aware of the activities performed by the COCOSAN or COLRED (where they existed). Vertical linkages between community-based and municipal organizations seem to be limited to some interactions between COCODEs and COCOSANes with the COMUSANes. For example, COCODEs and COCOSAN representatives might participate in the monthly meetings held by the COMUSAN, where a wide range of organizations and government departments come together for a morning or afternoon to present progress on their particular undertakings. The expected coordination between COCODEs and COMUDEs, and between COLREDes and COMREDes, happens very seldom or is non-existent. It is important to acknowledge PAISANO’s role in promoting and supporting the municipal government of Chiantla to prioritize food insecurity as a critical problem to be addressed. This has led to the allocation of financial resources from the municipal budget to support activities to reduce food insecurity in communities not covered by PAISANO. In other municipalities, efforts to coordinate with the municipal government have also resulted in concrete financial or material contributions to rehabilitate drinking water and irrigation systems. Despite these successes, PAISANO staff recognize that sometimes it is not easy to grab the attention and the interest of municipal governments because the program covers a relatively small geographic area, for example, 20 of the 90 cantones in Chichicastenango, Quiché. Social Accountability and Governance of Community Based Organizations Created/ Supported By the Project While accountability and transparency are topics covered by PAISANO in its core training curriculum to strengthen governance and advocacy skills of COCODEs and COMUDES, accountability and transparency practices were not observed by the ET or mentioned by those interviewed at community or municipal levels. It was obvious that most people were not aware of the amount of financial resources received by the municipal government from the central government, and even less aware of how those funds are used on a yearly basis. Based on the information gathered from interviews with COCODEs and COCOSAN members, no accountability practices are being followed regularly or systematically by either of these groups. On several 45 occasions, it was mentioned that assemblies are convened for other purposes such as presenting information about, and requesting support for, community project(s), the election of COCODE members, and/or during emergencies. In some of the communities visited, the ET saw program-provided information in Spanish regarding program activities and coverage (also in Spanish) displayed on the walls as part of the so-called “situation rooms/posters”. However, this information was not referred to in order to explain any aspect of the PAISANO program. This gives the impression that the information is generated and displayed as a program requirement but is not necessarily used for sharing, analysis and decision making. Water, Sanitation and Hygiene (WASH) The original RFA, as issued by FFP, did not articulate any linkage between WASH access and the high levels of chronic malnutrition (e.g., stunting) that have persisted in Guatemala’s western highland communities despite the long-term presence of development actors. As such, WASH interventions in the project proposal were largely limited to household hygiene campaigns and behavior change interventions. The original DFAP proposal did not include community-level WASH activities. The baseline assessment, completed in March 2014, revealed the magnitude of WASH needs (e.g., only 19.9 percent of households had access to improved drinking water, and only 51.6 percent to improved sanitation), and pointed to the need for more deliberate and strategic interventions (see Table 4). Diarrhea rates in children under 5 are exceedingly high – 28.8 percent in PAISANO areas – suggesting that the fecal-oral route of disease transmission is rampant, whether via drinking water or due to sanitation or hygiene practices. The baseline revealed a relatively strong understanding of the need for improved hygiene and sanitation, with beneficiaries self-identifying the need for better hygiene and sanitation practices. Many of the baseline WASH findings were re-confirmed during the MTE, such as the unreliability of existing drinking water systems for months each year, and the absence of formative research to identify and target the primary routes of fecal-oral disease transmission and diarrhea. These persistent issues severely inhibit progress on health and development indicators; absent a stronger focus and investment in WASH, they will likely continue to be serious challenges to improving health and nutrition status. While this is a moment to adjust the project strategy to be more responsive to WASH needs, true impact will require a concerted, large-scale effort on the part of government and other development actors. Table 4: Summary of Baseline WASH Conditions in PAISANO Project Area Percent of households Access to improved drinking water 21.5 % Access to improved sanitation 51.8 % Use of point-of-use treatment (boiling) 93.0 % Prevalence of handwashing stations 77.8 % Stunting (in children < 5 years) 74.7 % 46 Percent of households Wasting (in children < 5 years) 1.0 % Diarrhea incidence (in children < 5 years; two weeks preceding) 29.0 % Source: USAID/ICF International. 2014. Baseline study of Title II Development Food Assistance Programs of Guatemala. Uneven Integration of Baseline WASH Findings and Recommendations into the Project Strategy The project strategy has not fully taken into account or followed through on the baseline report’s WASH findings and explicit recommendations to increase the scope of WASH activities. There continues to be little articulation or understanding of WASH-nutrition linkages as they relate to project targets for chronic malnutrition and stunting. For example, the project has not executed formative research to trace the exceedingly high (e.g., 28 – 43 percent) diarrhea rates, nor has it developed holistic WASH strategies post-baseline (i.e., water + sanitation + hygiene) to address the low access levels in some communities. Additionally, the project has not developed a strategy for addressing water system reliability and/or water quality at a whole-of-community level. While it is challenging to make major strategic changes in the project midway through the implementation period, the long-term persistence of chronic malnutrition – in spite of direct nutritional assistance – calls for a stronger emphasis on nutrition-sensitive approaches, such as WASH, in the second half of the project. In PAISANO project areas, the striking baseline findings on WASH (e.g., 21 percent drinking water access; lower Height for Age Z scores (HAZ) scores were associated with piped water supply) do not appear to have been used to develop a water quality strategy or overall WASH strategy. During MTE investigations, SC/PCI management cited a partnership approach to improve WASH access at a community level (e.g., partnering with NutriSalud and the Nature Conservancy); however, the MTE team was later told by program staff that there are not specific partnerships or targets for improving water and sanitation access. Project staff report that water quality analysis is underway in every community, and that they had reached 50 percent of communities at the time of the midterm. Results from this analysis were unavailable at the time of the MTE, and follow-on actions from the results are not yet defined. Beyond the scope of point-of-use (POU) treatment (i.e., a subsidized water filtration system, Ecofiltros, provided to households), PAISANO is not working on water quality assurance or centralized treatment at a whole-of-community level. During field MTE investigations, PAISANO staff cited the development of a subsidy and distribution model for the Ecofiltros that included a rural supply chain and pricing agreement with the distributor to enable access to replacement parts and filters for rural communities. This was described as an innovation and lesson learned from their previous PROMASA II program, which lacked the element of sustainability and did not result in long-term demand generation for the filters. Upon later investigation and discussions, SC staff noted to FFP “that the program is not developing a rural supply chain with the distributor of water filters,” but will be adding this component in the remaining two years of the program. 47 However, the potential to impact community-level water quality, and thus, rates of diarrheal disease, is very low with this household approach, particularly because only “1,000 days” households are currently eligible for filters. As noted above, that the same filter technology was given away (using a 100 percent subsidy model) under the PROMASA II project, with limited demonstrated impact or sustainability. The MTE team spoke with Ecofiltro recipients from PROMASA II who explicitly stated that they had neither the resources to buy replacement filters (replacement cost: 200 GTQ or $25 USD), nor the ability to buy the filters locally. The MTE wishes to emphasize that centralized disinfection (e.g., chlorination) of existing water systems is the GoG’s preferred method for disinfecting drinking water. The MTE would like to encourage the PAISANO management to review pertinent USAID documents that designate awardee roles and responsibility for supporting local regulations related to environmental safeguards under WASH programming.38, 39 Across and within SC management levels, different perspectives exist regarding the most strategic entry point for WASH interventions, and regarding strategic engagement with existing GoG entities responsible for WASH. SC’s approach to WASH is primarily to intervene at the household level, promoting proper hygiene and sanitation practices. The ET recognizes this as a critical element of an effective WASH strategy, but also notes that best practices for WASH in reducing illness and malnutrition call for a comprehensive approach that also encompasses community-level interventions. A sustainable approach to the latter necessarily involves close coordination with government and community structures in the installation, monitoring and maintenance of water and sanitation infrastructure. In the GoG case, this coordination would involve Municipal Water and Sanitation Offices (OMAS by the Spanish acronym40) and local municipalities. Some field staff voiced concern to the ET that the current approach works in parallel to, rather than in coordination with, existing government initiatives and strategies for WASH, which raises questions for sustainability. The project also does not appear to adequately tailor its interventions to the existing WASH infrastructure context, and thus at times risks undermining its own efforts. For example, participants of 1000 days’ approach in at least one community with a centralized, chlorinated drinking water system are receiving Ecofiltros. This negates the purpose of chlorinating and providing residual disinfectant in drinking water. It also illustrates an inconsistency with the basic science (i.e., filtering removes the residual disinfectant) and certainly the best practices governing WASH. In developing a post-MTE activity plan, the ET would like the partner to incorporate the findings of the baseline evaluation, and to strongly consider whether a household-level approach to WASH will be sufficient to affect and sustain health and nutritional outcomes. 38 USAID Global Environmental Management Support Project (GEMS). www.usaidgems.org/bestpractice.htm 39 USAID (2013). USAID Water and Development Strategy. Available: https://www.usaid.gov/sites/default/files/documents/1865/USAID_Water_Strategy_3.pdf 40 Oficina Municipal de Agua y Saneamiento 48 WASH Hardware/Software Targets and Desired Outcomes Remain Undefined After the award was signed, the project was adjusted to include WASH infrastructure, such as drinking water systems. Consequently, the targets and desired outcomes for WASH activities were not identified or reviewed during the DFAP review process. When asked during midterm interviews, project staff could not articulate the end-of-project targets or goals for WASH access (e.g., number of communities or people gaining access; percentage of systems with improved water quality). Table 5: Tracking of PAISANO WASH activities Activity Proposal FY 2015 PREP Field Validation Household point-of-use water treatment (Ecofiltros) Yes Yes Yes SBCC (Essential Hygiene Actions) Yes Yes Yes Direct activities to improve water access No Yes No evidence in visited communities Partnering to improve water access No No No evidence in visited communities Rainwater harvesting No No Yes Food for Work: rainwater harvesting No Yes No evidence in visited communities Project Strategies Are Not Aligned With USAID/Guatemala CDCS WASH Strategies In alignment with the country’s decentralization efforts since the 1990s, the USAID/Guatemala CDCS has endorsed engagement and capacity building of local or municipal government bodies (e.g., OMAS, COCODEs, 41 and water committees) as the model for improving WASH access and governance. When asked, many community leaders (e.g., COCODE members), as well as PAISANO staff, reported low technical and managerial capacity within the OMAS and a need to strengthen this capacity to improve sustainability of any WASH intervention, which reinforces the appropriateness of the CDCS strategy for building government capacity and supporting GoG efforts. PAISANO does not support, and is largely working in parallel to, existing government initiatives and strategies for WASH. Conversations with country-level PAISANO staff and management revealed discord concerning the responsibility to provide drinking water in rural communities (i.e., GoG or NGO), and the level of engagement the project should attempt with OMAS and municipalities. SC and PCI are encouraged to speak with SEGAMIL project staff, who have 41 Consejo Comunitario de Desarrollo Urbano y Rural (COCODE) Community Council on Urban and Rural Development 49 reported a range of experiences in working with the OMAS bodies thus far suggesting that strengthening these structures is not universally possible, but should not be discounted completely. The 1,000 Days’ Participation Model Severely Limits the Impact Of WASH Activities Implementing WASH activities to only 1,000 Days’ participant mothers significantly limits the impact of the intervention in three ways: messaging or inputs reach only a small fraction of households in each community; there is no apparent targeting on the basis of poverty levels or baseline WASH access within a community; and non-1,000 days households in the community see it as a “mothers-only” project. As noted in the findings on agriculture and MCHN, the majority of inputs for WASH (e.g., Ecofiltros and handwashing kits) had thus far gone to the promoters, who are typically in the upper wealth bracket of a community, rather than reached the most vulnerable. This may be due to the fact that filters had only recently been introduced to many of the communities. FFP would encourage partners to evaluate the ability of the current 1,000 days approach for WASH to reach the majority of communities targeted by PAISANO. Demand-Driven WASH Strategies Are Absent From the Project The project is employing a subsidy model (60 percent subsidy) to provide WASH materials. The prevalence of free or significantly subsidized WASH inputs (e.g., handwashing kits and point-of-use filters) reduces the demand and/or willingness to pay for WASH services ex-post, and likely limits the sustainability of the approach. The subsidies seem further misplaced given the demonstrated capacity and willingness to pay for water services in many communities: piped water tariffs, collected in the majority of communities, ranged from 1.5 – 15Q (US$0.19 – US$2.00) per household per month. When asked, the majority of beneficiaries reported a willingness to pay for piped water services, as well as to contribute additional funds for repairs when needed. Further, there was a prevalence of latrines and basins (pilas) installed by families without subsidy or support (the baseline study reported 58 percent access to improved, unshared sanitation facilities42). PAISANO staff are strongly encouraged to review relevant literature on demand-driven WASH and on best practices on the use of subsidies to spur community development.43 Understaffing Of Technical WASH Personnel Undermines Project Goals PAISANO does not have dedicated WASH staff; 30 percent of the time of the engineer on staff is devoted to WASH (70 percent of his time is dedicated to roads and other infrastructure). PAISANO does have a behavior change specialist, but the MTE team was not able to assess what percentage of their time was devoted to WASH behavior change efforts. The ET determines that the current capacity of PAISANO is inadequate to promote positive WASH behaviors in any significant way, unless a deliberate partnership approach can be demonstrated. 42 FFP (2012). Baseline study of Title II DFAPs in Guatemala. 43 FFP (2014-2014). Technical Reference Chapters for Development Food Assistance Programs. Available: http://www.usaid.gov/sites/default/files/documents/1866/Technical%20References%20for%20FFP%20Developmen t%20Projects%204-23-15%20%282%29.pdf 50 Approaches for Strengthening Community Organizations Training seems to be the preferred approach adopted by program implementers to strengthen community and municipal organizational capacities. For example, at the beginning of the program, COCODE members participated in training events to learn about program goals, methodologies, activities and expected results. Subsequent trainings covered topics related to legal frameworks that support the organization and functioning of COCODEs and other community-based organizations, roles and responsibilities of members, leadership skills and good governance practices, among others. Similar training has been provided to COCOSANes and is being provided to COLREDes. One concern shared by several of the interviewed groups is the prolonged gaps between training events; this results in loss of interest and motivation, and a breakdown between training topics that are supposed to reinforce each other. The training events are not accomplishing the level of strengthening and empowerment that is necessary for these organizations and their members to have the vision, capacity and confidence to effectively advocate for attention to community needs, to secure local and external assistance, and to lead the implementation of community development initiatives. For example, even in those cases where the group has conducted a community diagnosis and identified priority problems to be addressed, it has not been able to implement any of the planned activities because the usual answer from municipal government – and mainly the mayor – is “there are no funds available.” This is where the group’s efforts usually stop. The organizations have no strength of conviction or confidence to pursue the dialogue, to maintain pressure, to explore options or to negotiate. A few of the people interviewed indicated that they gained useful experience and skills when they had the opportunity to participate in negotiations to obtain specific resources for community development and work on implementing their plans. This is an important observation, as it points out that the theoretical training currently provided will not be enough to strengthen community organization and empower their members unless it is also put into practice. IV. Gender PAISANO carried out a rapid gender assessment in 2014. The report investigated how families make decisions and who makes them, how decisions regarding family wellbeing are made, how families decide the division of labor, and who participates in various community events. While the report provides a descriptive status of the gender situation in the target area, it lacks critical analysis, and therefore is not helpful to understand the implications of gender dynamics for project activities. The involvement of men in PAISANO is extremely limited. In the eyes of the community, PAISANO is seen as a women’s project. The project has been trying to involve men through the nutrition schools to improve men’s participation, but with limited success. PAISANO has yet to carry out any assessments to understand the effect of women’s participation in project interventions on overall women’s wellbeing, or to develop strategies to improve women’s participation in decision making. 51 V. Monitoring and Evaluation M&E Staffing and Work Structure The PAISANO M&E system is managed by a team headed by the M&E specialist based in Guatemala City. In addition, there is one regional M&E specialist based in Quiché and one in Huehuetenango. At the community level, PAISANO has recruited M&E assistants. These staff oversee the data collection process and implement data quality assessments on a monthly basis. Figure 2: PAISANO M&E Organigram PAISANO set up a routine monitoring system to collect data for most of the annual monitoring indicators. The data for output indicators are collected monthly by the field staff (resilience, health, and livelihood technicians) using pre-designed forms. The data are then verified by the M&E technicians. According to field staff, PAISANO has yet to develop a consistent process for quality assessment. Data Collection Processes PAISANO developed a web-based Management Information System (MIS) system called SAMI. The MIS allows easy data analysis at department, town, and community levels. Once the monitoring data are verified, the M&E staff enter the data into Sistema Automatizado para el Manejo de Información (SAMI); these data include growth monitoring (wasting, stunting and underweight), training, and commodity distribution. The PAISANO database contains basic information on the participant and participant household and a participant photo for identification purposes. Each participant has a unique identification number, which allows PAISANO to track performance data by participant. For example, PAISANO collects and records anthropometric data – height, age and weight – which allows PAISANO to track linear growth of participant children. Annual monitoring data are collected through routine monitoring and a participant-based monitoring survey. The purpose of the latter is to measure progress against PAISANO’s annual targets for outcome and impact indicators. PAISANO contracted a consulting firm to conduct this survey, which collected data on MCHN, agriculture and livelihoods indicators. From a list of 10,600 M&E Specialist Regional M&E Specialist Coordinator (2) M&E Assistant 52 participant families in 44 target communities,44 492 mothers or caregivers of children less than 30 months (192 in El Quiché, 168 in Huehuetenango and 132 in Quetzaltenango) were randomly selected. The anthropometric data collected on children were analyzed by age group: 0-5 months, 6- 11 months and 12-17 months. For agriculture and livelihoods interventions, only 82 participants were interviewed (32 from El Quiché, 28 from Huehuetenango and 22 from Quetzaltenango). The main reason cited for the small sample size was inadequate budget; the evaluation report noted that the results for the agriculture and livelihoods section should be considered as a rapid assessment as opposed to conclusions from a formal investigation. Annex F shows the indicators for which data are collected in the annual survey. Figure 2: SAMI M&E System Strengths of M&E • The PAISANO M&E system is functional. The monitoring system has a defined structure and clear responsibilities for each implementing partner. PAISANO developed a clear process for data aggregation and reporting. • The MIS system developed by PAISANO should be considered as a best practice. The web￾based system is versatile: it can be accessed from anywhere and has the ability to respond to user needs. The system has the potential to increase data use and encourage evidence-based management. The MEAL coordinator was able to provide data as the ET needed. • There seems to be a high level of coordination between SC and PCI for data collection, recording, and entry. 44 PAISANO plans to gradually scale up its coverage to reach 198 communities; however, at the time of the annual survey, the project was supporting 10,600 families in 44 communities. (Executive Summary , Annual Evaluation PAISANO; October 23, 2014) 53 Challenges for M&E • The MIS system is underutilized compared to its potential. The staff identified several issues that limit optimal utilization. Among other challenges, many staff reported that they have limited knowledge and skills to use SAMI. The staff also noted lack of stable, consistent high-speed Internet connectivity. Field staff have limited time in the office to use SAMI; the lack of skills in combination with poor connection make it extremely challenging to use the system effectively. • The sample size in the annual survey was inadequate to make the estimates useful: it was much smaller than the sample size required to compute point estimates with a reasonable confidence level. Because of the smaller sample size, the confidence intervals around the point estimates are likely to be large. The consultants also noted in the report that they could not use an adequate sample size for agriculture and livelihoods components because of limited resources. This deficiency should have identified by PAISANO M&E staff, or the SC regional or HQ M&E backstop. The implications of such mistakes are that decisions based on this information could be misleading, and the point estimates reported by PAISANO in its annual results report are less precise. However, SC reported that it has switched the data collection methodology from survey to routine monitoring, therefore survey design will not be an issue moving forward. • The baseline report was not widely shared with staff and partners. Therefore, many staff are not aware of the baseline status of the target population in the areas of health, nutrition, water, sanitation, and food security. Thus, field staff are in a potentially disadvantaged position in that they implement project interventions without a clear understanding of why specific interventions are appropriate, and lack the empowerment to affect change to those interventions when appropriate. • The FFP baseline and end-line surveys use a population-based survey, which draws a sample from the entire population from a defined geographic area, not only from project participants. Positive change in food and nutrition security behaviors among the project direct beneficiaries may not necessarily be adequate to demonstrate a change at the population level. The project needs to develop strategies to promote secondary adoption. A greater understanding about how the project will be evaluated would help staff to start thinking about ways to promote secondary adoption. VI. Commodity Management The ET visited a number of commodity warehouses and found that the physical conditions and locations of the warehouses are acceptable. The warehouses are clean and well ventilated; commodities are stacked properly and in an orderly manner; all commodity stacks have pallets under them; non-food items are not stacked next to food items, and natural pest management tools are used. 54 The warehouses follow first-in, first-out method. The records and interviews with commodity management staff suggest that the country office carries out periodic inspections. PAISANO does not fumigate the warehouses because the project does not see a need: according to staff, natural pest management methods work well. The ET did not observe any signs of pest damage. The warehouses have cleaning materials, firefighting equipment, emergency lights, weighing scales and other necessary tools. The records are well maintained. Every stack has updated stack cards. The warehouses keep waybill and receiving notes, a warehouse inventory ledger, and physical count records. Warehouse managers prepare daily reports and physical count reports at the end of each delivery. The project delivers commodities from main warehouses to distribution centers located in targeted communities. Regardless of distance and/or transportation costs, all participants pay 13 GTQ to cover incentives for leaders and local facilities maintenance. COCOSANes manages the distribution. The sites visited by the ET were clean and well ventilated and seem well managed. VII. Sustainability According to Tufts University researchers, sustainability occurs when outcomes and impacts (and sometimes activities) are maintained – or even expanded – after a project withdraws its resources through the exit process. A sustainability plan should represent all the elements of project design that incorporate sustainability, and increase the likelihood that project outcomes, impacts and (where relevant) activities, continue. 45 PAISANO presented a sustainability strategy with its response to the RFA. The strategy identified criteria for exit, benchmarks, action steps, strategies to assess progress, and a set of illustrative indicators. The exit strategy indicates that PAISANO will improve the knowledge and capacity of beneficiaries and local institutions before exiting the communities. The PAISANO proposal stated that SC will develop a more detailed and rigorous plan for graduation, exit and sustainability during Year One of PAISANO, but this has yet to be developed. Regarding sustainability for agriculture efforts, the PAISANO proposal stated that it would design extension services in such a way that participating extension agents will be able to develop and establish “businesses” in their own geographical region. This model could allow the extension agents to continue to provide technical assistance and offer services and inputs to producers, such as poultry vaccinations, artificial insemination of goats, and sales of supplies and spare parts such as parts for micro-drip irrigation systems, fertilizers, and seeds. The ET believes that this market-based strategy has the potential to sustain service delivery to some extent, but this vision is not communicated to the field staff and PAISANO management were not able to articulate it to the ET. 45 Rogers, B.L. and Coates, J., 2015. Sustaining Development: A Synthesis of Results from a Four- Country Study of Sustainability and Exit Strategies among Development Food Assistance Projects/ Washington DC: FHI 360/ Food and Nutrition Technical Assistance III Project (FANTA). 55 In addition, the market-based strategy alone may not be adequate to fulfil the needs of the target communities, as only a handful of agriculture leaders may take up the envisioned business. Moreover, PAISANO has yet to implement any interventions to promote this approach. In the remainder of the project, PAISANO needs to focus more on strengthening local institutions and including government service providers in support of the strategy. Regarding SO 2, the proposal did not include any strategies to sustain the support services to continue access to health and nutrition services after the end of PAISANO. Beneficiaries stated that the rations were the most beneficial PAISANO activity; most were aware that the rations would end at some point after their child turned two years old. However, beneficiaries do not know how they would continue their involvement in PAISANO after the child turns two, even though many expressed interest in continuing with other project activities. Some beneficiaries reported that they would not have the time to continue in PAISANO nutrition sessions once the rations are finished. Yet to elicit sustainable change in household behaviors and practices, it is important to maintain contact with women after they stop receiving rations. Providing an opportunity to reach 1,000-day households before their food rations end, and to continue impacting participating households’ future 1,000-day children, will help elicit both household behavior change and community-level shifts in social norms. Currently most PAISANO staff are unaware of the project’s sustainability strategy, therefore the sustainability lens is not being used to design and implement project interventions. PAISANO project management has yet to articulate its vision and strategies (other than what was stated in the proposal) to sustain outcomes and essential services. In terms of SO 1, some agriculture promoters aim to continue providing services if they can develop an income-generating activity in the related subsector; however, dedicated strategies are required to facilitate and promote such intents. As far as SO 2, if the project can demonstrate a change in the chronic malnutrition rate for a particular participant cohort, the changes are likely to be permanent hence sustainable – for that cohort. However, without systems changes that facilitate the knowledge and means to access nutritious food and supportive behavior on an ongoing basis beyond the life of the project, the next cohort of children will not likely benefit from project investments. Beneficiaries expressed their willingness to continue practices learned through PAISANO activities – however, their ability to follow through is questionable, as there is a lack of clarity about technical/ extension support and continued access to inputs. Moreover, attendance at the various sessions appears to be incentivized by the provision of the ration rather than by being, in and of itself, a worthwhile return on the investment of the beneficiaries’ time. Anecdotal evidence indicates that SO 2 participants have withdrawn from savings groups or stopped participating in the sessions after becoming ineligible to receive rations. These findings suggest that the current approach is inadequate to sustain participant interest and investment in continued learning and adoption of promoted practices: the activities are primarily project-driven and community engagement heavily dependent on project presence. 56 VIII. Conclusions PAISANO has demonstrated progress on its strategic objectives and at midterm, appears on track to meet output indicator targets. The project is implementing several approaches or activities notable for attaining objectives at output level and the participants’ favorable reception. These include: • Homestead gardens, which the ET assessed to be contributing to nutrition and dietary diversity, and at times to allow households to generate income by selling surplus produce; • Cooking demonstrations, which are well linked to gardening activities and whose lessons appear to be taken up readily by project participants; • Savings and lending groups, which are facilitating savings among members and may instill a culture among some members; and • Engaging community members in training and empowerment-oriented activities. These are important successes. At the same time, they must be viewed in light of the project’s higher aspirations for outcomes, impact, and sustainability at project, population, and system levels. The MTE presents a moment for strategic reflection on what implications the observed presence or absence of progress to date – and more importantly, project strategies and processes – have for the short, medium, and long term. It is an opportunity to adjust mid-cycle, to optimize progress on different scales and timespans. It should be considered, also, that PAISANO is one in a decades￾long series of food security projects in Guatemala. However, these projects have shown only limited demonstrable, sustainable impact on chronic food insecurity, malnutrition and poverty among target populations. Despite government and donor investments in the western highlands, the rate of chronic malnutrition remains at an alarming level and continues to rise. The 2008-09 Reproductive Health Survey found that 49.8 percent of all Guatemalan children under age 5 are stunted.46 The rate of stunting among indigenous children was even higher (58 percent).47 The 2014 -15 National Maternal and Child Health Survey reported a slight decline in overall rate of stunting among Guatemalan children age below 5 years (46.5), but there is no change in the stunting rate for indigenous children.48 The baseline survey shows that more than three-quarters of children (75 percent) under 5 years of age suffer from moderate and severe stunting. Ethnicity is a key factor associated with food and income insecurity in Guatemala, as are suboptimal infant and young child feeding practices, low dietary diversity, food insecurity, lack of clean water and sanitation, poor hygiene, and poor access to health services. 49, 50 With regard to infant feeding, both the amount of 46 MSPAS Encuesta Nacional de Salud Materno Infantil 2008 (ENSMI 2008-2009). Ministerio de Salud Publica y Asistencia Social (MSPAS), Institución Nacional de Estadística (INE)/ Centro de Control y Prevención de Enfermedades (CDC), Guatemala 2010. WHO Child Growth Standards. 47 2008-2009 ENSMI, cited in USAID/Guatemala BEST Action Plan for Family Planning, MNCH, and Nutrition, Best Practices at Scale in the Home, Community, and Facilities, submitted to USAID/W/GH: April 27, 2011. 48 Ministerio de Salud Pública y Asistencia Social (MSPAS), Instituto Nacional de Estadística (INE), ICF Internacional, 2015. Encuesta Nacional de Salud Materno Infantil 2014-2015. Ciudad de Guatemala, Guatemala. 49 Chaparro, C. 2012. Household Food Insecurity and Nutritional Status of Women of Reproductive Age and Children under 5 Years of Age in Five Departments of the Western Highlands of Guatemala: An Analysis of Data from the National Maternal-Infant Health Survey 2008–09 of Guatemala. Washington, DC: FHI 360/FANTA. 57 food young children are fed and the variety of food they are given are inadequate to meet their nutritional needs. The baseline survey for PAISANO also reports poor dietary practices, sub optimal practices related to infant and young child feeding, and poor water, sanitation and hygiene behaviors. In this context, it is therefore both responsible and ethical to take a hard look at the drivers of food insecurity in the western highlands region and revisit the strategies being used to address chronic vulnerability to food insecurity and malnutrition. Taking this longer view, the ET finds several strategic and operational areas that need modification and strengthening. As far as project design, beneficiary targeting is foremost among areas to re￾assess. As noted in Section 4, PAISANO is viewed by communities as a “women’s project.” This stems from the 1,000 Days approach and corresponding emphasis on women. The ET agrees that women need to be a cornerstone of food security approaches. However, the quality of women’s engagement in learning and empowerment processes in the current project needs to be improved. Moreover, men’s role in these processes is not clearly articulated by the project, nor is it understood – and thus valued – by communities. In practice, project assumptions on synergies between SO 1 and SO 2 targeting do not bear out, as the de facto audience for health and nutrition interventions does not overlap significantly with the one for commercial agriculture, whether crops or livestock, and value chain work. Regardless of target group, adult education techniques and topics need to be more demand-driven, more flexible, and more engaging. Learning goals of all project activities need to be clearer to all participants: rations and material handouts are the draw for participation, which suggests that absent an improved understanding of how learning activities add value to current and future daily life, interest in applying project learning will fade when the project finishes. In addition, a strategy is needed for transferring knowledge beyond project participants to whole community, and for embedding this knowledge in community governance structures. Given its proven role in improving chronic health and nutrition outcomes, WASH—particularly community-level WASH— was identified as a strategic gap of the project. Moreover, WASH needs to be part of a larger learning and capacity development strategy that involves government and community structures. The data collection and organization aspects of PAISANO’s M&E system are well planned and well implemented. However the data and analysis generated from M&E processes, including the baseline study, have been poorly disseminated to staff and partners. This hinders the capacity to study and reflect on information that should be used to design, target, and adjust project strategies and interventions according to needs and achievements in project coverage areas on a regular basis. 50 De Pee, S. and Bloem, M.W. 2009. “Current and Potential Role of Specially Formulated Foods and Food Supplements for Preventing Malnutrition among 6 to 23 Month-Old Children and for Treating Moderate Malnutrition among 6 to 59 Month-Old Children.” Food and Nutrition Bulletin. Vol. 30 (3 Suppl), pp. S434–63. 58 Continuing PAISANO activities with a clearer eye on sustainability requires stronger consideration of contextual factors such as the role and capacity of Guatemalan institutions such as COCODES, OMAS, health centers and agricultural extension, as well as the private sector, to deliver the services essential to building and maintaining food security and nutrition. PAISANO needs to include, plan, coordinate and consult more with these entities and with the private sector to ensure that the services currently provided or facilitated primarily through PAISANO remain sufficiently resourced, particularly in the most vulnerable communities, and managed with adequate technical competency. PAISANO and partners also need to consider that demand for certain services or goods may need to be built or strengthened. This will require enhancing community awareness of returns on household investments, such as investment in agricultural inputs (e.g., improved seed varieties) and services (e.g., fee-for-service agricultural extension), or in health care (e.g., investing time and resources to attend pre-natal visits). The MTE finds “graduation” strategies at participant level, and exit strategies at project level, to be either poorly defined or poorly understood. PAISANO needs to work more, both project-internally and with local Guatemalan institutions and the private sector, to strategize how to optimize resource investments in the medium and long term in a context of chronic vulnerability. As implemented, PAISANO suffers to some extent from inclusion error in its targeting practices at the beneficiary level; this is one example of an area where effective use of M&E could assist in decisionmaking. Finally, the project has made important and promising connections with fellow USAID-funded initiatives, such as the Rural Value Chain Project, as well as with the Guatemalan private sector, especially in agriculture. There are also incipient relationships with entities that can play a role in capacity development in some technical areas after project exit. These collaborations, especially ones where the partner has a vested interest and commitment beyond PAISANO project life, can be an important part of the PAISANO phase-out and exit strategy; here again, meaningful engagement with native institutions in this planning is critical. The potential within PAISANO and in the stakeholder environment exists for consolidating project achievements and ensuring continued progress toward PAISANO objectives, which are commonly held objectives for food security in Guatemala’s western highlands. The remaining time in PAISANO would be well spent reworking implementation approaches – based on recommendations not only from this MTE but dating back to the baseline – and on solidifying the relationships and competencies in Guatemala that are critical to continued improvements in food security goals in the region. There is also an opportunity to enhance sharing and learning between SEGAMIL and PAISANO given the similarity of the projects and the challenges they face. IX. Recommendations Strategic Objective 1 Recommendations Recommendation 1: Provide More Tailored, Targeted and Context-Specific Livelihood Assistance to Target Communities 59 • Draw on existing knowledge and information about target communities to develop SO 1 support packages that respond to differences in the potential for promotion of specific agricultural crops (potato, coffee, goats, agroforestry, etc.) and to opportunities for communities where the greatest economic potential may lie outside the agricultural sector (e.g., handicrafts, skilled labor). • Given the limited reach of the goat “pass-on model,” and the fact that it is not likely to effectively benefit poorer households as currently implemented, evaluate the model for cost￾effectiveness and its contribution to achieving PAISANO’s overall food, nutrition and livelihood security objectives. Promoting poultry or other cost-effective productive assets is a more effective use of project resources and may result in greater improvements in household food security. • Target assistance based on appropriateness and need, verified through means testing. This will necessarily include inclusion of food insecure households with a need for livelihood support (e.g., assistance for agricultural production and/or livelihood diversification) that are currently ineligible to receive support under the 1,000-days criteria. Eliminate current criteria for participation in SO 1 activities (e.g., 25m2 minimal land size) that exclude the participation of the poorest, most food insecure households. • Increase support for establishing and training Women Empowered groups that PCI created and provide support to savings and lending as a means of strengthening participant capacity for agricultural production, accumulation of household savings, and access to credit for productive investments. • Further integrate agricultural training with financial literacy and asset accumulation promoted through savings groups, to promote livelihood diversification at the household level. Recommendation 2: Increase the Consistency and Quality of Support Provided to Participants in SO 1 Activities • Increase the frequency and consistency of SO 1 support to individual households. This may include limiting the number of SO 1 participants to those who demonstrate the greatest need and/or creating financial incentives for qualified community members to assume a greater role in capacity building under SO 1. • Seek a means of shifting from the current approach utilizing handouts (e.g., food, seeds and materials) as incentives for adopting improved practices, to one that provides targeted subsidies for participants. Strengthen the capacity of participants to use subsidized inputs and services as part of a coherent plan (e.g., a business plan) for improving household economic productivity while developing market linkages for sustainability. 60 • Utilize training and financial incentives to promote the provision of extension support by qualified community members on a fee-for-service basis, facilitating linkages to private sector suppliers. Recommendation 3: Improve the Current SO 1 Approach to SBCC • Increase the competency and capacity of field staff to facilitate participatory, demand-driven training to participants by strengthening their skills in SBCC. • Adopt a more holistic approach to SBCC by carrying out barrier analyses that clearly identify the obstacles to wider adoption of promoted agriculture, livestock, and natural resource management practices and potential means of overcoming them. • Increase the frequency and quality of the learning sessions. In addition to the methodology, learning plots play a key role in promoting learning. A sizeable plot allows all participants to practice on the plot, set up small trials, and practice different crops. Explore opportunities to access appropriate land with the community and community leaders in cases where access to such a plot is a challenge. • Promote greater understanding and adoption of promoted practices through adult learning methodologies that encourage simple experimentation and experiential learning. One potential means would be through consistent application of the Farmer Field School approach. Recommendation 4: Clarify the Extent to Which Linkages with Agricultural Value Chains Contribute to Improved Food, Nutrition and Livelihood Security Outcomes Among Project Participants • Carry out in-depth, participatory assessments of agricultural production capacity and marketing opportunities in a broad group of communities; assess intervention options in terms of their community-specific livelihood and nutritional outcomes and potential for sustainability. • Concentrate the product and geographic focus of value chain activities on products that have the greatest potential to contribute to household and community economic development (e.g., potato, onion). • Promote profitable participation all along the value chain. This may include building the capacity of participants to provide agricultural inputs (e.g., fodder, seeds and organic compost), services (unskilled and semi-skilled labor, transportation) or processing to add value. 61 • Evaluate whether the investment needed for developing goat milk processing capacity at the Centro de Producción Caprino del Altiplano, CEPROCAL (Highlands’ Goat Production Center) is an efficient and effective means of reaching SO 1 targets. The production model and complementary demand for CEPROCAL products (e.g., cajeta) is promising. However, the ET feels that the resources required for ensuring profitability and achieving the planned handover of the processing plant to Fundación Agros may stretch PAISANO’s resources. Complementary funds from other donors could be sought to cover these costs. Recommendation 5: Emphasize Improved Knowledge Management As A Means Of Achieving Project Impact • Dedicate financial and staff resources to documenting evidence of the relative efficiency and effectiveness of various SO 1 activities in meeting the overall program goal. Engage a university or research organization to support this endeavor. • Work with CRS, Caritas San Marcos, ADIPO and other organizations to exchange information on promising practices in promoting improved food, nutrition and livelihood security in the western highlands. Strategic Objective 2 Recommendations Recommendation 1: Implement an SBCC Strategy Grounded In Adult Education Principles • Increase the frequency of high-quality, systematic interactions with group members and include a harmonized approach for different regions. This should involve increasing the frequency and repetition of key topic areas introduced during SBCC activities. Use experiential learning techniques modelled after cooking demonstrations that seemed to be successful, and allow beneficiaries to choose topics of interest for SBCC nutrition sessions. • Continue to discourage the consumption of snacks that are highly processed or high in sugar or fat during PAISANO nutrition sessions. • Strengthen the quality of home visits so that they provide specialized care and counseling related to key behavioral practices during the 1,000-day window (e.g., from the nutrition wheel or based on household observation). Home visits should include a dialogue and holistic approach to identifying underlying determinants for malnutrition unique to that household. • Provide hands-on training and subsequent refreshers on participatory facilitation techniques, periodically monitoring the sessions and providing feedback. Continue providing supportive supervision visits to field technicians, transferring skills to develop sustainability plans with communities. 62 • To minimize health risks, consider supporting households with subsidized improved cook stoves or conduct the cooking demonstration sessions in open spaces. • Consider increasing the number of mother leaders and promoters and subsequently increasing the number of technicians to ensure quality and timely interaction between promoters and participants. Reevaluate and increase the incentive to both the mother leader and the promoters. • Address training needs of staff and community volunteers to ensure that they have adequate training to carry out SO 2 activities. • Review SBCC targeting for the maternal and child health indicators to understand why key behaviors have not been practiced/adopted by mothers. Consider targeting mothers-in-law as a supporting group to influence household behaviors. Recommendation 2: Implement a Graduation Strategy for Phasing Out or Phasing Over SO 2 Rations • Define a graduation/exit strategy for women whose children turn 24 months to ensure that they understand how to maintain optimal feeding practices, understand childhood illness warning signs, and are interested in continuing with nutrition-sensitive activities beyond the receipt of rations. • Limit one 1,000-day period per woman for ration distributions to ensure there is quality coverage and reach of activities. • Reduce significantly the ration size and eventually eliminate the household ration. Options for phasing out SO 2 rations: o Replace Title II commodities with commodity-based vouchers. In terms of sustainability, consider a voucher system that includes a tapering subsidy throughout the 1,000- day window. o Continue providing CSB+ and oil to pregnant and lactating women and children under 2 and implement a commodity-based voucher for the household protection ration. o Allow beneficiaries to receive Title II in-kind rations during pregnancy and lactation and implement a commodity-based voucher for the remainder of the 1,000 days. Recommendation 3: Strengthen Linkages to Health Centers and Health Posts That Cover the PAISANO Project 63 • Evaluate the feasibility of the pregnant-women-support-group approach and either increase the number of mother leaders or advocate for the MoH to implement this project with health center/post staff. • Evaluate the feasibility of linking MoH with SEGAMIL participants to avoid duplication of ration beneficiaries between those who are receiving them at the local health center and those receiving SEGAMIL rations. The MTE team acknowledges that given the current condition of MoH, the supply of Incaparina (a corn-soy flour mix) and Vitacereal could be unpredictable and interrupted, however, SEGAMIL should advocate for a predictable supply of Incaparina and Vitacereal at local health posts/centers for pregnant and lactating women and children under 2 under the GoG Zero Pact Hunger Program as part of a phase-over graduation strategy. Recommendation 4: Integrate Baseline WASH Findings into Program Strategies • Conduct formative research to trace primary routes of fecal-oral pathogen exposure in communities with high diarrheal rates. • Target interventions to beneficiary-identified WASH needs (e.g., latrine access, school WASH). • Define targets and desired outcomes for WASH activities (e.g., number of communities or people gaining access; fraction of systems targeted for improved water quality) – even if it is mid-way through the project cycle. • Target communities with the highest rates of diarrhea and/or stunting with targeted WASH interventions. • Implement community-level water quality interventions to address the striking baseline finding that lower HAZ scores are associated with piped drinking water in PAISANO target areas.51 This suggests that poor quality of piped drinking water may be a factor in the high rates of diarrheal disease and long-term nutritional deficits in the communities. For example, partners should consider building out strategies for follow-on to water quality assays that are underway. Recommendation 3: Align with Accepted Best Practices For WASH Access to Improve Impact and Sustainability • Expand current participation models (AMIGA) beyond pregnant and lactating women and children to reach, at a minimum, the most vulnerable members of a community – and ideally the entire community – with targeted WASH interventions. 51 USAID/ICF International. (2014). Baseline study of Title II Development Food Assistance Programs of Guatemala. p. 44. 64 • Employ unsubsidized models for WASH access, utilizing targeted subsidies only for the most poor/vulnerable households (e.g., demand-driven sanitation behavior change models such as Community Led Total Sanitation (CLTS) or Participatory Hygiene and Sanitation Transformation (PHAST). • Develop activities that work within, rather than parallel to, existing WASH infrastructure. Strategic Objective 3 Recommendations Recommendation 1: Target Institutional Capacity Development Activities to COMUDES and COCODES, and Support Shift of COCOSAN and COLRED Focus to More Sustainable Roles at The Community Level • Negotiate agreements, interventions, methods and work calendars with the recently elected municipal corporations. • Establish working agreements with the Municipal Planning Bureaus to join forces for strengthening the COMUDE. • Support the development of a municipal plan for reduction of food insecurity to guide the actions of all development organizations in the municipality. • At the community level, concentrate work on strengthening the central COCODE committee and its various commissions. • Begin a transition process to focus the work of the COLREDes on risk management at the community level, emphasizing implementation of mitigation activities with support from the COCODE. Advocate for municipal recognition of the COLREDes. • Strengthen the vertical links of the COCODEs with the COMUDE in order to facilitate project management, advocacy for community needs, and mutual accountability. • Systematize the processes for, and frequency of, sharing relevant information and accountability with the population through the community assemblies. • Explore opportunities for coordinating efforts with the Nexos Locales program. • Facilitate development of community disaster risk mitigation and local response plans. Support implementation of the mitigation plans by means of identification and use of local resources, and use of food for work and other resources that can be obtained from governmental and non-governmental organizations and the private sector. • Dialogue with COCOSAN representatives about their roles as established in the country’s legal framework, as opposed to their current roles, which correspond more to a community 65 committee to support implementation of PAISANO activities that would stop functioning at the end of the project. • Strengthen linkages and possible incorporation of the COCOSAN within the COCODES structure. Recommendation 2: Improve the Quality and Frequency of Training Based On Assessments • Provide training to all members of the groups and the COCODES (central committee and commissions) and not just the few representatives. Identify training topics based on practical, relevant issues; create opportunities to apply training in the implementation of project activities. Increase the frequency of training. • Review the training curriculum and methodology to ensure the development of essential abilities and skills for effectively carrying out group members’ and COCODES’ roles and responsibilities. • Facilitate the development and implementation of work plans with the COCODES using local materials and inputs, donated food, and funds generated from the cash contributions of the families who receive food rations. • Delegate the management and implementation of projects supported with food for work to COMUDEs and COCODEs to help strengthen their management, administration, and accounting abilities. Recommendation 3: Align With USAID/Guatemala CDCS to Strengthen Governance and Capacity Building as a Key Strategy to Improve WASH Access • Design approaches that contribute to, not bypass, state building and government accountability to citizens. • Facilitate a knowledge sharing session between SEGAMIL and PAISANO management and staff to share lessons learned from working with local and municipal government bodies. • Work collaboratively with FFP, USAID Mission in Guatemala, and SEGAMIL to strategize on the best way forward. • Incorporate financial transparency metrics at the municipal/community level. M&E Recommendations Recommendation 1: Improve Project Performance and Targeting of Interventions Using Learning from Baseline and Monitoring Results 66 • Widely share the baseline study report with staff at all levels to increase their understanding of what the project “start line” was in terms of all performance and contextual indicators, and what that means in terms of expected change by the end of the project. All staff from SC and PCI should clearly understand the baseline in order to strategically contribute to the success of the project goal. One way to effectively communicate the findings is to make presentations of the findings with interpretations to the staff, and discuss the implications for project implementation. • Consider developing and implementing an internal data quality assurance mechanism. This should take into consideration all data quality standards such as validity, reliability, precision, integrity, and timeline. Consult FFP M&E and Reporting Policy and Guidance and Automated Directives System (ADS) 203 for guidance in developing a data quality assurance system. • Take advantage of the baseline study to explain the methodology and the definition of the population in the population-based baseline and final evaluation surveys to help staff understand that the project will be evaluated not only based on direct project participants, but also on the community in which the project has been operating. Consider developing and implementing strategies to promote secondary adoption. • Consider developing strategies to increase data use for decision making. This would include conducting analysis tailored to the needs of various stakeholders (management staff, technical staff, and partners), interpreting the results, and communicating the results in an effective and timely manner. • Carefully review the need for annual monitoring and try to avoid using surveys. However if data for some indicators cannot be collected through routine monitoring, use a competent individual or firm to design a beneficiary-based survey and develop sampling methodologies. Follow guidance and requirements outlined in the FFP M&E and Reporting Policy and Guidance. Sustainability Recommendations Recommendation 1: Revise the Sustainability Strategy to Make It More Market-Oriented and Less Reliant on Government • In the absence of a strategic exit strategy that phases out free inputs, and/or transitions free or subsidized inputs over to fee-for-service, market, or well-functioning government service, providing free resources can pose a serious threat to the sustainability of activities and impact after the project ends. PAISANO should review its current provision of free or subsidized inputs, and the blanket distribution of 1,000-days food rations. Consider prioritizing households to receive subsidized support based on economic status through means testing. The support should be tailored to the individual’s livelihood strategy and not 67 based on their association, position in the group or any other criteria. Wherever possible, local markets should be used as a conduit for access to inputs and food, through vouchers or other means. To promote sustainability, one of the critical steps would be to identify the outcomes that should be sustainable and the critical services necessary to sustain them. • Consider investing in demand creation by supporting the delivery of high-quality services that lead to tangible benefits, developing the capacity of service providers, and ensuring a sustained source of resources for each input currently provided by the project. Potential sources include: o Agricultural and non-agricultural activities that are run profitably and can pay for services; o Funds leveraged through government operating budgets; o Funds provided by other donor agencies or NGOs; and o Contributions by community members in cash or in kind, or other types of innovative finance. • Explore potential fee-for-service models for providing ongoing technical support and services, using private input suppliers as sources of extension information, taking advantages of the savings groups to access inputs, and building the capacity of the MoA, MoH and other entities that have the potential for long-term service provision. • In the last year of PAISANO implementation, use market-based approaches to finance the critical services necessary to sustain key food security and nutritional outcomes. General Recommendations Recommendation 1: Mainstream Gender Integration into the Project Implementation Strategy • Involve men and youth into agriculture and livelihood activities so that they directly benefit from and participate in PAISANO services, which can be done if the SO 1 and the 1,000 day ration is delinked and developing training and other opportunities based on needs and the local economy. Potential areas of involvement could be including them in fee-for-service delivery initiatives, engaging them in production and marketing of produce that targets through the 1,000-days ration food voucher, and developing skills on alternative livelihood strategies. • Assess the time burden on women due to their participation in PAISANO activities and develop strategies to minimize the burden. • Consider revisiting the project’s gender analysis in view of the MTE recommendations, using headquarters, Mission, and FFP gender advisors as resources. 68 Annex A: Evaluation Protocol Draft Protocol for SEGAMIL and PAISANO Midterm Evaluation, Guatemala Introduction The evaluation team developed this methodology and protocol for the midterm evaluation of SEGAMIL and PAISANO programs in Guatemala. The two development food assistance programs funded by U.S. Agency for International Development (USAID) Office of Food for Peace (FFP) in FY 2012. FFP issued two awards: one to Catholic Relief Services’ (CRS) Seguridad Alimentaria Enfocada en los Primeros 1,000 Días 52 (SEGAMIL) program in San Marcos and Totonicapán departments; and the second to Save the Children’s (SC) Programa de Acciones Integradas de Seguridad Alimentaria y Nutricional del Occidente53 (PAISANO) program in Quiché, Huehuetenango, and Quetzaltenango departments. The main purpose of these two programs is to improve long-term food security and nutrition in the Western Highlands. Both of the programs seeks to address issues related to food insecurity, improve agricultural productivity and production, enhance farm and non-farm income to improve food availability and access and improve health and nutritional outcomes of poor and vulnerable households. Program activities include, training farmers and agricultural leaders in improved crop production techniques and natural resource management, creating micro-watershed communities, facilitating savings and lending groups, improving preventative health services and referrals, providing conditional food rations, and schools; strengthening community groups for disaster preparedness and emergency response establishing municipal-level early warning systems. FFP, Save the Children and CRS jointly implemented the midterm evaluation. Technical staff from FFP, Bureau of Food Security, CRS and Save the Children will form the evaluation team. The evaluation seeks to review, for each program, the effectiveness of the processes and approaches, offer the opportunity to identify effective activities and strategies for scaling up, but also modify those that are not serving the program’s goal. The evaluation will review, for each program, how well the delivery of an intervention matches the original, approved plans, and to identify factors that contribute to greater/lesser efficiency and quality of outputs and to the greater/lesser acceptance of the interventions and outputs in target communities. The evaluation will identify the lessons learned and best practices employed by either program that could be scaled up and/or incorporated into each program as appropriate. The evaluation will also review the implementation processes for producing planned outputs in the Detailed Implementation Plans (DIP). The evaluation will consider these processes and assess 52 Food Security Focused on the First 1,000 Days 53 Integrated Food Security and Nutrition Program in the West 69 whether the program logic reflected in the results framework, measured by indicators per result presented in the Indicator Performance Tracking Table (IPTT), is likely to be achieved. The evaluation will seek to identify factors in the program’s implementation or the context that appear to promote or impede the intended results, objective, and impact. The evaluation will offer prioritized recommendations to improve the effectiveness of project processes. It is expected that the projects will take the recommendations to adjust the strategies to maximize the impact on the project beneficiaries. Description of Programs to be Evaluated The two programs being evaluated are the following: 1. CRS SEGAMIL Program CRS implements the SEGAMIL program in coordination with local partners Asociación de Desarrollo Integral para el Occidente (ADIPO) and Social Pastorate/Caritas San Marcos. SEGAMIL serves 10,500 families each year, or approximately 23,500 families, over the life of the program (August 1, 2012 to July 31, 2018) in eight municipalities in the departments of San Marcos and Totonicapán. The overall goal of SEGAMIL program is to improve food security with an integrated approach in targeted micro-watersheds in eight municipalities in the Guatemalan Western Highlands. To achieve the goal CRS and its partner organizations hope to achieve improved food access of farmer households, reduce chronic malnutrition among vulnerable rural populations in targeted micro watersheds and improve local and municipal resilience systems to improve food security. SEGAMIL aims to empower women so that they can improve health and nutrition status for themselves and their children. 2. Save the Children PAISANO Program PAISANO program is implemented by Save the Children with the Project Concern International (PCI) in the 13 municipalities in the departments of Quiché, Quetzaltenango and Huehuetenango. Save the Children directly implements the program activities in eight municipalities of Quetzaltenango and Quiché while PCI implements the activities in five municipalities in Huehuetenango. PAISANO serves 26,500 vulnerable rural families in the region. The overall goal of PAISANO is to reduce food and nutrition insecurity for the most vulnerable rural families in the region. Specifically, the project targets rural communities suffering from food insecurity; at risk of suffering from food insecurity; and communities with a high prevalence of chronic malnutrition. To achieve the goal Save the Children and PCI hope to achieve increased household access to food, reduce malnutrition among girls and boys under 5 years and improve community resilience. The project aims to ensure gender equality, mitigate environmental consequences and influence behavior change. Evaluation Objectives and Primary Questions 70 The objectives of the midterm evaluation are to: 1. Assess the strengths and challenges of program implementation and outputs in terms of meeting terms agreed with FFP and their acceptability and usefulness to the targeted communities, and identify factors that appear to enhance or detract from the efficiency, quality, acceptability, and usefulness of the activities’ implementation and outputs. 2. Present, through qualitative methods and available routine monitoring data, evidence of changes (intended and unintended) associated with program activities and outputs, assess how well the observed changes support the logic of the results frameworks and identify factors in the implementation or context that impede or promote the achievement of target results . 3. Review SEGAMIL and PAISANO’s plan to make adjustments to programming per the recommendations from the USAID team based on the Assessment of the Western Highlands Integration Program (WHIP) in Guatemala. Review the recommendations in line with the findings from the midterm evaluation team and suggest adjustments if necessary. Assess the progress in coordination and collaboration with other actors within the Western Highland Integration Project, Department Development Committees, SESAN, Zero Hunger Pact, NutriSalud and Rural Value Chain Program. 4. Review logic of both program’s results frameworks to assess the hypothesis, risks, assumptions and the progress made by using the program service and delivery mechanisms. 5. Review the processes for capturing and documenting lessons learned, including feedback from the perspective of stakeholders and beneficiaries. Assess the systems in place to adjust program strategies to incorporate learning. Determine how well outcomes, systems and services are designed to be sustainable, and the progress towards implementing sustainability strategies. 6. Determine how well the programs considered gender-equality with regard to access to, participation in, and benefitting from the activities, and how the choices and focus of activities are gender-sensitive. The evaluation is expected to provide answers to the following key questions. The order of the question is not based on the importance or priority. 1. To what extent has the program adhered to the implementation plan, outputs, and targets reached? What factors promoted or inhibited adherence to schedules? How were problems and deterrents managed? 2. What are the strengths and challenges of the program design and management processes so far? What key factors of program management are associated with positively and negatively 71 influencing the quality of planned outputs? How SEGAMIL and PAISANO management and technical specialists use data to inform programmatic decisions, referral and follow up? What processes have been instituted to improve data collection and data quality? 3. How effective is the pass-on model54 to address food insecurity and nutrition needs of the most vulnerable population? Is there any evidence suggesting that the model is working? Does it have the potential to support the target community within the program life and help achieve the program strategic objective and intermediate results? 4. In each technical sector, what are the strengths and challenges to the implementation of activities and processes and the quality of outputs? How well do implementation processes adhere to the internationally acceptable underlying principles (for example participatory approaches, and experiential learning or learning-by-doing) and internationally established approaches, methodology and processes? What factors in the implementation and context are associated with greater/lesser efficiency in producing outputs of higher/lower quality? Which activities and implementation processes are more or less acceptable to members of the target communities and why? What business opportunities have been created to enable beneficiaries to participate into value chains? How they could be improved? Which FTF￾supported technologies, training, extension and other services were shared with FFP beneficiaries? Will FFP beneficiaries likely be ready to be incorporated into an FTF value chain activities by the end of the program? 5. How effective are the social and behavior change communication (SBCC) strategies and approaches? Are the strategies adequate to influence targeted practices and behaviors to promote optimum infant and young child feeding practices and health and sanitation practices of mothers, caretakers and children? Are the SBCC strategies and messages consistent, adequately contextualized, adequately disseminated and communicated? What is the perception of different target groups? Are there any evidences suggesting that the practices and behaviors promoted by the programs are being adopted by the target groups? 6. What changes do community members and other stakeholders associate with the program’s activities? Which factors appear to promote the apparent changes, and which have deterred intended changes? How do the changes correspond to those hypothesized by the program’s results framework? 54 The “pass-on model” is an approach where, for example, goats are loaned to a beneficiary, who pays in-kid by giving out the first two female offspring free to the next listed beneficiaries. The beneficiary also receives dairy entrepreneurial skills. The purpose is to ensure the spread of dairy goats in the communities involved, as the beneficiary recipients of goats are all expected to give their first two offspring free. 72 7. Based on the findings from 1-6, above, how could the program be modified to improve its acceptability to targeted communities or the efficiency and effectiveness of its implementation? How should the program’s results framework be refined or modified? The Evaluation Team The 14-person evaluation field team made up of USAID technical specialists and specialists from Save the Children, PCI and CRS with an equal mix of men and women, offers expertise in program evaluation, qualitative analysis, gender analysis, community mobilization, behavioral change communications, and resilience. The evaluation team consists of the following members: Arif Rashid, will assume overall responsibility for the evaluation, assuring that the SOW is carefully followed, that high quality information is collected and carefully analyzed, that the team maintains the objectivity and follows the proposed timeline (see below). Arif will take responsibility to analyze monitoring data collected by the projects, review M&E systems and management structure and functionality. Arif together with Adam Reinhart will take responsibility for analysis of community resilience implemented by Save the Children. Melanie Thurber, Nutritionist and Social and Behavioral Change Specialist, will review activities around infant and young child feeding practices, maternal health and nutritional practices, growth monitoring and behavioral change communication strategies to promote behaviors to reduce malnutrition. Adam Reinhart and Suzanne Poland, Agriculture and Livelihoods Specialist will review interventions related to agriculture and livelihoods promotions. They will review approaches designed and implemented by CRS, Save the Children, PCI and their partners to transfer knowledge and practices around agriculture and other livelihood strategies. Interface Between Qualitative and Quantitative Studies The midterm evaluation will apply a primarily qualitative approach, while project monitoring data (depending on the quality) may be used to measure the intermediate outcomes and evaluate the results framework and projects’ performance in achieving beneficiary-level targets. The evaluation team will design the overall approach and will employ a variety of qualitative primary data collection methods, including semi-structured in-depth-interviews, group discussions, key informant interviews, and direct observation. Desktop Review The team will conduct a desktop review of the following documents: 1. Program Proposal Narrative and relevant Attachments 2. Annual Reports and PIRS defining the calculations and sources of annual indicators 3. Pipeline and Resource Estimate Proposal narratives 73 4. Program routine monitoring records and reports 5. Program monitoring tools, including the IPTT and Performance Monitoring Plan 6. M&E plan 7. Food distribution reports: locations, type, quantities and time planned and actual quantities and time delivered 8. SEGAMIL and PAISANO conducted formative research and barrier analyses reports 9. Program commodity management reports and descriptions of processes and tools 10. SERGAMIL and PAISANO Baseline study report 11. Program monitoring data sets either in Excel or SPSS format for analysis by the evaluation team 12. Report on Key Takeaways and Recommendations from the Assessment of the Western Highlands Integration Program (WHIP) in Guatemala. January 11-17, 2015 13. Government of Guatemala Poverty Reduction and food security initiatives strategies and national nutrition plan 14. Feed the Future formative research and barrier analyses Types of Information to be collected The team will carry out interviews with staff of USAID, PVOs, partner NGOs, and host Government officials including the following: • Feed the Future, and Global Heath staff and other USG offices in Guatemala; • SEGAMIL and PAISANO staff and with their implementing partners at both country level office and field level; • Key staff implementing WHIP • Project participants, indirect beneficiaries, non-beneficiary community members, community leaders etc.; • Key Government of Guatemala representatives at both national and local levels; Site visits will include semi-structured in-depth interviews, focus group discussions (with beneficiaries and non-beneficiaries), and program constraints assessments with program managers, technical staff, service providers (frontline staff and volunteers) and beneficiaries focusing on the perceived program effectiveness, constraints inhibiting effectiveness and suggested means of overcoming these constraints. The evaluation team will purposively select an array of interview sites representing different livelihood strategies, program coverage, quality of service delivery, agro-ecological zones, and access to resources and services. 74 The Evaluation Process The entire evaluation process, covering an estimated three months’ time period including four weeks of in-country information collection will involve the following: • Team planning meetings • Scheduling • Development of protocols and data collection instruments • Interview beneficiary, non-beneficiary community members, project staff, volunteers, community leaders and Government officials • Observe knowledge and technology transfer sessions • Review monitoring and other management processes • Validating/ground-truthing meeting with concerned organizations and other primary stakeholders • In country debriefing to primary stakeholders • Preparation of a draft evaluation report • Preparation of a final evaluation report. Overview of Qualitative Data Collection Methodology The team will start reviewing documents and analyzing monitoring data in June before arriving Guatemala City on July 6, 2015. The first few days in Guatemala City will be spent meeting with USAID Mission officials and with program managers of participating organizations, local NGOs and relevant Government officials. The team will spend approximately two weeks in SEGAMIL and two weeks in PAISANO implementation areas. During an average data collection day, the team anticipates the following: • Twelve to fifteen in-depth interviews with some combination of local staff, service providers, implementing partners and volunteers, government counterparts, and community leaders. The team members will split into two groups, visit different communities and conduct interviews simultaneously; (Estimated time allotted for each interview = 30 minutes) • Four to six group meetings, each with an average of 10 participants; (Estimated time allotted for each group meeting = 90 minutes);55 • Four household interviews (carried out by two evaluation teams of two to three persons each) – with selection criteria based on purposive sampling with a particular effort to 55 Group meetings will be single sex and, in the case of women, will not combine older and younger women; separate group discussions with project participants and non-participant community members. 75 interview a reasonable cross-section of beneficiary households. (Estimated time allotted per interview= 45 minutes.) It is expected that some meetings with local staff will be carried out in English with all other meetings, group discussions and household interviews conducted in local language with the help of translators. Field translator/interpreter will also travel with the team during the field work to assist team members who doesn’t speak the local language and provide direct simulations interpreting of proceedings. Team member notes will be taken by individual team members during data collection process. Wherever possible, same sex interviewer and participant will be utilized. Team members will also use photography as a data collection tool. In addition to interviews and groups discussions, the team will observe growth monitoring and promotion (GMP) sessions, knowledge and technology transfer sessions, demonstration plots, pass￾on model demonstrations and other household or community-based activities. Data Analysis The advantage of qualitative methods is to have the ability to conduct real time analysis of the information. Every evening the team will meet to compare notes, triangulate data, share observations to identify patterns, develop hypothetical statements of what happened to look for similar events on the following days and see if the hypothesis holds true. If it doesn't, the team will revise hypothesis. The team members will analyze primary data using commonly used qualitative analytical methods including matrix, trend, and induction analysis and quasi statistics methods. The team will initiate the analysis of collected data as a whole after the completion of field work. Project monitoring data will be analyzed using SPSS. Topical Outline As qualitative methods will be used to collect primary data, instead of developing structured data collection instruments, the team members will develop topical outlines which include key thematic questions. These questions will be developed by individual team members based on the literature review and key evaluation questions. The only exception would be the semi structured interviews. The team will develop instruments for semi-structured interview which include a combination of open-ended and closed questions. The semi-structured interview instruments will also be developed based on the evaluation questions and literature review and will be ready to share one week before the field work begins. The topical outlines, however, indicate only the thematic major line of inquiry. All interesting or unusual responses will be pursued with follow-up questions. 76 The evaluation team will seek consent from the participants (See Appendix A for consent forms.) No names will be recorded in interviews and group discussions. No group session will be started until it is clear that the participants are homogeneous (i.e. no mixing of sexes or of older and younger women). The evaluation team will consult with the local authorities about these processes, and local authorization will be obtained if deemed necessary. The team will use Program Constraints Assessment (PCA), which has been used widely, to assess perceptions of project staff, volunteers and beneficiaries of the two programs, their estimates of efficiency and benefits, to identify constraints, and to elicit optimal means of addressing these constraints. The PCA is generally constructed from the following questions: 1. Are you familiar with the _______ project? (Describe the project, if the name is not familiar.) 2. Can you explain what this project seeks to accomplish? 3. How have you or other members of your household been involved in/affected by this project? For how long have you/they been involved? 4. Please describe any ways you or your family have benefited from the project. 5. Please describe any ways the project has reduced your wellbeing, or that of any member of your family. 6. How effective do you believe the project is in accomplishing its purposes? (These can be explained, if not adequately covered in Question 2.) 7. What constraints do you believe inhibit the project from fully accomplishing its purposes? 8. (For each of these constraints) what is your suggestion on means by which this constraint can be overcome? Note that for project staff and volunteers, the first five questions would be eliminated. The team will organize the constraints identified and the suggestions received during the PCAs in tabular form, with suggestions to address specific constraints grouped into categories of “technical” (those which lend themselves to adjustments at project sites); “policy and processes” (those require project wide methodological adjustments and higher level decisions); and “investigation and training” – where additional training or investigation is needed to address the constraint. Sampling and Participant Selection The evaluation will seek to collect data from each of the geographic areas being covered by each project – being sure to reach both readily accessible and less accessible areas. The evaluation team will request SEGAMIL and PAISANO management to provide lists of all target communities organized by departments. The lists should identify the following information: a) Categorize (near and far) the communities according to proximity to major infrastructure (markets, roads, towns) b) Categorize (early adopters, slow adopters) the communities according to their adoption of project promoted practices 77 c) Identify the communities based on the coverage of services (ag, livelihoods, MCHN, etc.) provided by the projects The evaluation team will purposively select villages/ communities from each categories to get a wider perspective of the challenges faced by the projects and the successful implementation of interventions. In each village/ community visited, the team will identify beneficiary households for interview with the assistance of project staff, volunteers and village leaders and will randomly select individuals for interview to assure that households selected represent the economic and geographic diversity of a village. Schedule The evaluation will proceed according to the timeline indicated in the Gantt Chart attached as Annex C. In changes to be made to the timeline, the evaluation team will discuss this with the partners and will update the Gantt Chart. The team will map out a tentative travel schedule (see Appendix C) which will be added to the protocol after the full team is assembled and discussion with the project managements and USAID Mission in Guatemala. The evaluation team plans to start with a qualitative data collection in the SEGAMIL (CRS) program area as requested by the partners. A physical map of project areas by department is included as part of Appendix D [to be added later on. We need the map from the partners]. Final Report The team’s final report, not to exceed 50 pages, will seek to meet the following criteria: • Represent a thoughtful and well-organized effort to objectively evaluate what has been working well and should be continued as is, what did not and why and how to modify? • Address all evaluation questions included in the scope of work; • Include the scope of work as an annex; • Explain the evaluation methodology in detail. All tools used in conducting the evaluation, such as questionnaires, and checklists will be included in an Annex in the final report; • Disclose limitations to the evaluation, with particular attention to the limitations associated with the evaluation methodology; • Findings should be specific, concise, and supported by evidence from the analyses of the quantitative secondary data from annual monitoring and/or qualitative interviews and observations; • Properly identify and list all sources of information in an annex; • Include recommendations that are supported by evidence and will inform adjustments to the remaining period of the project life. 78 Annex B: Illustrative Informed Consent Forms 1. Interviews with Beneficiaries Hello! My name is XXX. I am here from a team to understand the effectiveness of the services that you receive from PAISANO/SEGAMIL and what changes need to be made to improve effectiveness. You are one of several participants who will be interviewed through this process. During this interview, I will ask you questions related to SEGAMIL/ PAISANO program. Your experience as a beneficiary of the program will be very valuable to our understanding and knowledge of the program. The data collected through this evaluation may also be used for additional analysis purposes in the future. There are no wrong answers to the questions we will be asking in the interview. This evaluation will involve your participation in an interview that will take approximately 30 to 60 minutes. Your participation in this interview is completely voluntary. It is your choice whether to participate or not. There will be no risk as a result of your participation in the interview. You are free to withdraw your consent and stop participating in the interview at any time. Non￾participation will not affect the services/benefits that you usually get. All information given by you will be strictly treated as confidential and you will only be quoted if you give us the permission to do so. The information collected from this interview will be recorded but kept private and will not be shared with anyone outside of the evaluation team, and nothing will be attributed to you by name. Recorded interview/discussion will be destroyed after data retrieval is over. If you have any questions, please feel free to ask me or another member of the evaluation team. We highly appreciate your participation. Are you willing to participate in the study? 1. Yes 2. No Are you willing to be recorded? 1. Yes 2. No Signature of the interviewer: ______________________ Date: ____/_____/2015 Name of the participant: ________________________ Signature of the participant: ________________________ Date: ____/_____/2015 Contact information: Arif Rashid, Email: arashid@usaid.gov, Tel: (202) 216-3516 79 2. Interviews with Key Informants Hello! My name is XXX. I am here from a team to understand the effectiveness of the services that you receive from PAISANO/SEGAMIL and what changes need to be made to improve effectiveness. You are one of several key informants who will be interviewed through this process. During this interview, I will ask you questions related to PAISANO/ SEGAMIL program. Your experience as a ______ will be very valuable to our understanding and knowledge of the program. The data collected through this evaluation may also be used for additional analysis purposes in the future. There are no wrong answers to the questions we will be asking in the interview. We will use approximately 30 to 60 minutes of your time to collect all the information. Your participation in this evaluation is completely voluntary. It is your choice whether to participate or not. Your decision will not affect your job or job-related evaluations in any way. There will be no risk as a result of your participation in the evaluation. You are free to withdraw your consent and stop participating in the interview at any time. All information given by you will be strictly treated as confidential and you will not be quoted. The information collected from this interview will be recorded but kept private and will not be shared with anyone outside of the evaluation team, and nothing will be attributed to you by name. Recorded interview/discussion will be destroyed after data transcription is over. If you have any questions, please feel free to ask me or another member of the evaluation team. We highly appreciate your participation. Are you willing to participate in the evaluation? 1. Yes 2. No Are you willing to be recorded? 1. Yes 2. No Signature of the interviewer: ______________________ Date: ____/_____/2015 Name of the participant: _______________________ Signature of the participant: ________________________ Date: ____/_____/2015 Contact information: Arif Rashid, Email: arashid@usaid.gov, Tel: (202) 216-3516 80 3. Group Discussions Hello! My name is XXX. I am here from a team to understand the effectiveness of the services that you receive from PAISANO/SEGAMIL and what changes need to be made to improve effectiveness. You are one of several participants identified for the group session. During this session, we will ask you questions related to PAISANO/SEGAMIL program. Your experience as a beneficiary of the program [or a community member] will be very valuable to our understanding and knowledge of the program. The data collected through this meeting may also be used for additional analysis purposes in the future. There are no wrong answers to the questions we will be asking in the interview. This evaluation will involve your participation in a group discussion that will take approximately 90 minutes. All information given by you will be strictly treated as confidential and you will not be quoted. The information collected from this meeting will be recorded but kept private and will not be shared with anyone outside of the evaluation team, and nothing will be attributed to you by name. Non￾participation will not affect the services/benefits that you usually get. We cannot assure complete confidentially because other participants may share the information with people outside of the focus group. We request you and other participants to keep the discussions in the group confidential. Your participation in this meeting/discussion is completely voluntary. It is your choice whether to participate or not. There will be no risk as a result of your participation in the evaluation. You are free to withdraw your consent and stop participating in the discussion/interview at any time. Recorded interview/discussion will be destroyed after data transcription is over. If you have any questions, please feel free to ask me or another member of the evaluation team. We highly appreciate your participation. Are you willing to participate in the discussion? 1. Yes 2. No Are you willing to be recorded? 1. Yes 2. No Signature of the interviewer: ______________________ Date: ____/_____/2015 Signature of the interviewer: ______________________ Date: ____/_____/2015 Signature of the participants Date: ____/_____/2015 81 Name: ___________________________ Signature: _______________________ Name: ___________________________ Signature: _______________________ Name: ___________________________ Signature: _______________________ Name: ___________________________ Signature: _______________________ Name: ___________________________ Signature: _______________________ Name: ___________________________ Signature: _______________________ Name: ___________________________ Signature: _______________________ Name: ___________________________ Signature: _______________________ Name: ___________________________ Signature: _______________________ Name: ___________________________ Signature: _______________________ Name: ___________________________ Signature: _______________________ Name: ___________________________ Signature: _______________________ Name: ___________________________ Signature: _______________________ Name: ___________________________ Signature: _______________________ Contact information: Arif Rashid, Email: arashid@usaid.gov, Tel: (202) 216-3516 82 Annex C: Timeline 2 3 4 1 2 3 4 1 2 3 4 1 2 3 4 1 2 3 4 1 2 3 4 1 2 Finalize SoW Meeting with the partners Forming team Develop evaluation protocol Review literatrues and data analysis Tool development Field work - the first trip 6 to 17 July Initiatal analysis and ground truthing workshop July 18 Analsysis and synthesis of the findings in DC 20 through 24 July Meeting and itinerary - the second trip Field work - the second trip July 27 through 7 August Initial analysis and ground truthing workshop August 8 Presentation to USAID August 10 Analsysis and synthesis of the findings August 17 through 28 Drafting report August 24 through September 18 Finalizing report June July August September October Nov 83 Annex B: List of People Interviewed Name Title Organization Community Municipality/ Department 1 Francisco Tzul Program Manager, SEGAMIL ADIPO 2 Maria Rosabel Vasquer Manager Administration, SEGAMIL ADIPO 3 Estras Orozio Field Coordinator, SEGAMIL ADIPO 4 Luis Antonio Vilasquez Fuester Program Accountant, SEGAMIL ADIPO 5 Eunice Joachin Health Technician ADIPO 6 Dina Joachin Agriculture and Livelihoods Technician ADIPO 7 Selvin Estrada Community Organization Technician ADIPO 8 Fredy Buillen ADIPO 9 Demetrio Felix Commodity/ Food Technician ADIPO 10 Brenda Yes MEAL Specialist ADIPO 11 Vimieroi Lipoz Malbwado Program Manager, SEGAMIL Caritas 12 Jorge Armando Rodas Fernandez M&E Coordinator, SEGAMIL Caritas 13 Jorge Rodas MEAL Specialist Caritas 14 Byron Fernando Field Coordinator, SEGAMIL Caritas 15 Monica Rodriguez Chief of Party, SEGAMIL CRS 16 Fernando Erazo SBCC Specialist, SEGAMIL CRS 17 Cecelia Garua de Leon Health and Nutrition Specialist, SEGAMIL CRS 18 Libertad Peusita Gender Specialist, SEGAMIL CRS 84 Name Title Organization Community Municipality/ Department Rodriguez Lopez 19 Francisco Ruiz Head MEAL CRS 20 Gustavo Chew MEAL Coordinator, SEGAMIL CRS 21 Virginia Tzul Health Technician CRS 22 Edwar Garcia Agriculture y Natural Resources, SEGAMIL CRS 23 Patricia Velásquez Savings and Lending, SEGAMIL CRS 24 Miguel Orozco Agroenterprise Development, SEGAMIL CRS 25 Agriculture Coordinator, SEGAMIL CRS 26 Agriculture Leader, SEGAMIL CRS 27 Minra Herrera Lopez Volunteer Counselor PCI 28 Alizeth Vasquez Jimenes Volunteer Counselor PCI 29 Merly Rodriguez Mother Leader PCI 30 Trinidad Lourdes Silvestre Health and Nutrition Technician, PAISANO PCI 31 Maria Alejandro Health Specialist, PAISANO PCI 32 Israel ____ Infrastructure Specialist, PAISANO PCI 33 Lleandro Lopez Pasqual Resilience Coordinator, PAISANO PCI 34 Henry Gómez Women’s Empowerment Technician, PAISANO PCI 35 Luis Escobedo Organization and Food Logistics PCI 85 Name Title Organization Community Municipality/ Department Technician, PAISANO 36 Natan Rivas Livelihoods Technician, PAISANO PCI 37 Omar Gutiérrez Monitoring and Evaluation, PAISANO PCI 38 Erick Tambito Marketing Official, PAISANO PCI 39 Carlos Macario Livelihoods Specialist, PAISANO PCI 40 Rodrigo Arias Chief of Party, PAISANO Save the Children 41 Hector Veliz Warehouse Supervisor, PAISANO Save the Children 42 Francesco Zucaras Administration and Commodity Logistics Coordinator, PAISANO Save the Children 43 Esau Tzoc M&E Technician, PAISANO Save the Children 44 Felipe Perez Monitoring Technician Save the Children 45 Selvyn Fuentes M&E Specialist Save the Children 46 Leonardo Argueta Field Coordinator Save the Children 47 Byron Palacios Community Resilience Supervisor, PAISANO Save the Children 48 Alejandro Cali Livelihoods Specialist, PAISANO Save the Children 49 Leonso Godines Coordinator, North Region, PAISANO Save the Children 50 Wendy Garcia Marketing and Business Development Specialist, PAISANO Save the Children 51 Miguel Angel Chipel Agricultural and Livelihoods Save the Children 86 Name Title Organization Community Municipality/ Department Technician, PAISANO 52 Enrique Bulux Food Supervisor, PAISANO Save the Children 53 Sergio Tello Coordinator and Livestock Advisor CEPROCAL and PAISANO Save the Children 54 Martina Pablo Mendoza Promoter 55 Estela Lopez Health Promoter Tuilahala 56 German R. Gonzalez Dias Secretary SESAN Guatemala City Guatemala 57 Carlos Heer Sentinel Sites Coordinator SESAN Guatemala City Guatemala 58 Natalie Roehrs Manager and Owner Be Waffles Cafés Guatemala City Guatemala 59 Columba Sagastume Consultant for CRS Independent Guatemala City Guatemala 60 Mireya Palmieri Consultant for CRS Independent Guatemala City Guatemala 61 Dan McQuillan Program Manager-Agriculture CRS-Guatemala Guatemala City Guatemala 62 Lorena Méndez Gastronomy Director Saúl Restaurants Guatemala City Guatemala 63 Roly Godinez WASH Specialist, SEGAMIL Caritas Quetzaltenango Quetzaltenango 64 Jorge Sandoval Resilience Supervisor ADIPO Quetzaltenango Quetzaltenango 65 Salvador Casado Resilience Coordinator CRS Quetzaltenango Quetzaltenango 66 Marco Rivera Field Coordinator CRS Quetzaltenango Quetzaltenango 67 Juan Acabal Torres Community Mayor Municipal Authority Jutacaj/Xequemeyaj Momostenango, Totonicapán 68 Cristina Lool Agriculture Technician ADIPO Xequemeya Momostenango, Totonicapán 69 Catarina Health Promoter ECADI Xequemeya Momostenango, Totonicapán 70 Augusto Escobar Municipal Manager Municipal Government Momostenango Momostenango, 87 Name Title Organization Community Municipality/ Department Totonicapán 71 Esaú Guerra Municipal Representative SESAN Momostenango Momostenango, Totonicapán 72 Gabriel Velasquez Acabal President Municipal Authority Pasuc Momostenango, Totonicapán 73 Moise Pelico Chanchabaj Secretary Municipal Authority Pasuc Momostenango, Totonicapán 74 Santiago Coxaj Pelico President COCOSAN Pasuc Momostenango, Totonicapán 75 Lidia Zarat Lopez Coordinator COCOSAN/COLRED Xequemeya Momostenango, Totonicapán 76 Carmelita Zarate Assistant Coordinator COCOSAN/COLRED Xequemeya Momostenango, Totonicapán 77 Efrain Zarate Secretary COCOSAN/COLRED Xequemeya Momostenango, Totonicapán 78 Marino Hernández Resilience Technician ADIPO Momostenango Momostenango, Totonicapán 79 German Director Public School Tzanixnam Totonicapán 80 Hipolito Mayor Assistant Auxiliary Mayor Tzanixnam Totonicapán 81 Francisco Coordinator COLRED - COCOSAN Tzanixnam Totonicapán 82 Paulina Tacam Agriculture Promoter ECADI Tzanixnam Totonicapán 83 Isaias Garcia Tzoc Health Technician ADIPO Chuisactol Centro Santa Maria Chiquimula, Totonicapán 84 Maria León Castro Health Promoter ECADI Chuisactol Centro Santa Maria Chiquimula, 88 Name Title Organization Community Municipality/ Department Totonicapán 85 Juan Francisco WASH Specialist, SEGAMIL Caritas San Marcos San Marcos 86 Michelle Fuentes Risk Management Coordinator Caritas San Marcos San Marcos 87 Evelyn Macal Food Coordinator Caritas San Marcos San Marcos 88 Victor Santizo Agroenterprise Technician Caritas San Marcos San Marcos San Marcos 89 Raul Orozco Coordinator Productivity Component Caritas San Marcos San Marcos San Marcos 90 Juan Carlos Miranda Agriculture and Livestock Facilitator Caritas San Marcos San Marcos San Marcos 91 José Ignacio López Community Organization Facilitator Caritas San Marcos San Marcos San Marcos 92 Licda. Miriam Marleny Mérida Service Promotion Coordinator Health Area San Marcos San Marcos 93 Licda. Carolina Reyna de León Service Promotion Assistant Health Area San Marcos San Marcos 94 Licda. Mayra Isabel Krishe Nutrition Program Coordinator Health Area San Marcos San Marcos 95 Dr. Eunice Fabiola de León Reproductive Health Program Coordinator Health Area San Marcos San Marcos 96 Licda. Thelma de León Díaz Regional Coordinator SESAN San Marcos San Marcos 97 Fredy Velásco President COCODE Chanchicupe San Marcos 98 Lucio Antonio Mazariegos President Association Chanchi￾Miel Chanchcupe Tajumulco, San Marcos 99 Martin Aguilón López President Association for the Integral Development Aldea Piedra de Fuego Comitancillo, San Marcos 89 Name Title Organization Community Municipality/ Department of Piedra de Fuego Village, ADIPI 100 Isabel Pacheco Coordinator COCOSAN Xequixtun/Xemanzana Cunén, Quiche 101 Lidia Carmen Minutes and Finance COCOSAN Xequixtun/Xemanzana Cunén, Quiche 102 Petronila Pérez Food Logistics COCOSAN Xequixtun/Xemanzana Cunén, Quiche 103 Lucy Vicenta Nutrition Promoter COCOSAN Xequixtun/Xemanzana Cunén, Quiche 104 Candelaria López Livelihoods Promoter COCOSAN Xequixtun/Xemanzana Cunén, Quiche 105 Juan Tiu Tum President COCODE Xequixtun/Xemanzana Cunén, Quiche 106 Domingo Coj Secretary COCODE Xequixtun/Xemanzana Cunén, Quiche 107 Alcaldia Auxiliar Media Luna Mayoral Assistant Municipal Council Media Luna Cunén, Quiche 108 Serbando Rivas Pérez Legal Representative Asociación de Cooperación Integral Cunénense Trigales Cunén, Quiche 109 Carlos Joachin Resilience Specialist Save the Children Sta Cruz del Quiche Sta Cruz del Quiche, Quiche 110 Members of both COLRED and COCOSAN COLRED, COCOSAN Aldea las Marías Uspantan, Quiche 111 María Magadalena Saybin Lead Mother PAISANO Aldea las Marías Uspantan, Quiche 112 Oralia Blanca Saybin Nutrition Promoter PAISANO Aldea las Marías Uspantan, Quiche 113 Victor Chic Lead Farmer PAISANO Aldea las Marías Uspantan, Quiche 114 Santiago Braulio Cabrera President COCODE El Tizate San Juan Ostuncalco, Quetzaltenango 115 Daniel Braulio Member COCODE El Tizate San Juan 90 Name Title Organization Community Municipality/ Department Ostuncalco, Quetzaltenango 116 Bonifacio Sánchez Member COCODE El Tizate San Juan Ostuncalco, Quetzaltenango 117 Ema Gloria López Minutes and Finance COSAN El Tizate San Juan Ostuncalco, Quetzaltenango 118 Ofelia Juárez Food Logistics COSAN El Tizate San Juan Ostuncalco, Quetzaltenango 119 Eva Cabrera Nutrition Promoter PAISANO El Tizate San Juan Ostuncalco, Quetzaltenango 120 Maria Clarinez Lorenzo Nutrition Promoter PAISANO El Tizate San Juan Ostuncalco, Quetzaltenango 121 Maria Isabel Hernández Lead Mother PAISANO El Tizate San Juan Ostuncalco, Quetzaltenango 122 Manuel Sánchez Lead Farmer PAISANO El Tizate San Juan Ostuncalco, Quetzaltenango 123 Josué Rodas Municipal Representative SESAN San Juan Ostuncalco San Juan Ostuncalco, Quetzaltenango 124 Victoria Cuzán Nutritionist Health District San Juan Ostuncalco San Juan 91 Name Title Organization Community Municipality/ Department Ostuncalco, Quetzaltenango 125 Sandra Díaz de Gómez Cash Transfer Coordinator Ministry of Development San Juan Ostuncalco San Juan Ostuncalco, Quetzaltenango 126 Leandro López Resilience Specialist PCI Guatemala Huehuetenango Huehuetenango 127 Angel ____ Health and Nutrition Technician, PAISANO PCI Paya Huehuetenango 128 Milena Martínez Leader, Women Empowered Group Nojoya San Antonio Huista, Huehuetenango 129 Cipriani Cruz Livelihoods Representative COCOSAN Secheu Concepción Huista, Huehuetenango 130 Santos Alejandro Geronimo Agricultural Promotor Secheu Concepción Huista, Huehuetenango 131 Doroteo Mendoza Community Mobilization Promoter COCOSAN Tojchoj Todos Santos Cuchumatán, Huehuetenango 132 Felipe Mendoza Minutes and Finance COCOSAN Tojchoj Todos Santos Cuchumatán, Huehuetenango 133 Humberto Mendoza Livelihoods Promoter COCOSAN Tojchoj Todos Santos Cuchumatán, Huehuetenango 134 Modesta Mendoza Food Logistics Promoter COCOSAN Tojchoj Todos Santos 92 Name Title Organization Community Municipality/ Department Cuchumatán, Huehuetenango 135 Elsy Petronila Coordinator COCOSAN Ojo de Agua San Juan Cotzal, Quiché 136 Susan Noemi Chamal Minutes and Finance COCOSAN Ojo de Agua San Juan Cotzal, Quiché 137 Rosa Castro Livelihoods Promoter COCOSAN Ojo de Agua San Juan Cotzal, Quiché 138 Juana Chilel Health Promoter COCOSAN Ojo de Agua San Juan Cotzal, Quiché 139 Juan Castro López Food Logistics Promoter COCOSAN Ojo de Agua San Juan Cotzal, Quiché 140 Gaspar Chiroy Community Mobilization Promoter COCOSAN Ojo de Agua San Juan Cotzal, Quiché 141 Pedro Lux Secretary COCODE Ojo de Agua San Juan Cotzal, Quiché 142 Diego Chirol President COCODE Ojo de Agua San Juan Cotzal, Quiché 143 Miguel Ixcotoyaj Member COCODE Ojo de Agua San Juan Cotzal, Quiché 144 Juan Us Quiniyá Water Committee President COCODE Ojo de Agua San Juan Cotzal, Quiché 145 Margarita Sánchez Coordinator COCOSAN Juil San Gaspar Chajul, Quiché 146 Benito Santiago López Livelihoods Promoter COCOSAN Juil San Gaspar Chajul, Quiché 147 Juana Treasurer COCOSAN/COSAN Aldea chemal Chajul, Quiché 93 Name Title Organization Community Municipality/ Department 148 Sebastian Lopez Auxiliary Mayor/COSAN Coordinator COCOSAN/COSAN Aldea chemal Chajul, Quiché 149 Margarita Food Logistics COCOSAN/COSAN Aldea chemal Chajul, Quiché 150 Macaria Lopez Food/Nutrition Promoter COCOSAN/COSAN Aldea chemal Chajul, Quiché 151 Gaspar Lopez Food Logistics COCOSAN/COSAN Aldea chemal Chajul, Quiché 94 Annex C: List of Communities Visited Department Municipality Community Trip NGO 1 Totonicapán Santa Lucía La Reforma Canton Ichomchaj 1st CRS 2 Totonicapán Totonicapán Paraje Centro Tzanixnam 1st CRS 3 Totonicapán Momostenango Caserio Pasuc I 1st CRS 4 Totonicapán Momostenango Xequemeya Centro 1st CRS 5 Totonicapán Santa María Chiquimula Chuisactol Centro 1st CRS 6 Totonicapán Momostenango Racana-Chequemeya Centro 1st CRS 7 San Marcos Comitancillo Aldea Tuisacaja 2nd CRS 8 San Marcos Tajumulco Aldea Chanchicupe 2nd CRS 9 San Marcos Tajumulco Villa Hermosa 2nd CRS 10 San Marcos Comitancillo Aldea Piedra de Fuego 2nd CRS 11 Quiché Cunén Media Luna 1st SC 12 Quiché Cunén Xemanzana 1st SC 13 Quiché Cunén Trigales 2nd SC 14 Quiché Uspantan Las Marias 1st SC 15 Quiché Uspantan Choton 1st SC 16 Quiché Uspantan San Antonio Chiquito 1st SC 17 Quiché San Juan Cotzal Ojo De Agua 2nd SC 18 Quiché San Juan Cotzal Chichel 2nd SC 19 Quiché Chajul Xemal O Nuevo Xemal 1 2nd SC 20 Quiché Chajul Juil 2nd SC 21 Quetzeltenango Concepcion Chiquirichapa Talmax 1st SC 22 Quetzeltenango San Juan Ostuncalco El Tizate 1st SC 23 Huehuetenango Concepcion Huista Secheu 2nd SC 24 Huehuetenango San Antonio Nojoya 2nd SC 25 Huehuetenango San Antonio Cajul 2nd SC 26 Huehuetenango Todos Santos Tujchoj 2nd SC 95 Annex D: Annual Survey Indicators – PAISANO The annual survey collects data on the following indicators: 1. Percentage of farmers who have received training in crop and animal production 2. Percentage of farmers who have used at least two sustainable agricultural practices (crop management) practices during the past 12 months. 3. Percentage of farmers who have used at least two sustainable agricultural practices (livestock) practices during the past 12 months. 4. Percentage of farmers who have used at least two sustainable agricultural practices (natural resource management) practices during the past 12 months. 5. Percentage of farmers who have used improved storage practices during the past 12 months 6. Percentage of farmers who have used at least three sustainable agricultural practices (cumulative from above) practices during the past year. 7. Percentage of mothers who have received health and nutrition counseling during the past 12 months 8. Percentage of households that treat water prior to consumption in their homes / type of treatment utilized 9. Percentage of women who recognize at least two danger signs (or more) during pregnancy and detect the need to seek medical care 10. Percentage of mothers who recognize at least two danger signs (or more) in infants (< 28 days) and detect the need to seek medical care 11. Percentage of mothers who recognize at least two danger signs in children and detect the need to seek medical care 12. Percentage of pregnant women who sought medical care when danger signs were detected during their last pregnancy 13. Percentage of mothers or caregivers who sought medical care when danger signs were detected in their children 14. Percentage of mothers or caregivers who sought medical care when danger signs were detected in newborns 15. Prevalence of acute, global and chronic malnutrition in children under 30 months 16. Prevalence of exclusive breast-feeding among children less than six months of age 17. Prevalence of children between 6 and 23 months who receive a minimal accepted diet 18. Percentage of mothers of children under 30 months that have reported making decisions by themselves or jointly with their spouse in relation to seeking health care for themselves 19. Percentage of mothers of children under 30 months that have reported making decisions by themselves or jointly with their spouse in relation to seeking health care for their children