Mercy Corps Guatemala PROCOMIDA Program: Award Number: AID-FFP-A-09-00005-00 Final External Evaluation Report: PROCOMIDA’s Community Food Diversification Program for Mother and Child (Programa Communitario Materno Infantil de Diversificación Alimentaria) December 2016 Evaluation contributors: i PROCOMIDA EVALUATION DETAILS Intervention Title Programa Communitario Materno Infantil de Diversificación Alimentaria / Community Food Diversification Program for Mother and Child (PROCOMIDA) Award Number: AID-FFP-A-09-00005-00 Scope: Independent Final Evaluation Country: Guatemala Evaluation Date: 10 July 2015 (end of field data collection) Principle Consultants: Final Evaluation Re-analysis and Re-drafting Consultants: Evaluation Managers: PROCOMIDA Approval Date: July 2009 PROCOMIDA End Date: May 30, 2015 (field intervention) November 30, 2015 (closing date) Budget US$ 37.9 million Key Words: Nutrition, Food aid, Food security, Guatemala, PM2A, FANTA III, FFP, USAID, MYAP, M&E, MEL, Final Evaluation ii ACKNOWLEDGEMENTS , acting as an external international evaluation consultant team. The consultant team implemented evaluation fieldwork from April - July 2015 under the guidance of the PROCOMIDA team, Guatemala. , reviewed endline data for statistical relevance and comparison with baseline results, reconfigured data presentation, consolidated and integrated reviewer comments, and reorganized and edited the final evaluation report. The evaluation drew from the input provided by a wide range of individuals, both internal and external to the PROCOMIDA program. Special thanks goes to the PROCOMIDA management team including , to the monitoring and evaluation (M&E) team including , and , and to at Mercy Corps Headquarters. ABOUT THIS REPORT This report begins with background information on food insecurity and malnutrition in Guatemala, and the PROCOMIDA program. It then presents the endline evaluation methodology. The discussion of final evaluation findings begins with program characteristics, and then results based on Indicator Performance Tracking Table (IPTT) objectives by the percentage of target met in comparison to baseline figures. The report concludes with lessons and recommendations. 3 CONTENTS List of Figures .............................................................................................................................v List of Tables ............................................................................................................................. vi Acronyms.................................................................................................................................. vii Executive Summary ................................................................................................................... 1 Background ............................................................................................................................ 1 Methodology........................................................................................................................... 1 Results ................................................................................................................................... 2 1 Introduction ............................................................................................................................. 5 1.1 Guatemala Context........................................................................................................... 5 1.2 Program Intervention Logic............................................................................................... 6 2 Endline Evaluation .................................................................................................................. 7 2.1 Purpose and Scope .......................................................................................................... 7 2.2 Methods ........................................................................................................................... 7 2.2.1 Study Design.............................................................................................................. 7 2.3 Quantitative Surveys ........................................................................................................ 8 2.3.1 Sampling.................................................................................................................... 8 2.3.2 Data Collection........................................................................................................... 9 2.3.3 Monitoring and Evaluation Framework ......................................................................10 2.3.4 Analysis ....................................................................................................................10 2.4 Qualitative Fieldwork .......................................................................................................11 2.4.1 Sampling...................................................................................................................12 2.4.2 Data Collection..........................................................................................................12 2.4.3 Analysis ....................................................................................................................13 2.5 Limitations .......................................................................................................................13 3 Findings .................................................................................................................................14 3.1 Program Description........................................................................................................14 3.1.1 Key Baseline Findings...............................................................................................14 3.1.2 Program Design ........................................................................................................15 3.2 Program Research ..........................................................................................................21 3.3 Program Resources.........................................................................................................22 3.4 Program Participation ......................................................................................................23 3.5 Program Monitoring .........................................................................................................24 3.6 Program Sustainability.....................................................................................................24 3.7 Program Results..............................................................................................................25 3.7.1 IPTT..........................................................................................................................25 4 3.7.2 Discussion of Results................................................................................................30 3.7.3 Trigger Indicators ......................................................................................................51 4 Conclusions, Lessons, and Recommendations......................................................................53 4.1 Conclusions.....................................................................................................................53 4.2 Lessons and Recommendations......................................................................................56 References ...............................................................................................................................58 Appendices ...............................................................................................................................60 5 LIST OF FIGURES Figure 1: Key sampling and sample size process flow ............................................................... 9 Figure 2: Number of rations distributed .....................................................................................19 Figure 3: Participation rates at PROCOMIDA training events (%) .............................................20 Figure 4: Amount of voluntary contributions and use per year...................................................23 Figure 5: Household hunger scale at baseline and endline .......................................................33 Figure 6: Average HDDS at baseline and endline .....................................................................34 Figure 7: Increased mother knowledge of child nutrition at baseline and endline ......................36 Figure 8: Mother knowledge on child nutrition at baseline and endline......................................36 Figure 9: Mother knowledge of danger signs during pregnancy at baseline and endline ...........36 Figure 10: Mother knowledge of danger signs of childhood illnesses at baseline and endline ...38 Figure 11: Mothers receiving prenatal care assistance at baseline and endline ........................39 Figure 12: Mothers receiving minimum recommended antenatal and postnatal care at baseline and endline ...............................................................................................................................39 Figure 13: Institutions visited for prenatal care at baseline and endline.....................................40 Figure 14: Place of delivery and assisting staff during delivery at baseline and endline ............40 Figure 15: Prevalence of children immunized and measured for growth at baseline and endline .................................................................................................................................................41 Figure 16: Essential newborn care at baseline and endline.......................................................42 Figure 17: Children receiving minimum treatment for respiratory problems and diarrhea at baseline and endline .................................................................................................................42 Figure 18: Breastfeeding practices among mothers at baseline and endline.............................43 Figure 19: Percentage of children 0-6 months exclusively breastfed at baseline and endline....44 Figure 20: Dietary diversity in children 6-24 months at baseline and endline............................45 Figure 21: Complementary feeding practices at baseline and endline.......................................45 Figure 22: Percentage of children with measurements classified as wasting, stunting, and underweight, by age at baseline and endline ............................................................................47 Figure 23: Availability of basic services at CHCs at baseline and endline .................................49 Figure 24: CHC staff knowledge regarding danger signs and ORS use at baseline and endline .................................................................................................................................................49 Figure 25: CHC staff knowledge of child feeding practices at baseline and endline ..................49 Figure 26: Availability of medical supplies at CHCs at baseline and endline .............................51 6 LIST OF TABLES Table 1: Description and values of parameters used in the sample size calculation................... 9 Table 2: Distribution of communities and interviewed households per district ............................ 9 Table 3: Evaluation criteria for the exclusion of anthropometric outliers ....................................11 Table 4: Total valid observations in final databases ..................................................................11 Table 5: Sex and literacy levels of FGD participants per community (# and %).........................12 Table 6: Results framework.......................................................................................................16 Table 7: Distribution of monthly rations .....................................................................................18 Table 8: Summary of food aid program resources in US$ .........................................................22 Table 9: Amount and use of financial contributions from participants to PROCOMIDA..............23 Table 10: IPTT – baseline, target, endline, and percent achievement .......................................26 Table 11: Background characteristics of surveyed households, mothers, and children, at baseline and endline ...............................................................................................................................32 Table 12: Nutritional status of children at baseline and endline .................................................47 Table 13: Distribution of stunting among children 6-59 months (<-2 z-score height-for-age), by sex, at baseline and endline......................................................................................................47 Table 14: Adverse conditions and events affecting Alta Verapaz at baseline and endline .........52 Table 15: Evaluation ratings......................................................................................................53 vii ACRONYMS ACF Action Against Hunger BCC Behavior Change Communication BL Baseline BMI Body Mass Index CC Convergence Center CDC Centers for Disease Control and Prevention CHC Community Health Commission CHV Community Health Volunteer CHW Community Health Workers COCODE Consejo Comunitario de Desarrollo / Community Development Council CRS Catholic Relief Services CSB Corn-soy Blend CSI Coping Strategy Index DFAP Development Food Assistance Program EL Endline ENA Emergency Nutrition Assessment ENSMI Encuesta Nacional de Salud Materno Infantil / National Survey of Maternal and Child Health FAFSA Food Aid and Food Security Assessment FANTA Food and Nutrition Technical Assistance FEWSnet Famine Early Warning System Network FFP Food for Peace FGD Focus Group Discussion FY Fiscal Year GDP Gross Domestic Product H-A Height-for-age HAZ Height-for-age z-score HDDS Household Diet Diversity Scale HFIAS Household Food Insecurity Access Scale HHS Household Hunger Scale HP Health Post IFPRI International Food Policy Research Institute IPTT Indicator Performance Tracking Table IR Intermediate Result IY Implementation Year IYCF Infant and Young Child Feeding KG Kilogram KII Key Informant Interview KM Kilometers LNS Lipid-based Nutritional Supplements M&E Monitoring and Evaluation MCU Mother-Child Unit MCHN Maternal Child Health and Nutrition MIS Management Information System MNP Micronutrient Powder (Chispitas) MOH Ministry of Health and Social Assistance MT Metric Ton MYAP Multi-year Assistance Program N Sample Size NGO Non-governmental Organization OECD Organization for Economic Cooperation and Development ORS Oral Rehydration Salts PEC Programa de Extensión de Cobertura (Ministry of Health’s Extension of Coverage Program) PM2A Preventing Malnutrition in Children Under 2 Approach PPS Probability Proportional to Size PREP Pipeline and Resource Estimate Proposal PROCOMIDA Programa Comunitario Materno Infantil de Diversificación Alimentaria / Community Food Diversification Program for Mother and Child Q Quartiles SAM Severe Acute Malnutrition SBC Social and Behavior Change SD Standard Deviation SESAN Secretaria de Seguridad Alimentaria y Nutricional / Secretariat of Food and Nutritional Security SIAS Sistema Integral de Atención en Salud / Integrated Health Attention System SO Strategic Objective SOW Scope of Work SPSS Statistical Package for Social Sciences STATA Data Analysis and Statistical Software (branch name) USAID United States Agency for International Development WAZ Weight-for-age z-score WHO World Health Organization WHZ Weight-for-height z-score Y Year viii 1 EXECUTIVE SUMMARY Background Guatemala has some of the highest rates of chronic malnutrition in Latin America, and in the world. Conditions are historically most severe in the northern lowland and highland areas converging in and around Alta Verapaz. Within these geographic areas, the most vulnerable and affected by food insecurity and malnutrition are i) pregnant and lactating women, and ii) children under two years of age. This is particularly true amongst poor, marginalized, and vulnerable indigenous and ethnic communities, including the Q’eqchí. Programa Communitario Materno Infantil de Diversificación Alimentaria (Community Food Diversification Program for Mother and Child/PROCOMIDA) was a six-year program1 funded by the United States Agency for International Development (USAID) through the Bureau of Democracy, Conflict, and Humanitarian Assistance Office of Food for Peace (FFP), and implemented by Mercy Corps/Guatemala. PROCOMIDA began in July 2009, and field implementation (which began June 2010) concluded in May 2015. PROCOMIDA sought to improve the nutritional status of direct and indirect participants in 936 vulnerable communities in the Alta Verapaz Department, Guatemala. PROCOMIDA directly reached 60,605 mother-child units 2 in 1,180 vulnerable communities. The total number of direct and indirect beneficiaries was 266,000 community and family members, with an average family size of 5.3. PROCOMIDA worked with non-governmental organizations (NGO) and Ministry of Health and Social Assistance (MOH3 ) units to strengthen community-level health structures (convergence centers/CC) to improve culturally and technically appropriate health service provision, and to stimulate demand for these services. It provided participants at 356 CCs with monthly training sessions, health and nutrition counseling, and additional household visits combined with the provision of one of five different types of food rations. Methodology This final independent evaluation coincided with the conclusion of PROCOMIDA’s field implementation (service and food delivery) on 31 May 2015. The purpose was to evaluate the performance of PROCOMIDA in targeted populations, assessing for relevance, effectiveness, impact, and sustainability. This implies evaluating the program’s achievements in meeting objectives and targets set against baseline values. We intend lessons and recommendations for improvement of future Title II Development Food Assistance Programs and similar activities. The evaluation design was steered by general evaluation criteria from the Organization for Economic Cooperation and Development (OECD) Development Assistance Committee, guidelines for the evaluation of Preventing Malnutrition in Children under Two Years of Age Approach (PM2A) interventions, by USAID, and by the Food and Nutrition Technical Assistance (FANTA) project. 4 Field data collection occurred between April - July 2015. Activities included: i) a document review; ii) quantitative surveys; iii) anthropometric measurements; and iv) qualitative fieldwork. Analysis occurred in fall 2015, with an initial evaluation submitted in November 2015, and a re-drafted final evaluation report submitted in March 2017. 1 PROCOMIDA was originally defined as a ‘project.’ Given the scope and complexity of the intervention, it has the characteristics of a program, and thus this report refers to PROCOMIDA as a ‘program.’ 2 PROCOMIDA used a measurement unit of ‘Mother-Child Unit’ (MCU), which uniquely identified the beneficiary through the 1000 days approach (pregnant mother, lactating mother, child six to 24 months). 3 Local acronym is MSPAS. 4 The program implementation started during the FANTA II project and ended during the FANTA III project. Hence, in this report both project will be referred to as ‘FANTA project’. 2 Results PROCOMIDA improved the nutritional status and health of women and children vulnerable to food insecurity in northern Guatemala (program goal). PROCOMIDA’s two strategic objectives were: SO.1: Pregnant and lactating women, children under two, and malnourished children under five in program areas have improved and sustainable health and nutrition status Activities were appropriate, technically sound, and efficiently implemented. Activities and results materialized as expected, primarily operating through consistent nutritional supplementation of children and their mothers, along with increased dietary diversification among mother-child units (MCU). Core nutritional outcomes and their level of achievement against target include:  Indicator 1: % of children 0-59 months underweight – baseline 13.3%; endline 14.0% (71% achievement) – not statistically significant, p=0.2570.  Indicator 2: % of children 6-59 months stunted – baseline 59.2%; endline 53.5% (99% achievement) – statistically significant, p=0.002.  Indicator 3: Average Household Dietary Diversity Score (HDDS) – baseline 7.19; endline 7.26 (91% achievement) – not statistically significant, p=0.2899.  Indicator 5: % of children 6-24 months with minimum dietary diversity – baseline 60.1%; endline 71.6% (90% achievement) – statistically significant, p<0.0001.  Indicator 7: % of caregivers demonstrating increased nutritional knowledge – baseline 0%; endline 52.6% (75% achievement)  Indicator 9: % of children 0-6 months who were exclusively breastfed – baseline 65.1%; endline 75.7% (89% achievement) – statistically significant, p=0.0411. SO.2: Healthcare service providers at community through municipal levels have improved service quality and delivery PROCOMIDA’s activities focused on strengthening the quality and availability of primary healthcare services in rural areas through a model of NGO service delivery through the MOH Programa de Extensión de Cobertura (Extension of Coverage Program/PEC). Communities benefitted from strengthened local structures for self-management (establishment of health commissions and emergency funds; developed local health planning capacity) and improved infrastructure (store rooms, primary healthcare centers). Intervention success depended largely on the PEC’s budget and MOH provisions, which were outside of PROCOMIDA’s control. Issues of local and national governance interrupted primary healthcare service delivery during implementation periods, limiting the program’s ability to achieve outcomes, and exposing the fragility of the PEC model. The themes addressed under these two objectives were highly relevant, despite activities being limited to those pre-determined by the donor. Despite challenges, PROCOMIDA managed efficient product and service delivery in a complex geo-cultural environment, benefitting thousands of community members. While program participation was voluntary, 88.9% of eligible mothers and children benefitted from their engagement in PROCOMIDA during the implementation period. Key achievements include: Improved nutrition. PROCOMIDA presented an opportunity for participating children to improve their future potential. Econometric outcome assessment and undernutrition trends showed high long-term impact for children who participated, with positive results from nutritional improvements felt well beyond program graduation, specifically: 3  Improved maternal nutrition contributing to the improved growth and development of their children – over 34,000 pregnant and lactating women received food rations  Improved and consistent access to diverse and nutritious foods for children 6-24 months – 71.6% obtained a minimum dietary diversity; 57,590 received food rations  Reduced levels of stunting in children under five years, with enduring benefits through their lives The difference between participant and non-participant results is statistically significant. Households that participated in the program did so for a range of six to 30 months. Graduated households demonstrated increased efforts to reduce their own vulnerability to food insecurity: maintaining a diverse diet, improving household environments, and adopting behaviors related to improved infant and young child feeding (IYCF) and caring practices. Participating families expressed a desire for even further information on nutrition, childcare, illness warning signs, and family planning. They demonstrated an eagerness to continue using knowledge gained to benefit their current and future children. PROCOMIDA concentrated on support (skills, knowledge, resources, structures) at the local level and within existing health systems, and deliberately promoted the adoption of actionable behaviors to address barriers. While there is strong probability that nutritional gains for children under two years who participated in the program will endure, resulting in healthier and more productive adults, there is only moderate likelihood that: i) mothers and families will function in environments enabling application of program knowledge and skills (health behavior and capacities) to effectively protect future pregnancies and early childhood development; and ii) the public health sector will be able to continue program improvements given existing structural barriers (current health system, skills, resources). Increased food consumption ended with program graduation, and improved knowledge does not guarantee the adoption of desired behaviors, particularly in areas with extreme poverty and income insecurity. A stronger focus on community structures and exit strategies, that strengthened and complemented existing health structures may have compensated for this. Decreased stunting. PROCOMIDA achieved a decrease in annual stunting of 1.14 percentage points per year, a considerable improvement when compared against Alta Verapaz national survey results from 1999-2009 showing an average decrease of only 0.17%/year,5 and an overall national decrease of 0.55%/year during PROCOMIDA’s implementation period in survey results from 2014-2015. 6 A 2014 national survey conducted by SESAN for the Zero Hunger Pact in 166 prioritized municipalities (including all PROCOMIDA municipalities) found 60.6% stunting (H/A, rural areas 65.4%), 1.3% wasting (W/H, rural areas 1.4%), and 14.3% underweight (W/A, rural areas 15.6%) in children three to 59 months of age. 7 Although the latter age range is slightly different, the variance in stunting levels is indicative. When compared to the national and regional levels, PROCOMIDA was able to double the national annual stunting decrease. The average reduction for Latin American is 1.54%/year, and globally it is 1.32%/year. 8 5 MSPAS (2011). Encuesta Nacional de Salud Materno Infantil 2008 (ENSMI-2008/09). Ministerio de Salud Pública y Asistencia Social (MSPAS)/Instituto Nacional de Estadística (INE)/Centros de Control y Prevención de Enfermedades (CDC). Guatemala 6 MSPAS (2015). VI Encuesta Nacional de Salud Materno Infantil (ENSMI-2014/15). MSPAS/INE/Secretaría de Planificación y Programación de la Presidencia (Segeplán). Guatemala. 7 SESAN (2015). Tercera Encuesta de Monitoreo en los 166 municipios priorizados; Tomo I: Estado Nutricional de Menores de Cinco Años y Mujeres en Edad Fertil. Guatemala. 8 van Haeften, R, et.al. (2013). Second Food Aid and Food Security Assessment (FAFSA-2). Washington, DC: FHI 360/FANTA. 4 Healthy Timing and Spacing of Pregnancies. PROCOMIDA contributed to improved healthy pregnancy timing and birth spacing among program participants. Household survey data measured this, and sources that participated in qualitative fieldwork confirmed it. Coordinated and simultaneous interventions of other organizations (family planning, contraception distribution) complemented PROCOMIDA’s family planning activities in implementation areas. Gender empowerment. As a crosscutting programmatic theme, gender results show greater female participation, as learned primarily from qualitative fieldwork. In particular, women were more involved in decision-making around health and nutrition. Increases were noted in mother’s knowledge on child nutrition, danger signs of child illness, and danger signs during pregnancy. The development of home gardens and the overall increased role in food production enabled them to control dietary diversification for their families. Pertaining to gender equity, results were weaker. The almost exclusive concentration on the participant MCU limited involvement of other family members, including fathers, in-laws, and siblings. This was a lost opportunity to reinforce the drivers of change, only realized in the late stages of implementation when the program did initiate work with men and key family members including mothers-in-law. Strengthened community arrangements. Communities benefitted from strengthened community-based structures (health commission, emergency fund, health planning mechanisms) and improved infrastructure (program store rooms, health centers). It is unclear how well new first level health services will engage with community arrangements strengthened by the program. Community leaders intend to maintain and utilize infrastructure, co-opting them to serve as community centers with an aim of furthering programmatic messages and behaviors. 5 1 INTRODUCTION Food insecurity disproportionately affects the poor, indigenous, smallholder farmers, and landless populations in rural communities. In Latin America, including in Guatemala, food insecurity results in high rates of chronic child malnutrition. Programa Communitario Materno Infantil de Diversificación Alimentaria (Community Food Diversification Program for Mother and Child, or PROCOMIDA), a six-year program (2009 – 2015) funded by USAID through FFP, and implemented by Mercy Corps/Guatemala, aimed to improve the nutritional status and health of women and children vulnerable to food insecurity in northern Guatemala. 1.1 Guatemala Context Guatemala presents with some of the worst rates of chronic malnutrition in the world. Despite a strong institutional framework and established coordination structures supporting food security programming, Guatemala has the highest prevalence of stunting (49.8%) in children under five in Latin America, and among the worst rates of chronic malnutrition globally. 9 Conditions are most severe in the western highlands and the northern lowland and highland areas converging in and around the Alta Verapaz Department. Within this geographic area, the most vulnerable and affected by food insecurity and malnutrition are pregnant and lactating women, and children under two years of age, exacerbated amongst poor, marginalized, and vulnerable indigenous and ethnic communities, including the Q’eqchí. 10 In Alta Verapaz, malnutrition is especially harmful to children under two as they can experience lifetime impairment due to weak physical and cognitive development. Undernutrition causes 45% of child deaths globally. 11,12 In Alta Verapaz, 60% of children under five are chronically malnourished, while the infant mortality rate for 2014-2015 was 29 per 1,000 live births, putting Alta Verapaz at a ranking equivalent of 131 st (from best to worst) out of 195 countries.13 Overall, the maternal mortality ratio in Guatemala was 140 per 100,000, ranking equivalent to 122 nd out of 184 countries. 14,15 Evidence suggests that investments in nutrition improve national productivity and economic performance (gross domestic product/GDP per capita) by countering direct productivity losses, losses via poorer cognition, and losses via reduced schooling. 16 Despite macroeconomic stability and 6.5% GDP growth in Guatemala between January 2010 and December 2014, fiscal health structures remained unchanged, and poor and marginalized populations underserved. 17 Populations in Alta Verapaz largely pursue subsistence agriculture (corn and beans) with some cash crops (coffee, cocoa, cardamom), which suffered from pest-related diseases during periods of PROCOMIDA implementation. According to the coping strategy index (CSI), 18 by program end, 9 FAO (2014). Guatemala Country Fact Sheet on Food and Agriculture Policy Trends: Socio-economic Context and Role of Agriculture. http://www.fao.org/3/a-i4124e.pdf. 10 FEWSnet Guatemala Country Profile Website; FAO Socioeconomic Studies of Vulnerable Groups in Guatemala. 11 WHO (2016). Media Center: Children: Reducing Mortality Fact Sheet. http://www.who.int/mediacentre/factsheets/fs178/en/. 12 Black, R.E. et al., (2013). Maternal and child undernutrition and overweight in low-income and middle-income countries. The Lancet 382 (9890) pp. 427-451. 13 Murphy, L (2010). Mercy Corps and USAID mark one year of malnutrition prevention program in Guatemala. Guatemala. https://www.mercycorps.org/press-room/releases/mercy-corps-and-usaid-mark-one-year-malnutrition-prevention-program-guatemala 14 MSPAS (2015). VI Encuesta Nacional de Salud Materno Infantil (ENSMI-2014/15). MSPAS/INE/Secretaría de Planificación y Programación de la Presidencia (Segeplán). Guatemala. 15 World Bank Open Data. http://data.worldbank.org/indicator/SH.STA.MMRT and http://data.worldbank.org/indicator/SP.DYN.IMRT.IN 16 Black, R.E. et al., (2013). Maternal and child undernutrition and overweight in low-income and middle-income countries. The Lancet 382 (9890) pp. 427-451. 17 Cabrera, M., N. Lustig and H. Morán, Fiscal Policy, Inequality and Ethnic Divide in Guatemala CGD Working Paper No. 397, March 2015, page 22 - 23. http://www.cgdev.org/sites/default/files/CGD-Working-Paper-397-Cabrera-Lustig-Moran-Fiscal-Policy-Inequality-Ethnic-Divide￾Guatemala.pdf. 18 The CSI enumerates all consumption-related coping strategies commonly used by a population. It enumerates both the frequency and severity of coping strategies of households faced with short-term insufficiency of food. Hence, the CSI is a measure of food insecurity. The higher the score, the greater the food insecurity (see http://www.fsnnetwork.org/coping-strategies-index-field-methods-manual). Following the 6 populations were facing five times more difficulty and stress securing short-term food security than at program start. 1.2 Program Intervention Logic PROCOMIDA is part of FANTA Project research on Preventing Malnutrition in Children under 2 (PM2A) – a food-assisted approach to reducing the prevalence of child malnutrition in children under two years by targeting a health and nutrition intervention package to all pregnant women, lactating mothers of children from birth to six months, and children under two years, in food￾insecure areas, regardless of nutritional status. PM2A is built upon the premise that these women and children are the most nutritionally vulnerable members of the population, and that the ‘first 1,000 days’ represent the greatest potential for growth and development. Actions to increase household knowledge of nutrition and care, and strengthen health services, further bolster the universal approach of PROCOMIDA. To validate the PM2A approach, USAID’s FFP funded two parallel research Title II programs, in Burundi and Guatemala. They chose Mercy Corps to implement PROCOMIDA in Guatemala as a multi-year assistance program (MYAP). Mercy Corps began working globally in 1979 to alleviate suffering, poverty, and oppression by helping people build secure, productive, and just communities. In the mid-1980s, Mercy Corps began operations in Central America, and in 2001 expanded programming to northern Guatemala, targeting poor areas in Alta Verapaz. Mercy Corps pursued PROCOMIDA at a time when more than 60% of children under five in Alta Verapaz were chronically malnourished. Program implementation initially covered the period July 2009 – June 2014. USAID officially extended implementation to May 2015 through an agreement modification, signed June 2012 (the final modification covered through November 2015). rationale of the CSI, which by definition does not have a fixed maximum level (defining levels of low, moderate or severe food insecurity are context specific), the initial value measured for Alta Verapaz can be considered as low, while the final average level of 40 points would be considered to be at least at the upper level of a moderate food insecurity. 7 2 ENDLINE EVALUATION 2.1 Purpose and Scope The purpose of this final PROCOMIDA evaluation is to assess impact and sustainability potential. Specifically, it seeks to determine: i) results; ii) effectiveness and relevance of activities and implementation approaches; and iii) extent crosscutting themes (community participation, gender, and environment) 19 were integrated (see scope of work/SOW, Annex 1, A). Towards this, performance ratings are provided to compare achievements against the results framework (goal, objectives, performance, impact indicators: see Table 10). Achieved values are compared with targets and baseline findings to determine to what extent USAID-supported interventions contributed to reducing malnutrition and related health conditions. This evaluation further establishes plausible links between activities, outputs, and outcomes, identifying lessons and recommendations for the improvement of future Title II Development Food Assistance Programs (DFAP) and similar activities. General evaluation criteria of the OECD Development Assistance Committee, guidelines for the evaluation of PM2A interventions, and guidance from USAID and FANTA inclusive of rigorous statistical analysis, conclusions, and recommendations, led the evaluation approach. It covers the period July 2009 – May 2015. The SOW for re-drafting the final evaluation report (Annex 1, B) specified the need to: i) review endline data for statistical relevance in comparison with baseline results; ii) integrate reviewer comments; and iii) reorganize and edit the final report to comply with industry standards. 2.2 Methods 2.2.1 Study Design Comparability between the 2010 baseline and 2015 endline samples was assured by: i) comparable sampling frameworks and criteria; ii) data collection techniques that employed triangulation and stakeholder participation; iii) identical questionnaires; and iv) survey implementation at the same time/season and in the same geographic areas. 20 Note that the midterm evaluation and annual M&E surveys are not comparable with the baseline or endline, as their sampling frameworks were participant-based rather than population-based. Final evaluation activities covered two phases, over the period April – November 2015. Phase 1: Data Collection (April – July 2015) was conducted by a consortium between ACF (Action Against Hunger) and ECI Consultores, international and local consultant firms whose combined technical study team included eight members with backgrounds in economics, nutrition, management and public policy, social science, and knowledge management, and a further 6 field teams of 5 people each. Field team members spoke Spanish. Local translators assisted them where local Q’eqchí language was required. A team of local enumerators, all native Q’eqchí speakers, translated surveys from Spanish to Q’eqchí, and collected field data. The study team trained all enumerators in using tablets for data entry, and in using anthropometry equipment. 19 BCC is generally considered another crosscutting theme for Title II programs. However, BCC was a core PROCOMIDA component. 20 This refers to the original intervention areas and does not cover expansion sites. 8 ACF/ECI study team members, with Mercy Corps staff, developed quantitative endline data collection tools based on baseline tools, and designed by the International Food Policy Research Institute (IFPRI), to answer evaluation questions (see Annex 5, 6, 7). All field team members attended a two-week long training workshop to: i) become acquainted with the tools; ii) ensure uniform field application; iii) understand the study rationale and protocol; iv) receive advice on data collection procedures; v) practice conducting anthropometric measurement standardization tests with 12 voluntary children and their mothers; and vi) review digital data collection procedures using electronic tablets. 21 Data collection activities consisted of: Document review Quantitative field surveys  Household (participant level)  Knowledge survey (health workers, volunteers, health center – community level)  Health center (convergence center – cluster level) Anthropometric measurements  Children (0-59 months)  Mothers (caregiver) of same children Qualitative fieldwork  In-depth key informant interviews (KII)  Focus group discussions (FGD)  Direct observation (site visits) Phase 2: Data Cleansing, Analysis, and Reporting (August – November 2015) were completed by ACF/ECI, and reanalysis completed by Health and Development Consultants International, LLC (October 2016). The HDCi team included one global public health specialist, and one epidemiologist/biostatistician tasked with redrafting the final external evaluation report. 2.3 Quantitative Surveys The endline household survey was a cross-sectional study, comparable with the 2010 baseline. 2.3.1 Sampling Since the household survey aims to show results at the level of the entire population, the sampling frame was comprised of all households with children between zero and 59 months of age in March 2015, within selected clusters. The study team defined clusters using probability proportional to population methods equivalent to nearly one CC per cluster. They based the geographical sampling frame on the same 100 clusters used during the baseline. 22 Following a census conducted by PROCOMIDA staff within these 100 clusters, they defined the total sample size as 12,844 households 23 having children within the inclusion criteria age range. A sampling frame of 46 CCs (inter-cluster correlation = 0.01) were selected randomly using the probability proportional to size (PPS) algorithm (described in FANTA-2 guidelines) 24 out of the 100 clusters, taking into consideration that two CCs were larger communities that covered two clusters (thus, 46 CCs covered 48 clusters). The sample additionally represented with high fidelity the true geographic distribution of PROCOMIDA participants and non-participants. Standard statistical software (ENA 25 for cluster selection and STATA 26 for household selection) ensured that each household had the same probability of being selected. For a number of indicators, the survey only needed to interview a sub-population and not the entire sample of 21 Cybertech 8170 Tablets were used, and the operating system was Java for Android 2.2. 22 Refers to baseline survey stratum A to E. Baseline control groups and any alternative control groups were not included. 23 Census data revealed that non-participation was found to be 10.8% of households in the 100 clusters. 24 USAID (2009). Food and Nutrition Technical Assistance II Project-Annual Program Report Project Year One. June 2, 2008 – 30 Sept. 2009. 25 Emergency Nutrition Assessment for SMART, see www.nutrisurvey.net/ena/ena.html. 26 Data Analysis and Statistical Software (branch name). 9 households. The sampling framework was designed to produce statistically significant results for all sub-populations, specifically children between six and 24 months, and their mothers. The change in population prevalence of malnutrition was the core indicator used in the baseline study and hence determined the sample size requirement for the endline. In accordance with FANTA guidelines, 27 the study team used following equation (for follow-up surveys): n = D[(Zα + Zβ)² * (P1(1-P1) + P2(1-P2)) / (P2 – P1)²] Table 1: Description and values of parameters used in the sample size calculation Variable Description Value D Design effect (default value) 2 P1 Baseline prevalence (baseline) 0.566 P2 Endline expected prevalence (target) 0.504a Zα Z-score of the degree of confidence to ensure that the change occurred because of the 1.645 intervention (statistical significance Zβ Z-scored of the degree of confidence to ensure that the change has actually occurred 0.84 (statistical power) n Sample size (number of valid household interviews required) 1,592 a The expected decrease in malnutrition prevalence was 0.062. This was accurate with Mercy Corps. In calculating the minimum required sample size, the Type 1 error (α) was set at 0.05, and the power of study (1-β) at 0.80. Figure 1: Key sampling and sample size process flow 1,592 Minimum required sample size (of households) based on calculation Actual number of households sampled a No. of households interviewed (99 non-response, 5%) Final valid household dataset (23 with data issues, excluded) 1,843 1,744 1,721 1,830 1,850 Minimum required, accounting for 15% potential non-response Sampling strategy: 37 households x 50 clusters a One cluster only contained 30 households compared to the targeted 37, corresponding to a smaller community. 2.3.2 Data Collection Quantitative survey fieldwork occurred over several weeks between April 21 - May 18 in the municipalities of Cahabón, Cobán, Lanquín, and San Pedro Carchá, of the Alta Verapaz Department. All survey data was collected through a java-based digital application for Android on tablets, originally developed by Mercy Corps Staff, 28 and continuously crosschecked for accuracy. Field team surveyors sent survey and anthropometric data directly to field editors who used ENA software to check for consistency, and who returned Z values to the surveyors. Z values were double-checked during data analysis. Table 2: Distribution of communities and interviewed households per district District # of selected clusters # of selected CCs # of households sampled Cahabón 5 5 175 Cobán 11 10 376 27 USAID (1999). Fanta Sampling Guide. http://pdf.usaid.gov/pdf_docs/Pnacg172.pdf. 28 Original applications were designed for annual monitoring and for the Midterm Evaluation. Baseline data collection was held on a PDA platform through Mobile Data. 10 Lanquín 3 3 105 San Pedro Carchá 31 30 1088 TOTAL 50 48a 1,744b,c a Two CCs did not have any staff available for interviews, thus final sample size for the CC survey was 46. b Datasets from 23 households were incomplete, thus final sample size for household analysis was 1,721. c Four households had 2 mothers among the members, thus final number of mothers sampled was 1,748. The field teams also collected survey data from sampled healthcare facility staff, and community health workers (CHW) and volunteers (CHV) – from the same CCs as the household surveys.29 2.3.3 Monitoring and Evaluation Framework All surveys and information collected addressed the following topics (see Annex 5, 6, and 7 for more details): Household survey  Household composition  Dietary diversity (HDDS and Household Food Insecurity Access Scale/HFIAS)  Mother’s knowledge  Utilization of antenatal and post-natal care  Infant and young child feeding (IYCF practices)  Vaccination and growth monitoring  Field and household management plan  Annual monitoring (CSI) Anthropometric measurements  Mother (or caregiver) of 0-59 month old children (only one per household)  Children 0-59 months old (only children of the mother/caregiver)30 Knowledge survey of healthcare staff (CC level)  Education and basic knowledge  Pregnancy and complication  Antenatal and postnatal care  Maternal nutrition  Infant and Young Child Feeding and caring practices Health center survey  Responder information  Medicines and materials  Administration  Care services provided The study team did not conduct CC equipment surveys, which were part of the baseline. Mercy Corps/Guatemala negotiated the elimination of this indicator in the Fiscal Year (FY) 2014 pipeline and resource estimate proposal (PREP). USAID approved this in Modification 10. 2.3.4 Analysis The study team conducted initial data analysis in fall 2015 using Statistical Package for Social Sciences (SPSS). The HDCi team re-verified findings in fall 2016. Data and documentation were available in the original Spanish, and in some cases English translations. The review and analysis process included the following:  Comparison of IPTT data entry  Review of SPSS datasets, and cross-checking these against survey questionnaires 29 The healthcare center survey could not be carried out in Chinaichab, district of Cobán (HC code 73) and San Antonio I, district of San Pedro Carcha (HC code 280). The CHW was identified as an interview partner for the community healthcare center surveys. In both of these communities, the CHW was absent during the days of field visit in the respective communities. The information recollected for the final survey in 46 community health centers allows for full comparability with the baseline study of 45 community health centers. 30 Pregnant women were excluded from anthropometry. 11  Translation Spanish-English, where required  Review and re-running of the previous SPSS syntax files, where possible  Finalizing IPTT indicator list and presentation  Harmonizing any conflicting findings from draft reports  Generating correlation graphs for selected indicators  Running adding additional summary tables and data analysis, where relevant For anthropometric data, the study team used the World Health Organization’s (WHO) Growth Standard – 2006 31 data cleaning criteria. Table 3: Evaluation criteria for the exclusion of anthropometric outliers 32 Dimension of under nutrition Exclusion Criteria Wasting [-5 SD; +5 SD] Stunting [-6 SD; +6 SD] Underweight [-6 SD; +5 SD] For household survey variables, the study team applied usual data consistency controls, 33 providing a final and validated number of observations. Table 4: Total valid observations in final databases Sampling population and unit sub-strata Observations Minimum required observations Households with children 0-59 months 1,721 1,592 Children 0-6 months 189 176 Children 0-23 months 924 892 Children 6-23 months 735 694 Children 6-59 months 2,211 2,144 Children 0-59 months 2,406 2,365 CC’s (equipment and services available) 46 45 CC’s (health staff knowledge) 46 45 The results contain descriptive and econometric analysis of endline survey data in comparison with the baseline. Towards providing full methodological comparability 34 between baseline and endline surveys, all baseline data were re-estimated, with the following differences observed:35  The correct baseline value for I3 is 7.19 (was reported as 5.8 in 2010)  The correct baseline value for I13 is 50.0 (was reported as 40.5 in 2010)  The correct baseline value for I17 is 73.2 (was reported as 85.9 in 2010) PROCOMIDA discussed these differences with IFPRI (baseline implementer), and presented their agreement to USAID/FFP on October 8, 2015, after which IFPRI published an addendum to their baseline report. Throughout the analysis and presentation of findings, the team compared average values between two groups using the t-test. When analyzing the relationship between two categorical variables, they used the chi-square test. For the analysis of the percentage change among linked or paired data, they used the McNemar test. In all statistical analyses, a 95% significance level and a p-value of less than 0.05 indicates a statistically significant association. 2.4 Qualitative Fieldwork As per requirements, the study team complemented quantitative data analysis with qualitative field research for more comprehensive understanding of overall findings. Qualitative research 31 WHO (2006). WHO Child Growth Standards, Geneva. http://www.who.int/childgrowth/standards/Technical_report.pdf?ua=1. 32 Ibid. 33 Error detection through consistency checking using the consistency matrix tool. 34 In the sense of having used exactly the same definitions and steps for computing variables. 35 The evaluation team shared and discussed these findings intensively with both Mercy Corps and IFPRI (who conducted the baseline survey). IFPRI accepted the revised values and issued an errata for the baseline report. 12 allowed for triangulation and interpretive perspectives regarding PROCOMIDA, and the sociocultural and economic contexts of implementation geographies. The study team defined semi-structured key informant interview (KII) questionnaires and a thematic focus group discussion (FGD) list (see Annex 7) after reviewing the preliminary findings from the three surveys (household, health center services, and health center staff knowledge). 2.4.1 Sampling The study team identified 14 communities (see Annex 2) for qualitative research, from among the 46 CCs identified during quantitative surveys. They applied the following selection criteria (in no particular order): i) randomized selection; ii) geographic distribution proportional to all communities assisted by PROCOMIDA; iii) proportional distribution between communities nearby and far away from district capitals; and iv) proportional distribution between big and small communities. 2.4.2 Data Collection The study team utilized a variety of data collection techniques. Key Informant Interviews to gather perceptions from key stakeholders and participants included: i) PROCOMIDA team (management, M&E unit, nutritionists, other sector staff; n=11); ii) local authorities (head or members of the Consejo Comunitario de Desarrollo (Council for Local Development/COCODE; n=13); iii) CHWs (n=14); iv) midwives of visited communities (n=14); and v) peer counselors or instructors (n=3), for a total of 55 interviewees. Study field teams conducted community-level interviews in both Spanish and Q’eqchí (with local translators), while they conducted interviews with PROCOMIDA staff in Guatemala City and Coban in Spanish or English. Each interview lasted approximately 60 minutes (see Annex 7 for qualitative questionnaires). Focus Group Discussions were completed with 373 family members (Table 5), including 300 program participants. The goal was to gather information on behaviors, expectations, and satisfaction levels from: i) mothers receiving rations during PROCOMIDA’s field implementation (through May 2015); ii) mothers who ‘graduated’ from the program; iii) mothers with children under five who did not participate in or benefit from PROCOMIDA; and iv) fathers of participating families. In one community, a fifth group was comprised of five mothers who graduated from the program, and were also heads of household. 36 Table 5: Sex and literacy levels of FGD participants per community (# and %) Community Females Female Illiteracy (%) Males Male Illiteracy (%) TOTAL Female Participation (%) Male Participation (%) 1 21 50 7 21.4 28 75 25 2 21 57.1 7 14.3 28 75 25 3 21 42.9 6 0 27 77.8 22.2 4 20 70 7 0 27 74.1 25.9 5 20 50 6 50 26 76.9 23.1 6 19 15.8 7 42.9 26 73.1 26.9 7 20 20 6 16.7 26 76.9 23.1 8 21 57.1 6 0 27 77.8 22.2 9 20 50 7 57.1 27 74.1 25.9 10 19 10.5 7 0 26 73.1 26.9 11 20 55 7 28.6 27 74.1 25.9 12 24 45.8 2 0 26 92.3 7.7 36 In comparison to this group, all mothers from other groups live together with their spouses or partners, who are considered heads of household. The group of female single headed households was established in order to be able to determine if there were any important differences in their experiences and considerations regarding PROCOMIDA. There were no important differences found, but the evaluation had to check on this and could not just assume that experiences and considerations would be the same. 13 13 21 42.9 6 16.7 27 77.8 22.2 14 19 10.5 6 0 25 76 24 TOTAL 286 43 87 24.1 373 76.7 23.3 Discussions included seven participants on average, and lasted approximately 90 minutes. Themes discussed varied between participant cohorts. Direct Observation allowed for primary data collection across a wide range of contexts, captured a variety of community and local interactions, and facilitated on-site verification of KII and FGD findings. For example, the field teams observed hygiene conditions at community health centers (CHC), and confirmed the existence of home gardens. The field study teams visited 14 CHCs, two community centers, two schools, and three home gardens. 2.4.3 Analysis Translators transcribed KIIs and FGDs directly in the field by recording Q'eqchí interview notes in Spanish. They later coded discussion content by category for analysis using Atlas.ti 37 software. We highlight themes with greater frequency in this evaluation report. 2.5 Limitations There were a number of limitations in the endline evaluation process:  Delays to the contracting process required two rounds of solicitations. During the first round, Mercy Corps received only two solicitations, and neither passed the initial technical review. The second round involved an intensive process of promoting and sharing the scope of work with multiple firms. The process faced additional delays during the USAID/FFP approval process for the selected consultant.  There were multiple linguistic challenges with the need to cross-translate between Spanish, Q’eqchí, and English, both during field data collection, data management, and re-analysis of documents and datasets originally recorded in Spanish and ultimately needed in English.  Although the study team completed the evaluation with professionalism, strong statistical capacity, attention to detail, and an intensive feedback mechanism, USAID/FFP rejected the final report two times due to readability issues. Since Mercy Corps/Guatemala refrained from any significant direct editing of the document, to assure it maintained independency, final delivery of the document and approval of the final evaluation was delayed for over a year between reviews of FFP and responses from Mercy Corps. Ultimately, Mercy Corps contracted a separate consultant (HDCi) to rewrite the document and re-verify that all indicator calculations were correct, given the need ensure accurate and consistent data representation, use of professional language, and a polished presentation.  The virtual absence of services in 2014/2015, and changes to the MOH intervention/service model in early 2015, seriously affected outcomes at the community level and outcomes related to MOH services. This affected the search of qualified MOH staff at the CC level during the final evaluation process. In some communities, there were no CHVs, and in others, the study field teams had to return several times to find a newly hired CHV. 37 The Atlas.ti version 6 from 2011 was used, http://atlasti.com/. 14 3 FINDINGS 3.1 Program Description PROCOMIDA sought to improve the nutritional status of pregnant and lactating women and children under two in 936 vulnerable communities in selected municipalities 38 classified by the Famine Early Warning System Network (FEWSnet) as having poor food security and livelihoods, and some of the country’s highest stunting and infant and maternal mortality rates. 39 To ensure that PROCOMIDA’s activities strengthened existing health service structures, Mercy Corps worked at different levels in collaboration with the following implementing partners:  Local health commissions (community level)  NGOs implementing PEC40 (CC level)  Municipal health commissions and other indigenous programs (municipal levels)  MOH Secretaria de Seguridad Alimentaria y Nutricional (Secretariat for Food Security and Nutrition/SESAN41 – department level, and central level for coordination of overall strategies and work plans) These collaborations extended through two presidential administrations,42 four MOH minister turnovers, four MOH departmental director turnovers, and three SESAN secretary turnovers. It witnessed the breakdown of primary healthcare services under PEC in late 201443 through early 2015, when the MOH cancelled all NGO/private service provider contracts (sub-grantees of PROCOMIDA) in Alta Verapaz. Ramifications on the health and well being of local populations from this interruption were devastating, and affected several of PROCOMIDA’s expected results. Despite this, the implementation period coincided with increasing political attention and prioritization of nutrition and health issues. 44 This included Guatemala joining the Scaling Up Nutrition movement in December 2010, and the Presidential Initiative ‘Zero Hunger Pact’ in 2010, which aimed to reduce malnutrition in children under five by 10%. PROCOMIDA activities were included in the Zero Hunger Pact, resulting in intensified coordination with MOH/SESAN. 3.1.1 Key Baseline Findings To ensure guiding principles and actions were appropriate and operational for the specific local context, formative research (baseline) was conducted in early 2010 in 15 Alta Verapaz health districts to: i) understand food, nutrition, and health conditions and practices; and ii) identify key barriers that could prevent adoption of promoted practices. The baseline suggested that extreme poverty, large family size, limited land access, and limited income generation and employment opportunities created challenges for most households to access necessary food quality and quantity. Only limited fruit and vegetable production occurred in home gardens, and was sold 38 Implementation originally covered the municipalities of Cobán, San Pedro Carchá, Lanquin, Cahabon, and Senahú, expanded in August 2012 to include Chahal, Fray Barolomé de las casas, and Chisec were added. 39 WFP (2011). Evaluación de la seguridad alimentaria de los pueblos afrodescendientes, en las comunidades de las yungas de La Paz. http://es.wfp.org/sites/default/files/pma-afrobolivianosyyungas.pdf. 40 In Guatemala, the majority of rural communities access health services through the government System of Integrated Health Care, Coverage Extension Program (PEC). Instituted in 1996 as part of the Peace Accords, PEC aims to reach rural areas and improve primary healthcare. PEC is operated by MOH-contracted private service providers/NGOs, working with community support but paid with public funds to provide primary health services within communities. PEC provided services on an itinerant basis, with a doctor or nurse visiting each CC monthly. 41 The Law on Food and Nutritional Security System established SESAN as coordination structure that involves all relevant government ministries, donors, and civil society at national and local levels. 42 Government of Alvaro Colom (January 2008 to January 2012) and Otto Perez (January 2012 to January 2016). 43 Beginning in late 2012, reform processes in the health sector sought increased control of service provision through public institutions in an effort to achieve universal coverage. In early 2014, ministerial staff were expected to gradually take over services provided by NGOs through PEC. This transition process failed. In November 2014 NGO service provider contracts were canceled by the government in various departments, including Alta Verapaz, leaving the populace without primary healthcare during important parts of 2014 and early 2015. 44 The Guatemalan Constitution through Articles 2, 94, 97 and 99, provides specific provisions to everyone’s right to food. Similarly, Article 43 of the Health Code states that actions to ensure the availability, production, consumption, and adequate biological utilization of food should be promoted in order to support achieving food and nutrition of the Guatemalan population. 15 rather than consumed. General knowledge regarding optimal health and nutrition was limited. The suboptimal utilization of food, household resources, and health services contributed towards overall food insecurity. Several factors linked to the problem of undernutrition focused around two recurrent themes: i) behaviors that influence nutritional status; and ii) limited access to healthcare and related services. While household hunger was uncommon, household and child diet quality was poor, and IYCF practices suboptimal. Suboptimal growth and development of children under two years of age was compounded by repeated illness, poor hygiene and sanitation, and suboptimal IYCF. Low quality household and infant diets meant that children lacked essential micronutrients. Stunting was a major problem, with average height-for-age z-scores were significantly lower among boys compared to girls.45 While there was almost no wasting, and the prevalence of underweight was moderate, the majority of underweight was likely explained by high stunting. The baseline study further found that two-thirds of mothers were illiterate and did not speak Spanish, with a high prevalence overweight and obese (average body mass index/BMI 24.8). Mothers were short in stature, which places their fetus at increased risk for growth retardation in utero, and implies they experienced stunting during their own early childhoods (inter-generational pattern). Knowledge of optimal breastfeeding practices was high, but only half of mothers could identify the correct age to introduce complementary foods to infants. Early introduction of inappropriate foods and liquids, along with poor food hygiene, poor water quality, and poor food diversity, were a concern. These are associated with poor infant nutritional status and morbidity, and high rates of stunting and diarrhea. Feeding frequency was inadequate in many children, pointing to the fact that mothers’ improved knowledge did not always correspond to changes in actual practice. Few mothers knew the danger signs of pregnancy or childhood illness. While most attended a sufficient number of prenatal visits, postnatal care appeared inadequate with less than half of recommended visits attended. Growth monitoring and promotion in children 12 to 23 months was nearly universal, yet less than half of children received a mega-dose of vitamin A, and only half received iron and folic acid. Barely two-thirds of children 18 to 23 months were fully immunized, with most vaccinations received later than recommended. Only 60% of the baseline sample of children under two consumed four or more food groups, and only 52% consumed iron￾rich or fortified foods in the previous 24 hours. 3.1.2 Program Design PROCOMIDA followed recommended nutrition guidelines, sustained by technical and empirical evidence in line with global protocols on child nutrition. The innovative program employed a preventive methodology based on research and results from World Vision’s PM2A approach, which focused on protecting children at the most critical period in their development. To do this, PROCOMIDA worked with NGOs and the MOH to establish and/or strengthen community structures to improve the provision of culturally and technically appropriate health services, and specifically adequate maternal and child health and nutrition (MCHN) service coverage and quality. PROCOMIDA further provided nutrition and health education to empower mothers 46 to adopt best practices, including seeking health services, towards realization of the program goal and strategic objectives (Table 6). 45 There was no significant difference in the prevalence of stunting between boys and girls – boys were shorter in general, but no more were categorized as stunted. 46 ‘Mother’ is understood to be either the biological mother, or the primary caregiver. 16 Table 6: Results framework Goal: Improved nutritional status and health of women and children vulnerable to food insecurity in northern Guatemala Strategic Objectives Intermediate Results Activities 1. By 2015, pregnant and lactating women, children under 2, and malnourished children under 5 in program areas have improved and sustainable health and nutrition status 1.1 Increased consumption of energy and nutrient dense foods in households with pregnant/lactating women, a child 6-24 months of age, and/or a child 24-59 months with acute malnutrition 1.2 Improved household knowledge, attitudes and practices for key care behaviors 1.3 Increased engagement of households in reducing vulnerability to food insecurity  Consultation with government and civil society leaders  Establish / strengthen community health commissions  Disseminate criteria and selection of beneficiaries  Provide preventative rations to all pregnant or lactating women and all children aged 6-24 months  Management of acute malnutrition  Develop behavior change communication (BCC) messages  Enhance the capacity of community outreach actors (CHCs, CHVs, etc.) to deliver BCC  Form mothers’ groups  Improved practices reinforced through home visits, ration distribution and radio  Establish community emergency funds to facilitate transportation to health services  Connect with ongoing Mercy Corps food security programs 2. By 2015, healthcare service providers at community through municipal levels have improved Service quality and delivery 2.1 Increased communication and interaction between vulnerable populations and health service providers 2.2 Increased and improved provision of minimum standards in health services to women and children 2.3 Increased promotion and planning by healthcare providers to meet community healthcare needs  Enhanced technical capacity of health service providers to provide strong MCHN care  Improve MOH extension services through building capacity of SIAS (Sistema Integral de Atención en Salud / Integrated Health Attention System) NGOs  Strengthen recognition and treatment of children with severe acute malnutrition (SAM)  Facilitate regular meetings among community health commissions, community leaders, health service providers to plan around community health-related priorities and concerns  Facilitate orientation visits to CCs, health posts and health centers by community members  Strengthen health facility capacity to provide culturally and linguistically appropriate care  Build the capacity of SIAS￾implementing NGOs and MOH health post staff to prioritize, plan and advocate for community health priorities with key municipal, departmental and national actors and decision￾makers 17 In line with their results framework and proposed preventive activities, PROCOMIDA based program design on the following basic lines of action:  Universal coverage  Food distribution  Behavior change promotion  Health service strengthening Universal coverage. Under the universal preventive approach, all households with i) pregnant mothers, ii) lactating mothers of children under six months, and/or iii) children under two were targeted, not just those demonstrating signs of malnutrition. This approach is intended to improve growth and development within the first 1,000 days period (PM2A), or to prevent long-term consequences (diminished psycho-motor skills, work capacity, IQ, earning potential). 47 While the program defined the mother-child pair as its main participant beneficiary, the involvement of mothers was necessary at all levels of intervention, so the program measured beneficiaries in MCUs. A fourth participant group consisted of mothers with children between two and five years of age with acute malnutrition, who were temporarily enrolled for three-month periods when presenting with severe acute malnutrition (SAM). All participants were provided unique registration codes (ultimately assigned to each household), and received participant cards that they brought to trainings and food distribution points, allowing PROCOMIDA to track participation. Each participated for a minimum of six months and a maximum of 30 months. As of project implementation end (May 2015), 1,180 communities had participated in the program, reaching 60,605 direct participants, the vast majority children between six and 24 months of age. Though participation was voluntary, the final evaluation found that 88.9% percent of households surveyed had eligible mothers and children and had participated at some point in PROCOMIDA (between July 2010 - May 2015). During qualitative fieldwork, many participants expressed frustration that the program focused on children under two years. Former health interventions, from both the MOH and other NGOs, focused on children under 5 years, hence the reference to this age range. “I would say that (PROCOMIDA staff) should give us food until our children are five years old because when they turn two years old, this is when they (really) start eating” (graduated mother, Jobchacob). While program implementers and funders understand the scientific argument for focusing on children under two years, Mercy Corps staff could have better communicated the logic of the PM2A approach (and first 1,000 days) to beneficiary households and mothers. While PROCOMIDA’s design is based on the first 1,000 days concept (PM2A), the program was limited by the timely identification, enrollment, and retention of pregnant women. According to data provided by the program, the average duration of participation per MCU was 567 days. Had the project offered a suite of multi-sectoral activities, PROCOMIDA could have achieved more for these mothers and children. Food distribution. PROCOMIDA provided participants with a balanced ration to overcome food insecurity, differing according to cluster: individual rations of corn-soy blend (CSB - four kg), lipid￾based nutritional supplements (LNS), or micronutrient powder (MNP), and household rations of rice (six kg), pinto beans (four kg), and vegetable oil (two liters). The program provided individual rations to mothers during pregnancy and lactation until the child reached six months, after which it was transferred to the child until 24 months of age. Food supplementation was an incentive for participation, and a source of macro- and micro-nutrients for children between six and 24 months, and pregnant and lactating women. Because of the research focus, PROCOMIDA distributed five 47 FHI 360 (2010). FANTA-2 - Title II Technical Reference Materials. TRM-01: Preventing Malnutrition in Children Under 2 Approach (PM2A): A Food-assisted Approach. Washington, DC 18 different monthly rations: one standard ration and four alternate rations. For the research, the program randomly selected 120 CCs from the initial program intervention area. For each research arm, they selected 20 CCs and an additional 20 CCs as a control group. The latter did not receive any program interventions, including training or food rations. All non-research CCs received the standard ration (Ration A in Table 7). 48 Table 7: Distribution of monthly rations Research Arm A B C D (LNS replaces E (MNP replaces F (No CSB) CSB) intervention) # of CC’s 20 20 20 20 20 20 Ration Type Commodity KG KG KG KG KG KG Individual CSB 4 4 4 LNS MNP 0 Ration Family Ration Rice 6 3 0 6 6 0 Beans 4 3 0 4 4 0 Veg Oil 1.85 0.925 0 1.85 1.85 0 TOTAL 15.85 10.93 4 11.85 11.85 0 PROCOMIDA was highly effective at providing all goods and services under their control, including financial and material (food), on time. At program start, there were problems regarding micronutrient availability, which resulted in some delays. 49 Program staff quickly resolved them. Most Title II programs distribute commodities in 50 kg or 25 kg bags to distribution sites, where staff hand them out to participants who bring their own containers to transport home. Borrowing from learnings in social marketing, PROCOMIDA chose to pre-package food rations into individual household packages to better organize distributions, provide accurate weight and hygiene controls, and facilitate a user-friendly approach to distribution. They provided each household a ‘master’ ration bag allowing them to carry their food home in the traditional manner, using a strap around their forehead with the bag on their back. They printed 29 key messages (pictograms without text, consistent with the BCC strategy, see below) on the pre-packaged monthly ration bags, as daily program message reminders (Annex 9). The repacking operation occurred at a specially designed warehouse in Coban, Alta Verapaz, with 2,400 square meters of floor space to store and repack an annual average of 3,000 metric tons of donated food. Initially, the program distributed vegetable oil in four-liter bottles packaged in the United States and distributed every other month, based on a monthly ration of two liters. When the program’s field team noted that monthly attendance dropped on the months they did not distribute oil, PROCOMIDA recruited a local technical university to design and build a machine to bottle the oil into two-liter containers for monthly distribution. PROCOMIDA delivered 967,708 rations consisting of 15,172 metric tons of food donated during 58 months of uninterrupted monthly distributions (Figure 2). The monthly distributions used 48 transport routes during the first 16 workdays of each month. Field staff informed participants of the exact date for the next month’s distribution during the monthly BCC educational session. To ensure that food arrived on time, they dispatched three trucks each day from the central warehouse in Coban during the monthly distribution period. These trucks traversed an average of 5,726 kilometers (with a peak of 6,309 km) a month over mostly unpaved roads to deliver the rations, for a program total of 337,851 km covered. At the conclusion of the distribution period, 48 Initially the program had larger family rations (12 Kg Rice, 8 Kg beans, and 3.7 Kg vegetable oil for ration A), but after the first year they were reduced to the here mentioned ration sizes, as agreed with IFPRI and USAID/FFP. 49 IFPRI was responsible for the purchase and import of micronutrient, delays in distribution was due to delayed approval of the human research trails by MOH. 19 only 65 metric tons of different food items remained, which Mercy Corps donated to another Title II partner (Catholic Relief Services). 30,000 Figure 2: Number of rations distributed 25,000 20,000 15,000 10,000 5,000 0 Jul-2010 Jul-2011 Jul-2012 Jul-2013 Jul-2014 Behavior change promotion. To overcome identified gaps in knowledge and health seeking behavior amongst participating mothers, PROCOMIDA developed and implemented a comprehensive and targeted BCC component. PROCOMIDA utilized the BEHAVE Framework to design interventions, which consisted of both health education and the deliberate promotion of specific behaviors at household and community levels. Seventy-two key messages (included in Annex 9) were identified and organized around five themes: (i) food and health; (ii) exclusive breastfeeding; (iii) care of pregnant and lactating mothers; (iv) care of children from six to 24 months; and (v) sick and malnourished children. Materials included flipcharts, messages printed on ration bags, and pamphlets. Program field staff utilized them during monthly trainings (with the aid of a training guide). All training materials were pictorial with little or no text, and the BCC strategy used adult literacy techniques focused on collective participative learning through positive deviance. Enrollment and receipt of rations was conditional. Pregnant and lactating women were required to participate in trainings in order to qualify for rations (Figure 3). When food packages arrived in a community, they were stored in the storehouse until the training activity took place, and only distributed after the training’s conclusion. Depending on climate conditions and the scope of training activities, the time between food arrival and distribution averaged one to three days. While it was expected that the reception of rations would be the strongest incentive to high training participation, the qualitative endline research suggests that common situations made participation impossible, including mothers with ill children combined with the absence of fathers from homes (periodic labor migration). Participation rates averaged 78.6% between November of 2011 and May 2015. 50 50 The first month of food distribution was July 2010, and the last month was May 2015. Participation rates in training events have been measured systematically only since November 2011. 20 100 90 80 70 60 50 40 30 20 10 0 Figure 3: Participation rates at PROCOMIDA training events (%) Assistance level Absenteeism To ensure fluent communications between program staff and participants, as well as gender balance, PROCOMIDA organized field staff/educators into trainer teams of one woman and one man, speaking the native language or bilingual in Q’eqchí-Spanish. Trainers divided participants into four identified groups according to interest and cultural necessity: (i) pregnant women; (ii) lactating mothers with children from zero to six months of age; (iii) mothers with children from six to 24 months; and (iv) mothers of sick infants and children under 2. Trainings focused on a key message, coordinated with existing MOH messages. Training teams facilitated 60,995 short educational sessions with participating mothers over program implementation. PROCOMIDA utilized the Care Group approach and positioned women as change agents in their communities, enlisting throughout the duration of the program 2,549 mothers as Mother Leaders to model behaviors – starting in their own homes. Mother Leaders were equipped with knowledge and interpersonal communication skills towards promoting hygiene, health, and nutrition through specific behaviors, counsel, and advocacy. Educational sessions implemented by program staff included monthly recipe demonstrations for Mother Leaders to replicate with up to 20 peer neighbor participants assigned to each of them, ensuring transfer of knowledge to promote more balanced, nutritious, and diversified diets to participating families. Field staff and Mother Leaders conducted a combined 93,685 cooking demonstrations. Field and PEC staff complemented trainings with household visits, stressing the importance of attending trainings, strengthening key messages, monitoring food ration use, and following up on pregnant women and malnourished children. Over program implementation, they conducted 74,017 visits. They introduced the Household Action Plan during these home visits. The Household Action Plan is a pictorial diagram illustrating activities around their home to be 21 maintained or improved to ensure proper hygiene and sanitation, as well as home gardens to help diversify family diet by growing vegetables, fruit, and herbs. By program end, Mother Leaders implemented 11,587 home gardens. However, most of these home gardens provided vegetables during only one season given that seeds (purchased locally by communities themselves) were hybrids and thus not usable for seed reproduction. Field and PEC staff also promoted homemade hand washers called Ch’ajleb-uq’ (tippy taps) made from the program’s reused oil bottles during visits. Ch’ajleb-uq’ use only small amounts of water and help make hand washing fun and habitual. Health service strengthening. At the community and healthcare services level, staff implemented PROCOMIDA through CCs, which are rallying posts located in rural aldeas (small rural villages or communities) that do not have any other health services. CCs were part of the MOH first level of attendance, and managed by local health implementing NGOs contracted by the MOH. The main CC task included carrying out the Integrated Care of Prevalent Childhood Illnesses and Integrated Health Care for Women and Children at the Community Level Program, Guatemala’s integrated healthcare strategy established by the MOH. This program provided: i) integrated case management of childhood illness; ii) monthly growth monitoring and promotion for children; iii) standard preventive care for pregnant and lactating women; and iv) vaccinations and micronutrient supplements for pregnant and lactating women and children under five years of age. CCs were essential to the Zero Hunger Pact, but were limited in their ability to supply quality healthcare due to lack of necessary medical equipment and supplies (including drugs), and limited health staff knowledge. At its peak, staff implemented PROCOMIDA through 358 Training and Distribution Posts, including 344 CCs and 14 health posts. PROCOMIDA enhanced CC activities by providing trainings for CHVs working in intervention areas in themes such as nutritional evaluation, nutrition, and malnutrition. Staff invited decentralized health service providers (local NGO health implementers) and their teams to attend all trainings. The program extended additional institutional strengthening to these providers to improve their data management, reporting, financial, and organizational skills. All program field staff themselves received monthly trainings in either i) BCC, ii) nutrition, iii) monitoring and evaluation (M&E), iv) data management (management information system/MIS), v) taking standardized anthropometric measurements, vi) use of data collection forms, or vii) identification and referral of SAM. All of these activities strengthened local capacity and contributed towards sustainability. 3.2 Program Research In addition to direct activity implementation, PROCOMIDA incorporated a research component as part of FANTA’s research to validate the PM2A approach in two countries (Guatemala and Burundi). IFPRI implemented and coordinated the research in Guatemala with Mercy Corps, and in Burundi with Catholic Relief Services (CRS). The basic research question tried to identify the most effective mix of different foods in the monthly ration to be distributed, for reducing undernutrition at the lowest possible cost. 51 IFPRI randomly selected six clusters of 20 communities (total of 120 research sites), with one cluster as a control, and five covered by the program and receiving family and/or individual food rations (three arms of CBS, one arm of LNS, and one arm of MNP). Given the operations research focus, all data collected by PROCOMIDA was quantitative, though qualitative monitoring for specific areas (quality of services, adoption of practices) was conducted in years four through six. Towards a rigorous research design, USAID and the FANTA Project intentionally limited PROCOMIDA to a set of activities that excluded livelihood security (agriculture, income generation, livelihoods, water sanitation, hygiene) on the grounds that their inclusion would 51 This evaluation does not address the results of these research questions, answered through a separate report prepared by IFPRI. 22 complicate research analysis by adding too many variables. Thus, Guatemala and Burundi programs are not directly comparable to other Title II programs. The rigorous research component itself restricted the benefits that PROCOMIDA was able to offer, and when compared to programs of wider scope, was less able to meet local needs or fulfill individual and community expectations. This was difficult for participants to understand, despite PROCOMIDA’s explicit efforts to communicate the nature of the program to authorities and beneficiaries. To address participant confusion on the limits of the program, without compromising the provision of fully valid research results, PROCOMIDA implemented Household Action Plans in 2012, and took great effort to communicate the nature of the program to authorities and participants and support them in more sustainable ways. It was not a perfect solution, and some participants maintained a desire that the program should have provided more support in terms of agriculture and livelihood activities, especially considering the increasing violence and insecurity in Alta Verapaz. 3.3 Program Resources USAID, through the Bureau of Democracy, Conflict, and Humanitarian Assistance / Office of FFP, used monetization proceeds and Section 202(e) to fund PROCOMIDA. Overall resources mobilized throughout the entire program by FY reached a life-of-award budget of approximately inclusive of more than 35,000 metric tons (MT) of direct distribution and monetization commodities. Table 8: Summary of food aid program resources in US$ Fiscal Year Commodities Called Forward Commodities for Direct Distribution (MT) Commodities for Direct Distribution (cost & freight) US$ Commodities for Monetization (MS) Commodities for Monetization (cost & freight) US$ A unique feature of Title II programs in Guatemala is voluntary contributions made by program participants, normally used to cover food commodity transportation costs. In the case of PROCOMIDA, these Community Funds remained within the communities for local development needs, and managed by local health committees trained by the program. Communities held assemblies at program start to determine the nominal contribution amount per month, set at 10 and 15 Quetzales (US$1.25-2.00). Health committees then planned for annual Community Fund use, providing a fund report at quarterly public assemblies. Each community set aside a portion of their fund to pay for emergency health expenses, particularly those related to pregnancy and child illness. Access to the Community Fund was available to all community members during an emergency, regardless of program participation. When used, households received emergency funds as a loan to be repaid within three to six months. Participating communities collected over 12 million Quetzales (over US$1.6 million) over the life of the program towards their Community Fund, demonstrating that local communities are capable of raising, managing, and utilizing funds for their own development needs (Figure 4). This amount equals an impressive 4.5% of the overall program resources (commodities plus 23 operational costs), and coming from poor and extremely poor households, is a strong indication of the program’s importance to these communities. Of this Community Fund, 5% was dedicated to emergency funds, 63% to improve community infrastructure (CCs, CHCs, and local storerooms at distribution points used to temporarily store PROCOMIDA rations), and the remaining 32% covered operating costs (recipe ingredients; home garden seeds - Table 9). By program end, 632 communities had active emergency funds, and 1,490 emergency cases utilized funds. When fund management issues emerged (over contribution amounts, transparency, loan repayment), field staff mediated solutions with community managers. 52 “We had to pay Q15 on a monthly basis, which was used for improving the health center and for the emergency fund. This fund is very helpful, but there is a problem that once a family gets an emergency loan, it is very hard for them to pay it back. But we have to insist. We have still not benefited from the fund and we need it to be operational” (mother, Seconon). Figure 4: Amount of voluntary contributions and use per year 4,000,000.00 3,500,000.00 3,000,000.00 2,500,000.00 2,000,000.00 1,500,000.00 1,000,000.00 Storeroom Other Infrastructure Emergency Fund 500,000.00 0.00 IY2 (2010/11) IY3 (2011/12) IY4 (2012/13) IY5 (2013/14) IY6 (2014/15) Qualitative evaluation findings confirm that communities and participants appreciated infrastructure improvements, which remained with the communities after the program ended. They also appreciated the existence of the emergency funds, encouraged by the program and which continue to exist in a majority of the communities under local management.. “I think it was good to give the contribution, because the whole community benefited. And I would like to see that at least the emergency fund continues” (mother, Seconon). Table 9: Amount and use of financial contributions from participants to PROCOMIDA Year Emergency Fund Infrastructure 53 Storeroom Other 54 Total IY2 (2010/11) Q 28,830.43 Q 269,726.88 Q 555,772.82 Q 417,223.11 Q 1,271,553.24 IY3 (2011/12) Q 52,647.61 Q 492,551.78 Q 1,014,904.02 Q 761,896.59 Q 2,322,000.00 IY4 (2012/13) Q 76,636.05 Q 716,978.91 Q 1,477,336.61 Q 1,109,048.43 Q 3,380,000.00 IY5 (2013/14) Q 80,218.44 Q 750,494.49 Q 1,546,395.54 Q 1,160,891.53 Q 3,538,000.00 IY6 (2014/15) Q 348,465.52 Q 1,021,302.13 Q 236,621.03 Q 752,794.92 Q 2,359,183.60 Total Q 586,798.05 Q 3,251,054.19 Q 4,831,030.02 Q 4,201,854.58 Q 12,870,736.84 % of total 4.5% 25.3% 37.5% 32.6% 100% 52 There was one case where the food distribution was suspended for two months until the community reached an agreement and could continue within the program. 53 Investment in improvement of the convergence centers. 54 Costs of training events (inputs for cooking recipes) and procurement of home garden seeds. 24 Total in US$ 55 $74,750 $412,142 $615,410 $535,261 $1,639,560 3.4 Program Participation By program end, 88.9% of all households surveyed participated in the program, on a voluntary basis. For some remote households, the considerable distance to distribution points was a reason for non-participation. For others, the financial contribution to the Community Fund was an obstacle. “I did not participate in the program due to a lack of money. Participants had to pay a monthly contribution of Q.15, 56 and so I was unable to get in” (non-participant mother, Sacristal). Despite the voluntary and unconditional nature of contributions to reception of monthly food rations program, insufficient feedback mechanisms from program staff to community members and health commission staff regarding financial contributions resulted in confusion and led to avoidable non-participation. PROCOMIDA dealt with a range of challenges regarding the participation of women, including high levels of illiteracy and the cultural appropriateness of their participation. PROCOMIDA designed the BCC strategy to train and equip women in particular with knowledge and skills to improve their well being and the well being of their children. Where the program fell short regarding equitable gender relationships was underappreciating the value of such issues as the division of labor in child rearing, decision-making power, and access to and control of resources, to name a few. The baseline identified that culturally the Q’eqchí of Alta Verapaz allowed young women and mothers only limited autonomy. Respondents reiterated this during endline FGDs. Many participants noted that efforts to include fathers and other family members in trainings did not meet expectations, and made behavior change more difficult. “It would be good if training sessions include husbands, so that they can also hear and learn what we are learning” (graduated mother, Seconon). A major barrier to father participation was their lack of availability. Most men engage in subsistence agriculture. They were thus unable to participate in trainings that took place during farming hours. They also periodically migrate away from home in search of seasonal and other temporary work to supplement household incomes. Many fathers stated during qualitative fieldwork that they regretted not being able to participate more in PROCOMIDA activities, and wished trainings had been held in the evenings or on weekends. 3.5 Program Monitoring Mercy Corps/Guatemala designed a results based M&E framework to measure performance indicators, improve program implementation, and report the degree to which results were achieved. In addition to the 2010 baseline, a 2012 midterm review, and this 2015 endline, PROCOMIDA’s M&E unit carried out annual participant surveys to verify progress made against indicators in the IPTT. The program used results from these surveys, reviews, and other formative research to inform, and subsequently adjust, activities towards improved effectiveness and efficiency and, ultimately, results. Among the most important adjustments were:  Family ration size reductions  Message and BCC distribution channel changes for all training sessions  Household Action Plan implementation for all participating households  Home garden introduction  Oil ration repackaging 3.6 Program Sustainability Sustainability may be measured by whether partners (public sector and civil society sector) could i) continue to provide similar inputs to sustain the processes leading to desired results, and 55 Using exchange rate US$ 1 = Q 7.4955 as of August 30, 2015, www.oanda.com. 56 Q 15.00 = US$ 1.91, at an exchange rate of US$ 1 = Q 7.4955 as of August 30, 2015, www.oanda.com. 25 ii) ensure that results achieved under the program would be upheld and eventually replicated by new inputs. Towards this, partners must possess:  Institutional capacity (organizational structure, knowledge, expertise)  Institutional mandate and normative framework  Financial capacity to conduct expected tasks  Sufficient ownership and appropriation Regarding the public sector, this evaluation was unable to assess in any detail whether the local healthcare system, currently being rebuilt in Alta Verapaz, would be able to assume the functions listed above. The breakdown of local healthcare systems not only compromised PROCOMIDA’s outcomes, but also the possibility to successfully complete its comprehensive exit strategy. PROCOMIDA’s training efforts during the final months of fieldwork were immense, as shown by monitoring indicator 27 (health facility staff trained) in health and nutrition best practices (cumulative), which increased from 1,138 to 3,012 persons between June 2014 and June 2015. Only since May 2015 have health authorities been sufficiently organized to allow PROCOMIDA to provide some basic training to the new local healthcare staff. It is too early to determine whether this intense training effort will pay off, yet the new local health system - as designed - could meet its goals of providing quality services at reduced costs, if properly implemented. In order to increase the sustainability potential, PROCOMIDA transferred its training methods and materials to MOH and SESAN national, regional, and local staff, as well as local NGOs, during the last two years of the program. A third partner to sustainability is the community (household) sector, measured by how well they were able to maintain and share promoted behavior change. While evaluation findings demonstrate that community knowledge and capacity is improved, most households lacked the resources to apply knowledge gained through the program. Concerning nutritional improvements and undernutrition trends, optimal nutrition and healthy growth for children in the first 1,000 day period will have enduring benefits through their lives. Despite food security situations in households changing considerably in the absence of food rations, 41% of surveyed households (graduated from the program, and with children 24 to 59 months of age who were program participants at some point during their first two years of age) were able to maintain a positive nutritional status even in the absence of program interventions (rations). Findings further prove that knowledge increased significantly (irrespective of whether all targets were reached), but there was a delay in subsequent behavior change. Qualitative fieldwork suggests that the process of behavior change was further weakened following PROCOMIDA exit. Few structures were left behind for local multiplication of new knowledge from graduated participants to new mothers, and these are weak. Mothers rely primarily on midwives, future programs could increase replication of knowledge by including midwives at the design stage.PROCOMIDA staff anticipated that women involved in Care Groups would continue to practice the behaviors they promoted or adopted during the project period, and to influence other women. They hoped that future local public health staff will support their sustained performance, but there is no available information to allow for an informed judgment. 3.7 Program Results 3.7.1 IPTT This final evaluation implemented a series of household, knowledge, and healthcare center endline surveys to estimate achievement of the overall program goal and specific targets. PROCOMIDA provided other indicators related to the accomplishment of the program’s delivery. 26 Table 10 below shows the list of impact, monitoring, and trigger indicators with respect to the baseline, endline (achieved), and final target values. Table 10: IPTT – baseline, target, endline, and percent achievement Indicators Baseline (2010) Target (+/- direction) Endline (2015)a % of PROCOMIDA target achieved b # Indicator Objective 1: By 2015, mothers have the capacity to improve and maintain the health and nutritional status of household members, particularly for children less than 2 years of age, acutely malnourished children under 5 years of age, and pregnant and lactating women #1 Impact Indicator 1: 13.3% 10% (-) % children 0-59 months underweight 14.0% 60%* (<-2 z-score weight-for-age) #2 Impact Indicator 2: 59.2% 53% (-) % children 6-59 months stunted 57 53.5% 99% (<-2 z-score height-for-age) #3 Impact Indicator 3: 7.19 8 (+) Average Household Dietary Diversity Score 7.26 91% (HDDS) Intermediate Result 1.1: Increased consumption of energy and nutrient dense foods by pregnant and lactating women, all children 6-24 months of age, and acutely malnourished children between 24-59 months of age #4 Monitoring Indicator 1: 0 18,900 (+) # pregnant and lactating women receiving 34,337 182% food rations (accumulative) #5 Monitoring Indicator 2: 60.1% 80% (+) % children 6-24 months with minimum 71.6% 90% dietary diversity #6 Monitoring Indicator 3: 0 35,000 (+) # children aged 6-24 months receiving food 57,590 165% rations (accumulative) Intermediate Result 1.2: Improve household knowledge, attitudes and practices for key care behaviors #7 Monitoring Indicator 1: 0.0% 52.6% 75% 57 Note that the sample size for stunting is representative for both age ranges, 0-59 months and 6-59 months. We measured 0-59 months for comparison with the baseline reported by IFPRI, but the IPTT was 6-59 months from PROCOMIDA startup. 27 % mothers demonstrating increased nutritional knowledge 70% (+) #8 Monitoring Indicator 2: 75.0% 85% (+) % 69.5% 82%* of newborns who receive essential newborn care #9 Monitoring Indicator 3: 65.1% 85% (+) 75.7% 89% % children 0-6 months exclusively breastfed #10 Monitoring Indicator 4: 5.9% 40% (+) % mothers that know the danger signs of 41.7% 104% pregnancy #11 Monitoring Indicator 5: 4.9% 45% (+) % mothers with proper identification of 23.3% 52% childhood illness warning signs #12 Monitoring Indicator 6: 26.3% 70% (+) % children aged 0-23 months with diarrhea 44.6% 64% that received adequate treatment #13 Monitoring Indicator 7: 50.0% 70% (+) % 13.8% 20%* children aged 6-23 months with respiratory diseases that received adequate treatment #14 Monitoring Indicator 8: 82.2% 90% (+) % m 80.5% 89%* others receiving minimum recommended antenatal care #16 Monitoring Indicator 10: 26.9% 70% (+) % mothers receiving minimum 30.7% 44% recommended postnatal care #17 Monitoring Indicator 11: 73.2% 95% (+) 10.0% 11%* % children receiving full vaccinations #18 Monitoring Indicator 12: 18.3% 75% (+) 16.2% 22%* % children receiving routine health services Intermediate Result 1.3: Increased engagement of households in reducing vulnerability to food security #19 Monitoring Indicator 1: 1.8% 60% (+) % households with Household Actions 37.4% 62% Plans #20 Monitoring Indicator 2: 0 358 (+) # Training and Distribution Points with 211 59% emergency funds Objective 2: By 2015, healthcare service providers at the community through municipal levels have improved service quality and deliver #21 Impact Indicator 1: 6.5% 40% (+) % 0.0% 0%* health facility staff and community volunteers able to identify minimum number of core health and nutrition practices #22 Impact Indicator 2: 0.0% 40% (+) 0.0% 0% % of training and distribution points that offer mothers pre- and post-natal services and children growth monitoring and general health services 28 #30 Impact Indicator 3: % of local CHVs who meet minimum standards/thresholds for performance 0.0% 30% (+) 97.7% 326% Intermediate Result 2.1: Increased communication and interaction between community members and health service providers #23 Monitoring Indicator 1: 0 # health commissions with regular meetings Monitoring Indicator 2: 358 (+) 211 59% #24 #25 # health commissions demonstrating progress on action plans Monitoring Indicator 3: 0 358 (+) 0 2,100 211 59% #26 # of pregnant women in health facility orientation visits Monitoring Indicator 4: % deliveries at health facilities 35.9% (+) 2,124 101% 45% (+) 43.6% 97% Intermediate Result 2.2: Increased and improved provision of minimum standards in health services to women and children #27 Monitoring Indicator 1: # health facility staff trained in health and nutrition best practices Monitoring Indicator 3: 0 330 (+) 95% 3,012 913% #29 % of detected SAM cases referred per MOH protocols N/A (+) 98.7% 104% Intermediate Result 2.3: Increased promotion and planning by healthcare providers to meet community healthcare needs #31 Monitoring Indicator 1: # persons trained in planning and advocacy around food security and health 0 2,600 5,387 207% Trigger Indicators*: Trigger Indicator 1: #32 #33 Wasting: % children 6-59 months who are wasted (<-2 z-score weight for height) Trigger Indicator 2: Change in food prices Trigger Indicator 3: - <4.0% 1.3% n/a - 8.74 3.32 n/a 2,351 #34 Irregular rainfall Trigger Indicator 4: - 2,792 n/a #35 #36 Security Trigger Indicator 5: Coping strategies - 35.7 53.7 n/a - <10 40.1 n/a a Achievements that match or exceed the targeted values are shown in green while those that are below target are shown in red. b The endline value is compared to the target value set by the program, to show the percentage of the target achieved. Indicators that show an achievement of 90% and above are shown in this column in green, while those that are below 90% are in red. 29 * Denotes indicators that have endline values less than the baseline - showing an opposite directional change compared to the intended target. At the time of the endline evaluation, PROCOMIDA achieved 12 out of 29 impact and monitoring indicators (defined as having values within at least 90% of the target set, regardless of the baseline). Out of these, eight exceeded the target values.  IR 1.1: The distribution of food rations to pregnant and lactating women (indicator 4) reached close to 35,000 – nearly doubling the target set of 18,900. The distribution of food rations to children aged six to 24 months (indicator 6) exceeded the endline target by 65% (57,590 compared to the target of 35,000). Following agreed-upon ration reductions (after it was found that there were leftovers), the program was able to expand its interventions, hence increasing coverage. The program learned important lessons about strategic return on investment, minimum ration requirements, and balancing quantity and reach/coverage.  IR 1.2: Among indicators for household knowledge, attitudes, and practices, only the percentage of mothers with knowledge on the danger signs of pregnancy reached their targets (slightly exceeding), demonstrating that while the program had the potential to transfer knowledge to participating mothers as a result of the BCC strategy, more could have been done to improve attitudes and practices.  IR 1.3: The number of CCs and health posts (HP) with emergency funds (indicator 20) and the percentage of households with Household Action plans (indicator 19), only achieved about 60% of the target set.  SO 2: One of three indicators met the target. The target set for local CHVs who met minimum standards/thresholds for performance was 30%, but the achievement was close to 100%. It is worth noting, however, that the percentage of training and distribution points that offer pre- and post-natal services actually achieved the midterm target of 20%, but by the end of the program was reduced back to 0%. This is possibly (but not conclusively) linked to the observed last minute dip in post-natal care received (indicator 16).  IR 2.1: Both the number of health commissions with regular meetings (indicator 23), and those demonstrating progress on Household Action Plans (indicator 24), achieved only 59% of their targets. CHCs meet monthly with PROCOMIDA staff, and had half-yearly assemblies to present their progress towards annual work plans to their constituent communities.  IR 2.2: Indicator 29 (percentage of detected severe acute malnutrition, or SAM cases referred per MOH protocols) met the set target, likely due to the expansion and inclusion of educators and staff at HPs. The number of healthcare facility staff trained in health and nutrition best practices (indicator 27) exceeded the set target by about ten-fold, reported at 3,012 versus the targeted 330.  IR 2.3: The number of persons trained in planning and advocacy around food security and health (indicator 31) doubled the target, related directly to the expansion of the program.  SO 1: None of the anthropometric impact indicators met or exceeded target. The percentage of children 6-59 months stunted dropped from 59.2% to 53.5%, very narrowly missing the target of 53.0%. The percentage achievement, however, was 99%. The average HDDS also did not reach the targeted score, but did achieve 91% of the target.  IR 2.1: The percentage of deliveries at health facilities (indicator 26) increased significantly from baseline, but was slightly lower than the target set, at an achievement percentage of 97%. 30 Eighteen indicators did not reach the targets set, including seven indicators that showed values below those at baseline (marked * in the IPTT table) and clearly did not achieve endline targets. Of these, four indicators had unexpected results that were substantial.  SO 1: Although statistically insignificant, the percentage of children 0-59 months who were underweight increased from 13.3% at baseline to 14.0%, compared to 10.2% for Alta Verapaz, and 12.6% nationally (MSPAS 2015). The SESAN Zero Hunger Pact monitoring report for 2014 similarly reported 14.3% of children 3-59 months for all 166 prioritized municipalities were underweight. This worsening direction of change occurred in the last two years of the program; at midterm (year 3), the percentage underweight reduced to 12.3%,The national health system breakdown directly influenced the jump up to 14.0% in the second half of the program, and is an important finding for the country.  IR 1.2: Out of fifteen indicators, only two achieved target. The reduction in the MOH budget, issues of supply chain management, the MOH revamp of primary care delivery that caused gaps in health services at the community level for up to six months, all lowered coverage in Alta Verapaz for full and timely vaccinations and other interventions, and may explain this decrease. The breakdown of community health services through PEC severely hampered seven indicators. Other indicators had lower levels of achievement or reversed direction due to partial CC closeout, including: i) the percentage of newborns receiving essential newborn care (dropped by about 5%); ii) the percentage of children aged six to 23 months with respiratory diseases that received adequate treatment (dropped by more than 36%); and iii) the percentage of children receiving full vaccinations (dropped drastically from 73.2% to just 10%). Only indicators 7 (52% achievement) and 11 (52% achievement) were under the direct influence of the program. It is worth noting that, although the indicator on the percentage of mothers receiving minimum recommended antenatal care dropped only by about 2%, the indicator moved from a baseline figure that was already within 90% of achieving the target at the start of the program, to a final figure that fell just under 90%. It therefore failed to achieve the target.  SO 2: The percent of healthcare facility staff and CHVs able to identify a minimum number of core health and nutrition practices (indicator 21), and the percent of healthcare facilities meeting minimum standards for health and nutrition services and practices, was significantly unable to meet the endline target, and in fact dropped from baseline. This is linked intrinsically with changes in the MOH intervention strategy, and interim breakdown of the healthcare system around the time of this endline survey’s data collection period. Trigger indicators were measured through secondary data sources, mostly from government institutions, with the exception of indicator 36 (coping strategies), which was calculated using the CSI and measured through the annual household survey. Neither trigger indicators nor non-trigger indicators are linked directly to specific program activities, and therefore change cannot be attributable to either one. Information on security was unavailable for this final evaluation (indicator 35) from the Governor of Alta Verapaz. Coping strategies (indicator 36) increased, meaning that people in intervention areas had to implement more strategies to cope with food insecurity, even though they received food rations. Additionally, there were changes in food prices (indicator 33). 3.7.2 Discussion of Results The presentation of results provides a snapshot of two moments in time: April/May 2010, and April/May 2015. Between April/May 2010 and the end of 2013, there were some improvements in local healthcare services, reported in the program’s ARR through PROCOMIDA’s M&E unit. The task of this endline report, however, is to compare the snapshots including the virtual absence of services in 2014/2015. 31 Outcomes and results reference the objectives and indicators listed in the IPTT as the contextual framework for discussion. As such, Objective 1, which relates to parameters focused on the population-level indicators, covers household characteristics, dietary diversity, mothers’ knowledge of feeding care and health, childhood illness preventive healthcare practices, IYCF practices, and nutritional status of children, begins with the characteristics of households and mothers, while Objective 2, which concerns service providers, covers convergence centers, and community healthcare. Objective 1: By 2015, mothers have the capacity to improve and maintain the health and nutritional status of household members, particularly for children less than two years of age, acutely malnourished children under five years of age, and pregnant and lactating women Household Characteristics. Although no specific poverty indicators were assigned for the PROCOMIDA baseline and endline surveys, general characteristics of the population show that PROCOMIDA reached both poor and very poor households, and that the overall household characteristics did not change substantially during the implementation period. Table 11 presents several key household characteristics. In 2015, the average household size was 4.8 people, lower than in 2010, with an average of 2.4 members under the age of 18, and 1.4 children under the age of 60 months – a sampling effect since the household inclusion criteria was based on the household having at least one child aged zero to 59 months. Of all households, 11% had a child under six months of age, and 52% had a child between six and 23 months of age. Among the 2,419 sampled children in the zero to 59 months age group, 208 (8.6%) were from non-participant households, possibly due to the high PROCOMIDA participation rate. The percentage of pregnant women among the mothers sampled was lower in the endline survey at 5.2% compared to 10.6% in 2010. However, in both baseline and endline surveys, pregnancy among adolescents (pregnancy among mothers under 19 years of age) was at about 5%. A comparison of the age distribution of mothers between 2010 and 2015 shows that the share of pregnancies decreased for the age group 19 to 29, but remains unchanged for mothers under 19 and over 29 years of age. Family planning - Findings from the qualitative evaluation suggest that PROCOMIDA’s capacity building in family planning improved pregnancy timing and spacing. PROCOMIDA actively coordinated its activities with healthcare providers, NGOs, and others working in implementation areas. A reproductive health NGO, WINGS, trained PROCOMIDA field staff in family planning, enabling them to promote healthy timing and spacing of pregnancies among mothers reached by the program through family planning messages within the BCC component. Key informants and FGD participants noted that most mothers were planning and spacing their pregnancies, in agreement with their partners. This is important considering the strong community tradition of allowing pregnancies to occur naturally. Interviewed mothers appreciated their new family planning knowledge, highlighting the important role of midwives in this regard. Endline survey findings suggest that family planning efforts and contraceptive use was most successful for women between 19 - 29 years of age. Pregnancy among adolescents, however, continued to be an issue, indicating a need to focus on younger adolescent girls. Previously, public health institutions and NGOs in Alta Verapaz distributed contraceptives free of charge or at low cost, but the availability of free contraceptives through the public healthcare system decreased throughout 2014 and stopped completely towards the end of that year. Adolescents who were sexually active but not financially independent would not likely have had other means of obtaining contraceptives. “There are girls aged 13-14 already with their own babies, while they 32 themselves are still just little girls; but they have no alternative. Once they get together with a young man they quickly get pregnant, and these are often risky pregnancies because of their young age” (local midwife). 33 Table 11: Background characteristics of surveyed households, mothers, and children, at baseline and endline Characteristics Baseline Endline (2010) (2015) Household Average household size Mean: 5.3 Mean: 4.8 Average age of head of the household (years) Mean: 32 Percentage of pregnant women 10.6% 5.2% Number of children (siblings) in the 0- 59 months old age range (% n = 1,736 % households) 1 1,106 63.7 2 579 33.4 3 49 2.8 4 2 0.1 Mothers Total number of mothers 1,307 1,748 Average age of mother (years) 28 (± 7.2 SD) 28 (±7 SD) Percentage of mothers with male partners 93% 92% Mothers’ anthropometric classification (%) Overweight 35.8% 27.6% Underweight 0.7% 1.7% Children (0-59 months) Total number of children 1,840 2,419 Boys (%) 907 (49.3%) 1,234 (51.0%) Girls (%) 933 (50.7%) 1,185 (49.0%) Average age (months) 29.7 (±16.7) 30.3 (±16.6) Age groups (months) 0-5 (%) 189 7.8% 6-23 (%) 737 30.5% 24-59 (%) 1493 62.7% Household conditions - In 2015, housing conditions remained of poor quality for the vast majority, and access to basic services was poor. The majority had dirt floors (86% - a strong proxy for poverty in Guatemala), wooden walls (69%), and aluminum roofs (78%). The most common water source was rainwater (47%) and spring or river water (33%). Only 20% of households had access to an improved source of water (faucet, bottled, protected well, reservoir). The majority of households (92%) used latrines, an increase compared to 82% at baseline. This increase is likely due to campaigns to build or improve latrines by the municipality and NGOs in the area. Although the percentage of households with electricity increased considerably from 16% in 2010 to 30% in 2015, 70% of the population still did not have access to this critical service. Qualitative field visits confirmed the precarious living conditions reflected through the household survey. Only one of 14 communities visited had piped water. In the remaining communities, households collected rainwater or had to walk for 30 minutes to a nearby spring for water, often several times a day. This situation represented a severe impediment for the implementation of hygiene habits promoted by the program in order to provide proper and healthy conditions in the household. “There are children who often become sick. Even if we are being trained regarding hygiene habits, we have problems because of water shortage. There are children who do not bathe; the main problem is hygiene because of the lack of water” (CHW, Chajixim). 34 91.2% 87.7% 11.6% 0.7% 8.8% Per c enta g e (%) Household hunger scale - The Household Hunger Scale (HHS) is a household food deprivation scale based on perceptions and experiences of interviewed households, regarding their sensation of hunger. 58 At baseline, only 8.8% of households reported moderate hunger levels, and none for severe hunger. By 2015, although there is a slight shift of 11.6% reporting moderate hunger and 0.7% (12 households) severe hunger, the increase was not statistically significant. In the four weeks preceding the household survey, 28.7% of households reported that there were periods with no food in the house, 59 while 8.8% reported that at least one household member went to bed hungry, and 6.3% reported that at least one household member had not eaten for an entire day. Figure 5: Household hunger scale at baseline and endline 100% 80% 60% 40% Little or no hunger Moderate hunger Severe hunger 20% 0% Baseline 2010 Endline 2015 The qualitative evaluation confirmed that although most families do not suffer hunger for most of the year and have on average an acceptable dietary diversity, the poor economic conditions of most communities and families created important obstacles to people's access to safe and adequate food. The most dramatic cases are those of families who had no land available for growing food and consequently relied exclusively on income to feed their families. This situation may be aggravated if women are single heads of household. "In my case, my husband works away from home for 30 days at a time. For me it is complicated to be alone when my husband is away, because I have no money and I must find a way to feed my children" (graduated mother, Samanzana). Further, "Being a single mother is very difficult because we cannot find a husband and must always respond to what our fathers or siblings want. This is not right; we have to work at some point in order to have some Quetzales. When we grow older we will have to have our own household and home for our family; we cannot stay like this forever" (female head of household, San Antonio). 58 USAID (accessed October 2016). Household Hunger Scale (HHS): Indicator definition and Measurement Guide. http://www.fantaproject.org/monitoring-and-evaluation/household-hunger-scale-hhs. 59 Of the 28.7%, none of the respondents were active in PROCOMIDA at the moment of the survey. Some were participants in the program before and others never participated. 35 Average Household Diet ary Diversity Score Dietary Diversity. The HDDS, used as a proxy measure of household food access, counts the number of different food groups consumed rather than the number of different foods consumed. In 2015, households consumed on average 7.26 food groups out of 12 in the 24 hours before interview, only a slight increase from the 7.19 recorded in 2010 (Figure 6). The highest consumption (amount of food) and the most frequently consumed food groups were cereals, sugar, the miscellaneous group (junk food, instant food, coffee, tea, condiments), oils and fats, pulses and legumes, and eggs. Less than 40% of interviewed households reported the remaining food groups. The consumption of micronutrient-rich foods was low, with less than 40% of households having consumed meat, and the proportion of households reporting the consumption of fruits less as than 20%. Less than 10% of households reported the consumption of vegetables and dairy products. Figure 6: Average HDDS at baseline and endline 8.2 8 7.8 7.6 7.4 7.2 7 6.8 6.6 Target Baseline survey Final evaluation FGDs suggest that the main driver of improved dietary diversity was behavioral change regarding food consumption and optimal nutritional knowledge. New and additional food types were provided via home gardens and the spending of diverted household income. PROCOMIDA’s food ration package did not contribute to changes in diversity, in terms of their composition. Anecdotal reports suggest that the implementation of home gardens by the program contributed to improved dietary diversity, but these could not be confirmed econometrically since the home garden component was systematically implemented only during the second half of the program. Discussions also revealed that for most participants, they used home gardens during one season only and apparently did not extend them beyond that due to the use of hybrid seeds (purchased directly by communities, and likely unknowingly). FGDs revealed that measurements of dietary diversity might not apply consistently throughout the year. Measurements made in April (2010 and 2015) were at the end of yearly periods still characterized as modestly abundant, whilst there were at least three months per year (June to August) characterized by very low diversity and occasionally even a scarcity of food. As such, measurements of HDDS in this evaluation may not reflect the true dietary diversity of the population throughout the year. Program participants expressed their appreciation of new nutritional knowledge and recipes to cook more diverse foods, with the caveat that they were concerned about: i) the availability of ingredients, particularly once their participation in PROCOMIDA ended; and ii) their inability to use the cook book provided as it was prepared in Spanish. Mothers cited that they were only able to understand Q’eqchí and had to learn the recipes by memorizing them. 36 Meal times - Most FGD participants mentioned having a routine of three meals per day (breakfast, lunch, dinner) for the whole family, the most important one being breakfast, which occurred between five and seven a.m. Preparing large quantities of food in the morning demanded a considerable amount of time from women and girls, who were traditionally in charge of preparing food. “We have so much work to do that a day is simply not enough time for carrying out all our tasks, we would like to have more time, but it is hard for us because we have to take care of the house and the children” (graduated female, Jobchacob). Mothers with young children spent additional time preparing snacks between meals, while still complementing with breastfeeding. Older children may have received food at school, which reduced the pressure on households. Even though the coverage of school feeding programs (not part of PROCOMIDA) was officially countrywide, many schools, particularly those in remote areas, in reality did not have the program in place. Most FGDs indicated that at dinner, a light meal was prepared or leftovers from lunch consumed. The combination of a lack of electricity, durable goods, poor housing conditions, and often very low levels of cash income led to considerable limitations regarding the storage of food and hygiene when appropriate storage facilities were not available. Mothers’ Knowledge on Feeding, Care, and Health. The main aspects of breastfeeding learned or reinforced through PROCOMIDA, according to FGDs, were knowledge of the nutritional value and other properties of breast milk and colostrum versus bottle feeding, and about the amounts of food and suitable timing for feeding babies and small children at different stages in life. “Previously, I would give solid food to my three-month old baby, but when I entered the program I learned that I should not give him any solid foods before six months” (recently graduated mother, San Antonio I). Further, "The knowledge we obtained during the trainings includes the preparation of different foods, such as banana porridge during its season, and how to feed children and how to implement a home garden (...) and to not feed the kids only with what is in the shops, such as biscuits. We learned to wash the children's as well as our own hands and the importance of keeping them clean" (graduated mother, Secoyou). Lastly, "Knowledge improved regarding the quantities of food that should be given to babies and to feed when the baby wants to as opposed to only during fixed family mealtimes and to place the spoon in his mouth until he learns" (CHW). FGD participants noted that although they learned about childhood nutrition and appropriate feeding practices, they frequently encountered obstacles implementing the suggested practices. Many families had only one cooking pot, did not have the time (or firewood) to prepare three different meals a day, or simply did not have any food to prepare. PROCOMIDA achieved considerable improvements regarding mother knowledge on infant feeding (Figure 7). Although a high proportion of mothers had knowledge of each of the breastfeeding and feeding practices questions asked (ranging from 67% knowing that children less than six months should not receive semi-solid foods, to 92% knowing that children less than 37 % mothers de m onstrating incr e ased nutritional kno wledge six months should not be given colostrum), just over 50% had knowledge of all four infant feeding indicators. Figure 7: Increased mother knowledge of child nutrition at baseline and endline 80 70 Target 60 50 40 30 20 10 0 Baseline survey Final evaluation Figure 8: Mother knowledge on child nutrition at baseline and endline Children <6 m should not receive semi-solid foods Children <6 m should not drink other liquids 67.0% 83.3% 82.0% Children <6 m should be given colostrum 71.6% 92.0% 86.3% Children should be breastfed less than one hour after birth Mothers who have good knowledge of infant feeding 35.0% 53.0% 89.0% 82.5% Endline 2015 Baseline 2010 The baseline and endline surveys asked for nine different danger signs for pregnancy. Mother knowledge on proper identification of any four of these nine danger signs during pregnancy represents ‘proper identification’ of danger signs, and is presented in Figure 9 as an additional 38 comparison bar. In fact, proper identification of danger signs increased seven-fold during the project period. Only one danger sign (persistent backache) showed a decrease, most likely related to the fact that during the program period, Guatemalan health authorities removed ‘backache’ from the official catalogue of pregnancy danger signs (and thus the program no longer promoted this sign). Knowledge regarding all remaining danger signs improved, and these observed improvements are statistically significant at the level of 0.03 or below. Figure 9: Mother knowledge of danger signs during pregnancy at baseline and endline Trouble for breathing No fetal movement after the 5th month Continous contraction before the 37th wks Stronng headache and blurred vision Hands, face and body swelling Vaginal discharge of clear liquid Persistent backache Strong stomachache Haemorrhage and vaginal bleeding Mother with proper identification of pregnancy danger signs ( 4 or more) 9.0% 3.0% 4.0% 2.0% 3.0% 4.0% 6.0% 18.0% 24.0% 21.0% 30.0% 26.0% 31.0% 43.0% 41.0% 41.0% 42.0% 57.0% 76.0% 79.0% Endline 2015 Baseline 2010 The baseline and endline surveys asked for nine different danger signs for pregnancy. Mother knowledge on proper identification of any four of these nine danger signs during pregnancy represents ‘proper identification’ of danger signs, and is presented in The baseline and endline surveys asked for nine different danger signs for pregnancy. Mother knowledge on proper identification of any four of these nine danger signs during pregnancy represents ‘proper identification’ of danger signs, and is presented in Figure 9 as an additional comparison bar. In fact, proper identification of danger signs increased seven-fold during the project period. Only one danger sign (persistent backache) showed a decrease, most likely related to the fact that during the program period, Guatemalan health authorities removed ‘backache’ from the official catalogue of pregnancy danger signs (and thus the program no longer promoted this sign). Knowledge regarding all remaining danger signs improved, and these observed improvements are statistically significant at the level of 0.03 or below. Figure 9 as an additional comparison bar. In fact, proper identification of danger signs increased seven-fold during the project period. Only one danger sign (persistent backache) showed a decrease, most probably related to the fact that during the program period, Guatemalan health authorities removed ‘backache’ from the official catalogue of pregnancy danger signs, and the program consequently no longer promoted this sign. Knowledge regarding all remaining danger signs improved, and these observed improvements, are statistically significant at the level of 0.03 or below. 39 The results show that even if the PROCOMIDA trainings had emphasized knowledge regarding all of these signs, success in converting information (training) into knowledge is very different for different signs. PROCOMIDA was less successful in increasing knowledge regarding continuous contractions or lack of fetal movements. The qualitative evaluation was unable to clearly identify the underlying causes of these different learning outcomes, yet it appears plausible that the messages used for transmitting this specific knowledge on contractions or fetal movements was more difficult to understand or less convincing for participants compared to training on other danger signs. Childhood Illness. Regarding mother’s knowledge on proper identification of danger signs in childhood illness, the endline survey shows a high level of mothers with adequate capacity to recognize the classic signs of fever and vomiting (whose recognition was already high at 90%, before the program) (Figure 10). However, less than 30% of mothers were able to properly identify other symptoms, such as respiratory problems, although improvements were achieved for most signs compared to levels shown for 2010. All observed improvements are statistically significant at least at a 0.05 level. The symptom ‘Sicker, or not better’ was the only one with an observed decline at endline, but the difference was not statistically significant. When combining knowledge of all these signs into one indicator (the correct recognition of any three out of six childhood illness danger signs), it was observed that the knowledge level increased almost five-fold, from 5% to 23%. Figure 10: Mother knowledge of danger signs of childhood illnesses at baseline and endline Blood in stool Trouble to breath Rapid breathing Fever and vomiting Sicker or not better 5.0% 5.0% 6.0% 11.0% 24.0% 28.0% 23.0% 90.0% 90.0% 13.0% No food - consumption (breastfeeding, drinking and eating) 10.0% 24.0% Mother with proper identification of children illness warning signs (3 or more) 5.0% 23.0% Endline 2015 Baseline 2010 Public healthcare system - The achievement of the outcomes related to changes in minimum childcare practices depended exclusively on the performance of the public healthcare system (with a very small exception regarding the performance of midwives). As a result, the performance of public institutions affected all findings reported in this chapter. In November 2014, the MOH decided to discontinue contracts with local healthcare service providers, without being able to immediately reestablish services for ministerial healthcare staff to conduct. Services in many communities were virtually nonexistent for several months and were only slowly reestablished from March 2015 onward, and even later in some communities. This represented a virtual breakdown of all primary public healthcare services in the entire territory of Alta Verapaz during a key programmatic implementation period. Prenatal and postnatal services 40 % of mothers re c eiving the minimum recomm ended care Figure 11 shows that, in spite of the breakdown in the healthcare system, the indicators on mothers who attended four prenatal care visits, and those who received a tetanus shot, supplements, and blood pressure control, remained virtually unchanged. The sample included roughly 10% of non￾participants as well as roughly 42% of participants already graduated (with children who had turned two years of age by March 2015 or before). Most of these pregnancies took place before the eventual breakdown in the healthcare system. Even for the 48% of still active participants, a large share of their pregnancies and deliveries might have taken place before November 2014. Figure 11: Mothers receiving prenatal care assistance at baseline and endline Mother who received immunization and nutritional supplements 66.0% 67.0% Mother who attended prenatal care visits 81.0% 82.0% Mother who received minimum prenatal care 57.0% 57.0% Endline 2015 Baseline 2010 Figure 12: Mothers receiving minimum recommended antenatal and postnatal care at baseline and endline 100 90 80 70 60 50 40 30 20 10 0 Antenatal Postnatal Target Target Baseline survey Final evaluation Both the indicators on percentage of mothers receiving minimum recommended antenatal and postnatal care showed poor performance over the program duration. Although already relatively high at baseline (above 80%), the percentage of mothers receiving minimum antenatal care (defined as attending at least four antenatal visits) dropped by about 2% in 2015, and did not achieve the program target of 90%. The indicator on postnatal care also did not achieve the target set, and remained low at just 30.7% in 2015. Figure 13 shows that the community healthcare center (CHC) was by far the most important reference point for both 2010 and 2015, which is in line with the MOH’s recommendation to always 41 seek the nearest possible service point. There is also an interesting increase in visitations for prenatal services from public service providers, which presumably took place before the aforementioned breakdown in local healthcare services in November 2014. An additional increase in demand (albeit less strong) for private services can be observed (private hospitals, private clinics, others). Presumably, this increase took place only recently, in response to the crisis of public healthcare services. The same would apply to the observed increase of services sought from midwives or CHWs. According to mothers interviewed in FGDs, access to institutionalized antenatal and postnatal care and delivery was related directly to household income, notably the family’s ability to pay for transport from remote rural areas to healthcare facilities in urban centers – despite the Community Fund being implemented to counter this. “I had my delivery at the hospital because the midwife told me to go there. I went there by bus but had to walk two hours in order to get to a point where I could take the bus. I would not have been able to pay for an ambulance since this is very expensive, about 300 Quetzals” (mother, Chitoc). Figure 13: Institutions visited for prenatal care at baseline and endline Others Private Clinic Private Hospital APROFAM Social Security ( IGSS) Public Hospital Health Center 3.0% 7.0% 4.0% 3.0% 2.0% 2.0% 1.0% 1.0% 2.0% 2.0% 14.0% 27.0% Health Post Community Health Center Midwife or Community Health Facilitator 3.0% 1.0% 7.0% 3.0% 91.0% 91.0% Endline 2015 Baseline 2010 In 2015, 44.0% of deliveries occurred at healthcare centers, a significant increase from 34.0% in 2010. The remaining 56.0% of deliveries took place at home. Figure 14 shows differences in main assisting personnel during deliveries. Figure 14: Place of delivery and assisting staff during delivery at baseline and endline 42 Delivery with local health volunteers Delivery with midwife Delivery with doctor or nurse Delivery at health facilities Delivery at home 3.0% 7.0% 13.0% 35.0% 44.0% 34.0% 58.0% 56.0% 65.0% 84.0% Endline 2015 Baseline 2010 Interestingly, although a large (albeit decreasing) share of home deliveries continued to occur, assistance of these deliveries by more qualified medical staff seemed to be more frequent. Despite only 44% of deliveries taking place within healthcare facilities, either a doctor or a nurse, including ‘roving doctors and nurses,’ assisted 84% of all deliveries. Notably, the use of midwives or local CHVs decreased sharply. Preventive Healthcare Practices. Regarding the preventive care of children under two years, the proportion of children with proper immunization and their attendance at periodic growth monitoring and promotion were analyzed. The MOH, through local healthcare service providers, administered immunization services. Growth monitoring and promotion was conducted through a mixed system of local service providers assisted by the CHWs or CHVs from the community. Figure 15: Prevalence of children immunized and measured for growth at baseline and endline Number of children who attended growth monitoring and promotion 16.0% 18.0% Children 18 - 24 months fully immunization 10.0% 78.0% Children 12 -17 months fully immunization 10.0% 69.0% Children < 24 months fully immunization 10.0% 73.0% Endline 2015 Baseline 2010 Poor preventive services are a clear result of the decay (and subsequent breakdown) of community healthcare services provided by MOH, whether directly or indirectly. FGDs confirmed that participants of PROCOMIDA’s emergency funds viewed these funds as one way to access institutionalized healthcare services in cases of emergency, through the loan 43 for transportation and other costs. Regarding the care of newborns, the proportion of mothers who reported that their child had received all components of essential newborn care decreased overall ( Figure 16). For the purpose of the IPTT and program monitoring, PROCOMIDA defined essential newborn care through a composite indicator including drying the newborn immediately after birth, wrapping the newborn in a warm cloth or blanket immediately after birth, and putting the newborn to the breast within one hour of birth. The only newborn care practice that increased between 2010 and 2015 was early initiation of breastfeeding, which was widely promoted through PROCOMIDA. The overall decrease of newborn care suggests unsatisfactory performance of public healthcare staff. Thus, while institutional deliveries and professional assistance during deliveries (even for home deliveries) increased, it appears that the quality of newborn care decreased. As these activities form part of the national public health mandate, PROCOMIDA will not likely be accountable for these poor findings. The percentage of children under two years receiving the minimum treatment for diarrhea increased two-fold between 2010 and 2015, meanwhile the percentage of children receiving minimum treatment for respiratory problems decreased dramatically. Data analyzed indicate that both changes observed are statistically significant at the 0.01 level. Figure 16: Essential newborn care at baseline and endline Newborns wrapped immediately after birth 88.0% 99.0% Newborns dried immediately after birth 86.0% 98.0% Newborns breastfed within one hour of birth Newborns received essential care 81.0% 74.0% 70.0% 75.0% Endline 2015 Baseline 2010 Figure 17: Children receiving minimum treatment for respiratory problems and diarrhea at baseline and endline 44 Children 6 - 23 months receiving minimum treatment for respiratory problems Children 6 - 23 months receiving minimum treatment for diarrhea 14.0% 26.0% 45.0% 50.0% Endline 2015 Baseline 2010 Despite the breakdown in primary healthcare provision, changes were observed during the implementation of PROCOMIDA regarding increased treatment of diarrhea,. A key factor in these treatments consisted of the provision of oral rehydration salts (ORS) to families even without the presence of community center healthcare staff. The qualitative evaluation was able to confirm the availability of ORS in many of the community healthcare centers visited. In the absence of healthcare staff (in early 2015), CHWs administered ORS. Opposite to this example, adequate assistance in cases of respiratory problems required greater support from qualified staff. The combination of lower mother knowledge about respiratory problems (understood as a lower demand for respiratory therapeutic services), and the breakdown of local primary healthcare services, most likely contributed to decreases in this indicator. As in the previous examples regarding service demand (prenatal and postnatal control, institutionalized delivery, newborn care), data indicated that the training and sensitization efforts carried out by PROCOMIDA had the expected positive effect, increasing demand for these services. Local healthcare services were unable to meet increased demand in early 2015 given the systematic crisis in the healthcare sector. "Whatever affects our children, whether it is fever, cough, grains, diarrhea, flu, conjunctivitis, headache, we would visit the community health center, but there are shortages of medicines. For the last six months there was no health attention at all, including weighing children” (non-participating mother, Chitoc). Further, “The community feels abandoned in certain areas, such as health" (advisory mother, San Antonio 1). Infant and Young Child Feeding Practices. Figure 18 shows the comparisons between breastfeeding practices surveyed in 2010 and in 2015, with exclusive breastfeeding under six months rising from 65.1% to 75.7%. At baseline, a high proportion of children already received continued breastfeeding until one year of age. Three quarters of newborns received early initiation of breastfeeding. Statistics between the baseline and the endline increased regarding breastfeeding soon after childbirth. The baseline survey of 2010 indicated that 74% of babies were breastfed within one hour after birth, whereas the endline survey of 2015 suggests that 81% of babies were breastfed in the same timeframe, indicating a 7% increase (Figure 16). In the 24 hours before the interview, 75.7% of children under six months were exclusively breastfed. At the age of two, 88.5% of children continued to breastfeed, as recommended. Breastfeeding practices were extensive (98.7%), and the use of bottle feeding was low (14%). “We breastfeed all around the clock, because there is no specific time for breastfeeding. When (the babies) are already grown up we start to give them food and drinks but do not use bottle-feeding because PROCOMIDA taught us that we should not use bottles because children get sick” (graduated mother, Sacoyou). The program for continued breastfeeding at the age of two achieved an even greater improvement, which increased from approximately 77% to 89%. Figure 18: Breastfeeding practices among mothers at baseline and endline 45 % children 0-6 m onths exclusivel y breastfed Early initiation of breastfeeding (0-23 mths) (n=719/n=897) 78.1% 75.1% Exclusive breastfeeding under 6 mths (n=152/n=171) 75.7% 65.1% Continued breastfeeding at 1 yr (n=120/n=135) 97.0% 97.5% Continued breastfeeding at 2 yrs (n=128/n=165) 88.5% 77.3% Endline 2015 Baseline 2010 FGDs confirmed that early initiation of breastfeeding (but not necessarily exclusive) is traditionally practiced, but that the program successfully encouraged mothers to extend the amount of time they breastfed their children. “Before participating in PROCOMIDA, we stopped breastfeeding our babies as soon as we got pregnant again, even if they had still not reached two years of age. We thought breastfeeding during pregnancy would harm the unborn baby. Now we learned that this is not the case and continue breastfeeding” (graduated mother, Sexucti). Further, "Before the program we knew that we should breastfeed the child at birth. The only difference was that we did this less frequently. With the program we learned that we must breastfeed the child exclusively during the first six months of life, and that there is no fixed schedule for feeding a child" (graduated mother, Chitoc). Figure 19: Percentage of children 0-6 months exclusively breastfed at baseline and endline 90 80 Target 70 60 50 40 30 20 10 0 Baseline survey Final evaluation 46 % with minimum diet ary diversity (children 6-24 m onths) The reduced use of bottle feeding was the indicator that demonstrated the greatest improved performance since baseline. The proportion of bottle fed children decreased from 49% in 2010 to 14% in 2015, with strong statistical significance (chi-square p-value < 0.0001). A high proportion of children (84.2%) were introduced to soft or semi-solid foods at the age of six months, and solid foods at nine months, as recommended. Minimum meal frequency was appropriate for 70.9% of children, and minimum dietary diversity was appropriate, with the inclusion of more than four food groups, in 63.8% of children. About two-thirds (66.7%) of children consumed iron-rich foods in the 24 hours before interviews. "What we learned is that when we start to offer solid food to children of six months of age, we should begin with soft foods, mashed, or broths. We did not know this until the program came, and we are now applying what we have learned. From six months on we start giving food to children but always continue with breastfeeding" (graduated mother, Chitoc). Further, “Previously I thought a small child eats three times a day, just like us. With PROCOMIDA, I learned that my young son should eat five times a day and that even the amount of food is important and that I have to take my time and make him eat the entire ration every time so that he can gain weight and grow up” (graduated mother, Pinares). Figure 20: Dietary diversity in children 6-24 months at baseline and endline 90 80 Target 70 60 50 40 30 20 10 0 Baseline survey Final evaluation Figure 21: Complementary feeding practices at baseline and endline 47 Introduction of solid, semi-solid or soft foods (6-8 months) (n=96/n=39) Minimum dietary diversity (6-24 months) (n=572/n=700) 60.1% 71.6% 84.2% 89.6% Minimum meal frequency (6-24 months) (n=551/n=718) 63.8% 73.3% Minimum acceptable diet (6-24 months) (n=570/n=718) 46.7% 42.3% Consumption of iron-rich or iron-fortified foods (n=570/n=718) 51.7% 66.7% Endline 2015 Baseline 2010 By 2015, the majority of indicators for complementary feeding practices improved, namely, minimum dietary diversity, minimum acceptable diet, and consumption of iron rich foods (Figure 21). The percentage of children 6-24 months with minimum dietary diversity (71.6%) was higher than in 2010 (60.1%), but the difference was not statistically significant (chi-square test p- value = 0.1865). Only two indicators worsened, compared to 2010 (though the baseline was quite high): i) the proper introduction of solid and semisolid foods, and ii) meal frequency. The observed decrease in the percentage of children given a timely introduction of solid food is not statistically significant. The observed decrease in minimum meal frequency is significant (at 0.05 level). Both phenomena, however, could be inter-related.60 Nutritional status of children. Stunting, the main impact indicator defined for PROCOMIDA, decreased by 5.7 percentage points between 2010 and 2015. The average height-for-age z￾scores (HAZ) dropped from -2.24 z-scores in 2010 to -2.09 z-scores in 2015 (Table 12). The observed decrease is seen in both boys and girls, but is only statistically significant among boys (Table 13). Nevertheless, the general level of stunting remained high at over 50%. 60 The observed decrease of timely introduction of solid food is significant at a 0.14 level, which according to handbooks should not be accepted as statistically significant. Our understanding, however, is that given the sensitive nature of the observed topic (behavioral change) we should be more flexible regarding the interpretation of results. In other words, we could say that “in case the PROCOMIDA program would be repeated 100 times, 86 times a decrease in timely introduction of solid food would be observed”, which at least following common sense seems to be a result of important scope. Hence, if we accept that the introduction of solid food being postponed (for whatever reason, is a valid observation, such a behavior could very well lead to a decrease in the minimum meal frequency indicator, as observed in PROCOMIDA. 48 The prevalence of acute malnutrition measured by weight-for-height (WHZ) < -2 SD remained low and relatively stable for children six to 59 months, at 1.1% in 2010 and 1.3% in 2015. Table 12: Nutritional status of children at baseline and endline Nutritional status of Baseline Boys Girls Endline Boys Girls children (2010) (2015) Underweight (<-2 z-score weight-for-age) Age 0-59 months N 1,838 907 931 2,406 1,119 1,085 WAZ (mean ± SD) -1.01 (± - - -1.05 (± 0.96) 0.89) - - Percentage underweight 14.4% 13.5% (confidence interval) 13.3% 14.6% 11.2% 14.0% [11.4 – [12.5 – 14.4] (12.4-17.0) (9.3-13.4) 15.5] (12.4- (11.4- 16.7) 15.9) Stunting (<-2 z-score height-for-age) Age 6-59 months N 1,680 838 842 2,204 1,119 1,085 HAZ (mean ± SD) -2.24 (± -2.09 (± 1.01) - - 0.95) - - Percentage stunted 57.5% (confidence interval) 59.5% 61.5% 53.5% 54.7% 52.3% [57.1 – [50.9 – 61.8] (58.1-64.7) (54.1- 56.0] (51.6-57.8) (49.0-55.5) 60.8) Wasting (acute malnutrition) Age 6-59 months N 1,680 838 842 2,211 1,124 1,087 WHZ (mean ± SD) 0.28 (0.88) - - 0.15 (0.89) - - Percentage wasted 1.4% 0.7% 1.2% 1.4% (confidence interval) 1.1% 1.3% [0.7 – 1.7] [0.9 – 1.9] (0.8-2.5) (0.3-1.5) (0.7-2.2) (0.8-2.4) Table 13: Distribution of stunting among children 6-59 months (<-2 z-score height-for-age), by sex, at baseline and endline Baseline Endline (age 6-59 months) (age 6-59 months) n % stunted n % stunted (<-2 z-score h-a) (<-2 z-score h-a) Total 1,680 1,000 (59.5%) 2,204 1,179 (53.5%) Boys 838 515 (61.5%) 1,119 612 (54.7%) Girls 842 484 (57.5%) 1,085 567 (52.3%) In Figure 22, the average values for underweight, stunting, and wasting were plotted for each month of age of the children. Trend lines were then plotted based on 5-month moving averages. These trends were then compared between baseline and endline. 49 Trends regarding stunting and the comparison between 2010 and 2015 suggested that the prevalence of stunting at birth is lower in 2015. Further, the period of instability and steadily increasing levels of stunting (denoted by a pink background in the figures above) is shorter in 2015, stabilizing at an average of 20 months, compared to 38 months in 2010. Overall duration of the increase in stunting shortened as well, from 33 months to 20 months. Figure 22: Percentage of children with measurements classified as wasting, stunting, and underweight, by age at baseline and endline Objective 2: By 2015, healthcare service providers at the community through municipal levels have improved service quality and delivery Baseline 2010 Endline 2015 Convergence Centers. PROCOMIDA selected CCs using a two-step selection methodology. First, at the municipal level, a list of CCs was compiled from those identified by the government as being in food insecure communities (prioritized through SESAN). In consultation, SESAN and their Municipal Food Security Classification system, and the MOH, selected four municipalities of Alta Verapaz for implementation. Together with PEC, 221 extremely poor rural communities with year-round access to five-ton trucks were selected to participate. Then, the locations of these CCs were assessed for their accessibility by the delivery trucks, necessary for regular (monthly) ration distribution. PROCOMIDA later refined the methodology in consideration of smaller delivery trucks, reducing the issue of road access and increasing the number of eligible CCs including in remote locations to 358 covered by the program. In implementation year five, the program initiated a phased closeout, starting with closure of 141 CCs (July 2014). In the last implementation year, 211 CCs were attended, which explains the low coverage measured in the final evaluation. During the length of the program, the target of 358 CCs was achieved. Since each CC generally covers more than one community, note that many program participants still walk from surrounding communities to the CCs to attend training sessions and receive rations. Regarding household engagement in the effort to reduce vulnerability to food insecurity, one of two monitoring indicators was achieved. In Y5, all 358 CCs covered by the program had emergency funds. PROCOMIDA made good progress under the second indicator (Household Action Plans), but the target was not fully achieved since promotional activities regarding Household Action Plans were started in November 2011. During the baseline survey in 2010, only 1.8% of households reported having Household Action Plans, but the figure increased to 37.4% by 2015, achieving 62% of the targeted figure of 60%. 50 Community Healthcare. Although the target was to have 30% of local CHVs meet minimum standards of performance, the figure at endline showed more than 300% achievement of the target, with 97.7% of local CHVs meeting such standards. Due to changes in the basic healthcare service model, many of these CHVs are actually new and received training directly from the MOH before deployment. PROCOMIDA supported the MOH in these trainings, but these high levels cannot be exclusively attributed to the program. Two other indicators in Table 10 were recorded as 0% achieved in 2015. These were: i) the percentage of healthcare facilities meeting minimum standards for health and nutrition services and practices (Figure 23); and ii) the percentage of healthcare facility staff and CHVs able to identify a minimum number of core health and nutrition practices (Figure 24: CHC staff knowledge regarding danger signs and ORS use; and Figure 25: CHC staff knowledge regarding child feeding practices). These indicator results were likely driven by the way the indicators were defined, as both indicators refer to service provision and the knowledge of local healthcare staff. Taking into consideration the list of minimum knowledge and services to be available, failure in only a single element of the list brings the indictor down to zero. Figure 23: Availability of basic services at CHCs at baseline and endline Growth monitoring 98.0% 93.0% Preventive care (Children) 22.0% 40.0% Prenatal service 11.0% 20.0% Postnatal service 0.0% 2.0% Endline 2015 Baseline 2010 51 Figure 24: CHC staff knowledge regarding danger signs and ORS use at baseline and endline Use of ORS Of Dehydratation During Childbirth In Newborn In Children During postpartum During pregnancy 5.0% 15.0% 12.0% 13.0% 22.0% 29.0% 48.0% 60.0% 56.0% 56.0% 67.0% 80.0% 74.0% 89.0% Endline 2015 Baseline 2010 Figure 25: CHC staff knowledge of child feeding practices at baseline and endline Avoid semi-solid foods < 6 months Avoid other liquids < 6 months Avoid water < 6 months 71.0% 81.0% 81.0% 84.0% 93.0% Use of colostrum 80.0% 85.0% 96.0% Early Initiation Breastfeeding 93.0% 86.0% Endline 2015 Baseline 2010 Local CHC staff knowledge on the following topics was considered a minimum requirement: i) use of ORS; ii) danger signs of dehydration; iii) danger signs during childbirth; iv) danger signs in newborns; v) danger signs in children/childhood diseases; vi) danger signs during postpartum; and vii) danger signs during pregnancy. Eighty percent of staff reported having received training on these topics. The trainings that received the highest attendance were care of sick children (80%) and issues regarding pregnancy (75%). A high turnover of MOH personnel limited PROCOMIDA’s ability to contribute to an increase in their knowledge. Overall, trends are positive. By aggregating the different categories into a single indicator, none of the health staff in 2015 were able to recognize all of the danger signs, or correctly describe ORS use. The corresponding baseline value in 2010 was also low at 6.5%. In 2015, the highest levels of knowledge measured were on ORS use (89% of staff), danger signs of dehydration (80%), and danger signs during childbirth (74%). High levels of knowledge around the initiation of breastfeeding and exclusive breastfeeding were observed in 2015 (over 90% of the staff could adequately identify these recommended practices). 52 Per c enta g e O wn Building Bathroom Electricity ORS A moxicilin Trimetroprim-Sulfametoxazole Penincilin Albendazole Vitamin A Chispitas Iron Folic Acid Prenatal supplements The proportion of staff capable of properly identifying that water, other liquids and semi-solid foods should not be introduced in children under six months was higher than in 2010. Since 2010, 13% of CHCs improved their infrastructure or their access to basic services. By 2015, 96% of CHCs were located within their own buildings/property, and 89% had a toilet, although only 37% had electricity (Figure 26). In 2015, due to MOH limitations beyond the program’s influence, only 11% of CHCs had an available stock of nutritional supplements (micronutrients) for children. Supplements for mothers were more available (57% iron, 50% folic acid), however, levels of all supplements were only half (or even less) of those registered in 2010. The exception was vitamin A, whose availability in 2015 was higher than in 2010, but was still only available at 11% of centers. The same situation applied regarding the availability of medical supplies, which were far below 2010 levels. The most frequently encountered product, ORS, was only available in 50% of the CHCs visited. Medical supplies required for basic treatment of the most prevalent childhood diseases (diarrhea, respiratory infections) were available in fewer than 30% of CHCs visited. Figure 26: Availability of medical supplies at CHCs at baseline and endline 100.0% 80.0% 60.0% 40.0% 20.0% 0.0% Infrastructure Amenities and Medicines supply Children Supplements Supply Women Supplements Supply Baseline 2010 Endline 2015 Regarding the availability of service supplies, the following minimum services for pregnancies and childcare were considered:  Antenatal services – TT vaccination (Tetanus), nutritional supplements, diphtheria vaccination, guidance on danger signs during pregnancy and finding appropriate care, promotion of optimal nutritional practices, hygiene, maternal nutrition, safe sex, family planning, and breastfeeding  Postnatal services - home visits, nutritional supplements, and prenatal services listed above 53  Growth monitoring and promotion services - anthropometric measurements, development of growth curves, guidance on nutrition and health, immunizations  General preventive care services - centers offering all of the following services were considered: de-worming, counseling, anthropometric measures In 2015, CHCs offered a variety of services. While they did not offer post-natal care services, 20% of CHCs (almost twice that in 2010) did offer antenatal care services, and 22% offered general preventive childcare (much lower than in 2010). The availability of these services depended directly on the MOH. Growth monitoring and promotion services promoted and supported by PROCOMIDA (through trainings, achieving commitments from volunteers, and providing measurement equipment) were available in 98% of CHCs. 3.7.3 Trigger Indicators Trends in vulnerability contexts Indicators used by the program to reflect security and coping strategies suggest that the external living conditions for Alta Verapaz communities during the implementation period were less than favorable. Even though food prices showed a decrease at endline compared to baseline, pests and diseases affecting cash crops considerably reduced people’s ability to earn cash income, especially in recent years. Additionally, although average precipitation was normal, most of southern Alta Verapaz suffered from insufficient rainfall. An exploration of adverse conditions and events may provide a better understanding of the scope of the decline in external conditions, and the potential impact this had on poor rural households. Table 14: Adverse conditions and events affecting Alta Verapaz at baseline and endline Year Event/Conditions Description/Consequences 2010 Tropical storm Matthew Caused flooding in several municipalities northern Alta Verapaz. The damage was considerable in Coban, Carchá, Santa Cruz Verapaz, Raxruhá, Chisec, and San Cristobal Verapaz, with flooded areas and damaged crops. 2012/2013 Crisis of coffee rust plague 20% of the 2012/13 crop was lost, representing nearly 19% of income lost. Landless workers and small farmers were the most affected. If crop losses associated with grain quality are added, the income loss rises to 30% - 70%. June - July 2012 Prolonged drought A ‘prolonged heat wave’ resulted in significant rainfall deficits in municipalities in southern Alta Verapaz. San Cristobal, Santa Cruz, Chamelco, Tactic, Tamahú, and Tucurú were affected by a ‘strong’ drought. The impact was quantified as a 40%-50% loss of staple food production (grains, corn). Lower demand for labor and increased food prices (13%) resulted. 2014 Recidivism of prolonged heat wave A lack of rain of up to 29 to 45 consecutive days was recorded in July 2014, causing losses between 60% (corn) and 80% (beans). A lack of drinking water was also reported, and an increase of acute malnutrition in children under five years. Among all 22 Guatemalan Departments, the highest increase was reported in Alta Verapaz, with 554 cases. Lingering effects of coffee rust: Wages in the sector suffered a 50% decrease, and the levels and quality of the coffee harvest was poor. Alta Verapaz was declared to be in a state of ‘food insecurity crisis.’ Households relying on the coffee harvest (15%-20%) saw their revenues cut by 50%-55%. SESAN began distributing food rations and the World Food Program and Save the Children begin distributing cash transfers. Cardamom price drop The price of cardamom plummeted due to overproduction and the negative effect of a thrips outbreak. The incidence of the outbreak reached 40% of production, and 300,000 small producers. The effect of the outbreak reinforced smallholder problems – smallholders who had already suffered from a price drop resulting from overproduction by large-scale farmers. The average price per ‘quintal’ (100 pound bag) dropped from US$645-193. 2015 Coffee does not recover Despite projected increases in coffee rust control, the harvest continued to suffer. The sector showed a harvest below normal production for the third 54 consecutive year, with production 16% lower than the harvest recorded in 2012 - before rust infestation. The area remains in a situation of ‘food insecurity.’ 55 4 CONCLUSIONS, LESSONS, AND RECOMMENDATIONS 4.1 Conclusions It is the conclusion of this evaluation that, through analysis of key final evaluation findings, linked to strategic objectives and expected results, PROCOMIDA achieved its’ goal of improving the nutritional status and health of women and children vulnerable to food insecurity in northern Guatemala. Despite external challenges (economic, systemic, political), PROCOMIDA staff managed to deliver efficiently both products and services in a complex geo-cultural and logistical environment, benefitting tens of thousands of community members. While program participation was voluntary, 88.9% of eligible mothers and children ultimately engaged in program activities. The following evaluation scores are assigned to PROCOMIDA in view of endline results. Table 15: Evaluation ratings Evaluation Criteria Assigned Score Comment Thematic relevance and design Relevant The thematic relevance was extremely high, but given the research approach selected by the donor for this PM2A research program, the scope of the thematic intervention was limited. Regarding design, the program defined a set of targets for local healthcare service performance and the knowledge and capacities of local healthcare staff, which were dependent on MOH interventions outside program control. This evaluation understands that the target levels defined were desirable to support other outcomes of the program. From a technical point of view, the evaluation team considers it a mistake to have fixed targets for outcomes that were not within the program’s control. The program’s intention was to improve availability and quality of MOH services at the primary (community) level. That this was unsuccessful was outside of the program’s control – and indicates that defined targets should have been more under the program’s control. Effectiveness Satisfactory (minor shortcomings) for all outcomes within PROCOMIDA’s control. Unsatisfactory for the non-achievement of other important targets outside PROCOMIDA’s control. The expected results for required activities and effects for reduced undernutrition fully materialized for all activities fully controlled by PROCOMIDA (food distribution, trainings, improvement of local health services up until the second semester of 2014). It is plausible that without the substantive worsening of external conditions, effects on undernutrition would have been stronger. From a technical point of view, the program was less effective in achieving outcomes and targets in the sector of supporting public healthcare service provision. These results were not under the program’s control, and while PROCOMIDA’s support was technically sound and most likely had the potential to achieve defined targets, the non-achievement of the targets must be rated as unsatisfactory. Influence Significant Econometric analysis confirmed the significant influence on improving undernutrition levels and trends by contributing to: i) reduction of stunting at birth; ii) reduction of the growth faltering period under six months; and iii) reduced level of stunting for children under five years with an additional increased probability that much of the achieved improvement would endure over time. The difference between participants and non-participants regarding these results is statistically significant. For benefiting children, participation in PROCOMIDA represented an opportunity for their future, not likely possible in the absence of the program. Efficiency Highly satisfactory (no shortcomings) PROCOMIDA staff were highly efficient managers of product and service delivery in a complex environment (and topographic area) 56 managing extreme logistical challenges in hundreds of communities for tens of thousands of participants. Activity delivery was accomplished without shortcomings (which does not imply there were no problems, but rather that all problems were satisfactorily addressed and resolved in a timely manner). The program achieved participation rates above average for this kind of intervention. Sustainability Moderately likely (moderate risks) There is a high possibility that nutritional outcomes achieved for beneficiary participants under two years of age will endure into the future. There is, however, only a moderate likelihood that mothers and families might experience conditions to productively use their acquired knowledge (which translates into capacities and behavioral changes in a minor degree compared with what was anticipated by the program) in order to protect future pregnancies and early childhood development effectively. In addition, it is only moderately likely that the public sector will be able (given existing instructional capacities and available resources) to sustain service provisions, which enabled achievement of PROCOMIDA’s outcomes. To a certain degree, PROCOMIDA did not anticipate the need for deeper engagement through capacity building interventions and exit strategies with health districts, and concentrated too heavily on supporting local levels. This evaluation is aware that the fragility and virtual breakdown of primary healthcare provision in late 2014 widely determined this situation. Gender Significant Gender results were significant regarding empowerment, which was the main goal of the program. Gender results were, however, much weaker regarding equity results and inclusion of fathers and in-laws, including mothers-in-law. Overall Satisfactory (minor shortcomings) Overall, PROCOMIDA was relevant, well designed technically, and efficiently implemented. Results and activities materialized as expected and proved to have the power to achieve the expected scope of nutritional core results. This happened under an adverse external environment. Without the presence of PROCOMIDA, the nutritional situation in Alta Verapaz would likely have been worse. The program was committed to the achievement of results that depended on the performance of the public health system, which was not controlled by the program. The evaluation team considered this fact as most certainly an unavoidable yet important shortcoming, technically speaking, since the achievement of these outcomes most probably would have reinforced program outcomes even more. More specifically, PROCOMIDA:  Validated the PM2A approach, even when they implemented only health and nutritional interventions. Reductions in stunting, achieved through access to supplementary food rations and improved childcare and household environments, confirm this.  Improved nutrition and reduced undernutrition. Econometric assessment of outcomes, and undernutrition trends, show impact from nutritional improvements felt well beyond program graduation. There was statistical significance between program participants and non-participants in this area. A slight increase in the percentage of underweight children birth to 59 months was seen (13.3% baseline; 14% endline), yet the percentage of children six to 59 months presenting stunted fell (59.2% baseline; 53.5% endline), and the HDDS score improved (7.19 baseline; 7.26 endline). The annual decrease in stunting (1.14 percentage points) was a considerable improvement when compared against the greater Alta Verapaz Department national survey results, which 57 show only an average decrease of 0.17%/year (1999 to 2009), 61 and against overall national results during PROCOMIDA’s implementation period of 0.55%/year (2014- 2015). 62 While there is high sustainability potential for children and households who participated in the program to maintain nutritional outcomes, and become healthier and more productive adults, there is only moderate likelihood that: i) mothers and households will function in environments enabling application of program knowledge and skills to effectively protect future pregnancies and the early development of current and future children; and ii) the government’s primary healthcare sector will be able to maintain and advance program improvements given existing structural barriers (skills, resources).  Achieved key Objective 1 outcomes including improved household knowledge. Nutrition- and training-related activities were technically sound and, based on formative research, both necessary and culturally appropriate. PROCOMIDA staff implemented them efficiently and effectively. Consistent nutritional supplementation of children and/or their mothers, knowledge and skills on key hygiene practices and behaviors, and dietary diversification all contributed to varying degrees to the improved nutritional status of participating children. Participating families demonstrated improved knowledge regarding childcare, illness warning signs, family planning, and nutrition. Households that participated in program activities demonstrated increased efforts to reduce their own vulnerability to food insecurity through continued efforts to improve household conditions, diversify their diets, and adopt behaviors related to IYCF (exclusive breastfeeding, active feeding) independent of financial ability.  Achieved many Objective 2 outcomes including strengthened local structures and infrastructure. PROCOMIDA staff provided high quality technical support, and were largely on track to reach targets through FY 2014. Activities focused on strengthening the quality and accessibility of primary healthcare services in rural areas. Communities benefitted from strengthened local structures for self-management (establishment of health commissions and emergency funds; developed local health planning capacity) and improved infrastructure (store rooms, primary healthcare centers). Implementation, however, occurred in a context of increasingly critical external threats, which ultimately exposed the fragility of the PEC delivery model. When conditions beyond program control interceded, the PEC system broke down and many primary healthcare-related activities were suspended. While some indicators suggest that at least minimum health standards still increased (regarding service provision), after the redesign of the public healthcare system in Alta Verapaz (re-launched late spring 2015), only limited capacity and skills provided to local healthcare staff by the program remained given new service delivery structures. It is not yet possible to realistically speculate on how well new primary healthcare services might engage with community structures (health commissions, emergency funds) strengthened under PROCOMIDA. During evaluation fieldwork, community leaders expressed their intention to maintain and utilize infrastructure, co-opting them to support community health-related activities, and were optimistic that new primary healthcare services would engage with community structures initiated and/or strengthened by PROCOMIDA. 61 MSPAS (2011). Encuesta Nacional de Salud Materno Infantil 2008 (ENSMI-2008/09). Ministerio de Salud Pública y Asistencia Social (MSPAS)/Instituto Nacional de Estadística (INE)/Centros de Control y Prevención de Enfermedades (CDC). Guatemala 62 MSPAS (2015). VI Encuesta Nacional de Salud Materno Infantil (ENSMI-2014/15). MSPAS/INE/Secretaría de Planificación y Programación de la Presidencia (Segeplán). Guatemala. 58  Promoted healthier pregnancies. This was an important result of family planning BCC, leading to healthier birth spacing and pregnancy timing among program participants, as measured through household survey data and confirmed by qualitative fieldwork. Coordinated and simultaneous interventions of other organizations (family planning, contraception distribution) complemented the program’s family planning activities in implementation areas.  Gender empowerment. A crosscutting programmatic theme, gender results showed greater female participation. In particular, women were more involved in decision-making around health and nutrition. Increases were noted in mother’s knowledge on child nutrition, and danger signs of child illness and during pregnancy. Home gardens enabled women to be more involved in food production and dietary diversification decisions for their families. Pertaining to gender equity, results were weaker. The almost exclusive concentration on the participant mother-child unit limited involvement of other family members, including fathers, in-laws, and siblings. This was a lost opportunity to reinforce the drivers of change, only realized in the late stages of implementation when the program did initiate work with men and key family members including mothers-in-law. 4.2 Lessons and Recommendations We provide lessons and recommendations for the improvement of future Title II DFAP and similar activities.  The BCC strategy was crucial to achieving program outcomes. It was important that it was sensitive to local sociocultural and gender contexts, and flexible to periodic adjustment throughout implementation, based on new learning, monitoring evidence, and necessary environmental changes. That participating women benefited (e.g., healthier birth spacing and pregnancy timing) supports the importance of including social and behavior change (SBC) efforts in future Title II PM2A programs on topics of reproductive health and family planning. Information and messages validated over application of the program’s BCC strategy, as well as information gathered through M&E processes, has value and considerable potential for further program design. The initial BCC strategy included too many key messages, and the curriculum to cover all spread over two years. At midterm, the program identified that this was seriously affecting appropriation of these messages by participants, and decided to reduce the messages to half. In addition, the training sessions became routine, and active participation by women began to decline. More participatory processes put into place included Mother Leaders and Household Action Plans. CHCs, and CHVs, were more prominently involved, increasing retention of messages amongst participants. Future strategies should use a more inclusive SBC focus, prioritize more strictly key messages, and involve community members in SBC sessions.  Awareness-raising efforts were continuous throughout implementation regarding program scope and research components. Despite these efforts, however, participant expectations were higher than what was possible to deliver. Participants found it difficult to accept that: i) different communities received different food packages; ii) food rations were complementary (and hence did not cover full household food needs); iii) food rations ceased when children reached two years of age; and iv) outside of home gardens (which were not implemented through the entire program period), there were no livelihood components. This was particularly true for participants with exposure to other food aid programs. Enhanced communications, particularly with Mother Leaders and other village 59 opinion leaders, might have overcome some of these unmet expectation-related challenges. Of note, for the home gardens to be successful, communities must purchase non￾hybridized seeds to ensure seed saving opportunities.  Endorse the PM2A approach (predecessor to the first 1,000 days). PROCOMIDA added to evidence from similar programs, confirming that nutrition-specific interventions should: i) target women and children in the first 1,000 days; ii) deliver preventive nutrition-specific interventions; and iii) emphasize promotion and support of IYCF practices through BCC strategies based on formative research. Disclosure and dissemination of IFPRI research findings is a key step for further consolidation of PM2A. Given evidence already gathered on timing and duration of this approach, the details of which type of ration and food should be delivered is key to reaching greater efficiency and affordability.  Capacity strengthening interventions for community members, inclusive of a Household Action Plan, home garden promotion, and cooking demonstrations, should be implemented throughout program periods. Trainings and education should not only target mothers, but fathers, in-laws, and other caregivers, to promote across households the concept of responsible caregiving and parenthood, childcare, health, and nutrition. Special attention should be made to ensure that training materials are linguistically and literacy￾level compatible with target populations. Capacity strengthening for local public healthcare service units should focus on deeper involvement of mid-level administrators and healthcare service providers. 60 REFERENCES Black R.E. et al. (2013). Maternal and child undernutrition and overweight in low-income and middle-income countries. Maternal and child nutrition series. Lancet, 382:9890, 427-451. Black, R.E., et.al. (2008). Maternal and child undernutrition: global and regional exposures and health consequences. Lancet, 371(9608), 243-60. Cabrera, M., et.al. (2015). Fiscal policy, inequality, and ethnic divide in Guatemala. CGD Working Paper No. 397, 22-23. (http://www.cgdev.org/sites/default/files/CGD-Working-Paper￾397-Cabrera-Lustig-Moran-Fiscal-Policy-Inequality-Ethnic-Divide-Guatemala.pdf) CONEVAL (2010). Dimensiones de la seguridad alimentaria: Evaluación Estratégica de Nutrición y Abasto. Mexico DF (http://www.coneval.gob.mx/rw/resource/coneval/info_public/PDF_PUBLICACIONES/Dimensi ones_seguridad_alimentaria_FINAL_web.pdfhttp://www.coneval.gob.mx/rw/resource/coneval/ info_public/PDF_PUBLICACIONES/Dimensiones_seguridad_alimentaria_FINAL_web.pdf) Del Ángel-Pérez, A., J. Villagómez-Cortés (2014). Food, health and poverty in urban marginal areas: a case study in Veracruz-Boca del Rio, Mexico. Estud. Soc, vol.22 no.44, México jul./dic (http://www.scielo.org.mx/scielo.php?pid=S0188- 45572014000200001&script=sci_arttext). FAFSA-2 (2013). Second food aid and food security assessment. Washington DC, FHI 360. Fanta-2 Project (2009). Alternative sampling designs for emergency settings: A guide for survey planning, data collection and analysis. FANTA-2, Washington DC, Academy for Educational Development. FAO (2014). Measurement of dietary diversity for monitoring the impact of food based approaches. In Thompson, B., and L. Amoroso (Eds), Improving diets and nutrition: Food based approaches, 284-289. Rome: FAO. FEWSNET. Food security outlook 2011, 2011. Guatemala. MFEWS, FAO, USAID (2009). Guatemala: Perfil de Medios de Vida. Moratorell, R. (2012). Interventions and policy options for combating malnutrition in Guatemala. Inter-American Development Bank. (www.iadb.orghttp://www.iadb.org) MSPAS, INE, CDC (2010). Encuesta nacional de salud materno-infantil 2008 (ENSMI￾2008/2009). Guatemala. Oxfam (2012). Evaluación rápida. Impacto de la sequía meteorológica de 2012 en la seguridad alimentaria nutricional en el corredor seco de Guatemala. Guatemala. Palmieri, M., H.L. Delgado (2010). Análisis situacional de la malnutrición en Guatemala: sus causas y abordaje. Programa de las Naciones Unidas para el Desarrollo, Guatemala. Richter, S., et.al., (2010). Strengthening and evaluating the MP2A Approach in Guatemala: Cross-sectional baseline report PROCOMIDA. Mercy Corps, Guatemala. Swindale, A. (2004). The impact of Title II MCHN programs on the nutritional status of children USAID office Food for Peace. Occasional Paper No. 4. UN (2014). Guatemala: Sequía por canícula prolongada 2014. UN. Oficina de la coordinadora residente reporte de situación No. 01, Guatemala. 61 USAID (1000). Fanta Sampling Guide. (http://pdf.usaid.gov/pdf_docs/Pnacg172.pdfhttp://pdf.usaid.gov/pdf_docs/Pnacg172.pdf) WFP (2011). Evaluación de la seguridad alimentaria de los pueblos afrodescendientes, en las comunidades de las yungas de La Paz. (http://es.wfp.org/sites/default/files/pma￾afrobolivianosyyungas.pdf) World Bank (2009). Guatemala Poverty Assessment. WB Report No.43920-GT, Washington DC. (http://siteresources.worldbank.org/INTLACREGTOPPOVANA/Resources/GuatemalaPoverty AssessmentEnglish.pdf) Additional PROCOMIDA documents reviewed:  All ARRs  All PREPs  PROCOMIDA Proposal  Midterm Evaluation  BCC Strategy  Formative Research Study (Baseline) 62 APPENDICES Annex No. Title Page 1 Scope’s of Work (A, 2014; B, 2016) 62 2 List of PROCOMIDA staff and key stakeholders interviewed 86 3 Methodological aspects of the Impact Estimation 87 4 PROCOMIDA Results Framework and Nutritional Indicators 90 5 Survey Methodology and Sampling 95 6 Quantitative Survey Questionnaires 103 7 Qualitative Questionnaires 146 8 Data Collection Agenda 154 9 List of PROCOMIDA Key Messages 157